Method for generating detection model, monocular 3D target detection method and device thereof

Abstract

The application provides a generation method of a detection model, a monocular 3D target detection method and a device thereof. The generation method comprises the following steps: acquiring a first image set and a second image set, the images in the first and second image sets are collected by an image collection device on a vehicle, the image collection device is used for collecting images of the outside of the vehicle, the definition of the images in the first image set is higher than that of the images in the second image set, and each image in the second image set is labeled with a target; creating a detection model based on FCOS3D, in the detection model, a Velocity branch in FCOS3D is replaced by a blur attribute branch; and training the detection model based on the first and second image sets. The detection model can detect blurred targets.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, and in particular to a detection model generation method, a monocular 3D target detection method and device thereof. BACKGROUND

[0002] The input data of 3D target detection is divided into image and point cloud data, and laser, depth camera and monocular camera, binocular camera and other hardware are used respectively. In terms of cost, the monocular camera is the lowest in cost, and it is an algorithm that the industry urgently needs to research and develop (i.e. monocular 3D target detection algorithm).

[0003] The monocular 3D target detection algorithm is mainly used to obtain the 3D bbox (bounding box) of the object. The definition of 3D bbox is the smallest cuboid that encloses the target object in the real three-dimensional world. In theory, a 3D bbox has 9 degrees of freedom, 3 of which are position, 3 of which are dimension size, and 3 of which are rotation angle. The 3D target detection task can be divided into two parts from a macroscopic point of view: target positioning and target pose description. The mainstream monocular 3D detection algorithm generally includes: Deep3Dbox algorithm, DeepMANTA algorithm, GS3D algorithm introduction and M3D-RPN algorithm.

[0004] In many actual automatic driving scenarios, there are clear and blurred targets. The blurred target refers to the target whose boundary is not clear through the image. The cause of blurring is also many, such as bad weather (such as rainy day, foggy day), camera shaking or target fast moving, etc., so blurring cannot be avoided. This kind of target also needs to be detected, otherwise it is easy to cause accidents.

[0005] Therefore, in the case of blurred target, how to perform monocular 3D target detection is a problem to be solved. SUMMARY

[0006] The purpose of the present application is to provide a detection model generation method, a monocular 3D target detection method and device thereof.

[0007] In order to achieve one of the above-mentioned purposes, an embodiment of the present application provides a method for generating a detection model for monocular 3D target detection, comprising the following steps: obtaining a first image set and a second image set, the images in the first and second image sets are collected by an image collection device on a vehicle, the image collection device is used to collect images of the outside of the vehicle, the clarity of the images in the first image set is greater than that of the images in the second image set, and each image in the second image set is labeled with a target; creating a detection model based on FCOS3D, in which the Velocity branch in FCOS3D is replaced by a blur attribute branch; and training the detection model based on the first and second image sets.

[0008] As a further improvement of the embodiment of the present application, the output of the blur attribute branch is HxWx1.

[0009] As a further improvement of the embodiment of the present application, the labeling of each image in the second image set with a target specifically comprises: each image in the second image set is labeled with a target, the labeling information of each target includes: the X, Y and Z coordinate values of the center point of the target in the image collection device coordinate system, the size values of the target along the X, Y and Z axes, the azimuth angle yaw, the class label and the confidence score score of the target, and the blur degree value blur_type.

[0010] As a further improvement of the embodiment of the present application, the training of the detection model based on the first and second image sets specifically comprises: training the detection model based on the first and second image sets, the training process includes two stages; in the first stage, the initial learning rate is 0.001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the class, the weight of traffic_cone is 0.2, the remaining weights are all set to 1, and the training number of epoch is 20; in the second stage, the initial learning rate is 0.0001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the blur degree, and the training number of epoch is 10.

[0011] The embodiment of the present application also provides a generation device of a detection model for monocular 3D target detection, comprising the following modules: an image acquisition module, configured to acquire a first image set and a second image set, wherein the images in the first and second image sets are all acquired by an image acquisition device on a vehicle, the image acquisition device is configured to acquire images outside the vehicle, the definition of the images in the first image set is higher than that of the images in the second image set, and each image in the second image set is labeled with a target; a model creation module, configured to create a detection model based on FCOS3D, wherein the Velocity branch in FCOS3D is replaced by a blur attribute branch; and a training module, configured to train the detection model based on the first and second image sets.

[0012] As a further improvement of the embodiment of the present application, the output of the blur attribute branch is HxWx1.

[0013] As a further improvement of the embodiment of the present application, the image acquisition module is further configured to: each image in the second image set is labeled with a target, and each target is labeled with information including: the X, Y and Z coordinate values of the center point of the target in the image acquisition device coordinate system, the size values of the target along the X, Y and Z axes, the azimuth angle yaw, the category label and the confidence score score of the target, and the blur degree value blur_type.

[0014] As a further improvement of the embodiment of the present application, the training module is further configured to: train the detection model based on the first and second image sets, and the training process comprises two stages; in the first stage, the initial learning rate is 0.001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the category, the weight of traffic_cone is 0.2, the weights of the rest are all set to 1, and the training number of epoch is 20; in the second stage, the initial learning rate is 0.0001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the blur degree, and the training number of epoch is 10.

[0015] The embodiment of the present application also provides a monocular 3D target detection method for a vehicle, the front side, rear side, left front side, right front side, left rear side and right rear side of the vehicle are all provided with image acquisition devices, and the image acquisition devices are configured to acquire images outside the vehicle; the method comprises the following steps: executing the generation method, and obtaining the detection model; controlling all the image acquisition devices in the vehicle to acquire a to-be-processed image, and inputting each to-be-processed image into the detection model, so as to obtain a plurality of output targets; and fusing and displaying the plurality of output targets at the same position.

[0016] The embodiment of the present application also provides a monocular 3D target detection device for a vehicle, the front side, the rear side, the left front side, the right front side, the left rear side and the right rear side of the vehicle are provided with image acquisition devices, and the image acquisition devices are used for acquiring images of the outside of the vehicle; the monocular 3D target detection device comprises the following modules: a model generation module, which is used for the generation method and obtains the detection model; a processing module, which is used for controlling all the image acquisition devices in the vehicle to acquire a to-be-processed image, and inputting each to-be-processed image into the detection model, so as to obtain a plurality of output targets; and fusing and displaying the output targets at the same position.

[0017] Compared with the prior art, the technical effect of the present application is that: the embodiment of the present application provides a generation method of a detection model, a monocular 3D target detection method and a device thereof, the generation method comprises the following steps: acquiring a first image set and a second image set, the images in the first and second image sets are acquired by image acquisition devices on the vehicle, the image acquisition devices are used for acquiring images of the outside of the vehicle, the definition of the images in the first image set is higher than that of the images in the second image set, and each image in the second image set is labeled with a target; creating a detection model based on FCOS3D, in the detection model, the Velocity branch in FCOS3D is replaced by a blur attribute branch; and training the detection model based on the first and second image sets. The detection model can detect blurred targets. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is an example diagram of the image acquisition device in the embodiment of the present application;

[0019] Figure 2 and Figure 3 is an example diagram of the image in the embodiment of the present application;

[0020] Figure 4 is a structural schematic diagram of FCOS3D;

[0021] Figure 5 is a flow schematic diagram of the generation method of the image in the embodiment of the present application. DETAILED DESCRIPTION

[0022] The present application will be described in detail below with reference to the embodiments shown in the drawings. However, these embodiments do not limit the present application, and the changes in structure, method or function made by those skilled in the art according to these embodiments are all included in the protection scope of the present application.

[0023] As used herein, terms such as "upper," "above," "lower," "below," and the like, which describe a spatial relationship between one element or feature to another element or feature as shown in the figures, are utilized for convenience and for the purpose of illustrating the embodiments of the application. The spatially relative terms can be intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as being "below" or "beneath" other elements or features would then be oriented "above" the other elements or features. Thus, the exemplary term "below" can encompass both an orientation of above and below. The device can be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

[0024] Embodiment one of the present application provides a generation method of a detection model for monocular 3D target detection, as shown in the figure, comprising the following steps: Figure 5

[0025] Step 501: Obtain a first image set and a second image set, the images in the first and second image sets are all collected by an image collection device on the car, the image collection device is used to obtain the images outside the car, the definition of the images in the first image set is higher than that of the images in the second image set, and each image in the second image set is labeled with a target.

[0026] In practice, the weather conditions when the car is driving may be different (for example, rainy, foggy or sunny, daytime or night); in addition, the road conditions may also be different, which may cause the image collection device to shake; in addition, the speed of the objects outside the car may be very fast; it can be understood that these factors may cause the definition of the images obtained by the image collection device to be different, some clear and some blurred; at this time, someone can distinguish which images are clear and which images are blurred, or an image definition judgment method can be used to distinguish which images are clear and which images are blurred, and for blurred images, the target therein can be labeled by artificial; Figure 1 Part of the image collection device is shown.

[0027] Here, the essence of image blurring is the loss of high-frequency information (such as edges, textures and details, etc.) in the image, which is manifested as: the edges of the object become unclear, the transition area becomes wide; details (such as textures, small size objects) disappear or contrast is reduced; the overall change range (gradient) of pixel value is reduced. Therefore, the core of judging blurring is to quantify the richness of high-frequency information in the image.

[0028] ​Methods for judging image sharpness include: 1. Gradient / edge-based method, the edges of an image are formed by the abrupt change (gradient) of pixel values, and blur will cause the gradient to decrease. By calculating the gradient intensity distribution of the image, the blur degree can be judged; specifically including: Sobel operator / Laplacian operator and Tenengrad function. 2. Fourier transform-based method, Fourier transform converts an image from the spatial domain to the frequency domain, high-frequency components correspond to details and edges, and low-frequency components correspond to smooth areas. The high-frequency components of a blurred image will decay. 3. Statistical feature-based method, blur is judged by extracting statistical features (such as contrast, entropy) of the image. The contrast is the dynamic range (such as maximum-minimum) or variance of the pixel values in the image. The contrast of a blurred image is usually low (the difference between pixel values is small). Entropy reflects the richness of image information, and the entropy value of a blurred image will be lower than that of a clear image. 4. Machine learning-based method.

[0029] In Figure 2 In the illustrated case, the 3D bounding box in the radar point cloud is labeled. In Figure 3 In the illustrated case, the 3D box in the radar point cloud coordinate system can be projected onto the image.

[0030] Step 502: Create a detection model based on FCOS3D, in which the Velocity branch in FCOS3D is replaced by a blur attribute branch.

[0031] The 3D box in the point cloud coordinate system can be projected onto the image through the internal and external parameters. In the case of no truncation, it is 8 points, and in the case of truncation, it is less than 8 points.

[0032] The blur degree of a single image is divided into three grades: clear grade refers to the polygon boundary composed of 8 points (may be less than 8) of the target in the image being completely visible to the human eye, and the position being clearly visible; general grade refers to the polygon having unclear places; and blurred grade refers to the polygon being blurred.

[0033] The blur degree of a certain 3D target is the comprehensive degree of the blur degrees of 6 images, and is calculated according to the highest degree. That is, if there is one clear grade in the 6 images (there may be only one or two images on which the target can be seen), the target is clear grade, otherwise, whether there is a general grade is checked, and if there is no general grade, it is blurred grade.

[0034] Figure 4 The network structure diagram of FCOS3D is shown, wherein the black dashed box circled is the Velocity branch.

[0035] FCOS3D is an improved 3D object detection scheme based on FCOS. In the FCOS3D neural network model, the Velocity branch is mainly used to predict the velocity information of target objects, and is an important part of the model to achieve more comprehensive detection and understanding of 3D objects. Here is a specific introduction:

[0036] The Velocity branch mainly predicts the velocity components of target objects in a two-dimensional plane, usually represented by and , with units of m / s. Through the prediction of these two velocity components, the model can obtain the horizontal motion speed of the target, providing key information for subsequent target tracking, behavior prediction, and other tasks.

[0037] The detection head of the FCOS3D model is composed of 4 shared convolution blocks and small heads for different targets. The Velocity branch is the part of the detection head used for velocity prediction, which works together with other branches that predict target depth, size, angle, and other information to complete the prediction of 3D information of the target.

[0038] Usually, the feature map extracted and fused by the backbone network and the neck network is sent to the convolution layer corresponding to the Velocity branch. Through convolution operation, the input feature map is converted into single-channel output, and then the ReLU activation function is used to obtain the velocity prediction result.

[0039] Accurate velocity prediction is crucial for practical applications such as autonomous driving. For example, in the autonomous driving scenario, vehicles need to know the speed of surrounding target objects in real time to make reasonable decisions such as acceleration, deceleration, and avoidance. The Velocity branch enables the FCOS3D model to provide target speed information for autonomous driving systems, improving the safety and decision-making ability of the system. In addition, in the target tracking task, velocity information helps to more accurately predict the future position of the target, improving the accuracy and stability of tracking.

[0040] In the field of artificial intelligence, the "Fuzzy Attribute Branch" (Fuzzy Attribute Branch) is a model component that combines the idea of fuzzy logic (Fuzzy Logic), mainly used to handle tasks with fuzzy, uncertain, or continuous transition characteristics in target attributes.

[0041] The core idea is that traditional attribute prediction (such as "the color of the object is red / blue", "the size of the object is large / small", etc.) usually uses discrete labels and belongs to the "hard classification" of this or that. The "fuzzy attribute branch" draws on the concept of fuzzy logic and believes that many attributes are essentially continuous and have fuzzy boundaries (for example, "reddish purple", "medium size", etc.). Therefore, the core of this branch is not to predict attributes as a single discrete label, but to output the "membership degree" of the target belonging to multiple attribute categories, that is, a value between 0 and 1, indicating the degree to which the target meets the attribute. For example, for the "color" attribute, the model may output "red: 0.3, purple: 0.7", indicating that the target is more biased towards purple but with a red component.

[0042] In this detection model, the fuzzy attribute branch can predict fuzzy targets in the image, for example, fuzzy targets based on attributes with fuzzy boundaries such as object color (transition from red / pink / purple), texture (degree of smoothness / roughness), posture (intermediate state of standing / sitting), etc.

[0043] The fuzzy attribute branch usually outputs a vector, the vector dimension is equal to the number of attribute categories, each element represents the membership degree of the target belonging to the corresponding category, and the sum of all elements can not equal 1 (unlike probability, fuzzy membership does not require normalization). The loss function of the fuzzy attribute branch can be "fuzzy cross-entropy" and "mean square error", etc.

[0044] The shared Head does not directly output the above-mentioned target center point position xyz in the camera coordinate system, but the pixel coordinates xyz and depth of the target center point in the image coordinate system. A post-processing is also needed to convert the position in the image coordinate system to the output in the camera coordinate system through the intrinsic parameters. HxW on the diagram means that each cell of the feature map HxW has these values, and during post-processing, it is determined whether the position is a target according to the classification branch score and the fuzzy degree value.

[0045] The backbone uses resnet50 (fixed, searchable). The FPN neck is also commonly used, and the shared Head is some convolution operations.

[0046] Step 503: training the detection model based on the first and second image sets.

[0047] The detection model can detect fuzzy targets.

[0048] In this embodiment, the output of the fuzzy attribute branch is HxWx1.

[0049] In the embodiment, each image in the second image set is labeled with a target, and each target is labeled with information including: in the image acquisition device coordinate system, X coordinate value, Y coordinate value and Z coordinate value of the center point of the target, dimension value of the target along the X axis, dimension value of the target along the Y axis and dimension value of the target along the Z axis, azimuth yaw, category label and confidence score score of the target; and blur degree value blur_type.

[0050] The input is a single RGB image, and the output is (x, y, z, sx, sy, sz, yaw, label, score, blur_type) for each target, which respectively represent the position x / y / z of the target center point, the size sx / sy / sz, the azimuth yaw, the category label and the confidence score score in the image acquisition device coordinate system (note that it is not the image coordinate system nor the radar coordinate system). Compared with Fcos3D, the speed vx / vy is removed, and the blur degree is added.

[0051] Difference between blur_type and score: score is 1 / 0 in the training stage, indicating the presence or absence, blur_type is 0 / 1 / 2, indicating three degrees, and score is more inclined to the presence or absence of the target at the position, without considering the accuracy of the target position, the accuracy of the category, and the presence or absence of the cut, while blur_type represents the blur degree, which is a comprehensive evaluation of these indicators, and is more suitable for the self-driving scene.

[0052] In the inventor's experiment process, the processing of the training data includes the following steps:

[0053] Step 1: Sensor configuration, used for data annotation collection, 128-line mechanical radar installed on the roof, and 6 cameras installed on the front, rear, left front, right front, left rear and right rear.

[0054] Step 2: Sensor calibration, used for target position transformation between cameras, radar and images, calibration of camera intrinsic parameters (3x3 matrix) and camera-to-radar extrinsic parameters (4x4 matrix).

[0055] Step 3: Platform imports annotation data, imports point cloud, 6 images, intrinsic and extrinsic parameters into the platform, and a 3D box under the point cloud can directly display the projection effect on the image.

[0056] Step 4: Annotation: 3D box under the point cloud, including x / y / z / sx / sy / sz / yaw / label, blur degree on the image where the box is visible, several images with several blur degrees, and the blur degree of the same target may be different on different images.

[0057] Step 5: Data augmentation, not enough blurred data, need to create some. Specifically: the clear target is projected onto the image, and the polygon area composed of 8 points is subjected to different degrees of Gaussian blur. This target has a normal version and a blurred version. Process 3-5 targets on a picture to form a new frame of training data.

[0058] In this embodiment, the training of the detection model based on the first and second image sets specifically includes:

[0059] The detection model is trained based on the first and second image sets, and the training process includes two stages.

[0060] In the first stage, the initial learning rate is 0.001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the category, the traffic_cone weight is 0.2, and the rest of the weights are all set to 1, and the training number of epoch is 20.

[0061] In the second stage, the initial learning rate is 0.0001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the blur degree, and the training number of epoch is 10.

[0062] Staged training

[0063] First stage: normal training, initial learning rate of 0.001 decays, target Loss weight is calculated according to category, generally divided into 10 categories,

'car', 'truck', 'construction_vehicle', 'bus', 'trailer', 'barrier','motorcycle', 'bicycle', 'pedestrian', 'traffic_cone'

[0064] Second stage: enhanced training, initial learning rate of 0.0001 decays, target Loss weight is calculated according to blur degree, clear, general, and blurred target weights are set to 0.2, 0.5, and 1.0, respectively, and 10 epochs are trained. This stage can make the model focus on the detection of blurred targets.

[0065] Embodiment two of the present application provides a generation device of a detection model for monocular 3D target detection, comprising the following modules:

[0066] An image acquisition module is configured to acquire a first image set and a second image set, the images in the first and second image sets are acquired by an image acquisition device on the vehicle, the image acquisition device is configured to acquire images of the outside of the vehicle, the images in the first image set have higher definition than the images in the second image set, and each image in the second image set is labeled with a target.

[0067] A model creation module is configured to create a detection model based on FCOS3D, in which a Velocity branch in FCOS3D is replaced by a blur attribute branch.

[0068] A training module is configured to train the detection model based on the first and second image sets.

[0069] In this embodiment, the output of the blur attribute branch is HxWx1.

[0070] In this embodiment, the image acquisition module is further configured to label each image in the second image set with a target, and the labeling information of each target includes an X coordinate value, a Y coordinate value and a Z coordinate value of a center point of the target in a coordinate system of the image acquisition device, a size value of the target along the X axis, a size value of the target along the Y axis and a size value of the target along the Z axis, an azimuth angle yaw, a category label and a confidence score score of the target, and a blur degree value blur_type.

[0071] In this embodiment, the training module is further configured to train the detection model based on the first and second image sets, and the training process includes two stages; in the first stage, the initial learning rate is 0.001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the category, the weight of traffic_cone is 0.2, and the weights of the rest are all set to 1, and the training number of epoch is 20; in the second stage, the initial learning rate is 0.0001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the blur degree, and the training number of epoch is 10.

[0072] Embodiment three of the present application provides a monocular 3D target detection method for a vehicle, the front side, rear side, left front side, right front side, left rear side and right rear side of the vehicle are all provided with an image acquisition device, and the image acquisition device is configured to acquire images of the outside of the vehicle; the method comprises the following steps:

[0073] Step 1: execute the generation method in embodiment one to obtain the detection model.

[0074] Step 2: control all image acquisition devices in the car to obtain a to-be-processed image, input each to-be-processed image into the detection model, and obtain a plurality of output targets; fuse and display a plurality of output targets at the same position.

[0075] Embodiment four of the present application provides a monocular 3D target detection device for a car, the front side, rear side, left front side, right front side, left rear side and right rear side of the car are provided with image acquisition devices, the image acquisition devices are used to obtain images of the outside of the car; comprising the following modules:

[0076] The model generation module is used to execute the generation method in embodiment one, and obtain the detection model.

[0077] The processing module is used to control all image acquisition devices in the car to obtain a to-be-processed image, input each to-be-processed image into the detection model, and obtain a plurality of output targets; fuse and display a plurality of output targets at the same position.

[0078] It should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be combined appropriately to form other embodiments that those skilled in the art can understand.

[0079] The above series of detailed descriptions are only specific descriptions of the feasible embodiments of the present application, and are not intended to limit the protection scope of the present application, and any equivalent embodiments or changes made without departing from the spirit of the present application should be included in the protection scope of the present application.

Claims

1. A method for generating a detection model for monocular 3D object detection, characterized in that, The method comprises the following steps: obtaining a first image set and a second image set, the images in the first and second image sets are collected by an image collection device on a vehicle, the image collection device is used to collect images of the outside of the vehicle, the resolution of the images in the first image set is higher than that of the images in the second image set, and each image in the second image set is labeled with a target; creating a detection model based on FCOS3D, in which the Velocity branch in FCOS3D is replaced by a blur attribute branch; training the detection model based on the first and second image sets.

2. The generation method according to claim 1, wherein the output of the blur attribute branch is HxWx1.

3. The generation method of claim 1, wherein, each image in the second image set is labeled with a target, and the labeling information of each target comprises: the X, Y and Z coordinate values of the center point of the target in the image collection device coordinate system, the size values of the target along the X, Y and Z axes, the azimuth yaw, the category label and the confidence score score of the target, and the blur degree value blur_type. training the detection model based on the first and second image sets comprises:

4. The generation method of claim 1, wherein, training the detection model based on the first and second image sets, and the training process includes two stages; in the first stage, the initial learning rate is 0.001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the category, the weight of traffic_cone is 0.2, and the weights of the rest are all set to 1, and the training number of epoch is 20; in the second stage, the initial learning rate is 0.0001, and the learning rate is gradually decayed during the training process; the Loss weight of the target is divided according to the blur degree, and the training number of epoch is 10. The method comprises the following steps:

5. A device for generating a detection model for monocular 3D target detection, characterized in that, an image acquisition module for acquiring a first image set and a second image set, the images in the first and second image sets are collected by an image collection device on a vehicle, the image collection device is used to collect images of the outside of the vehicle, the resolution of the images in the first image set is higher than that of the images in the second image set, and each image in the second image set is labeled with a target; a model creation module for creating a detection model based on FCOS3D, in which the Velocity branch in FCOS3D is replaced by a blur attribute branch; a training module for training the detection model based on the first and second image sets.

6. The generation device according to claim 5, wherein the output of the blur attribute branch is HxWx1. the image acquisition module is further used for:

7. The generating device of claim 5, wherein, ​ Each image in the second image set is labeled with a target, and each target is labeled with information including: in the image acquisition device coordinate system, X coordinate value, Y coordinate value and Z coordinate value of the center point of the target, size value of the target along the X axis, size value of the target along the Y axis and size value of the target along the Z axis, azimuth yaw of the target, category label and confidence score score of the target, and blur degree value blur_type.

8. The generating device of claim 5, wherein, The training module is further configured to: train the detection model based on the first and second image sets, and the training process includes two stages; in the first stage, the initial learning rate is 0.001, and the learning rate is gradually decayed during the training process; the loss weight of the target is divided according to the category, the weight of traffic_cone is 0.2, the remaining weights are all set to 1, and the training number of epoch is 20; in the second stage, the initial learning rate is 0.0001, and the learning rate is gradually decayed during the training process; the loss weight of the target is divided according to the blur degree, and the training number of epoch is 10.

9. A monocular 3D target detection method for a vehicle, the front side, the rear side, the left front side, the right front side, the left rear side and the right rear side of the vehicle are provided with image acquisition devices, and the image acquisition devices are used to acquire images of the outside of the vehicle; characterized in that, The method comprises the following steps: performing the generation method of any one of claims 1-4 to obtain the detection model; controlling all image acquisition devices in the vehicle to acquire a to-be-processed image, inputting each to-be-processed image into the detection model to obtain a plurality of output targets, and fusing and displaying a plurality of output targets at the same position.

10. A monocular 3D target detection device for a vehicle, the front side, the rear side, the left front side, the right front side, the left rear side and the right rear side of the vehicle are provided with image acquisition devices, the image acquisition devices are used to acquire images of the outside of the vehicle; characterized in that, The method comprises the following modules: a model generation module configured to perform the generation method of any one of claims 1-4 to obtain the detection model; a processing module configured to control all image acquisition devices in the vehicle to acquire a to-be-processed image, input each to-be-processed image into the detection model to obtain a plurality of output targets, and fuse and display a plurality of output targets at the same position.

Images