A monocular three-dimensional target real-time detection method applied to intelligent driving

Abstract

The application provides a new intelligent driving real-time three-dimensional target recognition method based on monocular vision technology, which can operate two-dimensional images in an intelligent driving scene, regress key points of targets in the images, and calculate three-dimensional positions of the targets in a physical coordinate system in combination with geometric parameters of a camera. The application can be applied to the intelligent driving scene and meets real-time and stability requirements.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a monocular three-dimensional target real-time detection method applied to intelligent driving. BACKGROUND

[0002] The research on visual target detection technology is of great significance in the field of intelligent driving perception. Through this technology, the front obstacles can be detected in real time, providing a basis for the intelligent driving decision-making module.

[0003] Since images can only provide two-dimensional plane information, intelligent driving visual target detection algorithms are mainly two-dimensional rectangular frame detection, such as single-stage target detection algorithm YOLO, SSD, two-stage target detection algorithm Faster RCNN. This kind of algorithm is currently mature and widely used, but in the field of intelligent driving, three-dimensional information of obstacles needs to be recognized to make accurate decisions.

[0004] Currently, in the field of intelligent driving, three-dimensional target detection technology based on vision is an important research direction, and there are relevant research results published, such as ROI-10D algorithm, AM3D algorithm, GAM3D algorithm, etc. Some of these algorithms need to use independent network model to regress depth map, and then realize three-dimensional target detection through depth map and two-dimensional plane map. Some algorithms need to use two-level network to realize three-dimensional information extraction of targets, so it is usually difficult to meet the real-time and stability requirements in the intelligent driving scene.

[0005] Therefore, there is an urgent need for a monocular pure vision real-time three-dimensional target detection algorithm suitable for intelligent driving scenarios. SUMMARY

[0006] The purpose of the present application is to provide a monocular three-dimensional target real-time detection method applied to intelligent driving, aiming to improve the real-time and stability of monocular three-dimensional target recognition in intelligent driving scenarios.

[0007] According to the first aspect of the present application, a monocular three-dimensional target real-time detection method applied to intelligent driving is provided, characterized in that it comprises:

[0008] Step 10: data set preparation.

[0009] Step 20: target recognition neural network model construction.

[0010] Step 30: target recognition neural network model training.

[0011] Step 40: target recognition neural network model testing.

[0012] Step 10 comprises:

[0013] Step 11: image acquisition; based on monocular RGB camera, image acquisition during vehicle driving process, and using Zhang Zhengyou calibration method to calibrate the camera, obtain the projection matrix M, representing the coordinate conversion relationship between image pixel coordinates and physical coordinate system.

[0014] Step 12: image annotation; taking the vehicle as the target, annotating the target information on each image; target information includes: target category, target two-dimensional rectangular frame position Target vehicle heading direction dir g , target head or tail ratio Target driving direction ori g , target driving direction ratio Target three-dimensional size Target category includes 5 kinds: sedan, truck, bus, tricycle, non-motor vehicle; target vehicle heading direction includes 2 kinds: vehicle head forward, vehicle head backward; target driving direction includes 3 kinds: left, center, right.

[0015] Step 13: data set division, all annotated images are constructed into a data set, and divided into training set, validation set in a certain proportion.

[0016] Step 20 includes:

[0017] The target recognition neural network model includes a feature extraction network model and a pseudo three-dimensional target information regression network model.

[0018] Step 21: feature extraction network model construction; the feature extraction network inputs the image, uses 19 convolution layers and 5 pooling layers to extract high-dimensional features from the input image, uses spatial pyramid pooling SPP structure to fuse multi-scale features, and outputs 3 feature maps for detection. These 3 feature maps are for targets of different scales.

[0019] Step 22: pseudo 3D target information regression network model construction; the pseudo three-dimensional target information regression network inputs the 3 feature maps output by the feature extraction network model, uses 5 convolution layers to regress the pseudo three-dimensional information of the target for each feature map, and outputs the corresponding predicted target information for each cell unit in the feature map.

[0020] The regression target information includes: predicted target category label, predicted target rectangular frame bias (t x , t y , t w , t h ) 2d , predicted target rectangular frame confidence conf, predicted target vehicle heading direction dir, predicted target driving direction oir, predicted target head or tail ratio r w , predicted target driving direction ratio r h, predict target 3D size predict target 3D size belongs to 3D anchors class k.

[0021] where label, dir, oir, k are one-hot encoded.

[0022] Target rectangle frame regression uses 2D anchors design, through k-means clustering of target rectangle frame width and height in training data set, 9 groups of 2D anchors hyperparameters are obtained For each cell unit position (x p , y p ) in each feature map, the obtained predicted target rectangle frame bias (t x , t y , t w , t h ) 2d and the predicted target rectangle frame position (x, y, w, h)′ 2d The relationship is:

[0023]

[0024] Target 3D size regression uses 3D anchors design, through k-means clustering of target 3D size in training data set, 3 groups of 3D anchors hyperparameters are obtained For each target, the obtained predicted target 3D size bias and the predicted target 3D size (l, w, h)′ 3d The relationship is:

[0025]

[0026] where k is the 3D anchors class to which the predicted target 3D size belongs;

[0027] Step 23: target recognition neural network model loss function construction. Each sub-loss construction includes:

[0028] Target rectangle frame bias prediction uses mean square error loss:

[0029]

[0030] where KxK represents the output layer resolution; M represents 2D anchors, each output layer cell corresponds to 3; is the target rectangle frame bias true value, is the predicted target rectangle frame bias value, indicates whether the i, j position is a target.

[0031] The target rectangular bounding box confidence prediction adopts binary cross-entropy loss:

[0032]

[0033] wherein, denotes the true value of whether the i, j position is a target, C ij denotes the predicted value of whether the i, j position is a target.

[0034] The target class prediction adopts multi-class cross-entropy loss:

[0035]

[0036] wherein, denotes the probability true value of whether the i, j position is a class c, p ij (c) denotes the probability predicted value of whether the i, j position is a class c, and classes is a class set.

[0037] The target heading prediction adopts binary cross-entropy loss:

[0038]

[0039] wherein, denotes the true value of whether the i, j position is a front heading, denotes the predicted value of whether the i, j position is a front heading;

[0040] The target head or tail proportion prediction adopts mean square error loss.

[0041]

[0042] wherein, denotes the true value of the i, j position head or tail proportion, denotes the predicted value of the i, j position head or tail proportion.

[0043] The target driving direction prediction adopts multi-class cross-entropy loss:

[0044]

[0045] wherein, m denotes a vehicle driving direction oir class to which the target belongs, denotes the true value of the i, j position vehicle driving direction, denotes the predicted value of the i, j position vehicle driving direction.

[0046] The target driving direction proportion prediction adopts mean square error loss:

[0047]

[0048] wherein, represents the true value of the i, j position direction proportion, represents the predicted value of the i, j position direction proportion.

[0049] The target three-dimensional size category prediction adopts a multi-classification cross-entropy loss:

[0050]

[0051] wherein, represents the true value of the i, j position belonging to the rth in the 3D anchor, p ij (r) represents the predicted value of the i, j position belonging to the rth in the 3D anchor.

[0052] The target three-dimensional size bias prediction adopts a mean square error loss:

[0053]

[0054] wherein, represents the true value of the i, j position target three-dimensional size bias, represents the predicted value of the i, j position target three-dimensional size bias.

[0055] The total loss of the target recognition neural network model is the sum of each sub-loss:

[0056] loss(3d_det)

[0057] = loss(2d_box) + loss(conf) + loss(label) + loss(dir) + loss(r w )

[0058] + loss(ori) + loss(r h ) + loss(dim_class) + loss(dim_offset).

[0059] Further, the monocular three-dimensional target real-time detection method applied to intelligent driving provided by the application has the characteristics that step 30 further comprises: step 31: data set enhancement; the data set is enhanced by translation, scaling, and sharpening.

[0060] Step 32: setting training hyperparameters; the total training epochs are set to 100, the initial epoch is set to 0, the batch size is set to 32, the learning rate is set to 0.001, the non-maximum suppression parameter is set to 0.4, the target confidence parameter is set to 0.25, and the input layer model size is [704, 416].

[0061] Step 33: model training; reading a batch of data, calculating the output of the current target recognition neural network model through forward inference, calculating the total loss of the target recognition neural network model, performing back propagation according to the learning rate, adjusting the parameters of each layer of the network, until all data in the training set are trained, the epoch value is incremented by 1; inputting the data in the verification set into the current target recognition neural network model, calculating the recognition accuracy, and saving the model with the highest accuracy; if the epoch is greater than the epochs, the model training is completed, otherwise, repeat the step.

[0062] Further, the monocular three-dimensional target real-time detection method for intelligent driving provided by the application has the characteristics that step 40 further comprises:

[0063] Step 41: real-time prediction; obtaining images from a camera in real time, inputting the trained target recognition neural network model to obtain predicted target information output by the target recognition neural network model.

[0064] Step 42: predicted target information analysis; calculating the three-dimensional position information of the target in the current image according to the predicted target information.

[0065] According to the second aspect of the application, a computer device is provided, characterized by comprising:

[0066] a memory for storing instructions; and a processor for calling the instructions stored in the memory to execute the monocular three-dimensional target real-time detection method for intelligent driving of the first aspect.

[0067] According to the third aspect of the application, a computer readable storage medium is provided, characterized by storing instructions, which are executed by a processor to execute the monocular three-dimensional target real-time detection method for intelligent driving of the first aspect.

[0068] Compared with the prior art, the above technical solutions conceived by the application have at least the following beneficial effects:

[0069] 1. The application proposes a new intelligent driving real-time three-dimensional target recognition algorithm based on monocular vision technology, which can operate on two-dimensional images in an intelligent driving scene, regress the key points of the target in the image, and calculate the three-dimensional position of the target in the physical coordinate system in combination with the geometric parameters of the camera.

[0070] 2. The application adopts an end-to-end structure in design, and can detect three-dimensional targets in images in real time without laser radar and other point cloud information, and has very good portability.

[0071] 3、The application converts the intelligent driving three-dimensional target recognition problem into a two-dimensional rectangular frame detection and three-dimensional key point regression problem, greatly simplifies the model structure, and can be applied to forward, lateral and other intelligent driving visual perception scenes;

[0072] 4、The application combines the 2D anchor structure, adds the 3D anchor design in the target three-dimensional size detection, and improves the accuracy of the algorithm in calculating the target three-dimensional size;

[0073] 5、The application is deployed into an embedded system, and can realize real-time (30 frames / second) three-dimensional target recognition performance.

[0074] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0075] The drawings incorporated into the specification and constituting a part of the specification, show embodiments consistent with the application, and together with the specification serve to explain the principles of the application.

[0076] Figure 1 is a flowchart of the implementation steps according to an exemplary embodiment.

[0077] Figure 2 is a schematic diagram of a manually annotated image according to an exemplary embodiment.

[0078] Figure 3 is a schematic diagram of a feature extraction network model structure according to an exemplary embodiment.

[0079] Figure 4 is a schematic diagram of a pseudo three-dimensional target information regression network structure according to an exemplary embodiment.

[0080] Figure 5 is a schematic diagram of a target three-dimensional key point position according to an exemplary embodiment. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical scheme and advantages of the application clearer, the application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0082] As shown in Figure 1 , an embodiment of the application can include the following flow:

[0083] System building: Install the camera in the middle of the front windshield of the vehicle, the height is 1.0-1.6 meters, covering the front perception area, connecting with the controller through the video transmission line, and powering the entire system.

[0084] System initialization: The system starts, loads the driver, and performs a self-check of the hardware functions. If there is a hardware failure, an alarm is given and the system is exited. If the system self-check is normal, the next step is entered.

[0085] Camera parameter calibration: Zhang Zhengyou calibration method is used to calibrate the camera parameters, and the conversion relationship between the camera coordinate system and the physical coordinate system is calculated, which is recorded as the projection matrix M.

[0086] Image acquisition: Every second, the current camera-acquired image is saved to build a dataset containing 10,000 images.

[0087] Artificially annotated images include: two-dimensional rectangular frame position, class, vehicle heading, vehicle head or tail ratio, driving direction, direction ratio, and target three-dimensional size.

[0088] Dataset division: The dataset is divided into a training set and a validation set, with a data ratio of 7:3.

[0089] Neural network model design: Design a pseudo-three-dimensional target recognition algorithm model, mainly including feature extraction structure, pseudo-three-dimensional target information regression network structure, and loss function.

[0090] Model training: Set the training hyperparameters, send the data in the training dataset to the model in batches, adjust the model parameters, and make the model achieve the optimal detection accuracy; save the optimal model.

[0091] Model deployment: transplant the trained optimal model into the hardware platform.

[0092] Real-time inference: The system reads the current camera image in real time, sends it to the trained optimal model, and obtains the current network detection result from the model output end.

[0093] Three-dimensional target recognition result analysis: Calculate the network output result to obtain the three-dimensional position information of the target, complete the current frame recognition, and output the three-dimensional target recognition result.

[0094] If not exited, return to step 10). If exited, end.

[0095] Further, in some embodiments, the data set preparation step includes the following:

[0096] Image acquisition: based on RGB vision sensor, collect images in the process of vehicle driving, construct a data set containing 10,000 images, and calibrate camera parameters using Zhang Zhengyou calibration method, calculate projection matrix M, which represents the coordinate conversion relationship between image pixel coordinates and physical coordinate system.

[0097] Image annotation: manually annotate vehicles on each image, a total of five categories, including: sedan, truck, bus, tricycle, non-motor vehicle, and annotation content includes target two-dimensional rectangular frame Target head direction dir in camera view g (front head / rear head), target head or tail ratio Target driving direction ori in camera view g (left / center / right), direction ratio Target three-dimensional size Subsequently, the true value is constructed by the annotation data for model training; the annotation method is as shown in Figure 2 .

[0098] Further, in some embodiments, the algorithm model design includes the following contents:

[0099] Feature extraction network model design: the feature extraction network uses 19 convolutional layers and 5 pooling layers to extract high-dimensional features from images, and uses spatial pyramid pooling (SPP) structure to fuse multi-scale features and optimize feature expression; finally, 3 feature maps are output, and the 3 feature maps are responsible for large, medium and small targets of different scales; the network structure is as shown in Figure 3 .

[0100] Pseudo three-dimensional target information regression network structure design: 5 convolutional layers are used to regress the pseudo 3D information of the target for the 3 feature maps of the feature extraction network model, and the channel number of the last convolutional layer is 69, the calculation formula is: (classification category number 5 + 18) * 3, and finally 3 output layers are obtained. For each cell unit in the output layer, the following information is predicted:

[0101] Regression target rectangular frame bias (t x , t y , t w , t h ) 2dThe regression values ​​for target confidence (conf), target classification category label (5 categories), and other parameters are the same as in the 2D target detection algorithm. Specifically, the target classification category label uses one-hot encoding; the target bounding box bias regression uses a 2D anchors design, which clusters the width and height of the target bounding boxes in the training dataset using a k-means clustering algorithm, with 9 clusters, resulting in 9 sets of 2D anchors hyperparameters, denoted as . Then for the cell location (x) in each output layer p y p The target rectangle bias (t) that was regressed x , t y , t w , t h ) 2d With the predicted target bounding box position (x, y, w, h)′ 2d The relationship between them is shown in the following formula:

[0102]

[0103] Target vehicle heading category dir (2 categories: front-facing / back-facing), driving direction category ori (3 categories: left / center / right), regression vehicle heading or rear-facing proportion r w The regression direction ratio rh, where: the target vehicle heading direction category dir and the driving direction category ori are one-hot encoded;

[0104] The 3D size regression of the target employs a 3D anchors design. The 3D sizes of the targets in the training dataset are clustered using the k-means clustering algorithm, with 3 clusters per cluster, resulting in 3 sets of 3D anchors hyperparameters, denoted as [parameters to be filled in]. The target's 3D size regression mainly employs two structures: `dim_class` and `dim_offset`. `dim_class` represents the 3D anchor category to which the current target's 3D size belongs (three categories in this invention, using one-hot encoding). `dim_offset` represents the bias between the predicted 3D size of the target and the corresponding 3D anchor hyperparameter in the `dim_class` category, denoted as... The predicted three-dimensional dimensions (1, w, h)′ for each target 3d Offset from predicted 3D dimensions The calculation formula between them is as follows:

[0105]

[0106] wherein: k represents the 3D anchor category to which the target three-dimensional size belongs, given by the dim_class classification result.

[0107] The regression structure is shown as Figure 4

[0108] Further, in some embodiments, the loss function design includes the following parts:

[0109] The target rectangular frame bias prediction adopts mean square error loss, denoted as: loss(2d_box):

[0110]

[0111] wherein: KxK represents the output layer resolution; M represents 2D anchors, each output layer cell corresponds to 3; is the target rectangular frame bias true value, is the predicted target rectangular frame bias value, represents whether i, j position is the target.

[0112] The target rectangular frame confidence prediction adopts binary classification cross-entropy loss, denoted as: loss(conf):

[0113]

[0114] wherein, represents the true value of whether i, j position is the target, C ij represents the predicted value of whether i, j position is the target.

[0115] The target category prediction adopts multi-classification cross-entropy loss, denoted as: loss(label):

[0116]

[0117] wherein, represents the probability true value of i, j position being category c, p ij (c) represents the probability predicted value of i, j position being category c, classes is the category set, including: car, truck, bus, tricycle, non-motor vehicle.

[0118] The target vehicle heading prediction adopts binary classification cross-entropy loss, denoted as loss(dir):

[0119]

[0120] wherein, represents the true value of whether i, j position vehicle head is forward, represents the predicted value of whether i, j position vehicle head is forward.​

[0121] Target head or tail ratio prediction adopts mean square error loss, denoted as loss(r w ):

[0122]

[0123] wherein, represents the true value of the head or tail ratio at position i, j, represents the predicted value of the head or tail ratio at position i, j.

[0124] Target driving direction prediction adopts multi-classification cross-entropy loss, denoted as loss(ori):

[0125]

[0126] wherein, m represents the driving direction ori category to which the target belongs, represents the true value of the driving direction at position i, j, represents the predicted value of the driving direction at position i, j.

[0127] Target driving direction ratio prediction adopts mean square error loss, denoted as loss(r h ):

[0128]

[0129] wherein, represents the true value of the direction ratio at position i, j, represents the predicted value of the direction ratio at position i, j.

[0130] Target three-dimensional size category prediction adopts multi-classification cross-entropy loss, denoted as loss(dim_class):

[0131]

[0132] wherein, represents the true value of the probability that the position i, j belongs to the rth 3D anchor, p ij (r) represents the predicted value of the probability that the position i, j belongs to the rth 3D anchor.

[0133] Target three-dimensional size bias prediction adopts mean square error loss, denoted as loss(dim_offset):

[0134]

[0135] wherein, represents the true value of the target three-dimensional size bias at position i, j, represents the i, j position target three-dimensional size bias prediction value.

[0136] Finally, the loss of the entire three-dimensional target recognition network model is the sum of each loss, as follows:

[0137] loss(3d_det)

[0138] = loss(2d_box) + loss(conf) + loss(label) + loss(dir) + loss(r w )

[0139] + loss(ori) + loss(r h ) + loss(dim_class) + loss(dim_offset)

[0140] In some embodiments, the algorithm training includes the following steps:

[0141] Data set division: randomly divide the artificially labeled data set into a training set and a validation set in a ratio of 7:3.

[0142] Data augmentation: In order to improve the stability of the algorithm, image processing operations such as translation, scaling, and sharpening are used to enhance the data set.

[0143] Set the hyperparameters: the total training epochs is set to 100, the initial epoch is set to 0, the batch size is set to 32, the learning rate is set to 0.001, the non-maximum suppression parameter is set to 0.4, the target confidence parameter is set to 0.25, and the input layer model size is [704, 416].

[0144] Read a batch of data, calculate the current network output through forward inference, and calculate the loss(3d_det) with the true value. The error is propagated in the reverse direction with a learning rate, and the parameters of each layer of the network are adjusted until all data in the training set are trained, and the epoch value is incremented by 1.

[0145] Input the data in the validation set into the current model, calculate the recognition accuracy, and save the model with the highest accuracy.

[0146] If epoch > epochs, the model training is complete, otherwise continue to the 4) step.

[0147] In some embodiments, the target three-dimensional information extraction includes the following:

[0148] Computing key points: define four key points of the target in the image, based on the pseudo 3D target information output by the neural network, calculate the 2D coordinates of the four key points in the image, denoted as p1, p2, p3, p4, the key point positions are as shown in Figure 5

[0149] Based on the calibration matrix M of the camera before data acquisition and the ground plane assumption (the height of the point on the ground is 0), the physical distances of p1, p2, p3 can be calculated, and the projection relationship is as follows:

[0150]

[0151] wherein z 3d is equal to 0.

[0152] The line segment (p2, p4) is perpendicular to the ground, and according to the projection relationship and the camera intrinsic parameters, the physical length of the line segment can be obtained, that is, the vehicle height h 3d , and the projection formula is as follows:

[0153]

[0154] wherein d p2 represents the longitudinal distance from the point p2 to the camera, which is calculated in the previous step, f is the focal length of the camera, and h 2d is the pixel height of the line segment (p2, p4);

[0155] Finally, the physical lengths of the line segments (p1, p2), (p2, p3), (p2, p4) are approximately the three-dimensional dimensions of the target, and a logical judgment is made with the three-dimensional dimensions of the target predicted by the neural network to calculate the final three-dimensional dimensions (l, w, h)′ 3d .

[0156] Based on the target vehicle head direction category dir output by the neural network and the physical coordinate values of the line segment (p1, p2), the global orientation angle θ of the target is calculated, and the calculation formula is as follows:

[0157]

[0158] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the application is indicated by the following claims.

[0159] ​It should be understood that the application is not limited to the precise construction which has been described above and which shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should be limited only by the appended claims.

Claims

1. A monocular three-dimensional target real-time detection method applied to intelligent driving, characterized in that, Comprising: Step 10: data set preparation; Step 20: target recognition neural network model construction; Step 30: target recognition neural network model training; Step 40: target recognition neural network model testing; Step 10 comprises: Step 11: image acquisition; based on a monocular RGB camera, images during vehicle driving are acquired, and a Zhang Zhengyou calibration method is used to calibrate the camera to obtain a projection matrix M, representing the coordinate conversion relationship between the image pixel coordinates and the physical coordinate system; Step 12: image annotation; taking vehicle as target, annotating target information on each image; target information includes: target category, target two-dimensional rectangular box position Target vehicle head direction dir g Target head or tail proportion Target driving direction ori g Target driving direction proportion Target three-dimensional size Target category includes 5 kinds: sedan, truck, bus, tricycle, non-motor vehicle; target vehicle head direction includes 2 kinds: vehicle head forward, vehicle head backward; target driving direction includes 3 kinds: left, center, right; Step 13: data set division; all labeled images are constructed into a data set, and are divided into a training set and a validation set at a certain ratio; Step 20 comprises: The target recognition neural network model comprises a feature extraction network model and a pseudo three-dimensional target information regression network model; Step 21: feature extraction network model construction; the feature extraction network inputs an image, uses 19 convolutional layers and 5 pooling layers to extract high-dimensional features from the input image, uses a spatial pyramid pooling SPP structure to fuse multi-scale features, and outputs 3 feature maps for detection. These 3 feature maps are for targets of different scales; Step 22: pseudo 3D target information regression network model construction; the pseudo three-dimensional target information regression network inputs the 3 feature maps output by the feature extraction network model, uses 5 convolutional layers to regress the pseudo three-dimensional information of the target for each feature map, and outputs the corresponding predicted target information for each cell unit in the feature map; The regression target information includes: a predicted target category label, a predicted target rectangular frame offset (t x ,t y ,t w ,t h ) 2d , a predicted target rectangular frame confidence conf, a predicted target head direction dir, a predicted target running direction oir, a predicted target head or tail proportion r w , a predicted target running direction proportion r h , predicted target three-dimensional size A 3D anchor category k to which the predicted target three-dimensional size belongs. Wherein, label, dir, oir, k are one-hot encoded; The target rectangle regression adopts a 2D anchor design, and 9 groups of 2D anchor hyperparameters are obtained by using k-means clustering on the width and height of the target rectangle frame in the training data set For each cell unit position (x p ,y p ) in each feature map, the relationship between the obtained predicted target rectangle frame bias (t x ,t y ,t w ,t h ) 2d and the predicted target rectangle frame position (x, y, w, h)' 2d is: The target three-dimensional size regression adopts a 3D anchor design, and 3 groups of 3D anchor hyperparameters are obtained by using k-means clustering on the target three-dimensional sizes in the training data set For each target, the obtained predicted target three-dimensional size bias The relationship between the predicted target three-dimensional size (l, w, h) 3d is: Wherein, k is the 3D anchor category to which the predicted target three-dimensional size belongs; Step 23: target recognition neural network model loss function construction; each sub-loss construction comprises: The target rectangular frame bias prediction uses a mean square error loss: Wherein, KxK represents the output layer resolution; M represents 2D anchors, and each output layer has 3 cell units corresponding to the 2D anchors; a target rectangular frame bias value, a predicted target rectangular frame bias value, indicates whether the i,j position is a target; The target rectangular frame confidence prediction uses a binary classification cross-entropy loss: wherein, a true value indicating whether position i,j is the target, C ij a predicted value indicating whether position i,j is the target; The target category prediction uses a multi-classification cross-entropy loss: wherein, denotes the ground truth probability that the i,j position is of class c, p ij (c) denotes the predicted probability that the i,j position is of class c, classes being the set of classes; The target vehicle heading prediction uses a binary classification cross-entropy loss: wherein, is a true value indicating whether the vehicle head is facing forward at position i,j, is a predicted value indicating whether the vehicle head is facing forward at position i,j; The target head or tail proportion prediction uses a mean square error loss: wherein, represents the true value of the head or tail proportion at position i,j, represents the predicted value of the head or tail proportion at position i,j; The target driving direction prediction uses a multi-classification cross-entropy loss: wherein m represents a vehicle travel direction oir category to which the target belongs, a true value indicating a vehicle travel direction at the i, j position, a predicted value indicating a vehicle travel direction at the i, j position; The target driving direction proportion prediction uses a mean square error loss: wherein, represents the i, j position direction proportion true value, represents the i, j position direction proportion predicted value; The target three-dimensional size category prediction uses a multi-classification cross-entropy loss: wherein, Ptrue(i,j) represents the ground truth probability that the i,j position belongs to the rth of the 3D anchors, p ij Ppred(i,j) represents the predicted probability that the i,j position belongs to the rth of the 3D anchors. The target three-dimensional size bias prediction uses a mean square error loss: wherein, represents the true value of the target three-dimensional size bias at the i,j position, represents the predicted value of the target three-dimensional size bias at the i,j position; The total loss of the target recognition neural network model is the sum of each sub-loss: loss(3d_det) = loss(2d_box) + loss(conf) + loss(label) + loss(dir) + loss(r w ) + loss(ori) + loss(r h )+ loss(dim_class) + loss(dim_offset). 2.The monocular three-dimensional target real-time detection method applied to intelligent driving of claim 1, wherein, Step 30 further comprises: Step 31: data set enhancement; translation, scaling, and sharpening are used to enhance the data set; Step 32: setting training hyperparameters; the total training epochs are set to 100, the initial epoch is set to 0, the batch size is set to 32, the learning rate is set to 0.001, the non-maximum suppression parameter is set to 0.4, the target confidence parameter is set to 0.25, and the input layer model size is [704, 416]; Step 33: model training; read a batch of data, calculate the output of the current target recognition neural network model by forward inference, calculate the total loss of the target recognition neural network model, perform back propagation according to the learning rate, adjust the parameters of each layer of the network, until all data in the training set are trained, the epoch value is increased by 1; input the data in the verification set into the current target recognition neural network model, calculate the recognition accuracy, save the model with the highest accuracy; if epoch>epochs, the model training is completed, otherwise repeat the step. 3.The monocular three-dimensional target real-time detection method applied to intelligent driving of claim 1, wherein, Step 40 further comprises: Step 41: real-time prediction; real-time image acquisition from the camera, input into the trained target recognition neural network model, and obtain the predicted target information output by the target recognition neural network model; Step 42: prediction target information analysis; calculate the three-dimensional position information of the target in the current image according to the prediction target information.

4. A computer device, comprising: Comprise: A memory for storing instructions; A processor for calling the instructions stored in the memory to execute the monocular three-dimensional target real-time detection method for intelligent driving according to any one of claims 1-3.

5. A computer readable storage medium, characterized in that, The instructions are stored in the memory, and the instructions are executed by the processor to execute the monocular three-dimensional target real-time detection method for intelligent driving according to any one of claims 1-3.

Images