Three-dimensional target detection method based on stereoscopic vision technology

Abstract

The invention discloses a three-dimensional target detection method based on a stereoscopic vision technology, and the method comprises the following steps: obtaining a to-be-detected target image, and carrying out the preprocessing of the to-be-detected target image; acquiring depth information of the target image to be detected by adopting a binocular stereoscopic vision technology; extracting target features in the preprocessed to-be-detected target image by using the optimized CNN model; fusing the depth information of the to-be-detected target image with the target features; and identifying the fused target features by using an RPN algorithm based on a regional proposal network. The method solves the problems that in an existing three-dimensional target detection method, when depth information in an image is obtained, the image and point cloud information of the environment where a target object is located need to be obtained firstly, then coordinate information of the point cloud information in the image is obtained, and then the coordinate information is combined with 2D and 3D bounding box information in the image, so that the depth information in the image cannot be obtained. The process relates to a large amount of calculation, so that the calculation complexity is increased, and the efficiency of obtaining the depth information is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional target detection, and in particular to a three-dimensional target detection method based on stereo vision technology. Background Art

[0002] With the continuous development of autonomous driving technology, intelligent transportation systems and intelligent security, the demand for accurate and efficient three-dimensional target detection technology is becoming increasingly urgent. At present, the existing three-dimensional target detection method specifically obtains the color image and point cloud information of the environment in which the target object is located, and uses a deep neural network to detect the color image, thereby obtaining the two-dimensional (2D) bounding box and three-dimensional (3D) bounding box of the target object on the color image. Subsequently, the point cloud information is mapped to the image coordinate system of the color image. Through this mapping relationship, the precise coordinates of the point cloud information in the color image can be obtained. On this basis, the depth information of the 2D and 3D bounding boxes on the color image is obtained by combining the information of the 2D bounding box and the 3D bounding box. Finally, according to the dimensional characteristics of these depth information and point cloud information, the category, size and posture information of the target object can be accurately inferred. However, when obtaining the depth information in the image, the detection method needs to first obtain the image and point cloud information of the environment in which the target object is located, and then obtain the coordinate information of the point cloud information in the image, and then combine these coordinate information with the 2D and 3D bounding box information in the image. This process involves a lot of calculations, which not only increases the complexity of the calculations, but also may lead to reduced efficiency in obtaining depth information. Summary of the invention

[0003] In view of the above-mentioned defects, the present invention proposes a three-dimensional target detection method based on stereo vision technology, aiming to solve the problem that in the existing three-dimensional target detection method, when acquiring depth information in an image, it is necessary to first acquire the image and point cloud information of the environment in which the target object is located, and then acquire the coordinate information of the point cloud information in the image, and then combine the coordinate information with the 2D and 3D bounding box information in the image. This process involves a large amount of calculation, which not only increases the complexity of the calculation, but also may lead to reduced efficiency in acquiring depth information.

[0004] To achieve this object, the present invention adopts the following technical solutions:

[0005] A three-dimensional target detection method based on stereo vision technology comprises the following steps:

[0006] Step S1: Acquire several target images;

[0007] Step S2: labeling a number of target images to obtain a labeled data set;

[0008] Step S3: Establish a convolutional neural network CNN model;

[0009] Step S4: Use the labeled data set to train and optimize the CNN model to obtain an optimized CNN model;

[0010] Step S5: acquiring a target image to be detected, and preprocessing the target image to be detected to obtain a preprocessed target image to be detected;

[0011] Step S6: using binocular stereo vision technology to obtain depth information of the target image to be detected;

[0012] Step S7: input the preprocessed target image to be detected into the optimized CNN model for feature extraction, and output the target features;

[0013] Step S8: fusing the depth information of the target image to be detected with the target feature to obtain a fused target feature;

[0014] Step S9: Use the region proposal network (RPN) algorithm to identify the fused target features and identify the three-dimensional target object.

[0015] Preferably, in step S3, the CNN model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer;

[0016] The mathematical formula of the input layer is as follows:

[0017] input(H, W, C);

[0018] Among them, H represents the height of the image, W represents the width of the image, and C represents the number of channels of the image;

[0019] The mathematical formula of the convolutional layer is as follows:

[0020]

[0021] Among them, X(i, j) represents the input feature map, A(m, n) represents the filter, m represents the height of the convolution kernel, n represents the width of the convolution kernel, k represents the size of the filter, b represents the bias term, and Y1(i, j) represents the output feature map;

[0022] The mathematical formula of the pooling layer is as follows:

[0023]

[0024] Among them, Y2(i, j) represents the output feature map after pooling, X(i, j) represents the input feature map, m represents the height of the convolution kernel, and n represents the width of the convolution kernel;

[0025] The mathematical formula of the fully connected layer is as follows:

[0026] y=B·x+b

[0027] Among them, y represents the output vector, x represents the input vector, B represents the weight matrix, and b represents the bias term;

[0028] In the output layer, the Softmax activation function is used to output the probability distribution of the category, where the mathematical formula of the Softmax activation function is as follows:

[0029]

[0030] Among them, P(y=c k |x) represents the predicted probability of a category, z k represents the raw score of each category in the output layer, c k Represents the value of each category in the output layer.

[0031] Preferably, in step S4, the CNN model is optimized using the labeled data set, specifically comprising the following sub-steps: the weights of the CNN model are constrained using the L2 regularization method, specifically comprising the sub-steps:

[0032] Step S41: Calculate the total loss function L after adding the L2 regularization term total , the specific calculation formula is as follows:

[0033]

[0034] Among them, L(θ) represents the original loss function, which is used to measure the degree of fit of the CNN model to the labeled data set. represents the sum of squares of all model weights, θ i represents the i-th weight, λ represents the regularization strength hyperparameter; d represents the number of weights, L total Represents the total loss function after adding the L2 regularization term;

[0035] Step S42: Calculate the total gradient of the loss function to the weight of the CNN model. The specific calculation formula is as follows:

[0036]

[0037] in, represents the total gradient of the loss function after adding the L2 regularization term, represents the gradient of the original loss function, 2λθ i represents the gradient of the L2 regularization term;

[0038] Step S43: Use the gradient descent method to update the weights of the CNN model, where the calculation formula for the updated weights is as follows:

[0039]

[0040] Among them, θ i ′ represents the weight after the i-th update, and η represents the learning rate, which is used to determine the step size of each update.

[0041] Preferably, in step S4, the CNN model is optimized using the labeled data set, specifically including the following sub-steps: the parameters of the CNN model are optimized using an early stopping strategy, specifically including the following sub-steps:

[0042] Step S44: setting an iteration threshold and dividing the labeled data set into a training set, a validation set, and a test set;

[0043] Step S45: Initialize the parameters of the CNN model, and use the training set to perform several iterations of training on the CNN model;

[0044] Step S46: Calculate the validation loss of the CNN model on the validation set after each iteration of training;

[0045] Step S47: Record the validation loss of the CNN model on the validation set after each iterative training, and compare it with the best historical validation loss. If the validation loss of the CNN model on the validation set after the current iterative training is less than the best historical validation loss, update the best validation loss and the parameters of the best CNN model. If the validation loss of the CNN model on the validation set after the current iterative training is greater than the best historical validation loss, record the validation loss.

[0046] Step S48: Determine whether the verification loss of the CNN model on the verification set is less than the best historical verification loss after multiple consecutive iterative trainings, and the current cumulative number of iterations is greater than the iteration threshold. If so, stop the iterative training of the CNN model and return the parameters of the CNN model with the smallest verification loss during the training process. If not, continue the iterative training of the CNN model.

[0047] Preferably, in step S4, the following steps are also included: calculating the accuracy and recall of the CNN model on the test set to evaluate the performance of the CNN model, and generating a confusion matrix to evaluate the classification effect of the CNN model;

[0048] Among them, the calculation formula for the accuracy of the CNN model is as follows:

[0049]

[0050] Among them, Accuracy represents the accuracy of the CNN model, TP represents the number of samples correctly predicted as positive by the CNN model, TN represents the number of samples correctly predicted as negative by the CNN model, FP represents the number of samples incorrectly predicted as positive by the CNN model, and FN represents the number of samples incorrectly predicted as negative by the CNN model.

[0051] The calculation formula for the recall rate of the CNN model is as follows:

[0052]

[0053] Among them, Recall represents the recall rate of the CNN model;

[0054] The mathematical formula for the confusion matrix is ​​as follows:

[0055]

[0056] Preferably, step S8 specifically includes the following sub-steps: fusing the depth information of the target image to be detected with the target feature by weighted splicing to obtain the fused target feature.

[0057] Preferably, step S9 specifically includes the following sub-steps:

[0058] Step S91: using a sliding window algorithm to process the fused target features to obtain a number of sliding windows;

[0059] Step S92: generating a predetermined number of anchor frames for each sliding window;

[0060] Step S93: determine whether each anchor frame contains the target. If so, the anchor frame containing the target is used as a candidate frame. If not, the anchor frame not containing the target is not processed.

[0061] Step S94: Screening all candidate frames to obtain screened candidate frames;

[0062] Step S95: Remove duplicate or overlapping candidate frames through non-maximum suppression to obtain the final target detection frame.

[0063] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0064] In this scheme, the depth information of the target image is directly obtained through binocular stereo vision technology. Compared with the existing 3D target detection method, when obtaining the depth information in the image, it is necessary to first obtain the image and point cloud information of the target object's environment, and then obtain the coordinate information of the point cloud information in the image, and then combine these coordinate information with the 2D and 3D bounding box information in the image. This eliminates the steps of obtaining the point cloud information and calculating its coordinates in the image, thereby reducing the complexity of the entire depth information calculation process and improving the efficiency of obtaining depth information. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a step flow chart of a three-dimensional target detection method based on stereo vision technology. DETAILED DESCRIPTION

[0066] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0067] A three-dimensional target detection method based on stereo vision technology comprises the following steps:

[0068] Step S1: Acquire several target images;

[0069] Step S2: labeling a number of target images to obtain a labeled data set;

[0070] Step S3: Establish a convolutional neural network CNN model;

[0071] Step S4: Use the labeled data set to train and optimize the CNN model to obtain an optimized CNN model;

[0072] Step S5: acquiring a target image to be detected, and preprocessing the target image to be detected to obtain a preprocessed target image to be detected;

[0073] Step S6: using binocular stereo vision technology to obtain depth information of the target image to be detected;

[0074] Step S7: input the preprocessed target image to be detected into the optimized CNN model for feature extraction, and output the target features;

[0075] Step S8: fusing the depth information of the target image to be detected with the target feature to obtain a fused target feature;

[0076] Step S9: Use the region proposal network (RPN) algorithm to identify the fused target features and identify the three-dimensional target object.

[0077] This scheme is a three-dimensional target detection method based on stereo vision technology, such as Figure 1As shown, the first step is to obtain several target images. In this embodiment, several target images are collected by a camera, which is conducive to the subsequent processing of the target images. The second step is to annotate several target images to obtain a labeled data set. In this embodiment, the annotated content includes the location of the target object and the corresponding depth. By annotating several target images, it is conducive to subsequent model training. The third step is to establish a convolutional neural network CNN model. In this embodiment, the CNN model is a powerful tool in the field of deep learning, particularly suitable for image feature extraction and classification tasks. By establishing a CNN model, it is conducive to the subsequent extraction of target features. The fourth step is to train and optimize the CNN model using a labeled data set to obtain an optimized CNN model. In this embodiment, by training and optimizing the CNN model, it is conducive to improving the accuracy of target detection of the CNN model. The fifth step is to obtain the target image to be detected, and pre-process the target image to be detected to obtain the target image to be detected after pre-processing. In this embodiment, the pre-processing includes image correction, image denoising and image enhancement. By pre-processing the target image to be detected, it is conducive to improving the quality of the target image to ensure the accuracy and stability of subsequent target detection. Further explanation, image correction includes geometric correction, such as distortion removal and perspective correction, color correction, such as white balance correction and color space conversion and image registration. Image denoising includes spatial domain denoising techniques, such as mean filtering, median filtering and Gaussian filtering, frequency domain denoising techniques, such as Fourier transform and wavelet transform, and statistical methods, such as Wiener filtering and corrosion and expansion. Image enhancement includes contrast enhancement, such as histogram equalization and adaptive histogram equalization, brightness adjustment, such as gamma correction and linear stretching, and edge enhancement, such as Laplace filtering, Sobel filtering and high-pass filtering. The sixth step is to use binocular stereo vision technology to obtain the depth information of the target image to be detected. In this embodiment, binocular stereo vision technology is an important form of machine vision, which is based on the difference in viewing angles of two binocular cameras to calculate the depth information of objects in the scene. Obtaining the depth information of the target image to be detected through binocular stereo vision technology is conducive to the subsequent fusion of target features. The seventh step is to input the preprocessed target image to be detected into the optimized CNN model for feature extraction and output the target features. In this embodiment, the trained CNN model can extract key features in the target image, which are crucial for subsequent target recognition. The eighth step is to fuse the depth information of the target image to be detected with the target features to obtain the fused target features. In this embodiment, the depth information of the target image to be detected is fused with the target features so as to make full use of the depth information to improve the performance of three-dimensional target detection.The ninth step is to use the region proposal network (RPN) algorithm to identify the fused target features and identify the three-dimensional target object. In this embodiment, the region proposal network (RPN) algorithm is one of the target detection algorithms suitable for three-dimensional scenes. The RPN algorithm can be used to identify target objects in three-dimensional scenes.

[0078] In this scheme, the depth information of the target image is directly obtained through binocular stereo vision technology. Compared with the existing 3D target detection method, when obtaining the depth information in the image, it is necessary to first obtain the image and point cloud information of the target object's environment, and then obtain the coordinate information of the point cloud information in the image, and then combine these coordinate information with the 2D and 3D bounding box information in the image. This eliminates the steps of obtaining the point cloud information and calculating its coordinates in the image, thereby reducing the complexity of the entire depth information calculation process and improving the efficiency of obtaining depth information.

[0079] Preferably, in step S3, the CNN model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer;

[0080] The mathematical formula of the input layer is as follows:

[0081] input(H, W, C);

[0082] Among them, H represents the height of the image, W represents the width of the image, and C represents the number of channels of the image;

[0083] The mathematical formula of the convolutional layer is as follows:

[0084]

[0085] Among them, X(i, j) represents the input feature map, A(m, n) represents the filter, m represents the height of the convolution kernel, n represents the width of the convolution kernel, k represents the size of the filter, b represents the bias term, and Y1(i, j) represents the output feature map;

[0086] The mathematical formula of the pooling layer is as follows:

[0087]

[0088] Among them, Y2(i, j) represents the output feature map after pooling, X(i, j) represents the input feature map, m represents the height of the convolution kernel, and n represents the width of the convolution kernel;

[0089] The mathematical formula of the fully connected layer is as follows:

[0090] y=B·x+b

[0091] Among them, y represents the output vector, x represents the input vector, B represents the weight matrix, and b represents the bias term;

[0092] In the output layer, the Softmax activation function is used to output the probability distribution of the category, where the mathematical formula of the Softmax activation function is as follows:

[0093]

[0094] Among them, P(y=c k |x) represents the predicted probability of a category, z k represents the raw score of each category in the output layer, c k Represents the value of each category in the output layer.

[0095] In this embodiment, in the image classification task, the input of the input layer is usually image data, whose shape and size are H, W, C. The convolution layer is the most important layer in the CNN model, which extracts local features of the input data through convolution operations. Each convolution operation involves a filter that calculates the convolution output by sliding on the input image. The pooling layer usually follows the convolution layer to reduce the spatial dimension of the feature map, reduce the amount of calculation, and reduce overfitting. Maximum pooling is the most common pooling operation, which takes the maximum value in the local area. After the convolution layer and the pooling layer, there are usually one or more fully connected layers to flatten the feature map and map it to the final output space. For classification problems, the output layer uses the Softmax activation function to output the probability distribution of each category.

[0096] Preferably, in step S4, the CNN model is optimized using the labeled data set, specifically including the following sub-steps: the weights of the CNN model are constrained using the L2 regularization method, specifically including the sub-steps:

[0097] Step S41: Calculate the total loss function L after adding the L2 regularization term total , the specific calculation formula is as follows:

[0098]

[0099] Among them, L(θ) represents the original loss function, which is used to measure the degree of fit of the CNN model to the labeled data set. represents the sum of squares of all model weights, θ i represents the i-th weight, λ represents the regularization strength hyperparameter; d represents the number of weights, L total Represents the total loss function after adding the L2 regularization term;

[0100] Step S42: Calculate the total gradient of the loss function to the weight of the CNN model. The specific calculation formula is as follows:

[0101]

[0102] in, represents the total gradient of the loss function after adding the L2 regularization term, represents the gradient of the original loss function, 2λθ i represents the gradient of the L2 regularization term;

[0103] Step S43: Use the gradient descent method to update the weights of the CNN model, where the calculation formula for the updated weights is as follows:

[0104]

[0105] Among them, θ i ′ represents the weight after the i-th update, and η represents the learning rate, which is used to determine the step size of each update.

[0106] In one embodiment, the CNN model is optimized by L2 regularization, which is also called weight decay. It constrains the weights of the model to make the weights smaller, thereby simplifying the model and improving the generalization ability. Further explanation: a larger λ value will make the regularization effect stronger, resulting in smaller weights. The gradient of the L2 regularization term 2λθ i The role of is to weight θ i Push it towards 0 to avoid excessive weight.

[0107] Preferably, in step S4, the CNN model is optimized using the labeled data set, specifically including the following sub-steps: the parameters of the CNN model are optimized using an early stopping strategy, specifically including the following sub-steps:

[0108] Step S44: setting an iteration threshold and dividing the labeled data set into a training set, a validation set, and a test set;

[0109] Step S45: Initialize the parameters of the CNN model, and use the training set to perform several iterations of training on the CNN model;

[0110] Step S46: Calculate the validation loss of the CNN model on the validation set after each iterative training;

[0111] Step S47: Record the validation loss of the CNN model on the validation set after each iterative training, and compare it with the best historical validation loss. If the validation loss of the CNN model on the validation set after the current iterative training is less than the best historical validation loss, update the best validation loss and the parameters of the best CNN model. If the validation loss of the CNN model on the validation set after the current iterative training is greater than the best historical validation loss, record the validation loss.

[0112] Step S48: Determine whether the verification loss of the CNN model on the verification set is less than the best historical verification loss after multiple consecutive iterative trainings, and the current cumulative number of iterations is greater than the iteration threshold. If so, stop the iterative training of the CNN model and return the parameters of the CNN model with the smallest verification loss during the training process. If not, continue the iterative training of the CNN model.

[0113] In another embodiment, the CNN model is optimized by an early stopping strategy. In step S44, when the iteration threshold is set to 5 times, it means that if the performance of the CNN model on the validation set does not improve in 5 consecutive iterations, the training of the CNN model is stopped. This can avoid unnecessary early stopping due to single validation fluctuations. In step S46, by calculating the validation loss of the CNN model on the validation set after each iterative training, it is helpful to judge the performance difference of the CNN model on the validation set, so as to infer whether the CNN model has begun to overfit. In step S48, if the validation loss of the CNN model on the validation set after multiple consecutive iterative trainings is less than the historical best validation loss, and the current cumulative number of iterations is greater than the iteration threshold, it is considered that the CNN model has begun to overfit. By stopping the iterative training of the CNN model, it is helpful to avoid the performance degradation of the CNN model caused by continuing training after overfitting.

[0114] Preferably, in step S4, the following steps are also included: calculating the accuracy and recall of the CNN model on the test set to evaluate the performance of the CNN model, and generating a confusion matrix to evaluate the classification effect of the CNN model;

[0115] Among them, the calculation formula for the accuracy of the CNN model is as follows:

[0116]

[0117] Among them, Accuracy represents the accuracy of the CNN model, TP represents the number of samples correctly predicted as positive by the CNN model, TN represents the number of samples correctly predicted as negative by the CNN model, FP represents the number of samples incorrectly predicted as positive by the CNN model, and FN represents the number of samples incorrectly predicted as negative by the CNN model.

[0118] The calculation formula for the recall rate of the CNN model is as follows:

[0119]

[0120] Among them, Recall represents the recall rate of the CNN model;

[0121] The mathematical formula for the confusion matrix is ​​as follows:

[0122]

[0123] In this embodiment, the accuracy of the CNN model is the most intuitive performance metric in the classification problem, which indicates the proportion of samples correctly predicted by the CNN model to the total number of samples. The recall rate of the CNN model indicates the proportion of samples that are correctly predicted as positive by the CNN model among all samples that are actually positive. The recall rate emphasizes the samples that are missed. The confusion matrix is ​​a visualization tool that shows the distribution of actual categories and model-predicted categories. It is used to evaluate the classification effect of the classification model. For binary classification problems, the confusion matrix is ​​usually a 2×2 matrix.

[0124] Preferably, in step S8, the following sub-steps are specifically included: the depth information of the target image to be detected is fused with the target features by weighted splicing to obtain the fused target features. In this embodiment, the weighted splicing method specifically assigns weights to the depth features and the target features, splices the depth features and the target features in the channel dimension, and applies weights to the spliced ​​features. The fusion of features is helpful to improve the accuracy of subsequent three-dimensional target detection. In other embodiments, the depth information of the target image to be detected and the target features can also be fused by simple splicing, cross splicing, or pipeline parallelism.

[0125] Preferably, in step S9, the following sub-steps are specifically included:

[0126] Step S91: using a sliding window algorithm to process the fused target features to obtain a number of sliding windows;

[0127] Step S92: generating a predetermined number of anchor frames for each sliding window;

[0128] Step S93: determine whether each anchor frame contains the target. If so, the anchor frame containing the target is used as a candidate frame. If not, the anchor frame not containing the target is not processed.

[0129] Step S94: Screening all candidate frames to obtain screened candidate frames;

[0130] Step S95: Remove duplicate or overlapping candidate frames through non-maximum suppression to obtain the final target detection frame.

[0131] In this embodiment, in step S91, the sliding window algorithm slides multiple windows on the fused target feature map, and extracts and classifies features for the area within each window. In step S92, different anchor frames have different proportions and sizes to accommodate targets of various sizes that may appear in the image. In step S93, using the anchor frame containing the target as a candidate frame means only focusing on those areas that are most likely to contain the target, which helps to reduce false positives, that is, the situation where non-target areas are mistakenly identified as targets, thereby improving the accuracy of target detection. In step S94, the screening step includes filtering based on the confidence of the candidate frame to further reduce the number of candidate frames. In step S95, non-maximum suppression is used to process multiple highly overlapping candidate frames, retaining only the one with the highest confidence. By removing duplicate or overlapping candidate frames through non-maximum suppression, the duplicate candidate frames can be significantly reduced, thereby improving the accuracy of the detection results.

[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0133] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A three-dimensional target detection method based on stereo vision technology, characterized in that: The following steps are involved: Step S1: Acquire several target images; Step S2: labeling a number of target images to obtain a labeled data set; Step S3: Establish a convolutional neural network CNN model; Step S4: Use the labeled data set to train and optimize the CNN model to obtain an optimized CNN model; Step S5: acquiring a target image to be detected, and preprocessing the target image to be detected to obtain a preprocessed target image to be detected; Step S6: using binocular stereo vision technology to obtain depth information of the target image to be detected; Step S7: input the preprocessed target image to be detected into the optimized CNN model for feature extraction, and output the target features; Step S8: fusing the depth information of the target image to be detected with the target feature to obtain a fused target feature; Step S9: Use the region proposal network (RPN) algorithm to identify the fused target features and identify the three-dimensional target object.

2. The three-dimensional target detection method based on stereo vision technology according to claim 1, characterized in that: In step S3, the CNN model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer; The mathematical formula of the input layer is as follows: input(H, W, C); Among them, H represents the height of the image, W represents the width of the image, and C represents the number of channels of the image; The mathematical formula of the convolutional layer is as follows: Among them, X(i, j) represents the input feature map, A(m, n) represents the filter, m represents the height of the convolution kernel, n represents the width of the convolution kernel, k represents the size of the filter, b represents the bias term, and Y1(i, j) represents the output feature map; The mathematical formula of the pooling layer is as follows: Among them, Y2(i, j) represents the output feature map after pooling, X(i, j) represents the input feature map, m represents the height of the convolution kernel, and n represents the width of the convolution kernel; The mathematical formula of the fully connected layer is as follows: y=B·x+b Among them, y represents the output vector, x represents the input vector, B represents the weight matrix, and b represents the bias term; In the output layer, the Softmax activation function is used to output the probability distribution of the category, where the mathematical formula of the Softmax activation function is as follows: Where P(y=c k |x) represents the predicted probability of a category, z k represents the raw score of each category in the output layer, c k Represents the value of each category in the output layer.

3. The three-dimensional target detection method based on stereo vision technology according to claim 1, characterized in that: In step S4, the CNN model is optimized using the labeled data set, which specifically includes the following sub-steps: The weights of the CNN model are constrained using the L2 regularization method, which specifically includes the sub-steps: Step S41: Calculate the total loss function L after adding the L2 regularization term total , the specific calculation formula is as follows: Among them, L(θ) represents the original loss function, which is used to measure the degree of fit of the CNN model to the labeled data set. represents the sum of squares of all model weights, θ i represents the i-th weight, λ represents the regularization strength hyperparameter; d represents the number of weights, L total Represents the total loss function after adding the L2 regularization term; Step S42: Calculate the total gradient of the loss function to the weight of the CNN model. The specific calculation formula is as follows: in, represents the total gradient of the loss function after adding the L2 regularization term, represents the gradient of the original loss function, 2λθ i represents the gradient of the L2 regularization term; Step S43: Use the gradient descent method to update the weights of the CNN model, where the calculation formula for the updated weights is as follows: Among them, θ i ′ represents the weight after the i-th update, and η represents the learning rate, which is used to determine the step size of each update.

4. The three-dimensional target detection method based on stereo vision technology according to claim 1, characterized in that: In step S4, the CNN model is optimized using the labeled data set, which specifically includes the following sub-steps: The parameters of the CNN model are optimized using an early stopping strategy, which specifically includes the following sub-steps: Step S44: setting an iteration threshold and dividing the labeled data set into a training set, a validation set, and a test set; Step S45: Initialize the parameters of the CNN model, and use the training set to perform several iterations of training on the CNN model; Step S46: Calculate the validation loss of the CNN model on the validation set after each iterative training; Step S47: Record the validation loss of the CNN model on the validation set after each iterative training, and compare it with the best historical validation loss. If the validation loss of the CNN model on the validation set after the current iterative training is less than the best historical validation loss, update the best validation loss and the parameters of the best CNN model. If the validation loss of the CNN model on the validation set after the current iterative training is greater than the best historical validation loss, record the validation loss. Step S48: Determine whether the verification loss of the CNN model on the verification set is less than the best historical verification loss after multiple consecutive iterative trainings, and the current cumulative number of iterations is greater than the iteration threshold. If so, stop the iterative training of the CNN model and return the parameters of the CNN model with the smallest verification loss during the training process. If not, continue the iterative training of the CNN model.

5. The three-dimensional target detection method based on stereo vision technology according to claim 4, characterized in that: In step S4, the following steps are also included: calculating the accuracy and recall of the CNN model on the test set to evaluate the performance of the CNN model, and generating a confusion matrix to evaluate the classification effect of the CNN model; Among them, the calculation formula for the accuracy of the CNN model is as follows: Among them, Accuracy represents the accuracy of the CNN model, TP represents the number of samples correctly predicted as positive by the CNN model, TN represents the number of samples correctly predicted as negative by the CNN model, FP represents the number of samples incorrectly predicted as positive by the CNN model, and FN represents the number of samples incorrectly predicted as negative by the CNN model. The calculation formula for the recall rate of the CNN model is as follows: Among them, Recall represents the recall rate of the CNN model; The mathematical formula of the confusion matrix is ​​as follows:

6. The three-dimensional target detection method based on stereo vision technology according to claim 1, characterized in that: In step S8, the following sub-steps are specifically included: The depth information of the target image to be detected is fused with the target features by weighted splicing to obtain the fused target features.

7. The three-dimensional target detection method based on stereo vision technology according to claim 1, characterized in that: In step S9, the following sub-steps are specifically included: Step S91: using a sliding window algorithm to process the fused target features to obtain a number of sliding windows; Step S92: generating a predetermined number of anchor frames for each sliding window; Step S93: determine whether each anchor frame contains the target. If so, the anchor frame containing the target is used as a candidate frame. If not, the anchor frame not containing the target is not processed. Step S94: Screening all candidate frames to obtain screened candidate frames; Step S95: Remove duplicate or overlapping candidate frames through non-maximum suppression to obtain the final target detection frame.

Images