Target detection method and device in driving scene based on YOLOv8, storage medium and equipment

Abstract

The invention discloses a target detection method and device in a driving scene based on YOLOv8, a storage medium and computer equipment, and the method comprises the steps: obtaining a target detection model based on YOLOv8, a backbone network comprising a plurality of C2fDCNv4 modules, a neck network comprising a plurality of feature fusion layers of different scales, and a head network comprising a plurality of target detection modules of different scales; the target detection model completes training in advance based on an SODA-D data set; inputting a to-be-detected image in a driving scene into the target detection model, carrying out image feature extraction on the to-be-detected image through the backbone network to obtain an image feature corresponding to each C2fDCNv4 module, carrying out feature fusion on the plurality of image features through the neck network to obtain a fusion feature corresponding to each feature fusion layer, and carrying out feature fusion on the fusion feature corresponding to each feature fusion layer; target identification is carried out on the fusion features of each feature fusion layer through the head network, and a target detection result corresponding to each target detection module is obtained; and determining a detected target in the to-be-detected image according to the target detection result.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a target detection method and device, storage medium, and computer equipment in a driving scenario based on YOLOv8. Background Art

[0002] Small target detection has important applications in many fields, such as monitoring, security, and autonomous driving. With the continuous development of computer vision and image recognition technology, target detection algorithms have good performance in various fields, especially in the field of transportation. Such algorithms have faster reasoning speed, higher accuracy, and stronger robustness, which play an indispensable role in improving traffic congestion, reducing traffic accidents, recording illegal parking, and road driving safety. How to improve the accuracy of target detection has always been a hot issue in driving scenarios. Summary of the invention

[0003] In view of this, the embodiment of the present application provides a method and device for target detection in driving scenarios based on YOLOv8, a storage medium, and a computer device. The method effectively improves the detection capability of various targets in complex driving environments, especially the detection performance of small targets, by combining the advanced architecture of YOLOv8 and the training strategy optimized for driving scenarios. This is of great significance for applications such as autonomous driving and assisted driving, and can provide more accurate and reliable environmental perception capabilities.

[0004] According to one aspect of the present application, a method for target detection in a driving scenario based on YOLOv8 is provided, the method comprising:

[0005] Obtain a target detection model based on YOLOv8, wherein the target detection model includes a backbone network for extracting image features, a neck network for fusing image features, and a head network for detecting targets, the backbone network includes multiple C2f_DCNv4 modules, the neck network includes multiple feature fusion layers of different scales, and the head network includes multiple target detection modules of different scales; the target detection model is pre-trained based on the SODA-D (Small Object Detection dAtaset for Driving) data set;

[0006] The image to be detected in the driving scene is input into the target detection model, and the image features of the image to be detected are extracted through the backbone network to obtain the image features corresponding to each C2f_DCNv4 module, and the multiple image features are fused through the neck network to obtain the fusion features corresponding to each feature fusion layer, and the fusion features of each feature fusion layer are respectively identified through the head network to obtain the target detection results corresponding to each target detection module;

[0007] According to the target detection result, a detected target in the image to be detected is determined.

[0008] In an embodiment of the present application, optionally, the backbone network sequentially includes a first convolutional layer, a second convolutional layer, a first C2f_DCNv4 module, a third convolutional layer, a second C2f_DCNv4 module, a fourth convolutional layer, a third C2f_DCNv4 module, a fifth convolutional layer, a fourth C2f_DCNv4 module, and a SPPF (Spatial Pyramid Pooling Fast) module;

[0009] The image features of the image to be detected are extracted through the backbone network to obtain image features corresponding to each C2f_DCNv4 module, including:

[0010] The image to be detected is input into the backbone network to gradually extract image features, and the output of the first C2f_DCNv4 module is obtained as the first image feature, the output of the second C2f_DCNv4 module is obtained as the second image feature, the output of the third C2f_DCNv4 module is obtained as the third image feature, and the output of the SPPF module is obtained as the fourth image feature.

[0011] In the embodiment of the present application, optionally, the neck network includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer and a fourth feature fusion layer;

[0012] The first feature fusion layer includes a sixth convolutional layer, a first ASFF (Adaptive Spatial Feature Fusion) module, a first C2F (Convolution to Feature) module, a second ASFF module, a second C2F module, a third ASFF module, and a third C2F module;

[0013] The second feature fusion layer includes a seventh convolutional layer, a fourth ASFF module, a fourth C2F module, a fifth ASFF module, a fifth C2F module, a sixth ASFF module, and a sixth C2F module;

[0014] The third feature fusion layer includes an eighth convolutional layer, a seventh C2F module, a seventh ASFF module, an eighth C2F module, an eighth ASFF module, and a ninth C2F module;

[0015] The fourth feature fusion layer includes a ninth convolutional layer, a tenth C2F module, a ninth ASFF module, and an eleventh C2F module;

[0016] The neck network is used to fuse multiple image features to obtain fusion features corresponding to each feature fusion layer, including:

[0017] The first image feature, the second image feature, the third image feature and the fourth image feature are input into the sixth convolution layer, the seventh convolution layer, the eighth convolution layer and the ninth convolution layer respectively; the output results of the sixth convolution layer and the seventh convolution layer are processed by the first ASFF module and then input into the first C2F module; the output results of the sixth convolution layer and the seventh convolution layer are processed by the second ASFF module and then input into the second C2F module; the output result of the eighth convolution layer is input into the third C2F module; the output result of the ninth convolution layer is input into the tenth C2F module;

[0018] Input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the second ASFF module for processing, and input the output result of the second ASFF module into the second C2F module; input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the fifth ASFF module for processing, and input the output result of the fifth ASFF module into the fifth C2F module; input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the seventh ASFF module for processing, and input the output result of the fifth ASFF module into the seventh C2F module;

[0019] Input the output results of the second C2F module, the fifth C2F module, the eighth C2F module and the tenth C2F module into the third ASFF module, the sixth ASFF module, the eighth ASFF module and the ninth ASFF module for processing;

[0020] The output result of the third ASFF module is processed by the third C2F module to obtain the first fusion feature; the output result of the sixth ASSF module is processed by the sixth C2F module to obtain the second fusion feature; the output result of the eighth ASFF module is processed by the ninth C2F module to obtain the third fusion feature; the output result of the ninth ASFF module is processed by the eleventh C2F module to obtain the fourth fusion feature.

[0021] In an embodiment of the present application, optionally, the head network includes a first target detection module, a second target detection module, a third target detection module and a fourth target detection module;

[0022] The head network performs target recognition on the fused features of each feature fusion layer respectively, and obtains the target detection result corresponding to each target detection module, including:

[0023] The first fusion feature, the second fusion feature, the third fusion feature and the fourth fusion feature are respectively input into the first target detection module, the second target detection module, the third target detection module and the fourth target detection module to obtain the target detection result output by each target detection module.

[0024] In the embodiment of the present application, optionally, before obtaining the target detection model based on YOLOv8, the method further includes:

[0025] Build an initial target detection model based on YOLOv8;

[0026] A SODA-D dataset is obtained as a sample image set, wherein the sample image set includes a plurality of sample images with annotation information, the annotation information includes an annotation box and an annotation type of an object in the annotation box, and the annotation type includes a person, a rider, a bicycle, a car, a traffic sign, a traffic light, a traffic camera, and a warning cone;

[0027] The sample image set is divided into a training set and a test set, the initial target detection model is trained using the training set, and the trained initial target detection model is tested using the test set, and the initial target detection model that meets the test conditions is used as the target detection model.

[0028] In an embodiment of the present application, optionally, training the initial target detection model using the training set includes:

[0029] Using the initial target detection model to perform target detection on the training sample images in the training set, and determining the target detection results of the training sample images;

[0030] Determine a prediction score for each of the training sample images according to the target detection result and the annotation information of each of the training sample images, determine a sample division threshold according to the prediction score of each of the training sample images, and divide each of the training sample images into a positive sample or a negative sample according to the sample division threshold and the prediction score of each of the training sample images;

[0031] Different weights are respectively assigned to the positive samples and the negative samples, and the initial target detection model is trained using the training sample images based on a loss function, wherein during the training process, the weights corresponding to the training sample images are assigned to the loss function for loss calculation, and the weight of the negative samples is greater than the weight of the positive samples.

[0032] In an embodiment of the present application, optionally, determining a prediction score of each training sample image according to the target detection result and the annotation information of each training sample image includes:

[0033] The intersection area and the union area of ​​the prediction box and the annotation box of the training sample image are calculated, and the ratio of the intersection area to the union area is used as the prediction score of the training sample image.

[0034] According to another aspect of the present application, a target detection device in a driving scenario based on YOLOv8 is provided, the device comprising:

[0035] A model acquisition module, used to acquire a target detection model based on YOLOv8, wherein the target detection model includes a backbone network for extracting image features, a neck network for fusing image features, and a head network for detecting targets, the backbone network includes multiple C2f_DCNv4 modules, the neck network includes multiple feature fusion layers of different scales, and the head network includes multiple target detection modules of different scales; the target detection model is pre-trained based on the SODA-D (Small Object Detection dAtaset for Driving) data set;

[0036] A detection module is used to input the image to be detected in the driving scene into the target detection model, perform image feature extraction on the image to be detected through the backbone network, obtain image features corresponding to each C2f_DCNv4 module, perform feature fusion on multiple image features through the neck network, obtain fusion features corresponding to each feature fusion layer, perform target recognition on the fusion features of each feature fusion layer through the head network, and obtain target detection results corresponding to each target detection module;

[0037] The detection module is further used to determine the detected target in the image to be detected according to the target detection result.

[0038] In an embodiment of the present application, optionally, the backbone network sequentially includes a first convolutional layer, a second convolutional layer, a first C2f_DCNv4 module, a third convolutional layer, a second C2f_DCNv4 module, a fourth convolutional layer, a third C2f_DCNv4 module, a fifth convolutional layer, a fourth C2f_DCNv4 module, and a SPPF (Spatial Pyramid Pooling Fast) module;

[0039] The detection module is further used for:

[0040] The image to be detected is input into the backbone network to gradually extract image features, and the output of the first C2f_DCNv4 module is obtained as the first image feature, the output of the second C2f_DCNv4 module is obtained as the second image feature, the output of the third C2f_DCNv4 module is obtained as the third image feature, and the output of the SPPF module is obtained as the fourth image feature.

[0041] In the embodiment of the present application, optionally, the neck network includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer and a fourth feature fusion layer;

[0042] The first feature fusion layer includes a sixth convolutional layer, a first ASFF (Adaptive Spatial Feature Fusion) module, a first C2F (Convolution to Feature) module, a second ASFF module, a second C2F module, a third ASFF module, and a third C2F module;

[0043] The second feature fusion layer includes a seventh convolutional layer, a fourth ASFF module, a fourth C2F module, a fifth ASFF module, a fifth C2F module, a sixth ASFF module, and a sixth C2F module;

[0044] The third feature fusion layer includes an eighth convolutional layer, a seventh C2F module, a seventh ASFF module, an eighth C2F module, an eighth ASFF module, and a ninth C2F module;

[0045] The fourth feature fusion layer includes a ninth convolutional layer, a tenth C2F module, a ninth ASFF module, and an eleventh C2F module;

[0046] The detection module is further used for:

[0047] The first image feature, the second image feature, the third image feature and the fourth image feature are input into the sixth convolution layer, the seventh convolution layer, the eighth convolution layer and the ninth convolution layer respectively; the output results of the sixth convolution layer and the seventh convolution layer are processed by the first ASFF module and then input into the first C2F module; the output results of the sixth convolution layer and the seventh convolution layer are processed by the second ASFF module and then input into the second C2F module; the output result of the eighth convolution layer is input into the third C2F module; the output result of the ninth convolution layer is input into the tenth C2F module;

[0048] Input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the second ASFF module for processing, and input the output result of the second ASFF module into the second C2F module; input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the fifth ASFF module for processing, and input the output result of the fifth ASFF module into the fifth C2F module; input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the seventh ASFF module for processing, and input the output result of the fifth ASFF module into the seventh C2F module;

[0049] Input the output results of the second C2F module, the fifth C2F module, the eighth C2F module and the tenth C2F module into the third ASFF module, the sixth ASFF module, the eighth ASFF module and the ninth ASFF module for processing;

[0050] The output result of the third ASFF module is processed by the third C2F module to obtain the first fusion feature; the output result of the sixth ASSF module is processed by the sixth C2F module to obtain the second fusion feature; the output result of the eighth ASFF module is processed by the ninth C2F module to obtain the third fusion feature; the output result of the ninth ASFF module is processed by the eleventh C2F module to obtain the fourth fusion feature.

[0051] In an embodiment of the present application, optionally, the head network includes a first target detection module, a second target detection module, a third target detection module and a fourth target detection module;

[0052] The detection module is further used for:

[0053] The first fusion feature, the second fusion feature, the third fusion feature and the fourth fusion feature are respectively input into the first target detection module, the second target detection module, the third target detection module and the fourth target detection module to obtain the target detection result output by each target detection module.

[0054] In the embodiment of the present application, optionally, the model acquisition module is further used to:

[0055] Build an initial target detection model based on YOLOv8;

[0056] A SODA-D dataset is obtained as a sample image set, wherein the sample image set includes a plurality of sample images with annotation information, the annotation information includes an annotation box and an annotation type of an object in the annotation box, and the annotation type includes a person, a rider, a bicycle, a car, a traffic sign, a traffic light, a traffic camera, and a warning cone;

[0057] The sample image set is divided into a training set and a test set, the initial target detection model is trained using the training set, and the trained initial target detection model is tested using the test set, and the initial target detection model that meets the test conditions is used as the target detection model.

[0058] In the embodiment of the present application, optionally, the model acquisition module is further used to:

[0059] Using the initial target detection model to perform target detection on the training sample images in the training set, and determining the target detection results of the training sample images;

[0060] Determine a prediction score for each of the training sample images according to the target detection result and the annotation information of each of the training sample images, determine a sample division threshold according to the prediction score of each of the training sample images, and divide each of the training sample images into a positive sample or a negative sample according to the sample division threshold and the prediction score of each of the training sample images;

[0061] Different weights are respectively assigned to the positive samples and the negative samples, and the initial target detection model is trained using the training sample images based on a loss function, wherein during the training process, the weights corresponding to the training sample images are assigned to the loss function for loss calculation, and the weight of the negative samples is greater than the weight of the positive samples.

[0062] In the embodiment of the present application, optionally, the model acquisition module is further used to:

[0063] The intersection area and the union area of ​​the prediction box and the annotation box of the training sample image are calculated, and the ratio of the intersection area to the union area is used as the prediction score of the training sample image.

[0064] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the above-mentioned target detection method in a driving scenario based on YOLOv8 is implemented.

[0065] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein when the processor executes the program, the above-mentioned target detection method in a driving scenario based on YOLOv8 is implemented.

[0066] By means of the above technical scheme, a method and device for target detection in a driving scene based on YOLOv8, a storage medium, and a computer device are provided in the embodiment of the present application, and a target detection scheme in a driving scene based on YOLOv8 is provided. By adopting a special model architecture, a backbone network including a C2f_DCNv4 module, a neck network for multi-scale feature fusion, and a head network for multi-scale target detection, and training with a small target detection data set (SODA-D) for driving scenes, various types of targets, especially small targets, are accurately detected in a complex driving environment. The beneficial effects of the embodiment of the present application are: improving detection accuracy: using the C2f_DCNv4 module as the backbone network, by introducing deformable convolution (DCNv4), the adaptability of the model to changes in object shape is enhanced, and the detection accuracy of complex shape and posture targets is improved. The multi-scale feature fusion and detection mechanism enable the model to capture both detail information and global information in the image, thereby more accurately detecting targets of different sizes, especially small targets. Enhanced robustness: The model is trained on the SODA-D dataset designed for driving scenarios, making it more adaptable to complex driving environments, including different lighting conditions, weather conditions, occlusion conditions, etc., which enhances the robustness of the model. The stability and reliability of the detection results are further improved by removing duplicate and redundant detection frames through post-processing algorithms (such as non-maximum suppression). Improved real-time performance: YOLOv8 itself is an efficient target detection model. Combined with the optimized model architecture and training strategy, the solution maintains high accuracy while also maintaining a fast detection speed to meet real-time requirements. Furthermore, it provides effective technical support for the driving field: The solution provides more accurate and reliable environmental perception capabilities for autonomous driving and assisted driving systems, helping to improve the safety and intelligence of autonomous driving vehicles. Through accurate target detection, autonomous driving systems can better understand and respond to changes in the surrounding environment and make more reasonable decisions and actions.

[0067] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0069] Figure 1 A schematic diagram of a process of a target detection method in a driving scenario based on YOLOv8 provided in an embodiment of the present application is shown;

[0070] Figure 2 A schematic diagram of the structure of a target detection model provided in an embodiment of the present application is shown;

[0071] Figure 3 A schematic diagram of the structure of a C2f_DCNv4 module provided in an embodiment of the present application is shown;

[0072] Figure 4 A schematic diagram of the structure of a target detection device in a driving scenario based on YOLOv8 provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0073] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0074] In this embodiment, a target detection method in a driving scenario based on YOLOv8 is provided. Figure 1 As shown, the method includes:

[0075] Step 101, obtaining a target detection model based on YOLOv8, wherein the target detection model includes a backbone network for extracting image features, a neck network for fusing image features, and a head network for detecting targets, the backbone network includes multiple C2f_DCNv4 modules, the neck network includes multiple feature fusion layers of different scales, and the head network includes multiple target detection modules of different scales; the target detection model is pre-trained based on the SODA-D (Small Object Detection dAtaset for Driving) data set.

[0076] In the embodiment of the present application, the target detection method in the driving scene based on YOLOv8 significantly improves the detection ability of various targets in a complex driving environment, especially the detection of small targets, by optimizing the model structure and using a specific data set for training. The target detection model used is constructed based on the YOLOv8 model, which consists of three parts: a backbone network, a neck network, and a head network. Backbone network: multiple C2f_DCNv4 modules are used, which can not only effectively extract the basic features of the image, but also enhance the adaptability of the model to the change of object shape by introducing deformable convolution (DCNv4), which is particularly suitable for processing the variable object morphology in driving scenes. Neck network: contains multiple feature fusion layers, which are responsible for fusing feature maps of different scales to capture detailed information and global information in the image at the same time, which is crucial for detecting targets of different sizes. Head network: contains multiple target detection modules, each module performs target detection on a feature map of a specific scale to ensure accurate identification of targets from multiple scales. Dataset used for training model: SODA-D (Small Object Detection dAtaset for Driving) dataset is used for training. This dataset is designed for driving scenarios and contains a large number of pictures of small targets and complex backgrounds, which helps the model learn how to effectively detect small targets in complex environments.

[0077] In an embodiment of the present application, optionally, the backbone network includes a first convolutional layer, a second convolutional layer, a first C2f_DCNv4 module, a third convolutional layer, a second C2f_DCNv4 module, a fourth convolutional layer, a third C2f_DCNv4 module, a fifth convolutional layer, a fourth C2f_DCNv4 module and an SPPF (Spatial Pyramid Pooling Fast, fast spatial pyramid pooling) module; the neck network includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer and a fourth feature fusion layer; the first feature fusion layer includes a sixth convolutional layer, a first ASFF (Adaptive Spatial Feature Fusion, adaptive spatial feature fusion) module, a first C2F (Convolution to Feature, convolution to feature) module, a second ASFF module, a second C2F module, a third ASFF module, and a third C2F module; the second feature fusion layer includes a seventh convolution layer, a fourth ASFF module, a fourth C2F module, a fifth ASFF module, a fifth C2F module, a sixth ASFF module, and a sixth C2F module; the third feature fusion layer includes an eighth convolution layer, a seventh C2F module, a seventh ASFF module, an eighth C2F module, an eighth ASFF module, and a ninth C2F module; the fourth feature fusion layer includes a ninth convolution layer, a tenth C2F module, a ninth ASFF module, and an eleventh C2F module; the head network includes a first target detection module, a second target detection module, a third target detection module, and a fourth target detection module.

[0078] In the above embodiment, if Figure 2As shown in the figure, the backbone network introduces deformable convolution DCNv4. Traditional convolution operations use fixed convolution kernel weights, resulting in the size of the receptive field remaining unchanged when processing different areas of the image. However, since objects in the image may be located in different positions of the feature map due to different sizes or shapes, a network structure that can adaptively adjust the receptive field to adapt to different objects is required. Secondly, deformable convolution provides stronger robustness by fitting the size and shape of the object more closely, which standard convolution cannot do. In addition, for targets of small size and variable shape, traditional convolution may not be able to effectively detect them, affecting the overall performance of the model. DCNv1 adds a two-dimensional offset to the regular grid sampling points in the standard convolution, so that the sampling grid can be deformed freely. The offset is learned from the previous feature map through an additional convolution layer. Therefore, the deformable convolution constrains the input features in a local, dense and adaptive manner. In order to improve the adaptability of the model, DCNv2 uses a weighted sampling point offset. DCNv3 combines the sparse attention mechanism with the convolution operation, and performs well in processing spatial feature aggregation with locality, dynamic sampling points and input-dependent weights. However, there is a problem of slow running speed, which will limit its generalization ability. Therefore, DCNv4 reduces a large number of unnecessary memory accesses and removes Softmax normalization, enhancing the dynamic characteristics of DCNv4. The embodiment of the present application introduces the deformable convolution DCNv4 in the backbone network of YOLOv8, such as Figure 3 The figure shows a schematic diagram of a C2f_DCNv4 module, which is formed by replacing the ordinary convolution layer of the Bottleneck module in the C2f module with a deformable convolution. C2f_DCNv4 can adjust its sampling window more freely to adapt to the characteristic distribution of small targets, thereby improving the detection accuracy of small target objects, and thus significantly improving the model's detection performance for objects of various sizes and shapes. In particular, when the C2f modules corresponding to the P2, P3, and P4 layer feature maps (the second, third, and fourth C2f modules in the original YOLOv8 backbone network) are replaced with C2f_DCNv4 modules, the model's attention to medium, small, and tiny targets can be strengthened.

[0079] The neck network adopts a progressive feature fusion strategy to solve the problem of feature attenuation. Before feature fusion, features of different levels are extracted from the backbone. By gradually fusing shallow, deep and bottom feature information, the difference in feature information between non-adjacent layers is effectively reduced in this process, thereby alleviating the problem of a large amount of feature information loss between these layers, and introducing it into the Neck (neck network) of YOLOv8. In the progressive feature fusion strategy of the embodiment of the present application, after each adaptive spatial feature fusion (ASFF), the fused feature information is further processed by stacking four BasicBlocks continuously to further process the fused features, so that features of different scales can be better fused. ASFF assigns different spatial weights to features of different levels in the multi-level feature fusion process, enhances the importance of key levels, and reduces the impact of contradictory information from different objects. The head network includes four target detection modules (the first to fourth target detection modules), each of which is responsible for detecting targets of different scales. This multi-scale detection mechanism enables the model to cover targets in the image more comprehensively, improving the comprehensiveness and accuracy of detection. The embodiment of the present application optimizes the feature extraction capability of the backbone network and the multi-scale feature fusion mechanism of the neck network, as well as the multi-scale detection strategy of the head network, so that the model can more accurately detect various types of targets in driving scenes, especially small targets and targets with complex shapes. The deformable convolution capability of the C2f_DCNv4 module and the fast spatial pyramid pooling technology of the SPPF module make the model more robust and adaptable to complex driving environments such as lighting changes, occlusions, and object shape changes. Although the model structure is relatively complex, through efficient feature extraction and fusion mechanisms, as well as optimized target detection strategies, the model can still maintain a relatively fast detection speed and meet real-time requirements. The model provides more accurate and reliable environmental perception capabilities for autonomous driving and assisted driving systems, helps to improve the safety and intelligence of autonomous driving vehicles, and promotes the further development and application of related technologies.

[0080] Step 102: input the image to be detected in the driving scene into the target detection model, perform image feature extraction on the image to be detected through the backbone network, obtain image features corresponding to each C2f_DCNv4 module, perform feature fusion on multiple image features through the neck network, obtain fusion features corresponding to each feature fusion layer, perform target recognition on the fusion features of each feature fusion layer through the head network, and obtain target detection results corresponding to each target detection module.

[0081] In an embodiment of the present application, the image to be detected in the driving scene is input into the trained YOLOv8 model for feature extraction, feature fusion and target recognition. Feature extraction: The C2f_DCNv4 module in the backbone network extracts features from the input image to generate multiple levels of image features. Feature fusion: The neck network fuses these feature maps at different levels to generate a richer feature representation for subsequent detection. Target recognition: Each target detection module in the head network performs target detection on fused features of different scales, and outputs the target detection results of each detection module, including information such as the location, category and confidence of the target.

[0082] In an embodiment of the present application, image features are extracted from the image to be detected through the backbone network to obtain image features corresponding to each C2f_DCNv4 module, including: inputting the image to be detected into the backbone network to gradually extract image features, obtaining the output of the first C2f_DCNv4 module as the first image feature, the output of the second C2f_DCNv4 module as the second image feature, the output of the third C2f_DCNv4 module as the third image feature, and the output of the SPPF module as the fourth image feature.

[0083] The neck network is used to fuse multiple image features to obtain fusion features corresponding to each feature fusion layer, including: inputting the first image feature, the second image feature, the third image feature and the fourth image feature into the sixth convolution layer, the seventh convolution layer, the eighth convolution layer and the ninth convolution layer respectively, processing the output results of the sixth convolution layer and the seventh convolution layer through the first ASFF module and then inputting them into the first C2F module, processing the output results of the sixth convolution layer and the seventh convolution layer through the second ASFF module and then inputting them into the second C2F module, inputting the output result of the eighth convolution layer into the third C2F module, and inputting the output result of the ninth convolution layer into the tenth C2F module; inputting the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the second ASFF module for processing, and inputting the output result of the second ASFF module into the second C2F module; the output results of the first C2F module, the fourth C2F module and the seventh C2F module Input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the seventh ASFF module for processing, and input the output result of the fifth ASFF module into the seventh C2F module; input the output results of the second C2F module, the fifth C2F module, the eighth C2F module and the tenth C2F module into the third ASFF module, the sixth ASFF module, the eighth ASFF module and the ninth ASFF module for processing; process the output result of the third ASFF module by the third C2F module to obtain the first fusion feature; process the output result of the sixth ASSF module by the sixth C2F module to obtain the second fusion feature; process the output result of the eighth ASFF module by the ninth C2F module to obtain the third fusion feature; process the output result of the ninth ASFF module by the eleventh C2F module to obtain the fourth fusion feature.

[0084] The head network is used to perform target recognition on the fusion features of each feature fusion layer respectively, and obtain the target detection result corresponding to each target detection module, including: respectively inputting the first fusion feature, the second fusion feature, the third fusion feature and the fourth fusion feature into the first target detection module, the second target detection module, the third target detection module and the fourth target detection module, and obtaining the target detection result output by each target detection module.

[0085] In the above embodiment, image feature extraction is first performed through the backbone network. Step-by-step feature extraction: the image to be detected is input into the backbone network, preliminarily processed through the first convolution layer, the second convolution layer, etc., and then enters the C2f_DCNv4 module for more in-depth feature extraction. The output of each C2f_DCNv4 module (i.e., the first, second, and third image features) represents the feature representation of the image at different levels, and the output of the SPPF module (the fourth image feature) further enhances the spatial pyramid characteristics of the feature. Multi-level feature extraction ensures that the model can capture rich information in the image, from low-level edges and textures to high-level shapes and semantic information, providing a solid foundation for subsequent target detection. Secondly, feature fusion is performed through the neck network. Image features at different levels (the first to fourth image features) are input into the corresponding convolution layers, and adaptive spatial feature fusion and conversion are performed through the combination of the ASFF module and the C2F module. This multi-scale fusion method enables the model to combine feature information of different scales and improve the detection capability of targets of different sizes. During the feature fusion process, the ASFF module is responsible for adaptively adjusting the weights of different feature maps to achieve a better feature fusion effect. The C2F module is used to convert the fused features into a form suitable for subsequent detection. This complex fusion mechanism ensures the accuracy and effectiveness of the fused features. Multi-scale feature fusion can significantly improve the model's detection accuracy for targets in complex driving environments, especially for small targets and occluded targets. At the same time, adaptive spatial feature fusion also enhances the robustness and adaptability of the model. Finally, the head network is used for target recognition. The fused features (the first to fourth fused features) are input into the corresponding target detection module, and each module is responsible for detecting targets of different scales. This multi-scale detection strategy enables the model to cover the targets in the image more comprehensively and improve the comprehensiveness and accuracy of detection. The multi-scale target detection mechanism ensures that the model can detect large and small targets at the same time, improving the accuracy and efficiency of detection. At the same time, the output results of each target detection module also provide rich information support for subsequent decision-making and actions. Through the above steps, the target detection model in the driving scene based on YOLOv8 realizes the accurate detection of targets in complex driving environments. The optimized design of its backbone network, neck network and head network enables the model to perform well in feature extraction, feature fusion and target recognition, thereby improving the accuracy, robustness and real-time performance of detection. These beneficial effects are of great significance to the development of autonomous driving and assisted driving technologies.

[0086] Step 103: determining the detected target in the image to be detected according to the target detection result.

[0087] In the embodiment of the present application, the output results of each target detection module are summarized, and then the repeated and redundant detection frames can be removed by post-processing algorithms such as non-maximum suppression (NMS) to ensure that each target is detected only once. According to the processed detection results, the detected target in the image to be detected is determined, including information such as its position, category, and confidence. Thus, the detected target in the image to be detected and the type of the detected target are finally determined.

[0088] By applying the technical solution of this embodiment, a target detection solution in a driving scene based on YOLOv8 is provided. By adopting a special model architecture, a backbone network including a C2f_DCNv4 module, a neck network for multi-scale feature fusion, and a head network for multi-scale target detection, and using a small target detection data set (SODA-D) for driving scenes for training, various targets, especially small targets, are accurately detected in a complex driving environment. The beneficial effects of the embodiment of the present application are: improving detection accuracy: using the C2f_DCNv4 module as the backbone network, by introducing deformable convolution (DCNv4), the adaptability of the model to changes in object shape is enhanced, and the detection accuracy of complex shape and posture targets is improved. The multi-scale feature fusion and detection mechanism enable the model to capture detailed information and global information in the image at the same time, so as to more accurately detect targets of different sizes, especially small targets. Enhanced robustness: Training with the SODA-D data set designed for driving scenes makes the model more adaptable to complex driving environments, including different lighting conditions, weather conditions, occlusion conditions, etc., and enhances the robustness of the model. By removing duplicate and redundant detection frames through post-processing algorithms (such as non-maximum suppression), the stability and reliability of the detection results are further improved. Improve real-time performance: YOLOv8 itself is an efficient target detection model. Combined with the optimized model architecture and training strategy, the solution maintains high accuracy while also maintaining a fast detection speed to meet real-time requirements. Further, provide effective technical support for the driving field: This solution provides more accurate and reliable environmental perception capabilities for autonomous driving and assisted driving systems, helping to improve the safety and intelligence of autonomous driving vehicles. Through precise target detection, autonomous driving systems can better understand and respond to changes in the surrounding environment and make more reasonable decisions and actions.

[0089] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another method for generating a target detection model in a driving scenario based on YOLOv8 is provided, and the method includes:

[0090] Step 201, construct an initial target detection model based on YOLOv8.

[0091] Step 202, obtaining a SODA-D dataset as a sample image set, wherein the sample image set includes a plurality of sample images with annotation information, the annotation information includes an annotation box and an annotation type of an object in the annotation box, and the annotation types include a person, a rider, a bicycle, a car, a traffic sign, a traffic light, a traffic camera, and a warning cone.

[0092] Step 203, divide the sample image set into a training set and a test set, use the training set to train the initial target detection model, and use the test set to test the trained initial target detection model, and use the initial target detection model that meets the test conditions as the target detection model.

[0093] In the above embodiment, a method for building and training a target detection model based on YOLOv8 is provided. First, an initial target detection model based on YOLOv8 is constructed, and the model structure can be specifically as described above. Secondly, the data set is downloaded from the official website of the SODA-D data set or the specified data source. Make sure that the data set contains all necessary files, including image files and corresponding annotation files (such as XML, JSON, etc.). Convert the annotation file to a format that YOLOv8 can understand (usually a txt file, with annotation information of one sample per line, including category, center point x, center point y, width, height). Finally, train and test the model. Specifically, the data set can be divided first: the data set is divided into a training set and a test set using random or specific strategies (such as stratified sampling). Then use the training set to train the model, monitor the loss value, mAP and other indicators during the training process to evaluate the model performance. Adjust the training parameters as needed, such as learning rate, optimizer, regularization, etc., to optimize the training effect. Finally, test the model: use the test set to test the trained model and evaluate its performance on unseen data. Calculate and record indicators such as mAP, precision, and recall on the test set. If the model performance does not meet the requirements, you can return to the training stage for more iterations or adjustments.

[0094] In an embodiment of the present application, optionally, the initial target detection model is trained using the training set, including: using the initial target detection model to perform target detection on the training sample images in the training set to determine the target detection results of the training sample images; determining the prediction score of each of the training sample images based on the target detection results of each of the training sample images and the annotation information, and determining a sample division threshold based on the prediction score of each of the training sample images, and dividing each of the training sample images into positive samples or negative samples based on the sample division threshold and the prediction score of each of the training sample images; assigning different weights to the positive samples and the negative samples, respectively, and training the initial target detection model using the training sample images based on a loss function, wherein, during the training process, the weights corresponding to the training sample images are assigned to the loss function for loss calculation, and the weights of the negative samples are greater than the weights of the positive samples.

[0095] In this embodiment, a dynamic sample partitioning and weighting mechanism is introduced in the training process to improve the training efficiency and performance of the target detection model. This method is particularly suitable for processing unbalanced data sets, in which the number of samples of certain categories is much larger than that of other categories, or some samples are more important for model learning. The initial target detection model is used to perform target detection on each training sample image in the training set to obtain preliminary target detection results. According to the target detection results and annotation information (true labels) of each training sample image, the prediction score is calculated. The prediction score can be determined based on the matching degree of the detection result and the annotation information (such as IoU, i.e., intersection over union). According to the prediction scores of all training sample images, a sample partitioning threshold is determined. This threshold can be determined based on a statistical method (such as mean, median, or a value selected according to a specific strategy), and the purpose is to divide the training sample images into two categories: positive samples (images with higher prediction scores, indicating that the model predicts these images more accurately) and negative samples (images with lower prediction scores, indicating that the model predicts these images inaccurately or there are more false detections). Different weights are assigned to positive samples and negative samples, and the weight of negative samples is greater than that of positive samples, so as to enhance the model's learning ability for difficult samples (i.e., negative samples), because negative samples often contain information that the model currently fails to grasp well. During the training process, the weight corresponding to each training sample image is assigned to the loss function for loss calculation. The loss function is the key to model optimization, which measures the difference between the model prediction and the true annotation. By minimizing the loss function, the model can gradually learn how to detect the target more accurately. When calculating the total loss, the loss of negative samples will be amplified due to the increase in weights, thereby guiding the model to pay more attention to these samples during the back propagation process. Repeat the above process, update the weight of the model in each iteration, and re-divide the samples and calculate the weights according to the new prediction results. As the training proceeds, the model's ability to predict the training samples will gradually improve, and the prediction scores will change accordingly, thereby affecting the division and weight distribution of the samples. By dynamically adjusting the sample weights, the embodiment of the present application can focus more on those samples that are more critical to performance improvement, thereby improving training efficiency. Reinforcement learning of negative samples helps the model better handle samples that are prone to misdetection or omission, enhancing the robustness of the model. To a certain extent, it alleviates the problem of unbalanced data sets, allowing the model to have better detection capabilities for targets of different categories.

[0096] In an embodiment of the present application, optionally, the prediction score of each of the training sample images is determined based on the target detection result and the annotation information of each of the training sample images, including: calculating the intersection area and the union area of ​​the prediction box and the annotation box of the training sample image, and taking the ratio of the intersection area to the union area as the prediction score of the training sample image.

[0097] In the above embodiment, when training the model, there is a common problem of sample imbalance. The sample imbalance problem refers to the fact that in the training data, the number of samples that are easy to detect (simple samples) is far greater than the number of samples that are difficult to detect (difficult samples). This will cause the model to pay too much attention to simple samples during training and ignore difficult samples, thereby affecting the final detection performance. Simple samples and difficult samples are distinguished according to the IoU size of the predicted box and the true box. In order to reduce hyperparameters, the threshold μ is obtained by calculating the average value of the intersection and union (IoU) value of all candidate positive samples and the true annotation box. Samples greater than the threshold μ are used as positive samples, and samples less than the threshold μ are used as negative samples. When calculating the loss, different weights are applied to positive samples and negative samples, respectively, so that the weight of difficult samples (usually negative samples) in the loss function is higher. In this way, the Slide Loss function can effectively solve the problem of sample imbalance, so that the model can not only learn the features of simple samples during training, but also pay more attention to and learn the features of difficult samples, thereby improving the overall performance of the model. For each training sample image, first find all bounding boxes (prediction boxes) predicted by the model and the corresponding true bounding boxes (annotation boxes). For each pair of predicted boxes and labeled boxes, calculate their intersection area. The intersection area is the area of ​​the overlapping parts of the two boxes. Similarly, for each pair of predicted boxes and labeled boxes, calculate their union area. The union area is the total area of ​​the two boxes after merging, including the overlapping and non-overlapping parts. Divide the intersection area by the union area to get the IoU value. The IoU value is between 0 and 1. The higher the value, the higher the overlap between the predicted box and the labeled box, that is, the more accurate the prediction. In some cases, based on the situation where multiple predicted boxes correspond to the same labeled box, the predicted box with the highest IoU value may be selected as the final prediction, and the IoU value may be used as the prediction score of the training sample image (or more accurately, the prediction corresponding to the labeled box). If there are multiple labeled boxes in the training sample image (that is, there are multiple objects in the image), the above process needs to be repeated for each labeled box, and the prediction score of the entire image needs to be comprehensively determined according to certain strategies (such as average IoU, highest IoU, etc.). In practical applications, the IoU threshold (such as 0.5) is often used to determine whether the predicted box correctly detects the object. That is, if the IoU value is greater than or equal to the set threshold, the predicted box is considered to be a correct detection (positive sample); otherwise, it is considered to be an incorrect detection (negative sample or negligible). During the training process, the loss function is usually adjusted according to the IoU value to optimize the model parameters so that the predicted box is closer to the labeled box.

[0098] By applying the technical solution of this embodiment, 1) the deformable convolution DCNv4 improves the receptive field of the detection point on the feature map through its characteristic of adapting to the shape of the convolution kernel, and extracts the feature information of targets of different scales and shapes more flexibly and accurately. 2) The progressive feature pyramid strategy enables better fusion of feature information between non-adjacent layers, and at the same time, through the newly added P2 small target detection head, the network is more sensitive to small target objects. 3) Using the slide-loss loss function, the average value of the IoU values ​​of all bounding boxes is used as the threshold μ, and negative samples are taken for values ​​less than μ, and positive samples are taken for values ​​greater than μ, so that the model pays more attention to difficult samples and improves the detection ability of small targets.

[0099] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a target detection device in a driving scene based on YOLOv8, such as Figure 4 As shown, the device comprises:

[0100] A model acquisition module, used to acquire a target detection model based on YOLOv8, wherein the target detection model includes a backbone network for extracting image features, a neck network for fusing image features, and a head network for detecting targets, the backbone network includes multiple C2f_DCNv4 modules, the neck network includes multiple feature fusion layers of different scales, and the head network includes multiple target detection modules of different scales; the target detection model is pre-trained based on the SODA-D (Small Object Detection dAtaset for Driving) data set;

[0101] A detection module is used to input the image to be detected in the driving scene into the target detection model, perform image feature extraction on the image to be detected through the backbone network, obtain image features corresponding to each C2f_DCNv4 module, perform feature fusion on multiple image features through the neck network, obtain fusion features corresponding to each feature fusion layer, perform target recognition on the fusion features of each feature fusion layer through the head network, and obtain target detection results corresponding to each target detection module;

[0102] The detection module is further used to determine the detected target in the image to be detected according to the target detection result.

[0103] In an embodiment of the present application, optionally, the backbone network sequentially includes a first convolutional layer, a second convolutional layer, a first C2f_DCNv4 module, a third convolutional layer, a second C2f_DCNv4 module, a fourth convolutional layer, a third C2f_DCNv4 module, a fifth convolutional layer, a fourth C2f_DCNv4 module, and a SPPF (Spatial Pyramid Pooling Fast) module;

[0104] The detection module is further used for:

[0105] The image to be detected is input into the backbone network to gradually extract image features, and the output of the first C2f_DCNv4 module is obtained as the first image feature, the output of the second C2f_DCNv4 module is obtained as the second image feature, the output of the third C2f_DCNv4 module is obtained as the third image feature, and the output of the SPPF module is obtained as the fourth image feature.

[0106] In the embodiment of the present application, optionally, the neck network includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer and a fourth feature fusion layer;

[0107] The first feature fusion layer includes a sixth convolutional layer, a first ASFF (Adaptive Spatial Feature Fusion) module, a first C2F (Convolution to Feature) module, a second ASFF module, a second C2F module, a third ASFF module, and a third C2F module;

[0108] The second feature fusion layer includes a seventh convolutional layer, a fourth ASFF module, a fourth C2F module, a fifth ASFF module, a fifth C2F module, a sixth ASFF module, and a sixth C2F module;

[0109] The third feature fusion layer includes an eighth convolutional layer, a seventh C2F module, a seventh ASFF module, an eighth C2F module, an eighth ASFF module, and a ninth C2F module;

[0110] The fourth feature fusion layer includes a ninth convolutional layer, a tenth C2F module, a ninth ASFF module, and an eleventh C2F module;

[0111] The detection module is further used for:

[0112] The first image feature, the second image feature, the third image feature and the fourth image feature are input into the sixth convolution layer, the seventh convolution layer, the eighth convolution layer and the ninth convolution layer respectively; the output results of the sixth convolution layer and the seventh convolution layer are processed by the first ASFF module and then input into the first C2F module; the output results of the sixth convolution layer and the seventh convolution layer are processed by the second ASFF module and then input into the second C2F module; the output result of the eighth convolution layer is input into the third C2F module; the output result of the ninth convolution layer is input into the tenth C2F module;

[0113] Input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the second ASFF module for processing, and input the output result of the second ASFF module into the second C2F module; input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the fifth ASFF module for processing, and input the output result of the fifth ASFF module into the fifth C2F module; input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the seventh ASFF module for processing, and input the output result of the fifth ASFF module into the seventh C2F module;

[0114] Input the output results of the second C2F module, the fifth C2F module, the eighth C2F module and the tenth C2F module into the third ASFF module, the sixth ASFF module, the eighth ASFF module and the ninth ASFF module for processing;

[0115] The output result of the third ASFF module is processed by the third C2F module to obtain the first fusion feature; the output result of the sixth ASSF module is processed by the sixth C2F module to obtain the second fusion feature; the output result of the eighth ASFF module is processed by the ninth C2F module to obtain the third fusion feature; the output result of the ninth ASFF module is processed by the eleventh C2F module to obtain the fourth fusion feature.

[0116] In an embodiment of the present application, optionally, the head network includes a first target detection module, a second target detection module, a third target detection module and a fourth target detection module;

[0117] The detection module is further used for:

[0118] The first fusion feature, the second fusion feature, the third fusion feature and the fourth fusion feature are respectively input into the first target detection module, the second target detection module, the third target detection module and the fourth target detection module to obtain the target detection result output by each target detection module.

[0119] In the embodiment of the present application, optionally, the model acquisition module is further used to:

[0120] Build an initial target detection model based on YOLOv8;

[0121] A SODA-D dataset is obtained as a sample image set, wherein the sample image set includes a plurality of sample images with annotation information, the annotation information includes an annotation box and an annotation type of an object in the annotation box, and the annotation type includes a person, a rider, a bicycle, a car, a traffic sign, a traffic light, a traffic camera, and a warning cone;

[0122] The sample image set is divided into a training set and a test set, the initial target detection model is trained using the training set, and the trained initial target detection model is tested using the test set, and the initial target detection model that meets the test conditions is used as the target detection model.

[0123] In the embodiment of the present application, optionally, the model acquisition module is further used to:

[0124] Using the initial target detection model to perform target detection on the training sample images in the training set, and determining the target detection results of the training sample images;

[0125] Determine a prediction score for each of the training sample images according to the target detection result and the annotation information of each of the training sample images, determine a sample division threshold according to the prediction score of each of the training sample images, and divide each of the training sample images into a positive sample or a negative sample according to the sample division threshold and the prediction score of each of the training sample images;

[0126] Different weights are respectively assigned to the positive samples and the negative samples, and the initial target detection model is trained using the training sample images based on a loss function, wherein during the training process, the weights corresponding to the training sample images are assigned to the loss function for loss calculation, and the weight of the negative samples is greater than the weight of the positive samples.

[0127] In the embodiment of the present application, optionally, the model acquisition module is further used to:

[0128] The intersection area and the union area of ​​the prediction box and the annotation box of the training sample image are calculated, and the ratio of the intersection area to the union area is used as the prediction score of the training sample image.

[0129] It should be noted that for other corresponding descriptions of the functional units involved in the object detection device in a driving scenario based on YOLOv8 provided in the embodiment of the present application, please refer to Figure 1 to Figure 2 The corresponding description in the method will not be repeated here.

[0130] The embodiment of the present application also provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes a bus, a processor, a memory and a communication interface, and can also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0131] Those skilled in the art will appreciate that the structure of the above-mentioned computer device is only a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components, or combine certain components, or have a different arrangement of components.

[0132] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0133] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0134] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0135] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0136] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A target detection method in a driving scenario based on YOLOv8, characterized in that: The method comprises: Obtain a target detection model based on YOLOv8, wherein the target detection model includes a backbone network for extracting image features, a neck network for fusing image features, and a head network for detecting targets, the backbone network includes multiple C2f_DCNv4 modules, the neck network includes multiple feature fusion layers of different scales, and the head network includes multiple target detection modules of different scales; the target detection model is pre-trained based on the SODA-D (Small Object Detection dAtaset for Driving) data set; The image to be detected in the driving scene is input into the target detection model, and the image features of the image to be detected are extracted through the backbone network to obtain the image features corresponding to each C2f_DCNv4 module, and the multiple image features are fused through the neck network to obtain the fusion features corresponding to each feature fusion layer, and the fusion features of each feature fusion layer are respectively identified through the head network to obtain the target detection results corresponding to each target detection module; According to the target detection result, a detected target in the image to be detected is determined.

2. The method according to claim 1, characterized in that The backbone network includes a first convolutional layer, a second convolutional layer, a first C2f_DCNv4 module, a third convolutional layer, a second C2f_DCNv4 module, a fourth convolutional layer, a third C2f_DCNv4 module, a fifth convolutional layer, a fourth C2f_DCNv4 module and a SPPF (Spatial Pyramid Pooling Fast) module in sequence; The image features of the image to be detected are extracted through the backbone network to obtain image features corresponding to each C2f_DCNv4 module, including: The image to be detected is input into the backbone network to gradually extract image features, and the output of the first C2f_DCNv4 module is obtained as the first image feature, the output of the second C2f_DCNv4 module is obtained as the second image feature, the output of the third C2f_DCNv4 module is obtained as the third image feature, and the output of the SPPF module is obtained as the fourth image feature.

3. The method according to claim 2, characterized in that The neck network includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer and a fourth feature fusion layer; The first feature fusion layer includes a sixth convolutional layer, a first ASFF (Adaptive Spatial Feature Fusion) module, a first C2F (Convolution to Feature) module, a second ASFF module, a second C2F module, a third ASFF module, and a third C2F module; The second feature fusion layer includes a seventh convolutional layer, a fourth ASFF module, a fourth C2F module, a fifth ASFF module, a fifth C2F module, a sixth ASFF module, and a sixth C2F module; The third feature fusion layer includes an eighth convolutional layer, a seventh C2F module, a seventh ASFF module, an eighth C2F module, an eighth ASFF module, and a ninth C2F module; The fourth feature fusion layer includes a ninth convolutional layer, a tenth C2F module, a ninth ASFF module, and an eleventh C2F module; The neck network is used to fuse multiple image features to obtain fusion features corresponding to each feature fusion layer, including: The first image feature, the second image feature, the third image feature and the fourth image feature are input into the sixth convolution layer, the seventh convolution layer, the eighth convolution layer and the ninth convolution layer respectively; the output results of the sixth convolution layer and the seventh convolution layer are processed by the first ASFF module and then input into the first C2F module; the output results of the sixth convolution layer and the seventh convolution layer are processed by the second ASFF module and then input into the second C2F module; the output result of the eighth convolution layer is input into the third C2F module; the output result of the ninth convolution layer is input into the tenth C2F module; Input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the second ASFF module for processing, and input the output result of the second ASFF module into the second C2F module; input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the fifth ASFF module for processing, and input the output result of the fifth ASFF module into the fifth C2F module; input the output results of the first C2F module, the fourth C2F module and the seventh C2F module into the seventh ASFF module for processing, and input the output result of the fifth ASFF module into the seventh C2F module; Input the output results of the second C2F module, the fifth C2F module, the eighth C2F module and the tenth C2F module into the third ASFF module, the sixth ASFF module, the eighth ASFF module and the ninth ASFF module for processing; The output result of the third ASFF module is processed by the third C2F module to obtain the first fusion feature; the output result of the sixth ASSF module is processed by the sixth C2F module to obtain the second fusion feature; the output result of the eighth ASFF module is processed by the ninth C2F module to obtain the third fusion feature; the output result of the ninth ASFF module is processed by the eleventh C2F module to obtain the fourth fusion feature.

4. The method according to claim 3, characterized in that The head network includes a first target detection module, a second target detection module, a third target detection module and a fourth target detection module; The head network performs target recognition on the fused features of each feature fusion layer respectively, and obtains the target detection result corresponding to each target detection module, including: The first fusion feature, the second fusion feature, the third fusion feature and the fourth fusion feature are respectively input into the first target detection module, the second target detection module, the third target detection module and the fourth target detection module to obtain the target detection result output by each target detection module.

5. The method according to any one of claims 1 to 4, characterized in that Before obtaining the YOLOv8-based target detection model, the method further includes: Build an initial target detection model based on YOLOv8; A SODA-D dataset is obtained as a sample image set, wherein the sample image set includes a plurality of sample images with annotation information, the annotation information includes an annotation box and an annotation type of an object in the annotation box, and the annotation type includes a person, a rider, a bicycle, a car, a traffic sign, a traffic light, a traffic camera, and a warning cone; The sample image set is divided into a training set and a test set, the initial target detection model is trained using the training set, and the trained initial target detection model is tested using the test set, and the initial target detection model that meets the test conditions is used as the target detection model.

6. The method according to claim 5, characterized in that Using the training set to train the initial target detection model includes: Using the initial target detection model to perform target detection on the training sample images in the training set, and determining the target detection results of the training sample images; Determine a prediction score for each of the training sample images according to the target detection result and the annotation information of each of the training sample images, determine a sample division threshold according to the prediction score of each of the training sample images, and divide each of the training sample images into a positive sample or a negative sample according to the sample division threshold and the prediction score of each of the training sample images; Different weights are respectively assigned to the positive samples and the negative samples, and the initial target detection model is trained using the training sample images based on a loss function, wherein during the training process, the weights corresponding to the training sample images are assigned to the loss function for loss calculation, and the weight of the negative samples is greater than the weight of the positive samples.

7. The method according to claim 6, characterized in that Determining a prediction score for each of the training sample images according to the target detection result and the annotation information of each of the training sample images includes: The intersection area and the union area of ​​the prediction box and the annotation box of the training sample image are calculated, and the ratio of the intersection area to the union area is used as the prediction score of the training sample image.

8. A target detection device in a driving scene based on YOLOv8, characterized in that: The device comprises: A model acquisition module is used to acquire a target detection model based on YOLOv8, wherein the target detection model includes a backbone network for extracting image features, a neck network for fusing image features, and a head network for detecting targets, the backbone network includes multiple C2f_DCNv4 modules, the neck network includes multiple feature fusion layers of different scales, and the head network includes multiple target detection modules of different scales; the target detection model is pre-trained based on the SODA-D (Small Object Detection dAtaset for Driving) data set; A detection module is used to input the image to be detected in the driving scene into the target detection model, perform image feature extraction on the image to be detected through the backbone network, obtain image features corresponding to each C2f_DCNv4 module, perform feature fusion on multiple image features through the neck network, obtain fusion features corresponding to each feature fusion layer, perform target recognition on the fusion features of each feature fusion layer through the head network, and obtain target detection results corresponding to each target detection module; The detection module is further used to determine the detected target in the image to be detected according to the target detection result.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images