Target detection method and device, vehicle and storage medium

Abstract

The invention relates to a target detection method and device, a vehicle and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a to-be-detected target image; the target image comprises a plurality of target objects to be detected; inputting the target image into a target detection model to obtain a detection result of the target image; the target detection model comprises a feature fusion network of a channel fusion space attention mechanism; the detection result comprises a plurality of bounding boxes and a plurality of labels; the bounding box is used for marking the positions of the multiple target objects in the target image; the labels are used for representing categories of the multiple target objects of the bounding box. Therefore, the target detection model can more accurately capture the features of the small target object and the shielded target object in the target image based on a channel fusion space attention mechanism, and more effectively fuses the features of each target object in the target image at different levels, thereby improving the precision of target detection.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to the field of target detection technology in urban road scenes, and specifically to a target detection method, device, vehicle and storage medium. Background Art

[0002] With the continuous growth of the number of motor vehicles, road traffic accidents continue to occur, and road traffic conditions are becoming increasingly serious. Therefore, it is very important to build an advanced driving assistance system (ADAS) on the vehicle to ensure the safety of the driver. Particularly important in ADAS is the environmental perception module. The detection and analysis of the surrounding environment images obtained by the sensor through the road scene target detection algorithm is a prerequisite for ADAS to make decisions. At the same time, due to the complex environment in the road scene and the inconsistent scale of the target, the target detection algorithm needs to obtain the category and size information of the target in the road scene in real time and accurately, so that the vehicle can identify pedestrians and vehicles in the surrounding environment and avoid traffic accidents.

[0003] In the related art, the attention mechanism is introduced into the convolutional neural network, and the pictures to be detected are collected, the pictures are annotated to generate the target training set, and the target training set is input into the convolutional neural network for training, so that the pictures to be detected are input into the trained convolutional neural network for detection, and the detection results are obtained. This method introduces the attention mechanism into the convolutional neural network, adjusts the weights of the channels in the feature map extracted by the convolutional neural network, so as to improve the target detection accuracy, but the attention mechanism of this method only involves the attention to the channel, and the detection effect of occluded targets and smaller targets in the image to be detected is poor. Therefore, how to improve the detection accuracy of occluded targets and smaller targets in the image to be detected is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a target detection method, device, vehicle and storage medium to at least solve the technical problem in the related art that the detection effect of occluded targets and smaller targets in the image to be detected is poor. The technical solution of the present application is as follows:

[0005] According to a first aspect of the present application, a target detection method is provided, the method comprising: obtaining a target image to be detected; the target image comprises a plurality of target objects to be detected; inputting the target image into a target detection model to obtain a detection result of the target image; the target detection model comprises a feature fusion network of a channel fusion spatial attention mechanism; the detection result comprises a plurality of bounding boxes and a plurality of labels; the bounding box is used to mark the positions of the plurality of target objects in the target image; the label is used to characterize the categories of the plurality of target objects in the bounding box.

[0006] According to the above technical means, the target detection model can integrate multi-scale features from different levels in the target image through the feature fusion network of the channel fusion spatial attention mechanism, and strengthen the complementarity between different features, thereby improving the expression ability of the features after the target image is fused, enhancing the feature information of target objects in dense areas and long-distance small-scale target objects in complex road scenes, and improving the detection accuracy of the target detection model for target objects in dense areas and long-distance small-scale target objects.

[0007] In a possible implementation, the method further includes: obtaining a training data set; the training data set includes multiple training images and multiple categories of training images; based on the categories of the multiple training images, the training data set is re-divided to obtain multiple training data sets; based on the multiple training data sets and the loss of the target detection model, the target detection model is trained.

[0008] According to the above technical means, the target detection model can be trained based on multiple classified training data sets and the loss of the target detection model, so that the target detection model can learn the features of training images of different categories, so that the target detection model can more accurately identify multiple target objects in the target image, and improve the generalization ability of the target detection model for new data.

[0009] In another possible implementation, a target detection model is constructed based on a YOLO target detection model; the YOLO target detection model includes: a feature extraction network, a feature fusion network of a channel fusion spatial attention mechanism, and a detection head; the target image is input into the target detection model to obtain a detection result of the target image, including: based on the feature extraction network, extracting feature maps of the target image at different scales; based on the feature fusion network of the channel fusion spatial attention mechanism, fusing feature maps of the target image at different scales to obtain a target feature map; based on the detection head, expanding the number of channels of the target feature map, determining the bounding box positions and categories of multiple target objects in the target image; based on the bounding box positions and categories of multiple target objects in the target image, determining the detection result of the target image.

[0010] According to the above technical means, through the feature extraction network, the target detection model can capture the features of the target image at different scales, so as to detect target objects of different sizes in the target image. The feature fusion network of the channel fusion spatial attention mechanism enables the target detection model to focus on the most important areas in the target image, improve the detection performance of small-sized target objects or target objects in dense areas, and enhance the recognition ability of the target detection model for target objects in complex scenes, thereby improving the detection accuracy of the target detection model.

[0011] In another possible implementation, the feature extraction network includes a first cross-stage local structure and a second cross-stage local structure; based on the feature extraction network, feature maps of the target image at different scales are extracted, including: based on the first cross-stage local structure, extracting high-level semantic features of the target image; based on the second cross-stage local structure, extracting low-level visual features of the target image; and splicing the high-level semantic features and the low-level visual features to obtain feature maps of the target image at different scales.

[0012] According to the above technical means, the high-level semantic features of the target image include category-related information of multiple target objects in the target image, and the low-level visual features of the target image include detail information such as edges and textures of multiple target objects in the target image. By combining the high-level semantic features and low-level visual features of the target image, the target detection model can capture the multi-scale features of the target image from details to the global, thereby improving the detection accuracy of target objects of different sizes in the target image.

[0013] In another possible implementation, based on a feature fusion network of a channel fusion spatial attention mechanism, feature maps of a target image at different scales are fused to obtain a target feature map, including: fusing feature maps of a target image at different scales to obtain an aggregated feature map; based on a feature fusion network of a channel fusion spatial attention mechanism, determining the channel attention weight and the spatial attention weight of the aggregated feature map; and obtaining the target feature map based on the channel attention weight and the spatial attention weight.

[0014] According to the above technical means, the feature fusion network of the channel fusion spatial attention mechanism can fuse the feature maps of different scales of the target image in a weighted manner through the attention mechanism, more effectively integrate the feature information from different scales, and focus on the key areas in the target image, ignoring unimportant channels and spatial areas in the target image, avoiding the interference of redundant information, and improving the detection accuracy of the target object.

[0015] In another possible implementation, determining the channel attention weight of the aggregate feature map includes: determining the average pooling element and the maximum pooling element of the aggregate feature map; determining the channel attention map through a shared network based on the average pooling element and the maximum pooling element; the shared network includes a one-dimensional convolution with an adaptive convolution kernel size; and generating the channel attention weight based on the channel attention map using an activation function.

[0016] According to the above technical means, average pooling and maximum pooling can capture the global average information and the most significant feature points in the feature map respectively, which is helpful to integrate the multi-scale information of the target image, and through the pooling operation, the spatial dimension of the feature can be reduced, important feature information can be retained, and the complexity of the calculation can be reduced. In addition, the adaptive convolution kernel size of the shared network allows the feature fusion network to dynamically adjust the convolution kernel according to the content of the input feature map, improving the adaptability of the target detection model to different features. In the case of occluded or overlapping target objects in the target image, the accuracy of the target detection model in locating and classifying the target object can be improved through the channel attention weight.

[0017] In another possible implementation, determining the spatial attention weight of the aggregate feature map includes: multiplying the channel attention weight by the aggregate feature map to obtain a weighted aggregate feature map; performing global adaptive mean pooling on the weighted aggregate feature map according to the height and width of the weighted aggregate feature map to obtain a width feature map and a height feature map; concatenating and reducing the dimension of the width feature map and the height feature map to obtain a spatial feature map; based on the spatial feature map, using an activation function to generate a spatial attention weight; the spatial attention weight includes a height attention weight and a width attention weight.

[0018] According to the above technical means, through the height attention weight and width attention weight, the target detection model can integrate the feature information of the target image in the horizontal and vertical directions, focus on the key spatial area in the target image, and help the target detection model to more accurately determine the bounding box of the target object and improve the spatial positioning accuracy of the target object, especially the small-sized target object or the partially occluded target object. In addition, through global pooling and dimensionality reduction processing, the target detection model can compress high-dimensional spatial features into low dimensions, reduce the amount of calculation and highlight important features.

[0019] According to the second aspect provided by the present application, a target detection device is provided, which includes: an acquisition module and a processing module. The acquisition module is used to acquire a target image to be detected; the target image includes multiple target objects to be detected; the processing module is used to input the target image into a target detection model to obtain a detection result of the target image; the target detection model includes a feature fusion network of a channel fusion spatial attention mechanism; the detection result includes multiple bounding boxes and multiple labels; the bounding box is used to mark the positions of multiple target objects in the target image; the label is used to characterize the categories of multiple target objects in the bounding box.

[0020] In a possible implementation, the device further includes: a division module and a training module. The acquisition module is further used to acquire a training data set; the training data set includes multiple training images and multiple categories of training images; the division module is used to re-divide the training data set based on the categories of the multiple training images to obtain multiple training data sets; the training module is used to train the target detection model based on the multiple training data sets and the loss of the target detection model.

[0021] In another possible implementation, the target detection model is constructed based on the YOLO target detection model; the YOLO target detection model includes: a feature extraction network, a feature fusion network of a channel fusion spatial attention mechanism, and a detection head; a processing module, specifically used to extract feature maps of the target image at different scales based on the feature extraction network; based on the feature fusion network of the channel fusion spatial attention mechanism, the feature maps of the target image at different scales are fused to obtain a target feature map; based on the detection head, the number of channels of the target feature map is expanded, and the bounding box positions and categories of multiple target objects in the target image are determined; based on the bounding box positions and categories of multiple target objects in the target image, the detection result of the target image is determined.

[0022] In another possible implementation, the feature extraction network includes a first cross-stage local structure and a second cross-stage local structure; a processing module, specifically used to extract high-level semantic features of the target image based on the first cross-stage local structure; extract low-level visual features of the target image based on the second cross-stage local structure; and splice the high-level semantic features and the low-level visual features to obtain feature maps of the target image at different scales.

[0023] In another possible implementation, the processing module is specifically used to fuse the feature maps of the target image at different scales to obtain an aggregated feature map; determine the channel attention weights and spatial attention weights of the aggregated feature map based on a feature fusion network of a channel fusion spatial attention mechanism; and obtain the target feature map based on the channel attention weights and spatial attention weights.

[0024] In another possible implementation, the processing module is specifically used to determine the average pooling element and the maximum pooling element of the aggregated feature map; based on the average pooling element and the maximum pooling element, a channel attention map is determined through a shared network; the shared network includes a one-dimensional convolution with an adaptive convolution kernel size; based on the channel attention map, an activation function is used to generate a channel attention weight.

[0025] In another possible implementation, the processing module is specifically used to multiply the channel attention weight by the aggregate feature map to obtain a weighted aggregate feature map; perform global adaptive mean pooling on the weighted aggregate feature map according to the height and width of the weighted aggregate feature map to obtain a width feature map and a height feature map; concatenate and reduce the dimension of the width feature map and the height feature map to obtain a spatial feature map; based on the spatial feature map, use an activation function to generate a spatial attention weight; the spatial attention weight includes a height attention weight and a width attention weight.

[0026] According to the third aspect provided by the present application, a vehicle is provided, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to execute instructions to implement the method of the above-mentioned first aspect and any possible implementation manner thereof.

[0027] According to the fourth aspect provided by the present application, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by the processor of the vehicle, the vehicle is enabled to execute the method in the above-mentioned first aspect and any possible implementation method thereof.

[0028] According to the fifth aspect provided by the present application, a computer program product is provided, the computer program product comprising computer instructions, and when the computer instructions are executed on a vehicle, the vehicle executes the method of the above-mentioned first aspect and any possible implementation manner thereof.

[0029] Therefore, the above technical features of the present application have the following beneficial effects:

[0030] (1) The target detection model can integrate multi-scale features from different levels in the target image through the feature fusion network of the channel fusion spatial attention mechanism, and strengthen the complementarity between different features, thereby improving the expression ability of the features after the target image is fused, enhancing the feature information of target objects in dense areas and small-scale target objects at a long distance in complex road scenes, and improving the detection accuracy of the target detection model for target objects in dense areas and small-scale target objects at a long distance.

[0031] (2) The target detection model can be trained based on multiple classified training data sets and the loss of the target detection model, so that the target detection model can learn the features of training images of different categories, so that the target detection model can more accurately identify multiple target objects in the target image, and improve the generalization ability of the target detection model for new data.

[0032] (3) Through the feature extraction network, the target detection model can capture the features of the target image at different scales in order to detect target objects of different sizes in the target image. The feature fusion network of the channel fusion spatial attention mechanism enables the target detection model to focus on the most important areas in the target image, improve the detection performance of small-sized target objects or target objects in dense areas, and enhance the target detection model's ability to identify target objects in complex scenes, thereby improving the detection accuracy of the target detection model.

[0033] (4) The high-level semantic features of the target image contain the category-related information of multiple target objects in the target image, and the low-level visual features of the target image contain the detail information such as the edges and textures of multiple target objects in the target image. By combining the high-level semantic features and low-level visual features of the target image, the target detection model can capture the multi-scale features of the target image from details to the global, thereby improving the detection accuracy of target objects of different sizes in the target image.

[0034] (5) The feature fusion network of the channel fusion spatial attention mechanism can fuse the feature maps of different scales of the target image in a weighted manner through the attention mechanism, more effectively integrate the feature information from different scales, and focus on the key areas in the target image, ignoring unimportant channels and spatial areas in the target image, avoiding the interference of redundant information, and improving the detection accuracy of the target object.

[0035] (6) Average pooling and maximum pooling can capture the global average information and the most significant feature points in the feature map, respectively, which helps to integrate the multi-scale information of the target image. In addition, the pooling operation can reduce the spatial dimension of the feature, retain important feature information, and reduce the complexity of the calculation. In addition, the adaptive convolution kernel size of the shared network allows the feature fusion network to dynamically adjust the convolution kernel according to the content of the input feature map, thereby improving the adaptability of the target detection model to different features. In the case of occluded or overlapping target objects in the target image, the accuracy of the target detection model in locating and classifying the target object can be improved through the channel attention weight.

[0036] (7) Through the height attention weight and width attention weight, the target detection model can integrate the feature information of the target image in the horizontal and vertical directions and focus on the key spatial areas in the target image, which helps the target detection model to more accurately determine the bounding box of the target object and improve the spatial positioning accuracy of the target object, especially small-sized target objects or partially occluded target objects. In addition, through global pooling and dimensionality reduction processing, the target detection model can compress high-dimensional spatial features into low dimensions, reduce the amount of calculation and highlight important features.

[0037] It should be noted that the technical effects brought about by any implementation method in the second to fifth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here.

[0038] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application, and do not constitute an improper limitation on the present application.

[0040] Figure 1 is a flow chart of a target detection method according to an exemplary embodiment;

[0041] Figure 2 is a flow chart of another target detection method according to an exemplary embodiment;

[0042] Figure 3 is a flow chart of another target detection method according to an exemplary embodiment;

[0043] Figure 4 is a flow chart of another target detection method according to an exemplary embodiment;

[0044] Figure 5 is a flow chart of another target detection method according to an exemplary embodiment;

[0045] Figure 6 is a flow chart of another target detection method according to an exemplary embodiment;

[0046] Figure 7 is a flow chart of another target detection method according to an exemplary embodiment;

[0047] Figure 8 is a flow chart of another target detection method according to an exemplary embodiment;

[0048] Fig. 9 is a framework diagram of a target detection model according to an exemplary embodiment;

[0049] Fig.10 is a structural diagram of a channel fusion spatial attention mechanism according to an exemplary embodiment;

[0050] Fig.11 is a display diagram of a detection result according to an exemplary embodiment;

[0051] Fig.12 is a display diagram of another detection result according to an exemplary embodiment;

[0052] Fig.13 is a block diagram of a target detection device according to an exemplary embodiment;

[0053] Fig.14 is a block diagram of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0054] In order to enable ordinary persons in the art to better understand the technical solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0055] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0056] With the continuous growth of the number of motor vehicles, road traffic accidents continue to occur, and road traffic conditions are becoming increasingly serious. Therefore, it is very important to build ADAS on vehicles to ensure the safety of drivers. Particularly important in ADAS is the environmental perception module. The detection and analysis of the surrounding environment images obtained by the sensor through the road scene target detection algorithm is a prerequisite for ADAS to make decisions. At the same time, due to the complex environment in the road scene and the inconsistent scale of the target, the target detection algorithm needs to obtain the category and size information of the target in the road scene in real time and accurately, so that the vehicle can identify pedestrians and vehicles in the surrounding environment and avoid traffic accidents.

[0057] In the related art, the attention mechanism is introduced into the convolutional neural network, the weight of the useful channel in the feature map extracted by the feature extraction network is increased, and the weight of the useless channel is reduced, so as to improve the detection effect of the convolutional neural network in complex scenes. This method first constructs a convolutional neural network, introduces the attention mechanism into the convolutional neural network, collects the pictures to be detected, annotates the pictures to generate the target training set, inputs the target training set into the convolutional neural network for training, and then inputs the pictures to be detected into the trained convolutional neural network for detection to obtain the detection results. The attention mechanism of this method only involves the attention to the channel, and the detection effect of the occluded targets and smaller targets in the image to be detected is poor.

[0058] In another related technology, a channel and space attention mechanism module is introduced into the target detection model, wherein the spatial attention mechanism forms two spatial weight vectors by performing global pooling and average pooling on the input feature map, that is, all channels of the same feature point are pooled. The two spatial weight vectors are spliced ​​at the channel layer to obtain the feature map spatial weight. This method can assign weights to each point on the feature map through the feature map spatial weight, detect the image to be detected, and obtain the detection result. However, for some small-scale target objects in the image to be detected, the small-scale target pair itself can provide less feature information. The spatial attention mechanism implementation method of this method does not significantly enhance the weight information of some dense areas and small target objects, and the detection effect of occluded targets and smaller targets in the image to be detected is poor, resulting in inaccurate detection results. Therefore, how to improve the detection accuracy of occluded targets and smaller targets in the image to be detected is a technical problem that needs to be solved urgently.

[0059] In response to the above problems, the present application proposes a target detection method, in which the target detection model can integrate multi-scale features from different levels in the target image through a feature fusion network of a channel fusion spatial attention mechanism, and strengthen the complementarity between different features, thereby improving the expressiveness of the features after the target image is fused, enhancing the feature information of target objects in dense areas and long-distance small-scale target objects in complex road scenes, and improving the detection accuracy of the target detection model for target objects in dense areas and long-distance small-scale target objects.

[0060] For ease of understanding, the group obstacle detection method provided in this application is specifically introduced below with reference to the accompanying drawings.

[0061] Figure 1 is a flow chart of a target detection method according to an exemplary embodiment. Figure 1 As shown, the target detection method includes the following steps:

[0062] S101, obtaining a target image to be detected.

[0063] The target image includes a plurality of target objects to be detected.

[0064] As a possible implementation manner, the detection device may acquire a target image to be tested, and thereby detect multiple target objects in the target image to be detected based on a target detection model.

[0065] For example, during the operation of the vehicle, the vehicle's ADAS system can acquire images of the surrounding environment taken by the on-board camera, and send the acquired images as target images to be detected to a detection device for detection, so that the detection device can detect the positions and labels of multiple target objects in the target image. Thus, the vehicle's ADAS system can assist vehicle driving based on the detection results of the detection device to avoid traffic accidents.

[0066] It should be understood that obtaining the target image to be detected from the vehicle's on-board camera is only one way of obtaining the target image to be detected in the present application. The detection device can also obtain the target image to be detected input by the user or obtain the target image to be detected from the camera of other road-side equipment. The embodiments of the present application are not limited to this.

[0067] S102: Input the target image into the target detection model to obtain the detection result of the target image.

[0068] Among them, the target detection model includes a feature fusion network of a channel fusion spatial attention mechanism, and the detection results include multiple bounding boxes and multiple labels. The bounding boxes are used to mark the positions of multiple target objects in the target image, and the labels are used to characterize the categories of multiple target objects in the bounding boxes.

[0069] As a possible implementation manner, after the detection device obtains the target image to be detected, the target image can be input into the target detection model to obtain the detection result of the target image.

[0070] Exemplarily, the detection device can extract feature maps of the target image at different scales based on the feature extraction network of the target detection model, and fuse the feature maps of the target image at different scales based on the feature fusion network of the channel fusion spatial attention mechanism of the target detection model to obtain a target feature map, thereby expanding the number of channels of the target feature map based on the detection head of the target detection model, determining the bounding box positions and categories of multiple target objects in the target image, and obtaining the detection result of the target image.

[0071] In some embodiments, before the detection device determines the detection result of the target image based on the target detection model, it can also obtain a training data set of the target detection model, and train the target detection model based on the training data set to improve the detection accuracy of the target detection model. Figure 2 As shown, the target detection method provided in the embodiment of the present application also includes the following steps:

[0072] S201. Obtain a training data set.

[0073] The training data set includes multiple training images and multiple categories of training images.

[0074] As a possible implementation method, the detection device can obtain a training data set of the target detection model, and train the target detection model with the edited training data set.

[0075] It should be understood that the detection device can obtain a public vehicle target detection data set as a training data set for the target detection model, such as an open source data set for target detection tasks, a target detection data set of an online platform, etc. Alternatively, the detection device can also obtain a target detection data set customized by relevant technicians or managers as a training data set, which is not limited in the embodiments of the present application.

[0076] S202: Based on the categories of multiple training images, the training data set is re-divided to obtain multiple training data sets.

[0077] As a possible implementation method, after the detection device obtains the training data set, it can also re-divide the training data set based on the categories of the training images in the training data set to obtain multiple training data sets of different categories, so as to train the target detection model based on multiple training data sets of different categories.

[0078] Exemplarily, the detection device may first pre-process the training data set, remove the data of the miscellaneous category (Misc) and the irrelevant category (DontCare) in the training data set, so as to avoid affecting the training of the target detection model. Then, the detection device may reclassify the categories of the training images in the training data set, merge the sitting pedestrian category (Person_sitting) in the training data set into the pedestrian category (Pedestrain), and finally obtain six categories of the training data set: car category (Car), van category (Van), truck (Truck), tram (Tram), cyclist (Cyclist), and pedestrian (Pedestrian).

[0079] It should be understood that the training data set of the embodiment of the present application is a foreign open source data set, and the categories contained in it are not all applicable to domestic scenarios. Therefore, the data of the miscellaneous category (Misc) and the irrelevant category (DontCare) in the training data set are eliminated. Secondly, according to the category affiliation, the sitting pedestrian category (Person_sitting) originally belongs to the pedestrian category (Pedestrain), and the number of pictures of the sitting pedestrian category is relatively small. If it is trained as a separate category for the target detection model, the target detection model will learn less information about this category, and the prediction effect will be poor, affecting the overall performance of the target detection model.

[0080] S203: Training the target detection model based on multiple training data sets and the loss of the target detection model.

[0081] As a possible implementation method, the detection device re-divides the training data set based on the categories of multiple training images. After obtaining multiple training data sets, the target detection model can be trained based on the losses of the multiple training data sets and the target detection model.

[0082] Exemplarily, for each training data set in multiple training data sets, the detection device can input the training image in the data set into the target detection model to obtain the detection result of the training image, compare the detection result of the changed training image with the category of the training image, determine the loss of the target detection model, and adjust the parameters of the target detection model based on the loss of the target detection model, input the next training image into the adjusted target detection model again, determine the loss of the target detection model again, iterate this step until the loss of the target detection model is close to the minimum value and remains stable, save the parameters of the target detection model at this time, and use the parameters of the target detection model as the final target detection model.

[0083] In other embodiments, the target detection model is constructed based on the YOLO target detection model, which includes: a feature extraction network, a feature fusion network of a channel fusion spatial attention mechanism, and a detection head. In addition, the target detection model selects the fifth version of the YOLO target detection algorithm model (i.e., YOLOv5) as the baseline model, and introduces an improved channel fusion spatial attention mechanism (efficient channel coordinate attention, ECCA) into the feature fusion network of the target detection model to enhance the target detection model's effect on dense areas and small target detection in road scenes. The target detection model of this application is named: ECCA-YOLOv5. Therefore, Figure 3As shown, the above step S102, inputting the target image into the target detection model to obtain the detection result of the target image, can be specifically implemented as the following steps S1021-S1024:

[0084] S1021. Based on the feature extraction network, extract feature maps of the target image at different scales.

[0085] As a possible implementation method, the detection device can extract feature maps of the target image at different scales based on the feature extraction network of the target detection model.

[0086] Exemplarily, the detection device can extract high-level semantic features of the target image based on the first cross-stage local structure of the feature extraction network, and extract low-level visual features of the target image based on the second cross-stage local structure of the feature extraction network, thereby splicing the high-level semantic features and the low-level visual features to obtain feature maps of the target image at different scales.

[0087] S1022. A feature fusion network based on a channel fusion spatial attention mechanism fuses feature maps of the target image at different scales to obtain a target feature map.

[0088] As a possible implementation method, the detection device extracts feature maps of the target image at different scales based on a feature extraction network, and then can fuse the feature maps of the target image at different scales based on a feature fusion network of a channel fusion spatial attention mechanism of a target detection model to obtain a target feature map.

[0089] Exemplarily, the detection device can fuse feature maps of the target image at different scales to obtain an aggregated feature map, and determine the channel attention weight and spatial attention weight of the aggregated feature map based on a feature fusion network of a channel fusion spatial attention mechanism, thereby multiplying the channel attention weight with the aggregated feature map to obtain a weighted aggregated feature map, and multiplying the spatial attention weight with the weighted aggregated feature map to obtain a target feature map.

[0090] S1023. Determine bounding box positions and categories of multiple target objects in the target image based on the number of channels of the detection head-expanded target feature map.

[0091] As a possible implementation method, the detection device obtains the target feature map of the target image based on the feature extraction network and the feature fusion network of the channel fusion spatial attention mechanism. Then, the number of channels of the target feature map can be expanded based on the detection head of the target detection model to determine the bounding box positions and categories of multiple target objects in the target image.

[0092] Exemplarily, the detection device can expand the channel dimension of the target feature map based on the convolution with a convolution kernel size of 1×1 in the detection head, and the expanded channel dimension is: 3×(5+num_class). Among them, 3 represents the number of 3 anchor boxes allocated to each detection layer, and 5 corresponds to the horizontal coordinate, vertical coordinate, width, height and confidence of the center point of the bounding box, respectively. The confidence represents the credibility of the bounding box, and the value range is (0,1). The larger the confidence value, the more likely it is that the target object exists in the bounding box. num_class represents the number of categories of the target object.

[0093] It should be understood that the input image of the detection head is 640×640, and the output feature map size can be 80×80, 40×40, and 20×20 respectively. Among them, the large-scale feature map has a small receptive field range and is suitable for detecting small-sized target objects, while the small-scale feature map has a large receptive field range and is suitable for detecting large-sized target objects.

[0094] S1024: Determine a detection result of the target image based on the bounding box positions and categories of the multiple target objects in the target image.

[0095] As a possible implementation manner, after the detection device determines the bounding box positions and categories of multiple target objects in the target image, it can determine the detection result of the target image based on the bounding box positions and categories of the multiple target objects in the target image.

[0096] It should be understood that the target feature map obtained by processing the feature fusion network of the channel fusion spatial attention mechanism not only obtains channel attention enhancement but also captures the long-term dependencies between feature maps, which helps to determine the positioning of the target object in the target image. This implementation method can enhance the feature information of dense area targets and long-distance small-scale targets in complex road scenes.

[0097] In some other embodiments, the backbone network of the target detection model is a feature extraction network, in which a large number of cross-stage partial structures (CSP) are used, and two cross-stage partial structures are designed for the feature extraction network, that is, the feature extraction network includes a first cross-stage partial structure and a second cross-stage partial structure. The feature extraction network can extract feature maps of the target image at different scales based on the first cross-stage partial structure and the second cross-stage partial structure. Therefore, Figure 4 As shown, the above step S1021, based on the feature extraction network, extracts the feature maps of the target image at different scales, which can be specifically implemented as the following steps S1021a-S1021c:

[0098] S1021a. Extract high-level semantic features of the target image based on the first cross-stage local structure.

[0099] As a possible implementation method, the detection device can extract high-level semantic features of the target image based on the first cross-stage local structure in the feature extraction network.

[0100] Exemplarily, the first cross-stage local structure is located at the bottom of the feature extraction network, contains fewer convolutional layers or residual blocks to reduce the amount of computation, and extracts high-level semantic features of the target image through simple convolution operations.

[0101] S1021b, extracting low-level visual features of the target image based on the second cross-stage local structure.

[0102] As a possible implementation method, the detection device can also extract low-level visual features of the target image based on the second cross-stage local structure in the feature extraction network.

[0103] Exemplarily, the second cross-stage local structure is located at the upper layer of the feature extraction network, including more convolutional layers or residual blocks to extract richer features, and extracts low-level visual features of the target image through convolution operations and stacked residual blocks or stacked convolution blocks.

[0104] S1021c, concatenate high-level semantic features and low-level visual features to obtain feature maps of the target image at different scales.

[0105] As a possible implementation method, after the detection device extracts the high-level semantic features and low-level visual features of the target image through the first cross-stage local structure and the second cross-stage local structure respectively, the high-level semantic features and the low-level visual features can be spliced ​​to obtain feature maps of the target image at different scales.

[0106] Exemplarily, the detection device may concat the results of the first cross-stage local structure and the second cross-stage local structure, i.e., the high-level semantic features and low-level visual features of the target image, to obtain feature maps of the target image at different scales, thereby reducing the amount of parameters and speeding up the detection speed.

[0107] In some other embodiments, the detection device may fuse the feature maps of the target image at different scales to obtain an aggregated feature map, and determine the channel attention weight and spatial attention weight of the aggregated feature map to determine the target feature map. Figure 5 As shown, the above step S1022, the feature fusion network based on the channel fusion spatial attention mechanism, fuses the feature maps of the target image at different scales to obtain the target feature map, which can be specifically implemented as the following steps S1022a-S1022c:

[0108] S1022a, fusing the feature maps of the target image at different scales to obtain an aggregated feature map.

[0109] As a possible implementation method, the detection device can fuse feature maps of the target image at different scales to obtain an aggregated feature map.

[0110] Exemplarily, the detection device can utilize a feature pyramid network (FPN) or a similar structure to upsample the feature map of high-level semantic features (semantically rich but with low spatial resolution) of the target image to a lower resolution through a top-down path using upsampling (e.g., nearest neighbor or bilinear interpolation), and laterally connect or add the upsampled high-level feature map with the low-level feature map of the low-level visual features to achieve feature fusion of the target image at different scales and obtain an aggregated feature map.

[0111] S1022b. A feature fusion network based on a channel fusion spatial attention mechanism determines the channel attention weight and spatial attention weight of the aggregated feature map.

[0112] As a possible implementation method, after the detection device obtains the aggregated feature map, it can also determine the channel attention weight and spatial attention weight of the aggregated feature map based on the feature fusion network of the channel fusion spatial attention mechanism.

[0113] Exemplarily, the detection device can perform average pooling and maximum pooling on the aggregated feature map, respectively, determine the average pooling element and the maximum pooling element of the aggregated feature map, respectively, and based on the average pooling element and the maximum pooling element, determine the channel attention map through a shared network, thereby summing the elements in the channel attention map element by element, converting the summed result into a probability distribution through a SIGMOID function, and generating a channel attention weight.

[0114] After the detection device obtains the channel attention weight of the aggregated feature map, it can multiply the channel attention weight by the aggregated feature map to obtain a weighted aggregated feature map, and then perform global adaptive mean pooling on the weighted aggregated feature map according to the height and width of the weighted aggregated feature map to obtain a width feature map and a height feature map, and then concatenate and reduce the dimension of the width feature map and the height feature map to obtain a spatial feature map. The detection device can sum the elements in the spatial feature map element by element, convert the summed result into a probability distribution through the SIGMOID function, and generate the spatial attention weight.

[0115] S1022c. Based on the channel attention weight and the spatial attention weight, a target feature map is obtained.

[0116] As a possible implementation method, after the detection device determines the channel attention weights and spatial attention weights of the aggregated feature map, it can obtain a target feature map based on the channel attention weights and the spatial attention weights, and extract the positions and categories of multiple target objects in the target image by editing the target feature map to obtain the detection result of the target image.

[0117] Exemplarily, the detection device may multiply the aggregate feature map by the channel attention weight to obtain a weighted aggregate feature map, and multiply the spatial attention weight by the weighted aggregate feature map to obtain a target feature map.

[0118] In some other embodiments, the detection device may determine the channel attention weight of the aggregate feature map by determining the average pooling element and the maximum pooling element of the aggregate feature map. Figure 6 As shown, the above determination of the channel attention weight of the aggregated feature map can be specifically implemented as the following steps S601-S603:

[0119] S601, determining the average pooling element and the maximum pooling element of the aggregated feature map.

[0120] As a possible implementation method, the detection device can perform average pooling and maximum pooling on the aggregated feature map respectively to determine the average pooling element and the maximum pooling element of the aggregated feature map.

[0121] Exemplarily, the detection device can be based on the improved channel fusion spatial attention mechanism ECCA, using average pooling pool Avg (.) and max pooling Max (.) operation extracts spatial information on the aggregated feature map and generates two different spatial information descriptors: average pooling feature and max pooling elements Among them, the aggregate feature map of the improved channel fusion spatial attention mechanism ECCA input can be expressed as X∈R C×H×W , X represents the input aggregate feature map, C represents the input channel, H represents the height of the aggregate feature map, and W represents the width of the aggregate feature map. The detection device can calculate the average value of each region of the aggregate feature map through the improved channel fusion spatial attention mechanism ECCA, determine the average pooling element, and calculate the maximum value of each region of the aggregate feature map to determine the maximum pooling element.

[0122] S602: Based on the average pooling element and the maximum pooling element, determine the channel attention map through the shared network.

[0123] Among them, the shared network includes one-dimensional convolution with adaptive convolution kernel size.

[0124] As a possible implementation method, after the detection device determines the average pooling element and the maximum pooling element of the aggregated feature map, it can determine the channel attention map based on the average pooling element and the maximum pooling element through a shared network.

[0125] Exemplarily, the detection device can adjust the size of the adaptive convolution kernel in the shared network based on the size of the aggregated feature map, thereby inputting the average pooling element and the maximum pooling element of the two spatial information descriptors of the aggregated feature map into the shared network, and generating a channel attention map through the features processed by the shared network.

[0126] S603: Based on the channel attention map, an activation function is used to generate a channel attention weight.

[0127] As a possible implementation method, after the detection device generates a channel attention map, it can generate a channel attention weight based on the channel attention map using an activation function.

[0128] Exemplarily, the detection device can perform element-by-element summation of the elements in the channel attention map, convert the element-by-element summation result of the channel attention map into a probability distribution through the activation function sigmod, map the element-by-element summation result of the channel attention map to between 0 and 1, and generate a channel attention weight.

[0129] In some other embodiments, the detection device may further multiply the channel attention weight by the aggregate feature map to obtain a weighted aggregate feature map, and determine the spatial attention weight of the aggregate feature map based on the width and height of the weighted aggregate feature map. Figure 7 As shown, the above determination of the spatial attention weight of the aggregated feature map can be specifically implemented as the following steps S701-S704:

[0130] S701. Multiply the channel attention weight by the aggregate feature map to obtain a weighted aggregate feature map.

[0131] As a possible implementation method, the detection device may multiply the channel attention weight with the aggregate feature map to obtain a weighted aggregate feature map, so as to determine the spatial attention weight based on the weighted aggregate feature map.

[0132] Exemplarily, the detection device may multiply the channel attention weight and the aggregate feature map by the following expression to obtain a weighted aggregate feature map:

[0133]

[0134] Among them, X′ represents the aggregate feature map after channel attention weight enhancement, that is, the weighted aggregate feature map, X represents the aggregate feature map, Sigmod represents the activation function, Conv 1dRepresents one-dimensional convolution processing, pool Avg represents the average pooling operation, Represents the average pooling element, Pool Max represents the maximum pooling operation, Represents the maximum pooling element. The weighted aggregate feature map can also be expressed as X′∈R C×H×W , C represents the number of channels of the weighted aggregate feature map, H represents the height of the weighted aggregate feature map, and W represents the width of the weighted aggregate feature map.

[0135] S702: Perform global adaptive mean pooling on the weighted aggregate feature map according to the height and width of the weighted aggregate feature map to obtain a width feature map and a height feature map.

[0136] As a possible implementation method, after the detection device obtains the weighted aggregate feature map, it can perform global adaptive mean pooling on the weighted aggregate feature map according to the height and width of the weighted aggregate feature map to obtain a width feature map and a height feature map.

[0137] Exemplarily, the detection device may perform global adaptive mean pooling on the weighted aggregated feature map according to the height and width to obtain two feature maps of dimensions C×1×W and C×H×1, namely, a width feature map and a height feature map. The detection device may determine the width feature map and the height feature map by the following expression:

[0138]

[0139] Among them, Z h Represents the height feature map, pool avg represents the global adaptive mean pooling operation, X′ represents the weighted aggregate feature map, W represents the width of the weighted aggregate feature map, X′(h,i) represents the i-th element with height h in the weighted aggregate feature map, and Z w represents the width feature map, H represents the height of the weighted aggregate feature map, and X′(j,w) represents the jth element with width w in the weighted aggregate feature map.

[0140] S703: Concatenate and reduce the dimension of the width feature map and the height feature map to obtain a spatial feature map.

[0141] As a possible implementation method, after the detection device obtains the width feature map and the height feature map, the width feature map and the height feature map can be spliced ​​and reduced in dimension to obtain a spatial feature map.

[0142] Exemplarily, the detection device can splice the width feature map and the height feature map, perform dimensionality reduction processing on the spliced ​​feature map using a convolution with a convolution kernel size of 1×1, and obtain a feature map with a dimension size of C / r×(H+W)×1 through normalization and activation function processing. The detection device can also restore the obtained feature map to its original height and width using a convolution with a convolution kernel of 1×1 to obtain a spatial feature map.

[0143] S704: Based on the spatial feature map, an activation function is used to generate a spatial attention weight.

[0144] Among them, the spatial attention weight includes height attention weight and width attention weight.

[0145] As a possible implementation method, the detection device concatenates and reduces the dimension of the width feature map and the height feature map to obtain the spatial feature map. Then, the detection device can generate the spatial attention weight based on the spatial feature map by using an activation function.

[0146] Exemplarily, the detection device can perform element-by-element summation of the elements in the spatial feature map, convert the element-by-element summation result of the spatial feature map into a probability distribution through the activation function sigmod, map the element-by-element summation result of the spatial feature map to between 0 and 1, and generate a spatial attention weight. The detection device can determine the spatial attention weight by the following expression:

[0147]

[0148] Among them, g h represents the height of the spatial attention weight, σ represents the activation function sigmod, Conv 1×1 Indicates the convolution process with a convolution kernel of 1×1, Concat indicates the concatenation operation, and Z h Represents the height feature map, Z w represents the width feature map, g w Represents the spatial attention weight of width.

[0149] It should be understood that after the detection device obtains the spatial attention weight of the height and the spatial attention weight of the width of the spatial attention weight, the spatial attention weight of the height and the spatial attention weight of the width can be multiplied by the weighted aggregate feature map to obtain the target feature map. For example, the detection device can determine the target feature map using the following expression:

[0150] Y=X′*g h *g w ;

[0151] Among them, Y represents the target feature map, X′ represents the weighted aggregate feature map, and g h represents the height spatial attention weight, gw Represents the spatial attention weight of width.

[0152] Figure 8 is a flowchart of another target detection method according to an exemplary embodiment. Figure 8 As shown, the process includes the following steps: S801, constructing a data set and performing preprocessing (equivalent to the above steps S201 and S202), S802, constructing a complex road scene target detection model, S803, inputting training samples for training and retaining the optimal model parameters (equivalent to the above step S203), S804, using the optimal model to detect the input image (equivalent to the above step S102).

[0153] Fig. 9 is a framework diagram of a target detection model according to an exemplary embodiment. Fig. 9 As shown, the target detection model includes a feature extraction network, a feature fusion network and a detection head.

[0154] Among them, the feature extraction network includes: the starting layer Focus (320, 320, 32), which is used to reduce the resolution of the input image and increase the number of feature channels. The convolution block structure (cross bottleneck structure, CBS) (160, 160, 64), is used for further extraction and processing of features. CSP1_BottleNeck_1 (160, 160, 64) (equivalent to the first cross-stage local structure mentioned above), is used to extract features and reduce the amount of calculation. CBS (80, 80, 128). CSP1_BottleNeck_3 (equivalent to the second cross-stage local structure) (80, 80, 128). The output of CSP1_BottleNeck_3 can be sent to Concat (80, 80, 256) in the feature fusion network. CBS (40, 40, 256). CSP1_BottleNeck_3 (40, 40, 256). The output of CSP1_BottleNeck_3 can be sent to Concat(40,40,512) in the feature fusion network. CBS(20,20,512). Spatial pyramid pooling (SPP) (20,20,512) is used to extract multi-scale features and enhance the detection ability of the target detection model for targets of different sizes. CSP1_BottleNeck_1(20,20,512). The output of CSP1_BottleNeck_1 can be sent to CBS(20,20,256) in the feature fusion network.

[0155] The feature fusion network includes: CBS(20,20,256), the output of which can be sent to Concat(20,20,512). Upsampling layer Up_sample(40,40,256), used to increase the spatial size of the feature map. Concat(40,40,512), used to concatenate the feature maps of Up_sample(40,40,256) and CSP1_BottleNeck_3(40,40,256) along the channel dimension. CSP2_BottleNeck_1(40,40,256). Attention mechanism ECCA(40,40,256), used to emphasize important channel features and suppress unimportant channels. CBS(40,40,128). Up_sample(80,80,128). Concat(80,80,256), used to concatenate the feature maps of Up_sample(80,80,128) and CSP1_BottleNeck_3(80,80,128) along the channel dimension. CSP2_BottleNeck_1(80,80,128), used to output to the detection head for Conv(80,80,3*(5+num_class)). ECCA(80,80,128). CBS(40,40,128). Concat(40,40,256), used to concatenate the feature maps of CBS(40,40,128) and CBS(40,40,128) along the channel dimension. CSP2_BottleNeck_1(40,40,256), used to output to the detection head for Conv(40,40,3*(5+num_class)). ECCA(40,40,256). CBS(20,20,256). Concat(20,20,512), used to concatenate the feature maps of CBS(20,20,256) and CBS(20,20,256) along the channel dimension. CSP2_BottleNeck_1(20,20,512), used to output to the detection head for Conv(20,20,3*(5+num_class)) processing.

[0156] The detection heads include: Conv(80,80,3*(5+num_class)), Conv(40,40,3*(5+num_class)) and Conv(20,20,3*(5+num_class)).

[0157] Fig.10 is a structural diagram of a channel fusion spatial attention mechanism according to an exemplary embodiment. Fig.10As shown in the figure, the channel fusion spatial attention mechanism first processes the aggregate feature map extracted by the feature extraction network through average pooling Avg_pool and maximum pooling Max_pool to obtain the average pooling element and the maximum pooling element of the aggregate image. The average pooling element and the maximum pooling element are processed by one-dimensional convolution CONV1D, and the processed average pooling element and the maximum pooling element are added. The channel attention weight is generated by the activation function Sigmod, and the channel attention weight is multiplied by the aggregate feature map to obtain the weighted aggregate feature map. The weighted aggregate feature map is subjected to global adaptive mean pooling Avg_pool based on the width X and height Y, and the pooled structure is spliced ​​and processed by two-dimensional convolution Concat+CONV2D, and processed by batch normalization and non-linear activation function BatchNorm+Nor-linear to obtain the height feature map and width feature map. The height feature map and the width feature map are respectively processed by two-dimensional convolution CONV2D and activation function Sigmod to obtain the spatial attention weight, and the spatial attention weight is multiplied by the weighted aggregate feature map to obtain the target feature map.

[0158] Fig.11 is a display diagram of a detection result according to an exemplary embodiment. Fig.12 is another display diagram of detection results according to an exemplary embodiment, and the detection effect diagram of the target detection model is as follows Fig.11 and Fig.12 As shown, Fig.11 and Fig.12 The first row in the figure shows the detection effect of the target detection model ECCA-YOLOv5, the second row shows the detection effect of YOLOv5, the first column shows the heat map visualization, and the second column shows the actual detection effect. By comparing the detection effect graphs of the target detection models ECCA-YOLOv5 and YOLOv5, it can be found that the target detection model ECCA-YOLOv5 has a better detection effect on target objects in dense areas of road scenes and small-sized target objects.

[0159] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, the target detection device or vehicle includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0160] The embodiment of the present application can exemplarily divide the functional modules of the target detection device or vehicle according to the above method. For example, the target detection device or vehicle can include various functional modules corresponding to the various functional divisions, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0161] Fig.13 is a block diagram of a target detection device according to an exemplary embodiment. Fig.13 The target detection device 1300 includes: an acquisition module 1301 and a processing module 1302. The acquisition module 1301 is used to acquire a target image to be detected; the target image includes multiple target objects to be detected; the processing module 1302 is used to input the target image into a target detection model to obtain a detection result of the target image; the target detection model includes a feature fusion network of a channel fusion spatial attention mechanism; the detection result includes multiple bounding boxes and multiple labels; the bounding box is used to mark the positions of multiple target objects in the target image; the label is used to characterize the categories of multiple target objects in the bounding box.

[0162] In a possible implementation, the apparatus further includes: a division module 1303 and a training module 1304. The acquisition module 1301 is further used to acquire a training data set; the training data set includes multiple training images and multiple categories of training images; the division module 1303 is used to re-divide the training data set based on the categories of the multiple training images to obtain multiple training data sets; the training module 1304 is used to train the target detection model based on the multiple training data sets and the loss of the target detection model.

[0163] In another possible implementation, the target detection model is constructed based on the YOLO target detection model; the YOLO target detection model includes: a feature extraction network, a feature fusion network of a channel fusion spatial attention mechanism, and a detection head; a processing module 1302 is specifically used to extract feature maps of the target image at different scales based on the feature extraction network; based on the feature fusion network of the channel fusion spatial attention mechanism, the feature maps of the target image at different scales are fused to obtain a target feature map; based on the detection head, the number of channels of the target feature map is expanded to determine the bounding box positions and categories of multiple target objects in the target image; based on the bounding box positions and categories of multiple target objects in the target image, the detection result of the target image is determined.

[0164] In another possible implementation, the feature extraction network includes a first cross-stage local structure and a second cross-stage local structure; the processing module 1302 is specifically used to extract high-level semantic features of the target image based on the first cross-stage local structure; based on the second cross-stage local structure, extract low-level visual features of the target image; and splice the high-level semantic features and the low-level visual features to obtain feature maps of the target image at different scales.

[0165] In another possible implementation, the processing module 1302 is specifically used to fuse the feature maps of the target image at different scales to obtain an aggregated feature map; determine the channel attention weights and spatial attention weights of the aggregated feature map based on a feature fusion network of the channel fusion spatial attention mechanism; and obtain the target feature map based on the channel attention weights and spatial attention weights.

[0166] In another possible implementation, the processing module 1302 is specifically used to determine the average pooling element and the maximum pooling element of the aggregated feature map; based on the average pooling element and the maximum pooling element, determine the channel attention map through a shared network; the shared network includes a one-dimensional convolution with an adaptive convolution kernel size; based on the channel attention map, an activation function is used to generate a channel attention weight.

[0167] In another possible implementation, the processing module 1302 is specifically used to multiply the channel attention weight by the aggregate feature map to obtain a weighted aggregate feature map; perform global adaptive mean pooling on the weighted aggregate feature map according to the height and width of the weighted aggregate feature map to obtain a width feature map and a height feature map; concatenate and reduce the dimension of the width feature map and the height feature map to obtain a spatial feature map; based on the spatial feature map, use an activation function to generate a spatial attention weight; the spatial attention weight includes a height attention weight and a width attention weight.

[0168] According to the above technical means, the target detection model can integrate multi-scale features from different levels in the target image through the feature fusion network of the channel fusion spatial attention mechanism, and strengthen the complementarity between different features, thereby improving the expression ability of the features after the target image is fused, enhancing the feature information of target objects in dense areas and long-distance small-scale target objects in complex road scenes, and improving the detection accuracy of the target detection model for target objects in dense areas and long-distance small-scale target objects.

[0169] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0170] Fig.14 FIG. 1 is a block diagram of a vehicle according to an exemplary embodiment. Fig.14 As shown, vehicle 1400 includes, but is not limited to, a processor 1401 and a memory 1402 .

[0171] The memory 1402 is used to store executable instructions of the processor 1401. It can be understood that the processor 1401 is configured to execute instructions to implement the target detection method in the above embodiment.

[0172] It should be noted that those skilled in the art can understand that Fig.14 The vehicle structure shown in the figure does not constitute a limitation on the vehicle, and the vehicle may include Fig.14 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0173] The processor 1401 is the control center of the vehicle, which uses various interfaces and lines to connect various parts of the entire vehicle, and executes various functions of the vehicle and processes data by running or executing software programs and / or modules stored in the memory 1402, and calling data stored in the memory 1402, so as to monitor the vehicle as a whole. The processor 1401 may include one or more processing units. Optionally, the processor 1401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1401.

[0174] The memory 1402 may be used to store software programs and various data. The memory 1402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one functional module (such as a determination unit, a processing unit, etc.), etc. In addition, the memory 1402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0175] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1402 including instructions, and the above instructions can be executed by the processor 1401 of the vehicle 1400 to implement the target detection method in the above embodiment.

[0176] In actual implementation, Fig.13 The functions of the acquisition module 1301, the processing module 1302, the division module 1303 and the training module 1304 in Fig.14 The processor 1401 in the embodiment calls the computer program stored in the memory 1402 to implement. The specific execution process can refer to the description of the method part in the above embodiment, which will not be repeated here.

[0177] Optionally, the computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0178] In an exemplary embodiment, the embodiment of the present application also provides a computer program product including one or more instructions, which can be executed by the processor 1401 of the vehicle to complete the target detection method in the above embodiment.

[0179] It should be noted that when the instructions in the above-mentioned computer-readable storage medium or one or more instructions in the computer program product are executed by the vehicle's processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect as the above-mentioned method can be achieved. To avoid repetition, they will not be repeated here.

[0180] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0181] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0183] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0184] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the full classification part or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute the full classification part or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or CD and other media that can store program code.

[0185] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire a target image to be detected; the target image includes a plurality of target objects to be detected; The target image is input into a target detection model to obtain a detection result of the target image; the target detection model includes a feature fusion network of a channel fusion spatial attention mechanism; the detection result includes multiple bounding boxes and multiple labels; the bounding box is used to mark the positions of the multiple target objects in the target image; the label is used to characterize the categories of the multiple target objects in the bounding box.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a training data set; the training data set includes a plurality of training images and categories of the plurality of training images; Based on the categories of the multiple training images, the training data set is re-divided to obtain multiple training data sets; The object detection model is trained based on the multiple training data sets and the loss of the object detection model.

3. The method according to claim 1, characterized in that: The target detection model is built based on the YOLO target detection model; the YOLO target detection model includes: a feature extraction network, a feature fusion network of the channel fusion spatial attention mechanism and a detection head; The step of inputting the target image into a target detection model to obtain a detection result of the target image includes: Based on the feature extraction network, extracting feature maps of the target image at different scales; Based on the feature fusion network of the channel fusion spatial attention mechanism, the feature maps of the target image at different scales are fused to obtain a target feature map; Expanding the number of channels of the target feature map based on the detection head, determining bounding box positions and categories of multiple target objects in the target image; A detection result of the target image is determined based on bounding box positions and categories of multiple target objects in the target image.

4. The method according to claim 3, characterized in that The feature extraction network includes a first cross-stage local structure and a second cross-stage local structure; extracting feature maps of the target image at different scales based on the feature extraction network includes: Extracting high-level semantic features of the target image based on the first cross-stage local structure; Extracting low-level visual features of the target image based on the second cross-stage local structure; The high-level semantic features and the low-level visual features are concatenated to obtain feature maps of the target image at different scales.

5. The method according to claim 3, characterized in that: The feature fusion network based on the channel fusion spatial attention mechanism fuses the feature maps of the target image at different scales to obtain a target feature map, including: Fusing the feature maps of the target image at different scales to obtain an aggregated feature map; Determine the channel attention weight and the spatial attention weight of the aggregated feature map based on the feature fusion network of the channel fusion spatial attention mechanism; Based on the channel attention weight and the spatial attention weight, a target feature map is obtained.

6. The method according to claim 5, characterized in that The determining of the channel attention weight of the aggregated feature map comprises: Determine the average pooling factor and the maximum pooling factor of the aggregated feature map; Based on the average pooling element and the maximum pooling element, determining a channel attention map through a shared network; the shared network includes a one-dimensional convolution with an adaptive convolution kernel size; Based on the channel attention map, an activation function is used to generate the channel attention weight.

7. The method according to claim 5, characterized in that Determining the spatial attention weight of the aggregated feature map includes: Multiplying the channel attention weight by the aggregate feature map to obtain a weighted aggregate feature map; Performing global adaptive mean pooling on the weighted aggregate feature map according to the height and width of the weighted aggregate feature map to obtain a width feature map and a height feature map; The width feature map and the height feature map are concatenated and dimensionally reduced to obtain a spatial feature map; Based on the spatial feature map, an activation function is used to generate the spatial attention weight; the spatial attention weight includes a height attention weight and a width attention weight.

8. A target detection device, characterized in that: The device comprises: an acquisition module and a processing module; The acquisition module is used to acquire a target image to be detected; the target image includes a plurality of target objects to be detected; The processing module is used to input the target image into a target detection model to obtain a detection result of the target image; the target detection model includes a feature fusion network of a channel fusion spatial attention mechanism; the detection result includes multiple bounding boxes and multiple labels; the bounding box is used to mark the positions of the multiple target objects in the target image; the label is used to characterize the categories of the multiple target objects in the bounding box.

9. The device according to claim 8, characterized in that The device also includes: a division module and a training module; The acquisition module is further used to acquire a training data set; the training data set includes a plurality of training images and categories of the plurality of training images; The division module is used to re-divide the training data set based on the categories of the multiple training images to obtain multiple training data sets; The training module is used to train the target detection model based on the multiple training data sets and the loss of the target detection model.

10. A vehicle, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: When the computer-executable instructions stored in the computer-readable storage medium are executed by a processor of a vehicle, the vehicle is capable of performing the method of any one of claims 1 to 7.

12. A computer program product, characterized in that The computer program product comprises computer instructions which, when executed on a vehicle, cause the vehicle to perform the method according to any one of claims 1 to 7 .

Images