Target detection method and model training method, device, medium and electronic equipment

Abstract

The present disclosure provides a target detection method, relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and the technical field of image recognition. The specific implementation scheme is: obtaining N initial feature maps according to a first target feature map in M target feature maps of a target image, wherein the sizes of the N initial feature maps are different from each other, N is an integer greater than 1, and M is an integer greater than 1; extracting M sampling feature maps corresponding to the initial feature maps from the M target feature maps according to offset information of the initial feature maps; fusing the M sampling feature maps of the N initial feature maps respectively to obtain N fusion feature maps; and performing target detection according to the N fusion feature maps to obtain a detection result of the target image. The present disclosure also provides a training method and device of a target detection model, an electronic device and a storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of deep learning technology and image recognition technology, and can be applied to scenarios such as industrial quality inspection, intelligent transportation, and autonomous driving. More specifically, this disclosure provides an object detection method, an object detection model training method, an apparatus, an electronic device, and a storage medium. Background Technology

[0002] With the development of artificial intelligence technology, various models can be used to identify the location and category of objects in images. Features at multiple scales can be extracted from images. Based on these multi-scale features, object recognition or detection can be performed. Summary of the Invention

[0003] This disclosure provides an object detection method, an object detection model training method, an apparatus, a device, and a storage medium.

[0004] According to one aspect of this disclosure, a target detection method is provided, the method comprising: obtaining N initial feature maps based on a first target feature map in M ​​target feature maps of a target image, wherein the N initial feature maps have different sizes, N is an integer greater than 1, and M is an integer greater than 1; extracting M sampled feature maps corresponding to the initial feature maps from the M target feature maps based on the offset information of the initial feature maps; fusing the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps; and performing target detection based on the N fused feature maps to obtain a detection result of the target image.

[0005] According to another aspect of this disclosure, a training method for an object detection model is provided. The object detection model includes a feature extraction network, a feature generation network, and a detection network. The method includes: inputting a sample image into the feature extraction network to obtain M target feature maps of the sample image, where M is an integer greater than 1; performing the following operations on the M target feature maps using the feature generation network to obtain N fused feature maps: obtaining N initial feature maps based on the first target feature map in the M target feature maps, where the N initial feature maps have different sizes and N is an integer greater than 1; extracting M sampled feature maps corresponding to the initial feature maps from the M target feature maps based on the offset information of the initial feature maps; fusing the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps; inputting the N fused feature maps into the detection network to obtain the detection result of the sample image; and training the object detection model based on the detection result and the label of the sample image.

[0006] According to another aspect of this disclosure, a target detection apparatus is provided, comprising: a first obtaining module, configured to obtain N initial feature maps based on a first target feature map in M ​​target feature maps of a target image, wherein the N initial feature maps have different sizes, N is an integer greater than 1, and M is an integer greater than 1; an extraction module, configured to extract M sampled feature maps corresponding to the initial feature maps from the M target feature maps based on offset information of the initial feature maps; a fusion module, configured to fuse the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps; and a target detection module, configured to perform target detection based on the N fused feature maps to obtain a detection result of the target image.

[0007] According to another aspect of this disclosure, a training apparatus for an object detection model is provided. The object detection model includes a feature extraction network, a feature generation network, and a detection network. The apparatus includes: a second obtaining module for inputting a sample image into the feature extraction network to obtain M target feature maps of the sample image, where M is an integer greater than 1; an execution module for performing related operations on the M target feature maps using the feature generation network through the following sub-modules to obtain N fused feature maps: an obtaining sub-module for obtaining N initial feature maps based on a first target feature map in the M target feature maps, where the N initial feature maps have different sizes and N is an integer greater than 1; a third extraction sub-module for extracting M sampled feature maps corresponding to the initial feature maps from the M target feature maps based on the offset information of the initial feature maps; a fusion sub-module for fusing the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps; a third obtaining module for inputting the N fused feature maps into the detection network to obtain a detection result of the sample image; and a training module for training the object detection model based on the detection result and the label of the sample image.

[0008] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to this disclosure.

[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods provided according to this disclosure.

[0010] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to this disclosure.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 This is an exemplary system architecture diagram of an object detection method and apparatus applicable according to an embodiment of the present disclosure;

[0014] Figure 2 This is a flowchart of a target detection method according to an embodiment of the present disclosure;

[0015] Figure 3 This is a schematic diagram of a target detection model according to an embodiment of the present disclosure;

[0016] Figure 4 This is a flowchart of a training method for an object detection model according to an embodiment of the present disclosure;

[0017] Figure 5 This is a block diagram of a target detection apparatus according to an embodiment of the present disclosure;

[0018] Figure 6 This is a block diagram of a training apparatus for an object detection model according to an embodiment of the present disclosure; and

[0019] Figure 7 This is a block diagram of an electronic device according to an embodiment of the present disclosure, to which object detection methods and / or object detection model training methods can be applied. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] Object detection technology can identify objects of interest from images, as well as determine their location, size, and category. Object detection is one of the most important and challenging tasks in computer vision. It can be applied to scenarios such as industrial quality inspection, intelligent transportation, and autonomous driving.

[0022] Object detection methods can include dense detection methods and sparse detection methods. Dense detection can be implemented using one-stage or two-stage detectors. For example, dense detection can be implemented using a deep learning model, which can include a backbone network, a feature fusion network, and a detection network. The DarkNet53 network can be used as the backbone network.

[0023] Object detection tasks require detecting objects of various sizes in an image. Transformer models can output global features with strong expressive power, but they are difficult to output features at different scales and are also difficult to use as the backbone network of a one-stage detector.

[0024] Figure 1 This is a schematic diagram of an exemplary system architecture for applying a target detection method and apparatus according to an embodiment of this disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0025] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0026] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0027] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0028] It should be noted that the target detection method provided in this embodiment can generally be executed by server 105. Correspondingly, the target detection device provided in this embodiment can generally be located in server 105. The target detection method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the target detection device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0029] Figure 2 This is a flowchart of a target detection method according to an embodiment of the present disclosure.

[0030] like Figure 2 As shown, the method 200 may include operations S210 to S240.

[0031] In operation S210, N initial feature maps are obtained based on the first target feature map in the M target feature maps of the target image.

[0032] In this embodiment of the disclosure, multi-level feature extraction can be performed on the target image to obtain multiple feature maps. M target feature maps can be obtained from these multiple feature maps. For example, M can be an integer greater than 1. For instance, multiple cascaded Transformer layers can be used for multi-level feature extraction. Each Transformer layer can output a feature map. From the feature maps output by the multiple Transformer layers, M feature maps can be determined as the M target feature maps. Alternatively, the feature map output by the last Transformer layer can be used as the first target feature map.

[0033] In this embodiment of the disclosure, the sizes of the N initial feature maps are different from each other. For example, N can be an integer greater than 1. For example, the N initial feature maps may include a first initial feature map, a second initial feature map, and a third initial feature map. The size of the first initial feature map may be 20×20. The size of the second initial feature map may be 40×40. The size of the third initial feature map may be 80×80. It is understood that the sizes of the first to third initial feature maps are merely examples.

[0034] In operation S220, based on the offset information of the initial feature map, M sampled feature maps corresponding to the initial feature map are extracted from the M target feature maps.

[0035] In this embodiment of the disclosure, the offset information includes the offset values ​​of multiple feature points in the initial feature map. For example, the initial feature map may include multiple initial feature points. In the first initial feature map, the offset value of the initial feature point in the second row and second column may be 1.

[0036] In this embodiment of the disclosure, target feature points can be obtained from the first target feature map to the third target feature map based on the offset information of the initial feature map, and used as sampling feature points. A sampling feature map can be obtained based on multiple sampling feature points. For example, the M target feature maps may include a first target feature map, a second target feature map, and a third target feature map. For another example, in the first initial feature map, the offset value of the initial feature point in the second row and second column can be 1. The first target feature point in the third row and third column of the first target feature map can be used as the first sampling feature point corresponding to the initial feature point. The second target feature point in the third row and third column of the second target feature map can be used as the second sampling feature point corresponding to the initial feature point. The third target feature point in the third row and third column of the third target feature map can be used as the third sampling feature point corresponding to the initial feature point. For another example, a first sampling feature map can be obtained based on multiple first sampling feature points. A second sampling feature map can be obtained based on multiple second sampling feature points. A third sampling feature map can be obtained based on multiple third sampling feature points.

[0037] In operation S230, the M sampled feature maps of each of the N initial feature maps are fused to obtain N fused feature maps.

[0038] For example, M sampled feature maps corresponding to the first initial feature map can be fused to obtain a first fused feature map. M sampled feature maps corresponding to the second initial feature map can be fused to obtain a second fused feature map. M sampled feature maps corresponding to the third initial feature map can be fused to obtain a third fused feature map.

[0039] In operation S240, target detection is performed based on N fused feature maps to obtain the detection result of the target image.

[0040] For example, the N fused features have different scales. The N fused features correspond to N scales. Based on the N fused features, various methods can be used to perform target detection and obtain detection results.

[0041] In this embodiment, based on the offset information of the feature maps, sampled feature maps are extracted from M target feature maps at different depths, and these M sampled feature maps are fused to obtain a fused feature map corresponding to the initial feature map. This allows different fused feature maps to have different scale information, enabling the use of multiple fused features for target detection, thus improving the accuracy of target detection.

[0042] It is understandable that the target detection method provided in this disclosure can be implemented using the target detection model, which will be explained below.

[0043] Figure 3 This is a schematic diagram of a target detection model according to an embodiment of the present disclosure.

[0044] like Figure 3 As shown, the object detection model 300 may include a feature extraction network 310, a feature generation network 320, and a detection network 330. For example, the feature extraction network 310 may serve as the backbone network of the object detection model 300. The feature extraction network 310 may include multiple Transformer layers.

[0045] In some embodiments, in the above-described operation S210, obtaining N initial feature maps based on the first target feature map in the M target feature maps of the target image may include: performing multi-level feature extraction on the target image to obtain M target feature maps; and sampling the first target feature map N times to obtain N initial features.

[0046] For example, such as Figure 3 As shown, the target image 301 can be subjected to multi-level feature extraction using multiple Transformer layers of the feature extraction network 310, resulting in multiple feature maps. Based on these multiple feature maps, M target feature maps can be obtained. In one example, the feature extraction network 310 may include 12 cascaded Transformer layers. The feature map output by the 7th Transformer layer can be used as the third target feature map, the feature map output by the 9th Transformer layer can be used as the second target feature map, and the feature map output by the 12th Transformer layer can be used as the first target feature map. Next, the feature generation network 320 can be used to process the M target feature maps, which will be explained below.

[0047] For example, the first target feature map can be downsampled to obtain a first initial feature map T1. The first target feature map can also be downsampled to obtain a second initial feature map T2. The first target feature map can be upsampled to obtain a third initial feature map T3. Upsampling can be performed using methods such as pooling and interpolation, while downsampling can be performed using methods such as convolution. Through the embodiments of this disclosure, multiple basic features of different sizes can be generated based on the last target feature map containing global information, so that subsequent features with scale information can be generated based on these basic features.

[0048] In some embodiments, in the above-described operation S220, extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map may include: performing convolution on the initial feature map to obtain the offset information of the initial feature map.

[0049] In this embodiment, the offset information includes the offset values ​​of multiple initial feature points in the initial feature map. For example, each feature point may correspond to one offset value. Alternatively, the offset values ​​of multiple feature points may be the same or different from each other; this disclosure does not impose any limitations on this. For example, in the first initial feature map, the offset value of the feature point in the 3rd row and 3rd column may be 0.5. After obtaining the offset values, M sampled feature maps corresponding to the initial feature map can be extracted from M target feature maps based on the offset information. Through this embodiment, convolution is performed on the initial feature map to determine the offset values ​​of each initial feature point in the initial feature map, enabling the extraction of feature points from global feature maps of different depths, thereby facilitating the generation of a feature map with scale information based on the initial feature map.

[0050] In some embodiments, in the above-described operation S220, extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map may further include: interpolating the M target feature maps according to the offset information to obtain M interpolated feature maps.

[0051] In this embodiment of the disclosure, interpolation can be performed based on the offset values ​​of multiple feature points and multiple target feature points of the target feature map to obtain multiple interpolated feature points. An interpolated feature map can be obtained based on the multiple interpolated feature points and the multiple target feature points. For example, as described above, in the first initial feature map, the offset value of the feature point in the 3rd row and 3rd column can be 0.5. Interpolation can be performed on the first target feature map to obtain multiple interpolated feature points. The multiple interpolated feature points may include the interpolated feature point in the 3.5th row and 3.5th column. An interpolated feature map can be obtained based on the multiple interpolated feature points.

[0052] In this embodiment of the disclosure, based on the offset information of the first initial feature map, interpolation can be performed on the first target feature map, the second target feature map, and the third target feature map respectively to obtain a first interpolated feature map, a second interpolated feature map, and a third interpolated feature map corresponding to the first initial feature map T1. Based on the offset information of the second initial feature map, interpolation can be performed on the first target feature map, the second target feature map, and the third target feature map respectively to obtain a first interpolated feature map, a second interpolated feature map, and a third interpolated feature map corresponding to the second initial feature map T2. Based on the offset information of the third initial feature map, interpolation can be performed on the first target feature map, the second target feature map, and the third target feature map respectively to obtain a first interpolated feature map, a second interpolated feature map, and a third interpolated feature map corresponding to the third initial feature map T3.

[0053] In some embodiments, in the above-described operation S220, extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map may further include: extracting M sampled feature maps corresponding to the initial feature map from the M interpolated feature maps based on the offset information.

[0054] In this embodiment of the disclosure, multiple sampled feature points can be obtained from the interpolated feature map based on the offset values ​​of each initial feature point in the initial feature map. A sampled feature map can then be obtained based on these multiple sampled feature points. For example, in the first initial feature map T1, the offset value of the initial feature point in the 3rd row and 3rd column is 0.5. The interpolated feature point in the 3.5th row and 3.5th column can be obtained from the first interpolated feature map as a sampled feature point.

[0055] In this embodiment of the disclosure, the size of the M sampled feature maps corresponding to the initial feature map is the same as the size of the initial feature map.

[0056] For example, based on the offset information of the first initial feature map T1, the first interpolation feature map, the second interpolation feature map, and the third interpolation feature map corresponding to the first initial feature map T1 can be extracted respectively to obtain the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the first initial feature map T1. If the size of the first initial feature map T1 is 20×20, the size of the corresponding first sampled feature map can be 20×20, the size of the corresponding second sampled feature map can be 20×20, and the size of the corresponding third sampled feature map can be 20×20.

[0057] For example, based on the offset information of the second initial feature map T2, the first interpolation feature map, the second interpolation feature map, and the third interpolation feature map corresponding to the second initial feature map T2 can be extracted respectively to obtain the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the second initial feature map T2. If the size of the second initial feature map T2 is 40×40, the size of the corresponding first sampled feature map can be 40×40, the size of the corresponding second sampled feature map can be 40×40, and the size of the corresponding third sampled feature map can be 40×40.

[0058] For example, based on the offset information of the third initial feature map T3, the first interpolated feature map, the second interpolated feature map, and the third interpolated feature map corresponding to the third initial feature map T3 can be extracted respectively to obtain the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the third initial feature map T3. If the size of the third initial feature map T3 is 80×80, the size of the corresponding first sampled feature map can be 80×80, the size of the corresponding second sampled feature map can be 80×80, and the size of the corresponding third sampled feature map can be 80×80. Through the embodiments of this disclosure, interpolation is performed on the target feature map based on the offset information of the initial feature map, which can effectively extract feature points from target feature maps of different depths, and helps to generate feature maps with scale information.

[0059] In some embodiments, in the above-described operation S230, fusing the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps may include: weighting and fusing the M sampled feature maps according to their respective preset weights to obtain fused feature maps. For example, for a first initial feature map T1, according to the preset weights of the M sampled feature maps, the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the first initial feature map T1 can be fused to obtain a first fused feature map. For example, for a second initial feature map T2, according to the preset weights of the M sampled feature maps, the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the second initial feature map T2 can be fused to obtain a second fused feature map. For example, for a third initial feature map T3, according to the preset weights of the M sampled feature maps, the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the third initial feature map T3 can be fused to obtain a third fused feature map. Through the embodiments of this disclosure, multiple sampled feature maps sampled from target feature maps at different depths are fused according to preset weights, so that the fused feature map has scale information for target detection.

[0060] In some embodiments, during operation S240 described above, the first fused feature map, the second fused feature map, and the third fused feature map can be input into the detection network 330 to obtain a detection result. For example, the detection network 330 can be a Pyramid Attention Network (PAN). For example, the detection result can include a detection bounding box of the target object in the target image and the category of the target object; the detection bounding box indicates the location of the target object in the target image. Through embodiments of this disclosure, multiple fused feature maps have different scale information. Target detection based on multiple fused feature maps can yield more accurate detection results.

[0061] It is understood that the above description uses N=3 and M=3 as examples to illustrate the method of this disclosure, but this disclosure is not limited to this. N can be any integer greater than 1, and M can also be any integer greater than 1.

[0062] As can be understood, the object detection method of this disclosure has been described above, and the training method of the object detection model of this disclosure will be described below.

[0063] Figure 4 This is a flowchart of a training method for an object detection model according to another embodiment of the present disclosure.

[0064] In this embodiment of the disclosure, the target detection model may include a feature extraction network, a feature generation network, and a detection network.

[0065] like Figure 4 As shown, the method 400 may include operations S410 to S440.

[0066] In operation S410, the sample image is input into the feature extraction network to obtain M target feature maps of the sample image.

[0067] In this embodiment of the disclosure, M is an integer greater than 1. For example, the feature extraction network may include multiple cascaded Transformer layers. Multiple cascaded Transformer layers can be used for multi-level feature extraction. Each Transformer layer can output a feature map. From the feature maps output by the multiple Transformer layers, M feature maps can be determined as M target feature maps. It can be understood that the aforementioned target image can be used as a sample image. Manually labeling this sample image yields its label.

[0068] In operation S420, the feature generation network performs the following operations on M target feature maps to obtain N fused feature maps.

[0069] For example, a feature generation network can be used to perform operations S421 to S423 on M target feature maps.

[0070] In operation S421, N initial feature maps are obtained based on the first target feature map in the M target feature maps.

[0071] In this embodiment of the disclosure, the sizes of the N initial feature maps are different from each other, where N is an integer greater than 1.

[0072] For example, the N initial feature maps may include a first initial feature map, a second initial feature map, and a third initial feature map. The size of the first initial feature map can be 20×20. The size of the second initial feature map can be 40×40. The size of the third initial feature map can be 80×80. It should be understood that the sizes of the first to third initial feature maps are merely examples.

[0073] In operation S422, based on the offset information of the initial feature map, M sampled feature maps corresponding to the initial feature map are extracted from the M target feature maps.

[0074] In this embodiment of the disclosure, the offset information includes the offset values ​​of multiple initial feature points in the initial feature map. For example, the initial feature map may include multiple initial feature points. In the first initial feature map, the offset value of the initial feature point in the second row and second column can be 1.

[0075] In this embodiment of the disclosure, target feature points can be obtained from the first target feature map to the third target feature map based on the offset information of the initial feature map, and used as sampling feature points. A sampling feature map can be obtained based on multiple sampling feature points. For example, the M target feature maps may include a first target feature map, a second target feature map, and a second target feature map. For another example, in the first initial feature map, the offset value of the initial feature point in the second row and second column can be 1. The first target feature point in the third row and third column of the first target feature map can be used as the first sampling feature point corresponding to the initial feature point. The second target feature point in the third row and third column of the second target feature map can be used as the second sampling feature point corresponding to the initial feature point. The third target feature point in the third row and third column of the third target feature map can be used as the third sampling feature point corresponding to the initial feature point. For another example, a first sampling feature map can be obtained based on multiple first sampling feature points. A second sampling feature map can be obtained based on multiple second sampling feature points. A third sampling feature map can be obtained based on multiple third sampling feature points.

[0076] In operation S423, the M sampled feature maps of each of the N initial feature maps are fused to obtain N fused feature maps.

[0077] For example, M sampled feature maps corresponding to the first initial feature map can be fused to obtain a first fused feature map. M sampled feature maps corresponding to the second initial feature map can be fused to obtain a second fused feature map. M sampled feature maps corresponding to the third initial feature map can be fused to obtain a third fused feature map.

[0078] In operation S430, N fused feature maps are input into the detection network to obtain the detection results of the sample image.

[0079] For example, N fused feature maps have different scales. These N fused feature maps correspond to N scales. By inputting these N fused feature maps into a detection network, detection results can be obtained.

[0080] In operation of S440, a target detection model is trained based on the detection results and the labels of the sample images.

[0081] In this embodiment of the disclosure, the detection result may include the output detection box of the sample object in the sample image and the output category of the sample object. The output detection box may indicate the position of the sample object in the sample image. The label of the sample image may include the annotation detection box of the sample object and the true category of the sample object. The annotation detection box may indicate the true position of the sample in the sample image.

[0082] For example, based on the output and labeled detection boxes, various loss functions can be used to determine the detection box loss. Based on the output and ground truth classes, various loss functions can be used to determine the class loss. Based on the detection box loss and class loss, the total loss can be determined. The parameters of the object detection model are then adjusted until the total loss converges to train the object detection model.

[0083] In this embodiment, based on the offset information of the feature maps, sampled feature maps are extracted from M target feature maps at different depths, and these M sampled feature maps are fused to obtain a fused feature map corresponding to the initial feature map. This allows different fused feature maps to have different scale information, enabling the use of multiple fused features for target detection, thus improving the accuracy of target detection.

[0084] Furthermore, through the embodiments of this disclosure, the feature generation network can generate fused features at different scales. The feature generation network can be an end-to-end structure, which can be embedded as an independent module into various one-stage intensive detection processes, has a wide range of applications, and can also improve the accuracy of target detection.

[0085] As can be understood, the training method of the object detection model of this disclosure has been explained above, and the object detection model of this disclosure will be further explained below.

[0086] In some embodiments, the feature extraction network can serve as the backbone network of an object detection model. The feature extraction network may include multiple Transformer layers. For example, multiple Transformer layers of the feature extraction network can be used to perform multi-level feature extraction on a sample image, resulting in multiple feature maps. Based on these multiple feature maps, M target feature maps can be obtained. In one example, the feature extraction network may include 12 cascaded Transformer layers. The feature map output by the 7th Transformer layer can be used as the third target feature map, the feature map output by the 9th Transformer layer can be used as the second target feature map, and the feature map output by the 12th Transformer layer can be used as the first target feature map.

[0087] In some embodiments, in the above-described operation S421, obtaining N initial feature maps based on the first target feature map among the M target feature maps may include sampling the first target feature map N times to obtain N initial feature maps. For example, the first target feature map may be downsampled to obtain a first initial feature map. The first target feature map may be downsampled to obtain a second initial feature map. The first target feature map may be upsampled to obtain a third initial feature map. Upsampling may be performed using methods such as pooling or interpolation, and downsampling may be performed using methods such as convolution. Through the embodiments of this disclosure, multiple basic features of different sizes can be generated based on the last target feature map with global information, so that features with scale information can be generated subsequently based on the basic features.

[0088] In some embodiments, the feature generation network may include convolutional units.

[0089] In some embodiments, in the above operation S422, extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps according to the offset information of the initial feature map includes: performing convolution on the initial feature map using a convolution unit to obtain the offset information of the initial feature map.

[0090] In this embodiment, the offset information includes the offset values ​​of multiple initial feature points in the initial feature map. For example, each initial feature point may correspond to one offset value. Alternatively, the offset values ​​of multiple initial feature points may be the same or different from each other; this disclosure does not impose any limitations in this regard. For example, in the first initial feature map, the offset value of the initial feature point in the 3rd row and 3rd column may be 0.5. After obtaining the offset values, M sampled feature maps corresponding to the initial feature map can be extracted from M target feature maps based on the offset information. Through this embodiment, convolution is performed on the initial feature map to determine the offset values ​​of each initial feature point in the initial feature map, enabling the extraction of feature points from global feature maps of different depths, thereby facilitating the generation of a feature map with scale information based on the initial feature map.

[0091] In some embodiments, in the above operation S422, extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps according to the offset information of the initial feature map may further include: interpolating the M target feature maps according to the offset information to obtain M interpolated feature maps.

[0092] In this embodiment of the disclosure, interpolation can be performed based on the offset values ​​of multiple initial feature points and multiple target feature points of the target feature map to obtain multiple interpolated feature points. An interpolated feature map can be obtained based on the multiple interpolated feature points and the multiple target feature points. For example, as described above, in the first initial feature map, the offset value of the initial feature point in the 3rd row and 3rd column can be 0.5. Interpolation can be performed on the first target feature map to obtain multiple interpolated feature points. The multiple interpolated features may include the interpolated feature point in the 3.5th row and 3.5th column. An interpolated feature map can be obtained based on the multiple interpolated feature points.

[0093] In this embodiment of the disclosure, based on the offset information of the first initial feature map, interpolation can be performed on the first target feature map, the second target feature map, and the third target feature map respectively to obtain a first interpolated feature map, a second interpolated feature map, and a third interpolated feature map corresponding to the first initial feature map. Based on the offset information of the second initial feature map, interpolation can be performed on the first target feature map, the second target feature map, and the third target feature map respectively to obtain a first interpolated feature map, a second interpolated feature map, and a third interpolated feature map corresponding to the second initial feature map. Based on the offset information of the third initial feature map, interpolation can be performed on the first target feature map, the second target feature map, and the third target feature map respectively to obtain a first interpolated feature map, a second interpolated feature map, and a third interpolated feature map corresponding to the third initial feature map.

[0094] In some embodiments, in the above-described operation S422, extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map may further include: extracting M sampled feature maps corresponding to the initial feature map from the M interpolated feature maps based on the offset information.

[0095] In this embodiment of the disclosure, multiple sampled feature points can be obtained from the interpolated feature map based on the offset values ​​of each initial feature point in the initial feature map. A sampled feature map can then be obtained based on these multiple sampled feature points. For example, in the first initial feature map, the offset value of the initial feature point in the 3rd row and 3rd column is 0.5. The interpolated feature point in the 3.5th row and 3.5th column can be obtained from the first interpolated feature map as a sampled feature point.

[0096] In this embodiment of the disclosure, the size of the M sampled feature maps corresponding to the initial feature map is the same as the size of the initial feature map.

[0097] For example, based on the offset information of the first initial feature map, the first interpolated feature map, the second interpolated feature map, and the third interpolated feature map corresponding to the first initial feature map can be extracted respectively to obtain the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the first initial feature map. If the size of the first initial feature map is 20×20, the size of the corresponding first sampled feature map can be 20×20, the size of the corresponding second sampled feature map can be 20×20, and the size of the corresponding third sampled feature map can be 20×20.

[0098] For example, based on the offset information of the second initial feature map, the first interpolated feature map, the second interpolated feature map, and the third interpolated feature map corresponding to the second initial feature map can be extracted respectively to obtain the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the second initial feature map. If the size of the second initial feature map is 40×40, the size of the corresponding first sampled feature map can be 40×40, the size of the corresponding second sampled feature map can be 40×40, and the size of the corresponding third sampled feature map can be 40×40.

[0099] For example, based on the offset information of the third initial feature map, the first interpolated feature map, the second interpolated feature map, and the third interpolated feature map corresponding to the third initial feature map can be extracted respectively to obtain the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the third initial feature map. If the size of the third initial feature map is 80×80, the size of the corresponding first sampled feature map can be 80×80, the size of the corresponding second sampled feature map can be 80×80, and the size of the corresponding third sampled feature map can be 80×80. Through the embodiments of this disclosure, interpolation is performed on the target feature map based on the offset information of the initial feature map, which can effectively extract feature points from target feature maps of different depths, and helps to generate feature maps with scale information.

[0100] In some embodiments, in operation S423 described above, fusing the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps may include: weighting and fusing the M sampled feature maps according to their respective sample weights to obtain a fused feature map. For example, for a first initial feature map, according to the sample weights of the M sampled feature maps, the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the first initial feature map can be fused to obtain a first fused feature map. For example, for a second initial feature map, according to the sample weights of the M sampled feature maps, the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the second initial feature map can be fused to obtain a second fused feature map. For example, for a third initial feature map, according to the sample weights of the M sampled feature maps, the first sampled feature map, the second sampled feature map, and the third sampled feature map corresponding to the third initial feature map can be fused to obtain a third fused feature map. Through the embodiments of this disclosure, by fusing multiple sampled feature maps sampled from target feature maps at different depths according to their sample weights, the fused feature map can have scale information for target detection.

[0101] In some embodiments, during operation S430 described above, the first fused feature map, the second fused feature map, and the third fused feature map can be input into the detection network to obtain a detection result. For example, the detection network can be a pyramid attention network. Through embodiments of this disclosure, multiple fused feature maps have different scale information. Target detection based on multiple fused feature maps can yield more accurate detection results.

[0102] It is understood that the above description uses N=3 and M=3 as examples to illustrate the method of this disclosure, but this disclosure is not limited to this. N can be any integer greater than 1, and M can also be any integer greater than 1.

[0103] As you can understand, the above has described the object detection model disclosed herein, and the following will further explain some methods for training the object detection model.

[0104] In some embodiments, during operation S440, training the object detection model based on the detection results and the labels of the sample images may include adjusting the parameters of the convolutional units to train the object detection model. For example, as described above, the convolutional units can convolve the initial feature map and output the offset information of the initial feature map. The parameters of the convolutional units can be adjusted to converge the total loss described above to train the object detection model. Through the embodiments of this disclosure, the parameters of the convolutional units can be adjusted so that the convolutional units can output more accurate offset values, which helps to extract feature points more accurately from target feature maps of different depths and helps to generate more accurate feature maps with scale information based on the initial feature map.

[0105] In some embodiments, training the object detection model based on the detection results and the labels of the sample images may further include: adjusting the sample weights of the M sampled feature maps according to the detection results and the labels of the sample images to train the object detection model. For example, as described above, when fusing the M sampled feature maps, weighted fusion can be performed based on the sample weights. The sample weights can be adjusted to make the total loss converge to train the object detection model. Through the embodiments of this disclosure, the sample weights can be adjusted, and the sampled feature maps can be fused more accurately, which helps to generate more accurate feature maps with scale information based on the initial feature maps.

[0106] For example, the sample weights and parameters of the convolutional units can be adjusted to make the object detection model converge.

[0107] In some embodiments, the aforementioned feature extraction network comprising multiple Transformers can be used as the backbone network of the You Look Only Once (YOLO) model. The detection network of the YOLO model can be a pyramid attention network. The aforementioned feature generation network is deployed between the backbone network and the detection network of the YOLO model. After 30 epochs of iterative training, the mean average precision (mAP) of the YOLO model can reach 52%.

[0108] Figure 5 This is a block diagram of a target detection apparatus according to an embodiment of the present disclosure.

[0109] like Figure 5 As shown, the device 500 may include a first acquisition module 510, an extraction module 520, a fusion module 530, and a target detection module 540.

[0110] The first obtaining module 510 is used to obtain N initial feature maps based on the first target feature map in the M target feature maps of the target image. For example, the N initial feature maps have different sizes, N is an integer greater than 1, and M is an integer greater than 1.

[0111] The extraction module 520 is used to extract M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map.

[0112] The fusion module 530 is used to fuse the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps.

[0113] The target detection module 540 is used to perform target detection based on N fused feature maps to obtain the detection result of the target image.

[0114] In some embodiments, the first obtaining module includes: a feature extraction submodule, used to perform multi-level feature extraction on the target image to obtain M target feature maps; and a sampling submodule, used to sample the first target feature maps N times to obtain N initial feature maps.

[0115] In some embodiments, the extraction module includes: a convolution submodule, configured to convolve the initial feature map to obtain offset information of the initial feature map, wherein the offset information includes the offset values ​​of multiple initial feature points in the initial feature map; and a first extraction submodule, configured to extract M sampled feature maps corresponding to the initial feature map from M target feature maps based on the offset information.

[0116] In some embodiments, the extraction module includes: an interpolation submodule, configured to interpolate M target feature maps according to offset information to obtain M interpolated feature maps; and a second extraction submodule, configured to extract M sampled feature maps corresponding to the initial feature map from the M interpolated feature maps according to offset information, wherein the size of the M sampled feature maps corresponding to the initial feature map is the same as the size of the initial feature map.

[0117] In some embodiments, the fusion module includes a weighted fusion submodule, which is used to perform weighted fusion of the M sampled features according to the preset weights of each of the M sampled feature maps to obtain a fused feature map.

[0118] In some embodiments, the detection result includes a detection bounding box of a target object in the target image and the category of the target object, wherein the detection bounding box is used to indicate the location of the target object in the target image.

[0119] Figure 6 This is a block diagram of a training apparatus for an object detection model according to another embodiment of the present disclosure.

[0120] In this embodiment of the disclosure, the target detection model includes a feature extraction network, a feature generation network, and a detection network.

[0121] like Figure 6 As shown, the device 600 may include a second acquisition module 610, an execution module 620, a third acquisition module 630, and a training module 640.

[0122] The second acquisition module 610 is used to input the sample image into the feature extraction network to obtain M target feature maps of the sample image. For example, M is an integer greater than 1.

[0123] The execution module 620 is used to perform relevant operations on M target feature maps through the feature generation network via the following sub-modules to obtain N fused feature maps.

[0124] The submodule 621 is used to obtain N initial feature maps based on the first target feature map in the M target feature maps. For example, the N initial feature maps have different sizes, and N is an integer greater than 1.

[0125] The third extraction submodule 622 is used to extract M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map.

[0126] The fusion submodule 623 is used to fuse the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps.

[0127] The third acquisition module 630 is used to input N fused feature maps into the detection network to obtain the detection results of the sample image.

[0128] Training module 640 is used to train the target detection model based on the detection results and the labels of the sample images.

[0129] In some embodiments, the obtaining submodule includes: a sampling unit, used to sample the first target feature map N times to obtain N initial features.

[0130] In some embodiments, the feature generation network includes a convolutional unit, and the third extraction submodule includes: a first obtaining unit, configured to convolve the initial feature map using the convolutional unit to obtain offset information of the initial feature map, wherein the offset information includes the offset values ​​of multiple initial feature points in the initial feature map; and a first extraction unit, configured to extract M sampled feature maps corresponding to the initial feature map from M target feature maps based on the offset information.

[0131] In some embodiments, the training module includes: a first adjustment submodule, configured to adjust the parameters of the convolutional units based on the detection results and the labels of the sample images, in order to train the target detection model.

[0132] In some embodiments, the third extraction submodule includes: an interpolation unit, configured to interpolate the M target feature maps according to the offset information to obtain M interpolated feature maps; and a second extraction unit, configured to extract M sampled feature maps corresponding to the initial feature map from the M interpolated feature maps according to the offset information, wherein the size of the M sampled feature maps corresponding to the initial feature map is the same as the size of the initial feature map.

[0133] In some embodiments, the fusion submodule includes a weighted fusion unit, configured to perform weighted fusion of the M sampled features according to their respective sample weights to obtain a fused feature map.

[0134] In some embodiments, the training module includes a second training submodule, configured to adjust the sample weights of the M sampled feature maps according to the detection results and the labels of the sample images, in order to train the target detection model.

[0135] In some embodiments, the detection result includes the output detection box of the sample object in the sample image and the output category of the sample object. The output detection box is used to indicate the position of the sample object in the sample image. The label of the sample image includes the labeled detection box of the sample object and the true category of the sample object. The labeled detection box is used to indicate the true position of the sample in the sample image.

[0136] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0137] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0138] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0139] like Figure 7As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0140] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0141] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as object detection methods and / or object detection model training methods. For example, in some embodiments, the object detection methods and / or object detection model training methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the object detection methods and / or object detection model training methods described above can be performed. Alternatively, in other embodiments, the computing unit 701 may be configured by any other suitable means (e.g., by means of firmware) to perform object detection methods and / or training methods for object detection models.

[0142] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0144] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) monitor or an LCD (liquid crystal display)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0146] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0147] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0148] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0149] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A target detection method, comprising: Based on the first target feature map in the M target feature maps of the target image, N initial feature maps are obtained, wherein the sizes of the N initial feature maps are different from each other, N is an integer greater than 1, and M is an integer greater than 1; Based on the offset information of the initial feature map, extract M sampled feature maps corresponding to the initial feature map from the M target feature maps; The M sampled feature maps of each of the N initial feature maps are fused to obtain N fused feature maps; and Target detection is performed based on the N fused feature maps to obtain the detection result of the target image.

2. The method according to claim 1, wherein, The process of obtaining N initial feature maps based on the first target feature map among the M target feature maps of the target image includes: Multi-level feature extraction is performed on the target image to obtain M target feature maps; and The first target feature map is sampled N times to obtain N initial feature maps.

3. The method according to claim 1, wherein, The step of extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map includes: The initial feature map is convolved to obtain the offset information of the initial feature map, wherein the offset information includes the offset values ​​of each of the multiple initial feature points in the initial feature map; Based on the offset information, M sampled feature maps corresponding to the initial feature map are extracted from the M target feature maps.

4. The method according to claim 1, wherein, The step of extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map includes: Based on the offset information, interpolation is performed on the M target feature maps respectively to obtain M interpolated feature maps; Based on the offset information, M sampled feature maps corresponding to the initial feature map are extracted from the M interpolated feature maps, wherein the size of the M sampled feature maps corresponding to the initial feature map is the same as the size of the initial feature map.

5. The method according to claim 1, wherein, The step of fusing the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps includes: Based on the preset weights of the M sampled feature maps, the M sampled features are weighted and fused to obtain the fused feature map.

6. The method according to claim 1, wherein, The detection result includes the detection bounding box of the target object in the target image and the category of the target object. The detection bounding box is used to indicate the position of the target object in the target image.

7. A method for training an object detection model, the object detection model comprising a feature extraction network, a feature generation network, and a detection network, the method comprising: The sample image is input into the feature extraction network to obtain M target feature maps of the sample image, where M is an integer greater than 1; The feature generation network is used to perform the following operations on the M target feature maps to obtain N fused feature maps: Based on the first target feature map in the M target feature maps, N initial feature maps are obtained, wherein the sizes of the N initial feature maps are different from each other, and N is an integer greater than 1; Based on the offset information of the initial feature map, extract M sampled feature maps corresponding to the initial feature map from the M target feature maps; The M sampled feature maps of each of the N initial feature maps are fused to obtain N fused feature maps; The N fused feature maps are input into the detection network to obtain the detection results of the sample image; and The target detection model is trained based on the detection results and the labels of the sample images.

8. The method according to claim 7, wherein, The step of obtaining N initial feature maps based on the first target feature map in the M target feature maps includes: The first target feature map is sampled N times to obtain N initial features.

9. The method according to claim 7, wherein, The feature generation network includes convolutional units. The step of extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map includes: The initial feature map is convolved using the convolutional unit to obtain the offset information of the initial feature map, wherein the offset information includes the offset values ​​of each of the multiple initial feature points in the initial feature map; Based on the offset information, M sampled feature maps corresponding to the initial feature map are extracted from the M target feature maps.

10. The method according to claim 9, wherein, Training the target detection model based on the detection results and the labels of the sample images includes: Based on the detection results and the labels of the sample images, the parameters of the convolutional units are adjusted to train the target detection model.

11. The method according to claim 7, wherein, The step of extracting M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map includes: Based on the offset information, interpolation is performed on the M target feature maps respectively to obtain M interpolated feature maps; Based on the offset information, M sampled feature maps corresponding to the initial feature map are extracted from the M interpolated feature maps, wherein the size of the M sampled feature maps corresponding to the initial feature map is the same as the size of the initial feature map.

12. The method according to claim 7, wherein, The step of fusing the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps includes: Based on the sample weights of the M sampled feature maps, the M sampled features are weighted and fused to obtain the fused feature map.

13. The method according to claim 12, wherein, Training the target detection model based on the detection results and the labels of the sample images includes: Based on the detection results and the labels of the sample images, the sample weights of the M sampled feature maps are adjusted to train the target detection model.

14. The method according to claim 7, wherein, The detection result includes the output detection bounding box of the sample object in the sample image and the output category of the sample object. The output detection bounding box is used to indicate the position of the sample object in the sample image. The label of the sample image includes the bounding box of the sample object and the true category of the sample object. The bounding box is used to indicate the true location of the sample in the sample image.

15. A target detection device, comprising: The first obtaining module is used to obtain N initial feature maps based on the first target feature map in M ​​target feature maps of the target image, wherein the size of the N initial feature maps is different from each other, N is an integer greater than 1, and M is an integer greater than 1; The extraction module is used to extract M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map; A fusion module is used to fuse the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps; and The target detection module is used to perform target detection based on the N fused feature maps to obtain the detection result of the target image.

16. A training apparatus for an object detection model, the object detection model comprising a feature extraction network, a feature generation network, and a detection network, the apparatus comprising: The second acquisition module is used to input the sample image into the feature extraction network to obtain M target feature maps of the sample image, where M is an integer greater than 1; The execution module is used to perform relevant operations on M target feature maps using the feature generation network through the following sub-modules to obtain N fused feature maps: The submodule is used to obtain N initial feature maps based on the first target feature map in the M target feature maps, wherein the size of the N initial feature maps is different from each other, and N is an integer greater than 1; The third extraction submodule is used to extract M sampled feature maps corresponding to the initial feature map from the M target feature maps based on the offset information of the initial feature map; The fusion submodule is used to fuse the M sampled feature maps of each of the N initial feature maps to obtain N fused feature maps; The third acquisition module is used to input the N fused feature maps into the detection network to obtain the detection result of the sample image; and The training module is used to train the target detection model based on the detection results and the labels of the sample images.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 14.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 14.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 14.

Images