Object detection method and related device

Abstract

The embodiment of the present application discloses an object detection method and related devices, the method comprising: obtaining a feature map set based on an acquired image to be detected; then performing primary fusion according to the feature map set to obtain a primary fused feature map; and performing hierarchical feature fusion operation according to the primary fused feature map to obtain multiple data to be detected; finally, performing object detection based on the multiple data to be detected to obtain a detection result. In the embodiment of the present application, after the image to be detected is subjected to feature extraction, the obtained multiple human feature maps are further subjected to primary fusion and hierarchical feature fusion to extract the deep and shallow features in the image to be detected, thereby improving the detection accuracy of the target object in the image to be detected.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an object detection method and related devices. Background Art

[0002] Face detection is widely used in access control machines and video surveillance. For video surveillance scenes, due to the large changes in depth of field and the huge scale of face changes, detection is difficult. At present, traditional face detection methods usually predict the corresponding measurement target based on feature maps at different scales, but this method is prone to problems such as missed detection and false detection. Summary of the invention

[0003] The embodiments of the present application provide an object detection method and related devices to improve the accuracy of face detection.

[0004] In a first aspect, an embodiment of the present application provides an object detection method, comprising:

[0005] Obtain a feature map set based on the acquired image to be detected;

[0006] Performing primary fusion based on the feature map set to obtain a primary fused feature map;

[0007] Based on the primary fusion feature map, a hierarchical feature fusion operation is performed to obtain a plurality of data to be detected, wherein the hierarchical feature fusion operation refers to an operation of extracting feature data at different levels and fusing feature data at different levels;

[0008] Object detection is performed based on the multiple data to be detected to obtain a detection result.

[0009] In a second aspect, an embodiment of the present application provides an object detection device,

[0010] It includes a processing unit and a communication unit, wherein:

[0011] The processing unit is used to obtain a feature map set based on the image to be detected acquired by the communication unit; and to perform primary fusion based on the feature map set to obtain a primary fused feature map; and to perform a hierarchical feature fusion operation based on the primary fused feature map to obtain multiple data to be detected, wherein the hierarchical feature fusion operation refers to the operation of extracting feature data at different levels and fusing feature data at different levels; and to perform object detection based on the multiple data to be detected to obtain detection results.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps of any method in the first aspect of the embodiment of the present application.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiment of the present application. The computer program product may be a software installation package.

[0015] It can be seen that in the embodiment of the present application, the electronic device obtains a feature map set based on the acquired image to be detected; then performs primary fusion according to the feature map set to obtain a primary fused feature map; and performs a hierarchical feature fusion operation according to the primary fused feature map to obtain multiple data to be detected, and the hierarchical feature fusion operation refers to the operation of extracting feature data of different levels and fusing feature data of different levels; finally, object detection is performed based on the multiple data to be detected to obtain a detection result. In the embodiment of the present application, after the image to be detected is subjected to feature extraction, the obtained multiple human feature maps are further subjected to primary fusion and hierarchical feature fusion to extract the deep and shallow features in the image to be detected, thereby improving the detection accuracy of the target object in the image to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a schematic diagram of an object detection system provided in an embodiment of the present application;

[0018] Figure 2a is a flow chart of an object detection method provided in an embodiment of the present application;

[0019] Figure 2b It is a schematic diagram of a fusion process provided in an embodiment of the present application;

[0020] Figure 2c It is a schematic diagram of hierarchical feature fusion provided in an embodiment of the present application;

[0021] Figure 2d is a schematic diagram of an object detection framework provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0023] Figure 4 A block diagram of the functional units of an object detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] like Figure 1 As shown, Figure 1 The schematic diagram of an object detection system 100 is shown in FIG. 1 . The object detection system 100 includes an image acquisition device 110 and an image processing device 120. The image acquisition device 110 is connected to the image processing device 120. The image acquisition device 110 is used to acquire image data and send it to the image processing device 120 for processing. The image processing device 120 is used to process the image data and output the processing result. The object detection system 100 may include an integrated single device or multiple devices. For the convenience of description, the object detection system 100 is collectively referred to as an electronic device in this application. Obviously, the electronic device may include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem, as well as various forms of user equipment (UE), mobile stations (MS), terminal devices, etc.

[0026] Face detection is widely used in access control machines and video surveillance. For video surveillance scenes, due to the large change in depth of field and the huge change in the scale of face, detection is difficult. The current technical solution is usually to predict the corresponding measurement target with feature maps at different scales, but this approach is somewhat unreasonable and prone to problems such as missed detection and false detection. In surveillance scenarios, due to the large change in depth of field, the scale of the face changes greatly. In order to achieve multi-scale face detection, face detection is usually performed on feature maps at different scales. For example, in yolov3 (a detector), taking 416*416 input as an example, large-size faces are detected on 13*13 feature images; medium-size faces are detected on 26*26 feature images; and small-size faces are detected on 52*52 feature images. In this processing method, since the candidate box (Anchor box) is only related to the last convolutional layer, its receptive field is small, which easily leads to problems such as missed detection and false detection.

[0027] Based on this, an embodiment of the present application proposes an object detection method to solve the above-mentioned problem, and the embodiment of the present application is introduced in detail below.

[0028] See also Figure 2a , Figure 2a The present application provides a flow chart of an object detection method, which is applied to Figure 1 The electronic device shown in the figure, as shown in the figure, the object detection method includes:

[0029] 201: The electronic device obtains a feature map set based on the acquired image to be detected.

[0030] In the embodiment of the present application, the electronic device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the image processing method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0031] In a specific implementation, a convolutional network can be used to extract features from the image to be detected to obtain a fixed-scale feature map (i.e., a feature map with the largest scale), and then the fixed-scale feature map is downsampled to obtain a smaller-scale feature map, and then the smaller-scale feature map is downsampled to obtain an even smaller-scale feature map, and this cycle is repeated until a preset number of feature maps are obtained, which is equivalent to a large-scale feature map representing shallow-level features and a small-scale feature map representing deep-level features. The downsampling operation here may include convolution and / or pooling operations.

[0032] For example, assuming that the preset number is 3, that is, it includes 3 feature maps. After the image to be detected is subjected to feature extraction through the convolutional network, a first-level feature map with a scale of 52*52*256 is obtained, and then the first-level feature map is subjected to convolution and pooling operations to obtain a second-level feature map with a scale of 26*26*512, and then the second-level feature map is subjected to the above convolution and pooling operations to obtain a third-level feature map with a scale of 13*13*1024. Among them, the last digit of the scale represents the number of channels, such as 256 in the scale of the first-level feature map above represents 256 channels.

[0033] 202: The electronic device performs primary fusion based on the feature map set to obtain a primary fused feature map.

[0034] The above-mentioned primary fusion refers to the process of fusing a set of feature maps into one feature map, which is equivalent to the process of fusing deep features with shallow features.

[0035] In a specific implementation, the scale of the feature map set can be made the same through scale transformation, and then a plurality of feature maps with the same scale are added together to obtain the primary fused feature map.

[0036] 203: The electronic device performs a hierarchical feature fusion operation based on the primary fusion feature map to obtain a plurality of data to be detected, wherein the hierarchical feature fusion operation refers to an operation of extracting feature data at different levels and fusing feature data at different levels.

[0037] The hierarchical feature fusion operation is a process of performing multiple feature extractions and fusions step by step based on the primary fusion feature map to obtain data to be detected containing features at different levels.

[0038] For example, assume that the multiple data to be detected include two levels of data to be detected, namely the first data to be detected and the second data to be detected. Then the first data to be detected is: the detection data obtained after further feature extraction of the primary fusion feature map; the second data to be detected is: the detection data obtained after further feature extraction of the first data to be detected and the primary fusion feature map. Therefore, the second data to be detected contains deeper and more comprehensive feature information than the first data to be detected. That is, the more times the step-by-step feature extraction and fusion are performed, the more detailed and accurate the feature information contained in the obtained detection data, so that objects with smaller scales in the image to be detected can be detected.

[0039] 204: The electronic device performs object detection based on the multiple data to be detected to obtain a detection result.

[0040] In a specific implementation, after the multiple data to be detected are obtained, the multiple data to be detected are used respectively to perform object detection to obtain an object detection result corresponding to each data to be detected, and then the obtained detection results are merged to obtain a final detection result.

[0041] For example, the above object detection is to detect the face in the image to be detected. After obtaining the multiple data to be detected, the multiple data to be detected are used to perform face detection respectively to obtain the face detection results corresponding to each data to be detected. Since the feature levels contained in each data to be detected are different, the detection results corresponding to each detection data may be different, that is, the detection results of the detection data containing only shallow features may miss the detection of faces with smaller scales; and for the data to be detected that contains more deep features after multiple feature extractions, the number of shallow features contained is reduced due to the increase in the number of feature extractions, which leads to missed detection of large-scale face objects during face detection. Therefore, it is necessary to merge the detection results corresponding to the multiple data to be detected in order to obtain more accurate detection results.

[0042] It can be seen that in the embodiment of the present application, the electronic device obtains a feature map set based on the acquired image to be detected; then performs primary fusion according to the feature map set to obtain a primary fused feature map; and performs hierarchical feature fusion operation according to the primary fused feature map to obtain multiple data to be detected; finally, object detection is performed based on the multiple data to be detected to obtain a detection result. In the embodiment of the present application, after the image to be detected is subjected to feature extraction, the obtained multiple human feature maps are further subjected to primary fusion and hierarchical feature fusion to extract the deep and shallow features in the image to be detected, thereby improving the detection accuracy of the target object in the image to be detected.

[0043] In a possible example, the feature map set includes N feature maps of different scales, and the primary fusion is performed based on the feature map set to obtain a primary fused feature map, including: transforming the number of channels and the scale of the i-th fused feature map to obtain the i+1-th feature map to be fused, the i+1-th feature map to be fused has the same number of channels and scale as the i+1-th feature map, the i+1-th feature map is a feature map in the feature map set, i is an integer greater than or equal to 1 and less than or equal to N-1, and the 1st to i+1-th feature maps are different feature maps; wherein, in the case where i=1, the i-th fused feature map is the 1st feature map; fusing the i+1-th feature map to be fused and the i+1-th feature map to obtain the i+1-th fused feature map; in the case where i+1=N, determining the i+1-th fused feature map as the primary fused feature map.

[0044] In a specific implementation, the above-mentioned fusion of the i+1th feature map to be fused and the i+1th feature map to obtain the i+1th fused fused feature map means: adding the pixel values ​​of the corresponding positions of the two feature maps with the same scale (that is, the i+1th feature map to be fused and the i+1th feature map) to obtain the fused i+1th fused fused feature map, such as Figure 2b shown.

[0045] In a possible example, transforming the number of channels and the scale of the i-th fused feature map to obtain the i+1-th feature map to be fused includes: convolving the i-th fused feature map to obtain a fused feature map having the same number of channels as the i+1-th feature map; and upsampling the convolved i-th fused feature map to obtain the i+1-th feature map to be fused.

[0046] In a possible example, in the process of obtaining the primary fused feature map, the scales of the first feature map to the Nth feature map can be from small to large, from large to small, or random. That is, the fusion order of the feature maps can be fused according to size from small to large, from large to small, or randomly, that is, as long as all the feature maps are finally fused together to obtain the primary fused feature map, the specific fusion order is not limited in the embodiment of the present application.

[0047] In a specific implementation, the scales of the first feature map to the Nth feature map can be from small to large, and the electronic device performs primary fusion based on the feature map set to obtain a primary fused feature map, including: starting from the feature map with the smallest scale in the feature map set, first transforming the number of channels of the feature map with the smallest scale to be consistent with the number of channels of the feature map with the second smallest scale in the feature map set through 1*1 convolution; then further aligning and performing upsampling operations on corresponding scales so that the length and width of the feature map with the smallest scale are also consistent with the length and width of the feature map with the second smallest scale; then, fusing the feature map with the smallest scale after scale transformation with the feature map with the second smallest scale to obtain a fused feature map, that is, adding the feature map with the smallest scale after scale transformation to the feature map with the second smallest scale; then, performing the same operation on the current fused feature map as the feature map with the smallest scale on the upper scale to obtain a feature map with the scale consistent with the feature map with the third smallest scale, and fusing it with the feature map with the third smallest scale. The above steps are performed in order from small to large scales of the feature maps until the previously obtained fused feature map is scaled to be consistent with the scale of the feature map with the largest scale in the feature map set, and is fused with the feature map with the largest scale to obtain the preliminary fused feature map.

[0048] For example, by extracting features from the image to be detected, three feature maps N1, N2, and N3 with successively reduced scales are obtained, and the scales of N1, N2, and N3 are 52*52*256, 26*26*512, and 13*13*1024, respectively. First, a convolution operation is performed on N3 using a convolution kernel with a channel number of 512 and a scale of 13*13*1024, so that the scale of N3 becomes 13*13*512; then a 2*2 upsampling operation (such as bilinear interpolation) is performed on N3 so that the scale of N3 is transformed to 26*26*512, and then N3 is added to N2 (i.e., the above fusion) to obtain N32; then, a transformation operation similar to N1 is used to transform the scale of N32 to 52*52*256, and finally N32 is added to N1 to obtain the primary fusion feature map N.

[0049] In the embodiment of the present application, during the primary fusion process of the image to be detected, the primary fusion feature map obtained fuses feature maps of different levels, and the feature maps of different levels contain feature information of different depths; therefore, the primary fusion feature map obtained after the primary fusion operation also contains feature information of different depths, effectively reducing the loss of useful information in the feature extraction process. This can further make the subsequent object detection results based on the primary fusion feature map more accurate and improve the quality of object detection.

[0050] In a possible example, the number of the multiple data to be detected is M; the hierarchical feature fusion operation is performed based on the primary fusion feature map to obtain multiple data to be detected, including: fusing the primary fusion feature map and the kth data to be detected to obtain the k+1th fused feature data, where k is an integer greater than or equal to 1 and less than or equal to M-1; wherein, when k=1, the kth data to be detected is obtained by performing feature extraction on the primary fusion feature map; and feature extraction is performed on the k+1th fused feature data to obtain the k+1th data to be detected.

[0051] Wherein, in the process of the electronic device performing feature extraction on the k+1th fused feature data to obtain the k+1th data to be detected, the feature extraction process includes operations such as convolution, pooling and upsampling. Wherein, the upsampling here is to make the scale of the obtained data to be detected meet the requirements of object detection. The fusion of the primary fused feature map and the kth data to be detected to obtain the k+1th fused feature data refers to adding the values ​​of the corresponding positions of the primary fused feature map and the kth data to be detected to obtain the k+1th fused feature data.

[0052] For example, if Figure 2cAs shown, assuming that the preset number is 3, the primary fusion feature is N. First, N is subjected to three convolution operations to extract the features of N, and then three upsampling operations (i.e., the above-mentioned scale transformation) are performed on it to obtain the first data to be detected K1 that is consistent with the scale of the primary fusion feature map; then K1 is added to N, and the result of the addition of K1 and N is subjected to three convolution operations and upsampling operations to obtain the second data to be detected K2; finally, K2 is added to N, and the result of the addition of K2 and N is subjected to three convolution operations and upsampling operations to obtain the third data to be detected K3.

[0053] In object detection, shallow feature information has better detection effects on large-sized objects to be detected, while deep feature information has better detection effects on small-sized objects to be detected. In this solution, since the multiple data to be detected are obtained by extracting and fusing the primary fusion feature map step by step, that is, different data to be detected are obtained through different times of feature extraction and fusion, the more times of feature extraction and fusion, the deeper the feature information contained in the obtained data to be detected. Therefore, the multiple data to be detected obtained through the above-mentioned hierarchical feature fusion operation contain feature information of different levels. Furthermore, after object detection is performed on the multiple data to be detected, it can effectively meet the detection of large-sized objects and the detection of small and medium-sized objects, thereby effectively improving the quality of object detection.

[0054] In one possible example, the electronic device obtains a feature map set based on an input image to be detected, including: the electronic device downsampling the image to be detected to obtain an image to be detected with a preset resolution; and performing feature extraction on the image to be detected with the preset resolution to obtain the feature map set.

[0055] In the embodiment of the present application, in order to reduce the amount of calculation, before extracting features from the image to be detected, the electronic device performs a downsampling operation on the image to be detected to reduce the resolution of the image to be detected, so that the resolution of the image to be detected reaches a preset resolution. Then, a feature extraction operation is performed on the image to be detected with the preset resolution to obtain the feature map set.

[0056] In order to further reduce the amount of calculation, the structure of the feature extraction network can be changed to quickly reduce the feature map and improve the calculation efficiency.

[0057] For example, in order to reduce the size of the feature map more quickly, the first few convolutions of the feature extraction network can be modified. The first convolution layer uses a convolution with a kernel size of 7*7 and a stride of 4; the second convolution layer uses a convolution with a kernel size of 5*5 and a stride of 2; the third and fourth convolution layers use a convolution with a kernel size of 3*3 and a stride of 2; thus, the feature map can be reduced quickly and the amount of calculation can be effectively reduced. In order to further reduce the amount of calculation and quickly increase the depth of the feature map, the first three convolution layers pass the C.RELU activation function.

[0058] Based on the above object detection method, the embodiment of the present application also provides a system framework for object detection, including but not limited to face detection.

[0059] For details, see Figure 2d The object detection framework diagram shown in Figure 2d As shown, first, the image to be detected is preprocessed by a preprocessing module, wherein the preprocessing module can process the resolution of the image to be detected. After being processed by the preprocessing module, an image to be detected that meets the preset resolution can be obtained. Then, the image to be detected that meets the preset resolution is subjected to feature extraction by a feature extraction network module to obtain a first-level feature map, a second-level feature map, and a third-level feature map (feature maps of more levels can also be obtained as needed, such as a five-level feature map, a six-level feature map, etc.). Then, the obtained three-level feature maps are passed through a primary fusion module to obtain a primary fusion feature map. Then, the primary fusion feature map is input into a hierarchical feature fusion module to obtain a plurality of data to be detected. Finally, the obtained multiple data to be detected are subjected to object detection by a detection module to obtain an object detection result.

[0060] The specific implementation process of the feature network extraction module, the primary feature fusion module and the hierarchical feature fusion module can refer to the above Figure 2a-2c The description of the corresponding embodiments will not be repeated here.

[0061] It can be seen that in the embodiment of the present application, after the image to be detected is subjected to feature extraction, the obtained multiple person feature maps are further subjected to primary fusion and hierarchical feature fusion to extract deep and shallow level features in the image to be detected, thereby improving the detection accuracy of the target object in the image to be detected.

[0062] With the above Figure 2a For details on the embodiments shown in the drawings, please refer to Figure 3 , Figure 33 is a structural diagram of an electronic device 300 provided in an embodiment of the present application. As shown in the figure, the electronic device 300 includes an application processor 310, a memory 320, a communication interface 330, and one or more programs 321, wherein the one or more programs 321 are stored in the memory 320 and are configured to be executed by the application processor 310, and the one or more programs 321 include instructions for executing the following steps;

[0063] Based on the acquired image to be detected, a feature map set is obtained, the feature map set includes multiple feature maps of different scales; and based on the feature map set, primary fusion is performed to obtain a primary fused feature map; and based on the primary fused feature map, a hierarchical feature fusion operation is performed to obtain multiple data to be detected; and based on the multiple data to be detected, object detection is performed to obtain a detection result.

[0064] It can be seen that in the embodiment of the present application, the electronic device obtains a feature map set based on the acquired image to be detected, wherein the feature map set includes multiple feature maps of different scales; then performs primary fusion according to the feature map set to obtain a primary fused feature map; and performs hierarchical feature fusion operation according to the primary fused feature map to obtain multiple data to be detected; finally, performs object detection based on the multiple data to be detected to obtain a detection result. In the embodiment of the present application, after the image to be detected is subjected to feature extraction, the obtained multiple human feature maps are further subjected to primary fusion and hierarchical feature fusion to extract deep and shallow features in the image to be detected, thereby improving the detection accuracy of the target object in the image to be detected.

[0065] In one possible example, the feature map set includes N feature maps of different scales. In terms of performing primary fusion based on the feature map set to obtain a primary fused feature map, the instructions in the program are specifically used to perform the following operations: transform the number of channels and scale of the i-th fused feature map to obtain the i+1-th feature map to be fused, the i+1-th feature map to be fused has the same number of channels and scale as the i+1-th feature map, the i+1-th feature map is a feature map in the feature map set, i is an integer greater than or equal to 1 and less than or equal to N-1, and the 1st to i+1-th feature maps are different feature maps; wherein, in the case where i=1 is shown, the i-th fused feature map is the 1st feature map; the i+1-th feature map to be fused and the i+1-th feature map are fused to obtain the i+1-th fused feature map; the N-th fused feature map is determined as the primary fused feature map.

[0066] In one possible example, in terms of transforming the number of channels and the scale of the i-th fused feature map to obtain the i+1-th feature map to be fused, the instructions in the program are specifically used to perform the following operations: convolve the i-th fused feature map to obtain a fused feature map having the same number of channels as the i+1-th feature map; upsample the i-th fused feature map after the convolution to obtain the i+1-th feature map to be fused.

[0067] In one possible example, the number of the multiple data to be detected is M; in terms of performing a hierarchical feature fusion operation based on the primary fusion feature map to obtain multiple data to be detected, the instructions in the program are specifically used to perform the following operations: fuse the primary fusion feature map and the kth data to be detected to obtain the k+1th fused feature data, where k is an integer greater than or equal to 1 and less than or equal to M-1; wherein, when k=1, the kth data to be detected is obtained by performing feature extraction on the primary fusion feature map; and feature extraction is performed on the k+1th fused feature data to obtain the k+1th data to be detected.

[0068] In one possible example, in terms of obtaining a feature map set based on the input image to be detected, the instructions in the program are specifically used to perform the following operations: downsampling the image to be detected to obtain an image to be detected with a preset resolution; and performing feature extraction on the image to be detected with a preset resolution to obtain the feature map set.

[0069] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0070] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0071] Figure 4 4 is a functional unit block diagram of an object detection device 400 involved in an embodiment of the present application. The object detection device 400 is applied to an electronic device, and the electronic device includes a processing unit 401 and a communication unit 402, wherein:

[0072] The processing unit 401 is used to obtain a feature map set based on the image to be detected acquired by the communication unit 402, wherein the feature map set includes multiple feature maps of different scales; and, to perform primary fusion based on the feature map set to obtain a primary fused feature map; and, to perform a hierarchical feature fusion operation based on the primary fused feature map to obtain multiple data to be detected; and, to perform object detection based on the multiple data to be detected to obtain a detection result.

[0073] The object detection device 400 may further include a storage unit 403 for storing program codes and data of the electronic device. The processing unit 401 may be a processor, the communication unit 402 may be an internal communication interface, and the storage unit 403 may be a memory.

[0074] It can be seen that in the embodiment of the present application, the electronic device obtains a feature map set based on the acquired image to be detected, wherein the feature map set includes multiple feature maps of different scales; then performs primary fusion according to the feature map set to obtain a primary fused feature map; and performs hierarchical feature fusion operation according to the primary fused feature map to obtain multiple data to be detected; finally, performs object detection based on the multiple data to be detected to obtain a detection result. In the embodiment of the present application, after the image to be detected is subjected to feature extraction, the obtained multiple human feature maps are further subjected to primary fusion and hierarchical feature fusion to extract deep and shallow features in the image to be detected, thereby improving the detection accuracy of the target object in the image to be detected.

[0075] In a possible example, the feature map set includes N feature maps of different scales. In the aspect of performing primary fusion based on the feature map set to obtain a primary fused feature map, the processing unit 401 is specifically used to: determine the 1st feature map as the 1st fused feature map, and the 1st feature map is a feature map in the feature map set; transform the number of channels and the scale of the i-th fused feature map to obtain the i+1th feature map to be fused, the i+1th feature map to be fused has the same number of channels and scale as the i+1th feature map, and the i+1th feature map is a feature map in the feature map set, i is an integer greater than or equal to 1 and less than or equal to N-1, and the 1st to i+1th feature maps are different feature maps; fuse the i+1th feature map to be fused and the i+1th feature map to obtain the i+1th fused feature map; determine the Nth fused feature map as the primary fused feature map.

[0076] In a possible example, in terms of transforming the number of channels and the scale of the i-th fused feature map to obtain the i+1-th feature map to be fused, the processing unit 401 is specifically used to: convolve the i-th fused feature map to obtain a fused feature map with the same number of channels as the i+1-th feature map; upsample the i-th fused feature map after the convolution to obtain the i+1-th feature map to be fused.

[0077] In a possible example, the number of the multiple data to be detected is M; in terms of performing a hierarchical feature fusion operation based on the primary fused feature map to obtain multiple data to be detected, the processing unit 401 is specifically used to: perform feature extraction on the primary fused feature map to obtain the first data to be detected; fuse the primary fused feature map and the kth data to be detected to obtain the k+1th fused feature data, the kth data to be detected is obtained by performing feature extraction on the kth fused feature data, and k is an integer greater than or equal to 1 and less than or equal to M-1; perform feature extraction on the k+1th fused feature data to obtain the k+1th data to be detected.

[0078] In a possible example, in terms of obtaining a feature map set based on the input image to be detected, the processing unit 401 is specifically used to: downsample the image to be detected to obtain an image to be detected with a preset resolution; and perform feature extraction on the image to be detected with the preset resolution to obtain the feature map set.

[0079] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0080] The embodiment of the present application also provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes an electronic device.

[0081] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0082] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0083] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0084] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0086] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0087] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0088] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An object detection method, characterized in that: include: Obtaining a feature map set based on the acquired image to be detected, wherein the feature map set includes N feature maps of different scales; Performing primary fusion based on the feature map set to obtain a primary fused feature map, including: transforming the number of channels and scale of the i-th fused feature map to obtain the i+1-th feature map to be fused, the i+1-th feature map to be fused has the same number of channels and scale as the i+1-th feature map, the i+1-th feature map is a feature map in the feature map set, the i is an integer greater than or equal to 1 and less than or equal to N-1, and the 1st to i+1-th feature maps are different feature maps; wherein, in the case where i=1, the i-th fused feature map is the 1st feature map; fusing the i+1-th feature map to be fused and the i+1-th feature map to obtain the i+1-th fused feature map; in the case where i+1=N, determining the i+1-th fused feature map as the primary fused feature map; Based on the primary fusion feature map, a hierarchical feature fusion operation is performed to obtain multiple data to be detected, and the number of the multiple data to be detected is M; the hierarchical feature fusion operation refers to the operation of extracting feature data of different levels and fusing feature data of different levels, including: fusing the primary fusion feature map and the kth data to be detected to obtain k+1th fused feature data, where k is an integer greater than or equal to 1 and less than or equal to M-1; wherein, in the case of k=1, the kth data to be detected is obtained by performing feature extraction on the primary fusion feature map; performing feature extraction on the k+1th fused feature data to obtain the k+1th data to be detected; Object detection is performed based on the multiple data to be detected to obtain a detection result.

2. The method according to claim 1, characterized in that The transforming the number of channels and the scale of the i-th fused feature map to obtain the i+1-th feature map to be fused includes: Convolving the i-th fused feature map to obtain a fused feature map having the same number of channels as the i+1-th feature map; The i-th fused feature map after convolution is upsampled to obtain the i+1-th feature map to be fused.

3. The method according to claim 1 or 2, characterized in that: The step of obtaining a feature map set based on the input image to be detected includes: Downsampling the image to be detected to obtain an image to be detected with a preset resolution; Feature extraction is performed on the image to be detected with the preset resolution to obtain the feature map set.

4. An object detection device, characterized in that, It includes a processing unit and a communication unit, wherein: The processing unit is used to obtain a feature map set based on the image to be detected acquired by the communication unit, wherein the feature map set includes N feature maps of different scales; and, based on the feature map set, primary fusion is performed to obtain a primary fused feature map; and, based on the primary fused feature map, a hierarchical feature fusion operation is performed to obtain a plurality of data to be detected, wherein the number of the plurality of data to be detected is M, and the hierarchical feature fusion operation refers to an operation of extracting feature data of different levels and fusing feature data of different levels; and, based on the plurality of data to be detected, object detection is performed to obtain a detection result; In the aspect of performing primary fusion based on the feature map set to obtain a primary fused feature map, the processing unit is specifically used to: determine the first feature map as the first fused feature map, the first feature map is a feature map in the feature map set; transform the number of channels and scale of the i-th fused feature map to obtain the i+1-th feature map to be fused, the i+1-th feature map to be fused has the same number of channels and scale as the i+1-th feature map, the i+1-th feature map is a feature map in the feature map set, the i is an integer greater than or equal to 1 and less than or equal to N-1, and the 1st to i+1-th feature maps are different feature maps; fuse the i+1-th feature map to be fused and the i+1-th feature map to obtain the i+1-th fused feature map; determine the N-th fused feature map as the primary fused feature map; In terms of performing a hierarchical feature fusion operation based on the primary fusion feature map to obtain multiple data to be detected, the processing unit is specifically used to: perform feature extraction on the primary fusion feature map to obtain the first data to be detected; fuse the primary fusion feature map and the kth data to be detected to obtain the k+1th fused feature data, where the kth data to be detected is obtained by performing feature extraction on the kth fused feature data, and k is an integer greater than or equal to 1 and less than or equal to M-1; perform feature extraction on the k+1th fused feature data to obtain the k+1th data to be detected.

5. An electronic device, characterized in that: The method comprises a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 3.

Images