A target detection method and system

Abstract

The present invention discloses a target detection method and system, which extracts multi-scale features from an image to be detected, and fuses features of multiple scales to obtain multiple fused feature maps; performs target detection processing on the multiple fused feature maps to obtain a target detection result, and the target detection result includes location information and classification information of at least one target. A target detection method and system of the present invention can realize simultaneous detection of targets of different scales while ensuring detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection method and system. Background Art

[0002] At present, when detecting small objects in images, such as heads and faces, detection is generally based on traditional image features and cascade classifiers, which have low detection accuracy, poor generalization and robustness. There are also detections based on deep learning, but they cannot achieve simultaneous detection of objects of different scales. Summary of the invention

[0003] In order to solve the above problems, an object of the present invention is to provide a target detection method and system, which can realize simultaneous detection of targets of different scales while ensuring detection accuracy.

[0004] The present invention provides a target detection method, comprising:

[0005] Extract multi-scale features from the image to be detected, and fuse the features at multiple scales to obtain multiple fused feature maps;

[0006] The fused multiple feature maps are subjected to target detection processing to obtain a target detection result, wherein the target detection result includes location information and classification information of at least one target.

[0007] As a further improvement of the present invention, multi-scale features are extracted from the image to be detected, and features of multiple scales are fused to obtain multiple fused feature maps, including:

[0008] Down-sampling the image to be detected, extracting multi-scale features, and obtaining first feature maps of multiple scales;

[0009] Upsampling the first feature maps of the multiple scales to obtain multiple second feature maps with the same scales as the first feature maps;

[0010] The first feature map and the second feature map of the same scale are subjected to feature fusion to obtain multiple fused feature maps.

[0011] As a further improvement of the present invention, target detection processing is performed on the fused multiple feature maps to obtain target detection results, including:

[0012] Performing detection processing on the multiple fused feature maps respectively, and correcting the aspect ratio of the detection frame by using the aspect ratio calibration coefficients of multiple sizes, to obtain the position information and classification information of at least one target in each fused feature map; performing size transformation on the position information of at least one target in each fused feature map to restore it to the original image size, to obtain multiple position information of each target on the original image;

[0013] The multiple position information of each target on the original image is sorted from large to small according to the classification information, and deduplication is performed through the NMS algorithm to obtain the target detection result of each target.

[0014] As a further improvement of the present invention, the method is implemented by a neural network, and the method further includes: training the neural network according to a training set.

[0015] As a further improvement of the present invention, the neural network includes: a grouped convolution layer, a deconvolution layer and a fusion layer, and the grouped convolution layer and the deconvolution layer are used to extract multi-scale features from the image to be detected.

[0016] As a further improvement of the present invention, the training set includes multiple sample images, each of the sample images includes annotation information, and the method also includes: determining the width and height calibration ratio coefficients of multiple sizes based on the annotation information of each of the sample images, wherein the width and height calibration ratio coefficients of the multiple sizes are used to correct the aspect ratio of the detection box during the neural network training process and the target detection process.

[0017] As a further improvement of the present invention, the annotation information includes an annotation frame, wherein, according to the annotation information of each of the sample images, the width and height calibration ratio coefficients of multiple sizes are determined, including:

[0018] S11, select k initial target boxes Ci of different clusters in the data set, i∈{1,2,..,k}, where the initial target box Ci represents a detection box surrounded by a width wi and a height hi and a coordinate origin (0,0);

[0019] S21, determining the distance Di from each annotation box Xi to each initial target box Ci, and assigning each annotation box Xi to the cluster represented by the nearest initial target box Ci according to the distance Di, wherein each annotation box Xi represents the annotation box enclosed by the width and height of each sample image in the data set and the coordinate origin (0, 0);

[0020] Where Di = 1-GIOU, GIOU represents the generalized intersection-over-union ratio of the annotation box and the initial target box;

[0021] S31, taking the mean of the width and height of all the labeled boxes Xi assigned to the cluster represented by the initial target box Ci as the width wi of the new target box CI of the cluster / and high / , calculate the width and height difference S between the initial target frame Ci and the new target frame CI, and so on, determine the new target frame of each cluster;

[0022] S41, repeat S21-S31 until the width and height difference S between the initial target frame Ci and the new target frame CI is less than a preset threshold, and determine the width and height calibration ratio coefficients of multiple sizes.

[0023] As a further improvement of the present invention, the method further comprises:

[0024] Determine the minimum enclosing area C of the annotation box and the initial target box;

[0025] Determine the intersection over union (IOU) of the annotation box and the initial target box, where IOU=|A∩B| / |A∪B|, A represents the area of ​​the annotation box, and B represents the area of ​​the initial target box;

[0026] Determine the proportion of the area of ​​the minimum enclosed area C that does not belong to the annotation box and the initial target box to the minimum enclosed area C |C\(A∪B)| / |C|;

[0027] Determine the generalized intersection-over-union (GIOU) of the annotation box and the initial target box, where GIOU=IOU-|C-(A∪B)| / |C|.

[0028] As a further improvement of the present invention, the training set includes an original training set, and the method further includes: performing data enhancement on the original training set to obtain a fused image training set,

[0029] Wherein, training the neural network according to the training set includes:

[0030] Training the neural network according to the original training set, and obtaining a pre-trained neural network when a first training condition is met;

[0031] The pre-trained neural network is trained according to the fused image training set, and when a second training condition is met, a trained neural network is obtained.

[0032] As a further improvement of the present invention, the original training set includes a plurality of sample images, each of the sample images includes annotation information.

[0033] The original training set is subjected to data enhancement to obtain a fused image training set, including:

[0034] Acquire at least one application scene image;

[0035] Fusing each of the application scene images with each of the sample images to obtain a plurality of fused images, and determining the annotation information of the sample images used for the fusion processing as the annotation information of the corresponding fused images;

[0036] The multiple fused images are combined into a fused image training set.

[0037] As a further improvement of the present invention, each of the application scene images is fused with each of the sample images to obtain a plurality of fused images, including:

[0038] Determining the aspect ratio coefficient of each of the sample images, wherein the aspect ratio coefficient includes a width / height coefficient and a height / width coefficient;

[0039] Scaling each of the application scene images in equal proportions of aspect and height according to a target coefficient to obtain each scaled application scene image, wherein the target coefficient is a larger coefficient between the width / height coefficient and the height / width coefficient;

[0040] The scaled application scene images are respectively subjected to weighted fusion processing with the sample images to obtain a plurality of fused images.

[0041] As a further improvement of the present invention, the target includes a human head.

[0042] The present invention also provides a target detection system, comprising:

[0043] The multi-scale feature module is used to extract multi-scale features from the image to be detected and fuse the features of multiple scales to obtain multiple fused feature maps;

[0044] The target detection module is used to perform target detection processing on the fused multiple feature maps to obtain a target detection result, which includes location information and classification information of at least one target.

[0045] As a further improvement of the present invention, the multi-scale feature module is used for:

[0046] Down-sampling the image to be detected, extracting multi-scale features, and obtaining first feature maps of multiple scales;

[0047] Upsampling the first feature maps of the multiple scales to obtain multiple second feature maps with the same scales as the first feature maps;

[0048] The first feature map with the same scale is respectively fused with the second feature map to obtain multiple fused feature maps.

[0049] As a further improvement of the present invention, the target detection module is used for:

[0050] Performing detection processing on the multiple fused feature maps respectively, and correcting the aspect ratio of the detection frame by using the aspect ratio calibration coefficients of multiple sizes, to obtain the position information and classification information of at least one target in each fused feature map; performing size transformation on the position information of at least one target in each fused feature map to restore it to the original image size, to obtain multiple position information of each target on the original image;

[0051] The multiple position information of each target on the original image is sorted from large to small according to the classification information, and deduplication is performed through the NMS algorithm to obtain the target detection result of each target.

[0052] As a further improvement of the present invention, the system is implemented by a neural network, and the system further includes: training the neural network according to a training set.

[0053] As a further improvement of the present invention, the neural network includes: a grouped convolution layer, a deconvolution layer and a fusion layer, and the grouped convolution layer and the deconvolution layer are used to extract multi-scale features from the image to be detected.

[0054] As a further improvement of the present invention, the training set includes multiple sample images, each of the sample images includes annotation information, and the system also includes: determining width and height calibration ratio coefficients of multiple sizes based on the annotation information of each of the sample images, wherein the width and height calibration ratio coefficients of the multiple sizes are used to correct the aspect ratio of the detection box during the neural network training process and the target detection process.

[0055] As a further improvement of the present invention, the annotation information includes an annotation frame, wherein, according to the annotation information of each of the sample images, the width and height calibration ratio coefficients of multiple sizes are determined, including:

[0056] S11, select k initial target boxes Ci of different clusters in the data set, i∈{1,2,..,k}, where the initial target box Ci represents a detection box surrounded by a width wi and a height hi and a coordinate origin (0,0);

[0057] S21, determining the distance Di from each annotation box Xi to each initial target box Ci, and assigning each annotation box Xi to the cluster represented by the nearest initial target box Ci according to the distance Di, wherein each annotation box Xi represents the annotation box enclosed by the width and height of each sample image in the data set and the coordinate origin (0, 0);

[0058] Where Di = 1-GIOU, GIOU represents the generalized intersection-over-union ratio of the annotation box and the initial target box;

[0059] S31, taking the mean of the width and height of all the labeled boxes Xi assigned to the cluster represented by the initial target box Ci as the width wi of the new target box CI of the cluster / and high / , calculate the width and height difference S between the initial target frame Ci and the new target frame CI, and so on, determine the new target frame of each cluster;

[0060] S41, repeat S21-S31 until the width and height difference S between the initial target frame Ci and the new target frame CI is less than a preset threshold, and determine the width and height calibration ratio coefficients of the multiple sizes.

[0061] As a further improvement of the present invention, the system further comprises:

[0062] Determine the minimum enclosing area C of the annotation box and the initial target box;

[0063] Determine the intersection-over-union (IOU) ratio of the annotation box M and the initial target box, where IOU=|A∩B| / |A∪B|, A represents the area of ​​the annotation box, and B represents the area of ​​the initial target box;

[0064] Determine the proportion of the area of ​​the minimum enclosed area C that does not belong to the annotation box and the initial target box to the minimum enclosed area C |C\(A∪B)| / |C|;

[0065] Determine the generalized intersection-over-union (GIOU) of the annotation box and the initial target box, where GIOU=IOU-|C-(A∪B)| / |C|.

[0066] As a further improvement of the present invention, the training set includes an original training set, and the system further includes: performing data enhancement on the original training set to obtain a fused image training set,

[0067] Wherein, training the neural network according to the training set includes:

[0068] Training the neural network according to the original training set, and obtaining a pre-trained neural network when a first training condition is met;

[0069] The pre-trained neural network is trained according to the fused image training set, and when a second training condition is met, a trained neural network is obtained.

[0070] As a further improvement of the present invention, the original training set includes a plurality of sample images, each of the sample images includes annotation information.

[0071] The original training set is subjected to data enhancement to obtain a fused image training set, including:

[0072] Acquire at least one application scene image;

[0073] Fusing each of the application scene images with each of the sample images to obtain a plurality of fused images, and determining the annotation information of the sample images used for the fusion processing as the annotation information of the corresponding fused images;

[0074] The multiple fused images are combined into a fused image training set.

[0075] As a further improvement of the present invention, each of the application scene images is fused with each of the sample images to obtain a plurality of fused images, including:

[0076] Determining the aspect ratio coefficient of each of the sample images, wherein the aspect ratio coefficient includes a width / height coefficient and a height / width coefficient;

[0077] Scaling each of the application scene images in equal proportions of aspect and height according to a target coefficient to obtain each scaled application scene image, wherein the target coefficient is a larger coefficient between the width / height coefficient and the height / width coefficient;

[0078] Performing weighted fusion processing on each of the scaled application scene images and each of the sample images to obtain a plurality of fused images;

[0079] And so on, the multiple fused images are obtained.

[0080] As a further improvement of the present invention, the target includes a human head.

[0081] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the described method.

[0082] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described.

[0083] The beneficial effect of the present invention is that it can realize simultaneous detection of targets of different scales while ensuring detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0085] Figure 1 A schematic diagram of a process of a target detection method according to an exemplary embodiment of the present disclosure;

[0086] Figure 2 for Figure 1 Schematic diagram of the process of S1;

[0087] Figure 3 This is a schematic diagram of target detection at three scales, large, medium and small, according to an exemplary embodiment of the present disclosure;

[0088] Figure 4 A schematic diagram of a neural network according to an exemplary embodiment of the present disclosure;

[0089] Figure 5 It is a schematic diagram of a flow chart of determining width and height calibration ratio coefficients of multiple sizes according to the annotation information of each sample image according to an exemplary embodiment of the present disclosure;

[0090] Figure 6 A schematic diagram of a GIOU according to an exemplary embodiment of the present disclosure;

[0091] Figure 7 This is a schematic diagram of a process of performing data enhancement on an original training set according to an exemplary embodiment of the present disclosure;

[0092] Figure 8 A schematic diagram of a data enhancement result according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0093] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0094] It should be noted that if the embodiments of the present disclosure involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0095] In addition, in the description of the present disclosure, the terms used are only for illustrative purposes and are not intended to limit the scope of the present disclosure. The terms "include" and / or "comprise" are used to specify the existence of the elements, steps, operations and / or components, but do not exclude the existence or addition of one or more other elements, steps, operations and / or components. The terms "first", "second" and the like may be used to describe various elements, do not represent the order, and do not limit these elements. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "multiple" is two and more than two. These terms are only used to distinguish one element from another element. In conjunction with the following drawings, these and / or other aspects become apparent, and it is easier for those of ordinary skill in the art to understand the description of the embodiments described in the present disclosure. The accompanying drawings are used to describe the embodiments described in the present disclosure for illustrative purposes only. Those skilled in the art will easily recognize from the following description that, without departing from the principles described in the present disclosure, alternative embodiments of the structures and methods shown in the present disclosure can be adopted.

[0096] A target detection method according to an embodiment of the present disclosure is as follows: Figure 1 As shown, including:

[0097] S1, extract multi-scale features from the image to be detected, and fuse the features of multiple scales to obtain multiple fused feature maps;

[0098] S2, performing target detection processing on the fused multiple feature maps to obtain a target detection result, wherein the target detection result includes location information and classification information of at least one target.

[0099] In an optional implementation, multi-scale features are extracted from the image to be detected, and features of multiple scales are fused to obtain multiple fused feature maps, such as Figure 2 As shown, including:

[0100] Down-sampling the image to be detected, extracting multi-scale features, and obtaining first feature maps of multiple scales;

[0101] Upsampling the first feature maps of the multiple scales to obtain multiple second feature maps with the same scales as the first feature maps;

[0102] The first feature map and the second feature map of the same scale are subjected to feature fusion to obtain multiple fused feature maps.

[0103] In the prior art, target (such as a human head) detection based on deep learning generally uses a position branch to regress the detection box border after feature extraction, and uses a classification branch to determine whether there is a target (such as a human head) in the detection box. The detection speed of this classification branch is relatively slow. Since an image may contain targets of multiple scales, such as large, medium, and small, the existing single-scale network (such as the FCHD network) cannot detect targets of different scales at the same time. The method disclosed in the present invention can extract and fuse features of different scales in an image to obtain target information of different scales and achieve better detection effects. For example, Figure 3 As shown, the image to be detected is downsampled to obtain first feature maps of large, medium and small scales, and then the first feature maps of the three scales are upsampled to obtain three second feature maps. The first feature maps of large, medium and small scales are fused with the second feature maps of large, medium and small scales, respectively, to obtain three fused feature maps, which are used for large, medium and small scale target detection, respectively.

[0104] Among them, when the first feature maps of the multiple scales are upsampled respectively, the first feature map of the largest scale may not be processed, and the first feature maps of other scales are upsampled. For example, after the small-scale first feature map is upsampled, a medium-scale second feature map is obtained, and after the medium-scale first feature map is upsampled, a large-scale second feature map is obtained. The medium-scale second feature map and the medium-scale first feature map are feature fused, and the large-scale second feature map and the large-scale first feature map are feature fused, so that the reverse fusion of deep features and shallow features can be achieved.

[0105] In an optional implementation, target detection processing is performed on the fused multiple feature maps to obtain a target detection result, including:

[0106] Performing detection processing on the multiple fused feature maps respectively, and correcting the aspect ratio of the detection frame by using the aspect ratio calibration coefficients of multiple sizes, to obtain the position information and classification information of at least one target in each fused feature map; performing size transformation on the position information of at least one target in each fused feature map to restore it to the original image size, to obtain multiple position information of each target on the original image;

[0107] The multiple position information of each target on the original image is sorted from large to small according to the classification information, and deduplication is performed through the NMS algorithm to obtain the target detection result of each target.

[0108] Among them, each fused feature map contains at least one target detection frame (position information) and category (classification information). There may be overlapping detection frames for each target in the multiple fused feature maps. The NMS algorithm is used to deduplicate the detection frames to obtain the unique detection frame with the highest confidence for each target category.

[0109] In an optional implementation, the method may be implemented by a neural network, and the method further comprises: training the neural network according to a training set.

[0110] In an optional implementation, the neural network includes: a grouped convolution layer, a deconvolution layer and a fusion layer, and the grouped convolution layer and the deconvolution layer are used to extract multi-scale features from the image to be detected.

[0111] In the prior art, when performing feature extraction for target detection based on deep learning, feature extraction is generally performed through a convolutional layer (conv2 layer), an activation layer (relu layer), and a maximum pooling layer (maxpool layer), which will result in excessive computational effort during the extraction process, excessive loss of spatial information, slow detection speed, and inaccurate positioning of the detection frame. The method disclosed in the present invention replaces the maxpool layer with a grouped convolutional layer, which can improve the ability to extract spatial information while achieving downsampling, reduce network parameters and computational effort, and improve network detection speed and positioning accuracy. Upsampling is achieved through a deconvolution layer, and reverse fusion of deep features and shallow features is achieved through a fusion layer. Detection is performed separately on multiple fused feature maps to improve the detection effect of targets of different scales, large, medium, and small.

[0112] For example, Figure 4 As shown, three grouped convolutional layers (hereinafter referred to as GConv layers) are used for downsampling to obtain first feature maps of large, medium and small scales. The first feature maps processed by the second GConv layer and the third GConv layer are upsampled by a deconvolution layer (hereinafter referred to as DeConv layer) to obtain a large-scale second feature map and a medium-scale second feature map. The first feature map processed by the first GConv layer and the second feature map processed by the first DeConv layer are fused by a first fusion layer (hereinafter referred to as Concat layer) to perform feature fusion. The first feature map processed by the second GConv layer and the second feature map processed by the second DeConv layer are fused by a second Concat layer. The large-scale feature map fused by the first Concat layer, the medium-scale feature map fused by the second DeConv layer, and the small-scale feature map processed by the third GConv layer are output respectively to achieve large-, medium- and small-scale target detection.

[0113] In an optional embodiment, the training set includes multiple sample images, each of the sample images includes annotation information, and the method further includes: determining width and height calibration ratio coefficients of multiple sizes based on the annotation information of each of the sample images, wherein the width and height calibration ratio coefficients of the multiple sizes are used to correct the aspect ratio of the detection box during the neural network training process and the target detection process.

[0114] By determining the width and height calibration ratio coefficients of multiple sizes through the annotation information of each of the sample images, the positioning accuracy of target detection frames of different scales can be improved during the training of the neural network and when performing target detection on the image to be detected.

[0115] In an optional implementation, the width and height calibration ratio coefficients of the detection frames of multiple sizes are determined according to the annotation information of each of the sample images, such as Figure 5 As shown, including:

[0116] S11, select k initial target boxes i∈{1,2,..,k} of different clusters in the data set, where the initial target box Ci represents a detection box surrounded by a width wi and a height hi and a coordinate origin (0,0);

[0117] S21, determining the distance Di from each annotation box Xi to each initial target box Ci, and assigning each annotation box Xi to the cluster represented by the nearest initial target box Ci according to the distance Di, wherein each annotation box Xi represents the annotation box enclosed by the width and height of each sample image in the data set and the coordinate origin (0, 0);

[0118] Where Di = 1-GIOU, GIOU represents the generalized intersection-over-union ratio of the annotation box and the initial target box;

[0119] S31, taking the mean of the width and height of all the labeled boxes Xi assigned to the cluster represented by the initial target box Ci as the width wi of the new target box CI of the cluster / and high / , calculate the width and height difference S between the initial target frame Ci and the new target frame CI, and so on, determine the new target frame of each cluster;

[0120] S41, repeat S21-S31 until the width and height difference S between the initial target frame Ci and the new target frame CI is less than a preset threshold, and determine the width and height calibration ratio coefficients of the multiple sizes.

[0121] In the prior art, when detecting small targets such as human heads, the anchor frame ratio (width-height calibration ratio coefficient) used is single, generally 1:1. However, when the shooting angle and shooting distance are different, the aspect ratio of the human head is not necessarily 1:1. This makes the detection frame positioning inaccurate when the fixed-ratio width-height calibration ratio coefficient is used to correct the aspect ratio of the detection frame, affecting the detection result. The present disclosure uses the kmeans++ clustering algorithm to generate width-height calibration ratio coefficients of multiple sizes to adapt to targets with different aspect ratios (such as human heads), which can improve the accuracy of target detection in scenes with different shooting angles and different shooting distances.

[0122] In an optional embodiment, the method further includes:

[0123] Determine the minimum enclosing area C of the annotation box and the initial target box;

[0124] Determine the intersection over union (IOU) of the annotation box and the initial target box, where IOU=|A∩B| / |A∪B|, A represents the area of ​​the annotation box, and B represents the area of ​​the initial target box;

[0125] Determine the proportion of the area of ​​the minimum enclosed area C that does not belong to the annotation box and the initial target box to the minimum enclosed area C |C\(A∪B)| / |C|;

[0126] Determine the generalized intersection-over-union ratio G|OU of the annotation box and the initial target box, where GIOU=IOU-|C-(A∪B)| / |C|.

[0127] like Figure 6 As shown, area A represents the area of ​​the annotation box, area B represents the area of ​​the initial target box, area C represents the minimum closure area of ​​the annotation box and the initial target box, the gray area represents the area of ​​A intersecting B (A∩B), and C\(AUB) represents C minus (the area of ​​A and B). The present disclosure uses GIOU (Generalized Intersection over Union) to measure the overlap between the annotation box and the initial target box. Different from IOU (Intersection over Union) which only focuses on the overlap area between the annotation box and the initial target box, GIOU not only focuses on the overlap area between the annotation box and the initial target box, but also focuses on other non-overlapping areas, which can better reflect the overlap between the annotation box and the initial target box.

[0128] In an optional implementation, the training set includes an original training set, and the method further includes: performing data enhancement on the original training set to obtain a fused image training set,

[0129] Wherein, training the neural network according to the training set includes:

[0130] Training the neural network according to the original training set, and obtaining a pre-trained neural network when a first training condition is met;

[0131] The pre-trained neural network is trained according to the fused image training set, and when a second training condition is met, a trained neural network is obtained.

[0132] In an optional implementation, the original training set includes a plurality of sample images, each of the sample images includes annotation information.

[0133] The original training set is subjected to data enhancement to obtain a fused image training set, including:

[0134] Acquire at least one application scene image;

[0135] Fusing each of the application scene images with each of the sample images to obtain a plurality of fused images, and determining the annotation information of the sample images used for the fusion processing as the annotation information of the corresponding fused images;

[0136] The multiple fused images are combined into a fusion training set.

[0137] When the method described in the present disclosure fuses the application scene image and the sample image, for example, an application scene image can be obtained, and the application scene image and each sample image in the original training set are fused respectively to obtain multiple fused images. For example, an application scene image can be obtained, and the application scene image and some sample images in the original training set are fused respectively to obtain at least one fused image. For example, multiple application scene images can be obtained, and for each application scene image, it is fused with each sample image in the original training set respectively to obtain multiple fused images. For example, multiple application scene images can be obtained, and for each application scene image, it is fused with some sample images in the original training set respectively to obtain at least one fused image. By fusing the application scene images to perform data enhancement on the original training set, the detection performance of the neural network in different application scenarios can be improved.

[0138] In an optional implementation, each of the application scene images is fused with each of the sample images to obtain a plurality of fused images, including:

[0139] Determining the aspect ratio coefficient of each of the sample images, wherein the aspect ratio coefficient includes a width / height coefficient and a height / width coefficient;

[0140] Scaling each of the application scene images in equal proportions of aspect and height according to a target coefficient to obtain each scaled application scene image, wherein the target coefficient is a larger coefficient between the width / height coefficient and the height / width coefficient;

[0141] The scaled application scene images are respectively subjected to weighted fusion processing with the sample images to obtain a plurality of fused images.

[0142] Since the image scenes in the original training set of the network are very different from the actual application scenes, if the scenes in the original training set are used alone to detect the objects in each application scene, the detection accuracy will be reduced. The present disclosure enhances the data by weighted fusion of the images in the original training set and the images in the actual application scene, retaining the annotation information of the sample images, and improving the generalization performance of the network. The present disclosure does not specifically limit the weighting coefficient. For example, Figure 7 As shown in the figure, a sample image Timg and an actual application scene image Simg are weighted fused with different weight coefficients a, and the annotation information of the sample image Timg is retained to obtain multiple different fused images TSimg. When scaling the actual application scene image Simg, the width and height (Tw, Th) of the sample image Timg are used as parameters to calculate the aspect ratio coefficient, and the larger coefficient is selected to scale the application scene image Simg proportionally, and the short side is filled with 0 to obtain the scaled application scene image. The fused image is shown in Figure 8 As shown, the completely black part is obtained by padding the short side with 0 during the scaling process.

[0143] In an optional embodiment, the target includes a human head.

[0144] The method disclosed in the present invention can realize target detection of multiple scales (such as head detection of large, medium and small scales) through multi-scale feature fusion. Compared with classification branch judgment, the real-time detection can be improved, and missed detection and false detection due to different target sizes can be reduced. In the network structure, group convolution can reduce network parameters and calculation amount, improve network feature extraction speed, and improve network detection speed. The aspect ratio of targets of different scales is analyzed, GIOU is used as the metric of the clustering algorithm, and a more suitable width-height calibration ratio coefficient is generated to adapt to targets of different aspect ratios, which can improve the positioning accuracy of the detection frame and improve the network's detection performance for targets at different angles and distances. The sample image and the application scene image are weighted fused at different ratios and the annotation information of the sample image is retained. Transfer learning is performed on the fused data to improve the generalization ability of the network and the actual detection performance.

[0145] An object detection system according to an embodiment of the present disclosure includes:

[0146] The multi-scale feature module is used to extract multi-scale features from the image to be detected and fuse the features of multiple scales to obtain multiple fused feature maps;

[0147] The target detection module is used to perform target detection processing on the fused multiple feature maps to obtain a target detection result, which includes location information and classification information of at least one target.

[0148] In an optional implementation, the multi-scale feature module is used to:

[0149] Down-sampling the image to be detected, extracting multi-scale features, and obtaining first feature maps of multiple scales;

[0150] Upsampling the first feature maps of the multiple scales to obtain multiple second feature maps with the same scales as the first feature maps;

[0151] The first feature map and the second feature map of the same scale are subjected to feature fusion to obtain multiple fused feature maps.

[0152] In the prior art, target detection based on deep learning generally uses the position branch to regress the detection box border after feature extraction, and uses the classification branch to determine whether there is a target (such as a human head) in the detection box. The detection speed of this classification branch is relatively slow. Since an image may contain multiple targets of different sizes, such as large, medium, and small, the existing single-scale network (such as the FCHD network) cannot detect targets of different sizes at the same time. The system disclosed in the present invention can extract and fuse features of different scales in an image to obtain target information of different scales and achieve better detection effects. For example, Figure 3 As shown, the image to be detected is downsampled to obtain multi-feature maps of large, medium and small scales, and then the first feature maps of the three scales are upsampled to obtain three second feature maps. The first feature maps of large, medium and small scales are fused with the second feature maps of large, medium and small scales, respectively, to obtain three fused feature maps, which are used for large, medium and small scale target detection, respectively.

[0153] Among them, when the first feature maps of the multiple scales are upsampled respectively, the first feature map of the largest scale may not be processed, and the first feature maps of other scales are upsampled. For example, after the small-scale first feature map is upsampled, a medium-scale second feature map is obtained, and after the medium-scale first feature map is upsampled, a large-scale second feature map is obtained. The medium-scale second feature map and the medium-scale first feature map are feature fused, and the large-scale second feature map and the large-scale first feature map are feature fused, so that the reverse fusion of deep features and shallow features can be achieved.

[0154] In an optional implementation, the target detection module is used to:

[0155] Performing detection processing on the multiple fused feature maps respectively, and correcting the aspect ratio of the detection frame by using the aspect ratio calibration coefficients of multiple sizes, to obtain the position information and classification information of at least one target in each fused feature map; performing size transformation on the position information of at least one target in each fused feature map to restore it to the original image size, to obtain multiple position information of each target on the original image;

[0156] The multiple position information of each target on the original image is sorted from large to small according to the classification information, and deduplication is performed through the NMS algorithm to obtain the target detection result of each target.

[0157] During target detection, each fused feature map contains at least one target detection frame (position information) and category (classification information). There may be overlapping detection frames for each target in the multiple fused feature maps. By removing duplicates with the NMS algorithm, a unique detection frame with the highest confidence level for each target category can be obtained.

[0158] In an optional implementation, the system is implemented by a neural network, and the system further includes: training the neural network according to a training set.

[0159] In an optional implementation, the neural network includes: a grouped convolution layer, a deconvolution layer and a fusion layer, and the grouped convolution layer and the deconvolution layer are used to extract multi-scale features from the image to be detected.

[0160] In the prior art, when performing feature extraction for target detection based on deep learning, feature extraction is generally performed through a convolutional layer (conv2 layer), an activation layer (relu layer), and a maximum pooling layer (maxpool layer), which results in excessive computational effort and excessive loss of spatial information during the extraction process, resulting in a slow detection speed and inaccurate positioning of the detection frame. The system disclosed in the present invention replaces the maxpool layer with a grouped convolutional layer, which can improve the ability to extract spatial information while achieving downsampling, reduce network parameters and computational effort, and improve network detection speed and positioning accuracy. Upsampling is achieved through a deconvolution layer, and reverse fusion of deep features and shallow features is achieved through a fusion layer. Detection is performed separately on multiple fused feature maps to improve the detection effect of targets of different scales, large, medium, and small.

[0161] For example, Figure 4As shown, three grouped convolutional layers (hereinafter referred to as GConv layers) are used for downsampling to obtain first feature maps of large, medium and small scales. The first feature maps processed by the second GConv layer and the third GConv layer are upsampled by a deconvolution layer (hereinafter referred to as DeConv layer) to obtain a large-scale second feature map and a medium-scale second feature map. The first feature map processed by the first GConv layer and the second feature map processed by the first DeConv layer are fused by a first fusion layer (hereinafter referred to as Concat layer) to perform feature fusion. The first feature map processed by the second GConv layer and the second feature map processed by the second DeConv layer are fused by a second Concat layer. The large-scale feature map fused by the first Concat layer, the medium-scale feature map fused by the second DeConv layer, and the small-scale feature map processed by the third GConv layer are output respectively to achieve large-, medium- and small-scale target detection.

[0162] In an optional embodiment, the training set includes multiple sample images, each of the sample images includes annotation information, and the system further includes: determining width and height calibration ratio coefficients of multiple sizes based on the annotation information of each of the sample images, wherein the width and height calibration ratio coefficients of the multiple sizes are used to correct the aspect ratio of the detection box during the neural network training process and the target detection process.

[0163] By determining the width and height calibration ratio coefficients of multiple sizes through the annotation information of each of the sample images, the positioning accuracy of target detection frames of different scales can be improved during the training of the neural network and when performing target detection on the image to be detected.

[0164] In an optional implementation, determining the width and height calibration ratio coefficients of multiple sizes according to the annotation information of each of the sample images includes:

[0165] S11, select k initial target boxes i∈{1,2,..,k} of different clusters in the data set, where the initial target box Ci represents a detection box surrounded by a width wi and a height hi and a coordinate origin (0,0);

[0166] S21, determining the distance Di from each annotation box Xi to each initial target box Ci, and assigning each annotation box Xi to the cluster represented by the nearest initial target box Ci according to the distance Di, wherein each annotation box Xi represents the annotation box enclosed by the width and height of each sample image in the data set and the coordinate origin (0, 0);

[0167] Where Di = 1-GIOU, GIOU represents the generalized intersection-over-union ratio of the annotation box and the initial target box;

[0168] S31, taking the mean of the width and height of all the labeled boxes Xi assigned to the cluster represented by the initial target box Ci as the width wi of the new target box CI of the cluster / and high / , calculate the width and height difference S between the initial target frame Ci and the new target frame CI, and so on, determine the new target frame of each cluster;

[0169] S41, repeat S21-S31 until the width and height difference S between the initial target frame Ci and the new target frame CI is less than a preset threshold, and determine the width and height calibration ratio coefficients of the multiple sizes.

[0170] In the prior art, when detecting small targets such as human heads, the anchor frame ratio (width-height calibration ratio coefficient) used is single, generally 1:1. However, when the shooting angle and shooting distance are different, the aspect ratio of the human head is not necessarily 1:1. This makes the detection frame positioning inaccurate when the fixed-ratio width-height calibration ratio coefficient is used to correct the aspect ratio of the detection frame, affecting the detection result. The present disclosure uses the kmeans++ clustering algorithm to generate appropriate width-height calibration ratio coefficients of multiple sizes to adapt to targets with different aspect ratios (such as human heads), which can improve the accuracy of target detection in scenes with different shooting angles and different shooting distances.

[0171] In an optional embodiment, the system further includes:

[0172] Determine the minimum enclosing area C of the annotation box and the initial target box;

[0173] Determine the intersection over union (IOU) of the annotation box and the initial target box, where IOU=|A∩B| / |A∪B|, A represents the area of ​​the annotation box, and B represents the area of ​​the initial target box;

[0174] Determine the proportion of the area of ​​the minimum enclosed area C that does not belong to the annotation box and the initial target box to the minimum enclosed area C |C\(A∪B)| / |C|;

[0175] Determine the generalized intersection-over-union (GIOU) of the annotation box and the initial target box, where GIOU=IOU-|C-(A∪B)| / |C|.

[0176] like Figure 6As shown, area A represents the area of ​​the annotation box, area B represents the area of ​​the initial target box, area C represents the minimum closure area of ​​the annotation box and the initial target box, the gray area represents the area of ​​A intersecting B (A∩B), and C\(AUB) represents C minus (the area of ​​A and B). The present disclosure uses GIOU (Generalized Intersection over Union) to measure the overlap between the annotation box and the initial target box. Different from IOU (Intersection over Union) which only focuses on the overlap area between the annotation box and the initial target box, GIOU not only focuses on the overlap area between the annotation box and the initial target box, but also focuses on other non-overlapping areas, which can better reflect the overlap between the annotation box and the initial target box.

[0177] In an optional implementation, the training set includes an original training set, and the system further includes: performing data enhancement on the original training set to obtain a fused image training set,

[0178] Wherein, training the neural network according to the training set includes:

[0179] Training the neural network according to the original training set, and obtaining a pre-trained neural network when a first training condition is met;

[0180] The pre-trained neural network is trained according to the fused image training set, and when a second training condition is met, a trained neural network is obtained.

[0181] In an optional implementation, the original training set includes a plurality of sample images, each of the sample images includes annotation information.

[0182] The original training set is subjected to data enhancement to obtain a fused image training set, including:

[0183] Acquire at least one application scene image;

[0184] Fusing each of the application scene images with each of the sample images to obtain a plurality of fused images, and determining the annotation information of the sample images used for the fusion processing as the annotation information of the corresponding fused images;

[0185] The multiple fused images are combined into a fusion training set.

[0186] When the system described in the present disclosure fuses the application scene image and the sample image, for example, an application scene image can be obtained, and the application scene image and each sample image in the original training set are fused separately to obtain multiple fused images. For example, an application scene image can be obtained, and the application scene image and some sample images in the original training set are fused separately to obtain at least one fused image. For example, multiple application scene images can be obtained, and for each application scene image, it is fused separately with each sample image in the original training set to obtain multiple fused images. For example, multiple application scene images can be obtained, and for each application scene image, it is fused separately with some sample images in the original training set to obtain at least one fused image. By fusing the application scene images to perform data enhancement on the original training set, the detection performance of the neural network in different application scenarios can be improved.

[0187] In an optional implementation, each of the application scene images is fused with each of the sample images to obtain a plurality of fused images, including:

[0188] Determining the aspect ratio coefficient of each of the sample images, wherein the aspect ratio coefficient includes a width / height coefficient and a height / width coefficient;

[0189] Scaling each of the application scene images in equal proportions of aspect and height according to a target coefficient to obtain each scaled application scene image, wherein the target coefficient is a larger coefficient between the width / height coefficient and the height / width coefficient;

[0190] The scaled application scene images are respectively subjected to weighted fusion processing with the sample images to obtain a plurality of fused images;

[0191] Since the image scenes in the original training set of the network are very different from the actual application scenes, if the scenes in the original training set are used alone to detect the objects in each application scene, the detection accuracy will be reduced. The present disclosure enhances the data by weighted fusion of the images in the original training set and the images in the actual application scene, retaining the annotation information of the sample images, and improving the generalization performance of the network. The present disclosure does not specifically limit the weighting coefficient. For example, Figure 7 As shown in the figure, a sample image Timg and an actual application scene image Simg are weighted fused with different weight coefficients a, and the annotation information of the sample image Timg is retained to obtain multiple different fused images TSimg. When scaling the actual application scene image Simg, the width and height (Tw, Th) of the sample training image Timg are used as parameters to calculate the aspect ratio coefficients, and the larger coefficient is selected to scale the application scene image Simg proportionally, and the short side is filled with 0 to obtain the scaled application scene image. The fused image is shown in Figure 8 As shown, the completely black part is obtained by padding the short side with 0 during the scaling process.

[0192] In an optional embodiment, the target includes a human head.

[0193] The system disclosed in the present invention can realize target detection of multiple scales (such as head detection of large, medium and small scales) through multi-scale feature fusion. Compared with classification branch judgment, the real-time detection can be improved, and missed detection and false detection due to different target sizes can be reduced. In the network structure, group convolution can reduce network parameters and calculation amount, improve network feature extraction speed, and improve network detection speed. The aspect ratio of targets of different sizes is analyzed, GIOU is used as the metric of the clustering algorithm, and a more suitable width-height calibration ratio coefficient is generated to adapt to targets of different aspect ratios. The positioning accuracy of the detection frame can be improved, and the network's detection performance for targets at different angles and distances can be improved. The sample image and the application scene image are weighted fused at different ratios and the annotation information of the sample image is retained. Transfer learning is performed on the fused data to improve the generalization ability of the network and the actual detection performance.

[0194] The present disclosure also relates to an electronic device, including a server, a terminal, etc. The electronic device includes: at least one processor; a memory connected to the at least one processor; and a communication component connected to the storage medium, wherein the communication component receives and sends data under the control of the processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the target detection method in the above embodiment.

[0195] In an optional embodiment, the memory is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory, that is, realizing the target detection method.

[0196] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store a list of options, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0197] One or more modules are stored in the memory, and when executed by one or more processors, the target detection method in any of the above method embodiments is executed.

[0198] The above-mentioned product can execute the target detection method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the target detection method provided in the embodiment of the present application.

[0199] The present disclosure also relates to a computer-readable storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above-mentioned target detection method embodiments.

[0200] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0201] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0202] In addition, it will be understood by those skilled in the art that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present disclosure and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0203] It will be appreciated by those skilled in the art that, although the present disclosure has been described with reference to exemplary embodiments, various changes may be made and elements thereof may be replaced with equivalents without departing from the scope of the present disclosure. In addition, many modifications may be made to adapt specific circumstances or materials to the teachings of the present disclosure without departing from the substantive scope of the present disclosure. Therefore, the present disclosure is not limited to the specific embodiments disclosed, but the present disclosure will include all embodiments falling within the scope of the appended claims.

Claims

1. A target detection method, characterized in that: include: Extract multi-scale features from the image to be detected, and fuse the features at multiple scales to obtain multiple fused feature maps; Performing target detection processing on the fused multiple feature maps to obtain a target detection result, wherein the target detection result includes location information and classification information of at least one target; The method is implemented by a neural network, and the method further comprises: training the neural network according to a training set, wherein the training set comprises an original training set; The original training set includes a plurality of sample images, each of which includes annotation information. The method further includes: performing data enhancement on the original training set to obtain a fused image training set, including: acquiring at least one application scene image; fusing each of the application scene images with each of the sample images to obtain a plurality of fused images, and determining the annotation information of the sample images used for fusion processing as the annotation information of the corresponding fused image; and composing the plurality of fused images into a fused image training set; wherein, Each of the application scene images is fused with each of the sample images to obtain a plurality of fused images, including: Determine the aspect ratio coefficient of each of the sample images, wherein the aspect ratio coefficient includes a width / height coefficient and a height / width coefficient; scale the aspect ratio of each of the application scene images according to a target coefficient to obtain scaled application scene images, wherein the target coefficient is the larger coefficient of the width / height coefficient and the height / width coefficient; perform weighted fusion processing on each of the scaled application scene images and each of the sample images to obtain a plurality of fused images; wherein a sample image and an application scene image are weightedly fused with different weighting coefficients to obtain a plurality of fused images.

2. The method of claim 1, wherein: Extract multi-scale features from the image to be detected, and perform feature fusion of multiple scales to obtain multiple fused feature maps, including: Down-sampling the image to be detected, extracting multi-scale features, and obtaining first feature maps of multiple scales; Upsampling the first feature maps of the multiple scales to obtain multiple second feature maps with the same scales as the first feature maps; The first feature map and the second feature map of the same scale are subjected to feature fusion to obtain multiple fused feature maps.

3. The method of claim 1, wherein: Performing target detection processing on the fused multiple feature maps to obtain target detection results, including: Performing detection processing on the multiple fused feature maps respectively, and correcting the aspect ratio of the detection frame by using the aspect ratio calibration coefficients of multiple sizes, to obtain the position information and classification information of at least one target in each fused feature map; performing size transformation on the position information of at least one target in each fused feature map to restore it to the original image size, to obtain multiple position information of each target on the original image; The multiple position information of each target on the original image is sorted from large to small according to the classification information, and deduplication is performed through the NMS algorithm to obtain the target detection result of each target.

4. The method of claim 1, wherein: The neural network comprises: a grouped convolution layer, a deconvolution layer and a fusion layer, wherein the grouped convolution layer and the deconvolution layer are used to extract multi-scale features from an image to be detected.

5. The method of claim 1, wherein: The training set includes a plurality of sample images, each of the sample images includes annotation information, and the method further includes: According to the annotation information of each of the sample images, the aspect ratio coefficients of multiple sizes are determined, wherein the aspect ratio coefficients of the multiple sizes are used to correct the aspect ratio of the detection frame during the neural network training process and the target detection process.

6. The method of claim 5, wherein: The annotation information includes an annotation frame, wherein the width and height calibration ratio coefficients of multiple sizes are determined according to the annotation information of each sample image, including: S11, select k initial target boxes Ci of different clusters in the data set, i∈{1,2,...,k}, where the initial target box Ci represents the detection box surrounded by the width wi and height hi and the coordinate origin (0,0); S21, determining the distance Di from each annotation box Xi to each initial target box Ci, and assigning each annotation box Xi to the cluster represented by the nearest initial target box Ci according to the distance Di, wherein each annotation box Xi represents the annotation box enclosed by the width and height of each sample image in the data set and the coordinate origin (0, 0); Where Di = 1-GIOU, GIOU represents the generalized intersection-over-union ratio of the annotation box and the initial target box; S31, taking the mean of the width and height of all the labeled boxes Xi assigned to the cluster represented by the initial target box Ci as the width wi of the new target box CI of the cluster / and high / , calculate the width and height difference S between the initial target frame Ci and the new target frame CI, and so on, determine the new target frame of each cluster; S41, repeat S21-S31 until the width and height difference S between the initial target frame Ci and the new target frame CI is less than a preset threshold, and determine the width and height calibration ratio coefficients of the multiple sizes.

7. The method of claim 6, wherein: The method further comprises: Determine the minimum enclosing area C of the annotation box and the initial target box; Determine the intersection-over-union (IOU) ratio of the annotation box and the initial target box, where IOU=IA∩B| / IA∪B|, A represents the area of ​​the annotation box, and B represents the area of ​​the initial target box; Determine the proportion of the area of ​​the minimum enclosed area C that does not belong to the annotation box and the initial target box to the minimum enclosed area C |C\(A∪B)| / |C|; Determine the generalized intersection-over-union (GIOU) of the annotation box and the initial target box, where GIOU=IOU-|C-(A∪B)I / |C|.

8. The method of claim 1, wherein: Training the neural network according to the training set includes: Training the neural network according to the original training set, and obtaining a pre-trained neural network when a first training condition is met; The pre-trained neural network is trained according to the fused image training set, and when a second training condition is met, a trained neural network is obtained.

9. The method of claim 1, wherein: The targets include human heads.

10. A target detection system, characterized in that: include: The multi-scale feature module is used to extract multi-scale features from the image to be detected and fuse the features of multiple scales to obtain multiple fused feature maps; A target detection module, used to perform target detection processing on the fused multiple feature maps to obtain a target detection result, wherein the target detection result includes location information and classification information of at least one target; The system is implemented by a neural network, and the system further comprises: training the neural network according to a training set, wherein the training set comprises an original training set; The original training set includes a plurality of sample images, each of which includes annotation information. The system further includes: performing data enhancement on the original training set to obtain a fused image training set, including: acquiring at least one application scene image; fusing each of the application scene images with each of the sample images to obtain a plurality of fused images, and determining the annotation information of the sample images used for fusion processing as the annotation information of the corresponding fused image; and composing the plurality of fused images into a fused image training set; wherein, The application scene images are fused with the sample images to obtain a plurality of fused images, including: Determine the aspect ratio coefficient of each of the sample images, wherein the aspect ratio coefficient includes a width / height coefficient and a height / width coefficient; scale the aspect ratio of each of the application scene images according to a target coefficient to obtain scaled application scene images, wherein the target coefficient is the larger coefficient of the width / height coefficient and the height / width coefficient; perform weighted fusion processing on each of the scaled application scene images and each of the sample images to obtain a plurality of fused images; and so on to obtain the plurality of fused images; wherein a sample image and an application scene image are weightedly fused with different weighting coefficients to obtain a plurality of fused images.

11. The system of claim 10, wherein: The multi-scale feature module is used to: Down-sampling the image to be detected, extracting multi-scale features, and obtaining first feature maps of multiple scales; Upsampling the first feature maps of the multiple scales to obtain multiple second feature maps with the same scales as the first feature maps; The first feature map and the second feature map of the same scale are subjected to feature fusion to obtain multiple fused feature maps.

12. The system of claim 10, wherein: The target detection module is used to: Performing detection processing on the fused multiple feature maps respectively, and correcting the aspect ratio of the detection frame by using aspect-height calibration ratio coefficients of multiple sizes, to obtain position information and classification information of at least one target in each fused feature map; Resizing the position information of at least one target in each fused feature image to restore it to the original image size, so as to obtain multiple position information of each target on the original image; The multiple position information of each target on the original image is sorted from large to small according to the classification information, and deduplication is performed through the NMS algorithm to obtain the target detection result of each target.

13. The system of claim 10, wherein: The neural network comprises: a grouped convolution layer, a deconvolution layer and a fusion layer, wherein the grouped convolution layer and the deconvolution layer are used to extract multi-scale features from an image to be detected.

14. The system of claim 10, wherein: The training set includes a plurality of sample images, each of the sample images includes annotation information, and the system further includes: According to the annotation information of each of the sample images, the aspect ratio coefficients of multiple sizes are determined, wherein the aspect ratio coefficients of the multiple sizes are used to correct the aspect ratio of the detection frame during the neural network training process and the target detection process.

15. The system of claim 14, wherein: The annotation information includes an annotation frame, wherein the width and height calibration ratio coefficients of multiple sizes are determined according to the annotation information of each sample image, including: S11, select k initial target boxes Ci of different clusters in the data set, i∈{1,2,..,k}, where the initial target box Ci represents the detection box surrounded by the width wi and height hi and the coordinate origin (0,0); S21, determining the distance Di from each annotation box Xi to each initial target box Ci, and assigning each annotation box Xi to the cluster represented by the nearest initial target box Ci according to the distance Di, wherein each annotation box Xi represents the annotation box enclosed by the width and height of each sample image in the data set and the coordinate origin (0, 0); Where Di = 1-GIOU, GIOU represents the generalized intersection-over-union ratio of the annotated measurement box and the initial target box; S31, taking the mean of the width and height of all the labeled boxes Xi assigned to the cluster represented by the initial target box Ci as the width wi of the new target box CI of the cluster / and high / , calculate the width and height difference S between the initial target frame Ci and the new target frame CI, and so on, determine the new target frame of each cluster; S41, repeat S21-S31 until the width and height difference S between the initial target frame Ci and the new target frame CI is less than a preset threshold, and determine the width and height calibration ratio coefficients of the multiple sizes.

16. The system of claim 15, wherein: The system further comprises: Determine the minimum enclosing area C of the annotation box and the initial target box; Determine the intersection-over-union (IOU) of the annotation box and the initial target box, where IOU=IA∩B| / IA∪BI, A represents the area of ​​the annotation box, and B represents the area of ​​the initial target box; Determine the proportion of the area of ​​the minimum enclosed area C that does not belong to the annotation box and the initial target box N to the minimum enclosed area C; IC\(A∪B)| / |C|; Determine the generalized intersection-over-union (GIOU) of the annotation box and the initial target box, where GIOU=IOU-|C-(A∪B)| / |C|.

17. The system of claim 10, wherein: Training the neural network according to the training set includes: Training the neural network according to the original training set, and obtaining a pre-trained neural network when a first training condition is met; The pre-trained neural network is trained according to the fused image training set, and when a second training condition is met, a trained neural network is obtained.

18. The system of claim 10, wherein: The targets include human heads.

19. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 9.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the method according to any one of claims 1 to 9.

Images