A small target detection method and device based on residual enhancement and a medium

Abstract

The application discloses a small target detection method and device based on residual enhancement and a medium, and the method comprises the following steps: acquiring a small target detection image; configuring a residual feature enhancement algorithm; removing noise from the small target detection image by using a noise removal network to obtain a first to-be-processed image; and performing layer-by-layer target prediction processing on the first to-be-processed image by calling the residual feature enhancement algorithm to obtain a small target prediction result. The application can combine high-level feature maps into low-level feature maps layer by layer based on the residual feature enhancement algorithm, thereby transferring context information from the high level to the low level, performing deep supervision learning to reduce the dimension, then readjusting the size of the input image by using deconvolution, reducing the weight of easily identifiable samples by using a hierarchical boosting network, directly enhancing the representation ability of the target detection object and the background, and finally enabling the detection model to pay more attention to the detection samples that are difficult to identify.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, in particular, the present application is applied to the field of small target detection in automatic driving, and particularly relates to a small target detection method and device based on residual enhancement and medium. BACKGROUND

[0002] At present, for small target detection in complex traffic scenes, the commonly used detection method is: based on a target detection model, a complex deep network model is used to learn features from massive data and perform classification and positioning, and the complexity of the model makes it difficult for people to understand the decision results of the model, resulting in the model being non-interpretable; the non-interpretable nature of the model can exist many safety risks, and deployment in different fields will be greatly limited, and the applicability is poor and the limitation is high; at the same time, the shallow feature extraction of the model is chaotic, and key feature extraction cannot be performed, which affects the detection efficiency and detection result accuracy. SUMMARY

[0003] The present application aims at the above-mentioned problems in the prior art, and provides a small target detection method and device based on residual enhancement and medium, thereby solving the problems of poor detection applicability, high detection limitation, low detection efficiency and low detection result accuracy of the small target detection method in the prior art due to the non-interpretable nature of the detection model and the chaotic shallow feature extraction of the detection model.

[0004] To solve the above technical problems, the specific technical solutions of the present application are as follows:

[0005] On the one hand, the present application provides a small target detection method based on residual enhancement, comprising the following steps:

[0006] Obtaining a small target detection image; configuring a residual feature enhancement algorithm;

[0007] Using a noise removal network to perform noise removal processing on the small target detection image to obtain a first to-be-processed image;

[0008] Calling the residual feature enhancement algorithm to perform layer-by-layer target prediction processing on the first to-be-processed image to obtain a small target prediction result.

[0009] As an improved scheme, the residual feature enhancement algorithm is:

[0010] Performing hierarchical feature extraction on the first to-be-processed image to obtain a plurality of initial feature images;

[0011] Respectively performing feature merging processing on a plurality of the initial feature images to obtain a plurality of to-be-adjusted feature images;

[0012] perform dimension reduction processing on the plurality of to-be-adjusted feature images to obtain a plurality of to-be-predicted feature images;

[0013] perform target prediction based on the plurality of to-be-predicted feature images to obtain the small target prediction result.

[0014] As an improved solution, the layer-by-layer feature extraction on the first to-be-processed image to obtain a plurality of initial feature images comprises:

[0015] extracting, by a plurality of convolutional networks from bottom to top, a plurality of initial feature images about the first to-be-processed image;

[0016] The feature levels of the plurality of initial feature images are respectively matched with the plurality of convolutional networks.

[0017] As an improved solution, the feature merging processing on the plurality of initial feature images respectively to obtain a plurality of to-be-adjusted feature images comprises:

[0018] dividing the plurality of initial feature images into non-adaptive merging layer images and adaptive merging layer images according to the feature levels;

[0019] fusing the initial feature image corresponding to the non-adaptive merging layer image and the high-level layer feature image of the initial feature image to obtain a plurality of first fused feature images;

[0020] performing adaptive fusion processing on the initial feature image corresponding to the adaptive merging layer image to obtain a second fused feature image;

[0021] arranging the second fused feature image and the plurality of first fused feature images to obtain the plurality of to-be-adjusted feature images.

[0022] As an improved solution, the plurality of initial feature images respectively are: a first-level feature image, a second-level feature image, a third-level feature image, a fourth-level feature image, and a fifth-level feature image;

[0023] The initial feature image corresponding to the non-adaptive merging layer image comprises: a first-level feature image, a second-level feature image, a third-level feature image, and a fourth-level feature image;

[0024] The initial feature image corresponding to the adaptive merging layer image comprises: a fifth-level feature image;

[0025] The fusing the initial feature image corresponding to the non-adaptive merging layer image and the high-level layer feature image of the initial feature image to obtain a plurality of first fused feature images comprises:

[0026] Fusing features of the first hierarchical feature image and the second hierarchical feature image to obtain a first fused image;

[0027] Fusing features of the second hierarchical feature image and the third hierarchical feature image to obtain a second fused image;

[0028] Fusing features of the third hierarchical feature image and the fourth hierarchical feature image to obtain a third fused image;

[0029] Fusing features of the fourth hierarchical feature image and the fifth hierarchical feature image to obtain a fourth fused image;

[0030] Let the first fused image, the second fused image, the third fused image and the fourth fused image be a plurality of first fused feature images respectively;

[0031] The adaptive fusion processing on the initial feature image corresponding to the adaptive merging layer image to obtain a second fused feature image comprises:

[0032] Identifying a feature weight of the fifth hierarchical feature image;

[0033] Fusing the fifth hierarchical feature image and the feature weight to obtain a sixth hierarchical feature image;

[0034] Fusing features of the sixth hierarchical feature image and the fifth hierarchical feature image to obtain a final fifth hierarchical feature image;

[0035] Let the fifth hierarchical feature image be the second fused feature image.

[0036] As an improved scheme, the dimension reduction processing on a plurality of the to-be-adjusted feature images to obtain a plurality of to-be-predicted feature images comprises:

[0037] Respectively inputting a plurality of the to-be-adjusted feature images into a convolution filter for dimension reduction processing;

[0038] When performing the dimension reduction processing, a deconvolution is used to respectively adjust second image parameters of a plurality of the to-be-adjusted feature images to match first image parameters of the small target detection image, to obtain a plurality of the to-be-predicted feature images.

[0039] As an improved scheme, the target prediction based on a plurality of the to-be-predicted feature images to obtain the small target prediction result comprises:

[0040] Respectively introducing a plurality of the to-be-predicted feature images into a hierarchical boosting network of an FPHBN model to generate a plurality of target object prediction maps respectively corresponding to a plurality of the to-be-predicted feature images;

[0041] calculate a plurality of loss functions respectively corresponding to the plurality of target object prediction maps;

[0042] confirm a final target prediction map in the plurality of target object prediction maps based on the plurality of loss functions, and take the final target prediction map as the small target prediction result.

[0043] As an improved scheme, before the noise removal network is used to perform noise removal processing on the small target detection image, the noise removal network is initialized based on parameters of a feature extraction layer in the noise removal network and a VGG network trained on an ImageNet.

[0044] In another aspect, the present application also provides a small target detection device based on residual enhancement, comprising:

[0045] An initialization unit is configured to obtain a small target detection image and configure a residual feature enhancement algorithm.

[0046] A noise removal unit is configured to use a noise removal network to perform noise removal processing on the small target detection image to obtain a first to-be-processed image.

[0047] A target prediction unit is configured to call the residual feature enhancement algorithm to perform layer-by-layer target prediction processing on the first to-be-processed image to obtain a small target prediction result.

[0048] In another aspect, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the small target detection method based on residual enhancement.

[0049] The technical scheme of the present application has the following advantages:

[0050] The small target detection method based on residual enhancement can add the residual feature enhancement algorithm to the fifth layer of the bottom-up module in the feature pyramid and the hierarchical boosting network, and in application, the high-layer feature map is merged into the low-layer feature map layer by layer, so that the context information is transmitted from the high layer to the low layer, and deep supervision learning is performed to reduce the dimension, and then the size of the input image is adjusted using deconvolution, and the hierarchical boosting network is used to reduce the weight of easily recognizable samples, which directly enhances the representation ability of the target detection object and the background, so that the detection model can pay more attention to the difficult-to-recognize detection samples, improve the applicability of the detection model, avoid feature extraction confusion, improve the detection efficiency and detection accuracy, and has high application value.

[0051] The small target detection device based on residual enhancement can add the residual feature enhancement algorithm to the fifth layer of the bottom-up module of the feature pyramid and the hierarchical boosting network through the cooperation of the initialization unit, the noise removal unit and the target prediction unit, and in application, the high-level feature map is merged into the low-level feature map layer by layer, so as to transfer the context information from the high layer to the low layer, and perform deep supervision learning to reduce the dimension, and then the size of the input image is adjusted by using the deconvolution, the weight of the easily recognizable sample is reduced by using the hierarchical boosting network, the representation ability of the target detection object and the background is directly enhanced, and finally the detection model can pay more attention to the detection sample which is difficult to recognize, the applicability of the detection model is improved, the feature extraction confusion is avoided, the detection efficiency and the detection accuracy are improved, and the application value is extremely high.

[0052] The computer readable storage medium can realize the cooperation of the initialization unit, the noise removal unit and the target prediction unit, and further realize the small target detection method based on residual enhancement. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0054] Figure 1 It is a flowchart of the small target detection method based on residual enhancement according to the embodiment 1 of the present application.

[0055] Figure 2 It is a logic principle schematic diagram of the residual feature enhancement algorithm in the small target detection method based on residual enhancement according to the embodiment 1 of the present application.

[0056] Figure 3 It is an architecture schematic diagram of the small target detection device based on residual enhancement according to the embodiment 2 of the present application.

[0057] The marks in the drawings are explained as follows:

[0058] 711, initialization unit; 712, noise removal unit; 713, target prediction unit. DETAILED DESCRIPTION

[0059] The preferred embodiments of the present application will be described in detail below with reference to the drawings, so that the advantages and features of the present application can be more easily understood by those skilled in the art, and the scope of protection of the present application can be more clearly defined.

[0060] In the description of the present application, it should be noted that the embodiments described in the present application are part of the embodiments of the present application, but not all the embodiments; based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0061] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or apparatus including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or apparatus. Embodiment 1

[0062] The present embodiment provides a small target detection method based on residual enhancement, as shown in Figure 1 and Figure 2 The method comprises the following steps:

[0063] S100, acquiring a small target detection image; configuring a residual feature enhancement algorithm;

[0064] S200, removing noise from the small target detection image using a noise removal network to obtain a first to-be-processed image;

[0065] S300, calling the residual feature enhancement algorithm to perform layer-by-layer target prediction processing on the first to-be-processed image to obtain a small target prediction result.

[0066] As an embodiment of the present application, in steps S100-S300 of the embodiment, first, the feature map of the fifth layer is subjected to adaptive pooling related to the image scale, and the target image is predicted for each layer of the side network; in the initial noise removal network, the parameters in the feature extraction layer are initialized by the VGG network trained on ImageNet, so as to reduce overfitting and speed up the training process; at the same time, the parameters of other layers are initialized by random noise, so as to finally realize the enhanced recognition and detection of small targets and solve the loss balance of the recognition samples; in the method, the deep supervision learning method does not perform the highest level of recognition task, but uses the side network to perform target prediction at each level;

[0067] As an embodiment of the present application, before the small target detection image is subjected to noise removal processing by the noise removal network, the noise removal network is initialized based on the parameters of the feature extraction layer in the noise removal network and the VGG network trained on ImageNet (a picture database for image / visual training), and then the first image to be processed after noise removal processing is subjected to the following operations;

[0068] As an embodiment of the present application, specifically, the residual feature enhancement algorithm is as follows:

[0069] S210, performing hierarchical feature extraction on the first image to be processed based on a convolutional network to obtain a plurality of initial feature images;

[0070] S220, performing feature merging processing on a plurality of the initial feature images according to levels respectively to obtain a plurality of adjusted feature images;

[0071] S230, then performing dimension reduction processing on a plurality of the adjusted feature images to obtain a plurality of predicted feature images;

[0072] S240, finally performing target prediction based on a plurality of the predicted feature images to obtain the small target prediction result.

[0073] As an embodiment of the present application, the hierarchical feature extraction on the first image to be processed to obtain a plurality of initial feature images comprises: extracting a plurality of the initial feature images about the first image to be processed through a plurality of convolutional networks from bottom to top; a plurality of the initial feature images have different levels of features, that is, the feature levels of a plurality of the initial feature images are matched with a plurality of the convolutional networks respectively, and specifically, there are 5 levels in total;

[0074] As an embodiment of the present application, the feature merging processing is performed on the initial feature images respectively to obtain a plurality of to-be-adjusted feature images, comprising:

[0075] The initial feature images are divided into non-adaptive merging layer images and adaptive merging layer images according to the feature levels; for the non-adaptive merging layer image, the initial feature image corresponding to the non-adaptive merging layer image and the high-level layer feature image of the initial feature image are fused to obtain a plurality of first fused feature images; for the adaptive merging layer image, the initial feature image corresponding to the adaptive merging layer image is adaptively fused based on an adaptive spatial fusion module to obtain a second fused feature image; the second fused feature image and the plurality of first fused feature images are arranged to obtain a plurality of to-be-adjusted feature images.

[0076] As an embodiment of the present application, the initial feature images respectively are: first-level feature images, second-level feature images, third-level feature images, fourth-level feature images and fifth-level feature images divided in sequence; the initial feature image corresponding to the non-adaptive merging layer image comprises: first-level feature images, second-level feature images, third-level feature images and fourth-level feature images; the initial feature image corresponding to the adaptive merging layer image comprises: fifth-level feature images;

[0077] As an embodiment of the present application, the initial feature image corresponding to the non-adaptive merging layer image and the high-level layer feature image of the initial feature image are fused to obtain a plurality of first fused feature images, comprising: correspondingly, the initial feature image corresponding to the non-adaptive merging layer image and the high-level layer feature image of the initial feature image are fused, that is, the feature image corresponding to the non-adaptive merging layer image is fused with the feature image of the previous layer, so the first-level feature image and the second-level feature image are fused to obtain a first fused image; the second-level feature image and the third-level feature image are fused to obtain a second fused image; the third-level feature image and the fourth-level feature image are fused to obtain a third fused image; the fourth-level feature image and the fifth-level feature image are fused to obtain a fourth fused image; so the first fused image, the second fused image, the third fused image and the fourth fused image are respectively a plurality of first fused feature images.

[0078] The initial feature image corresponding to the adaptive merging layer image is adaptively fused to obtain a second fused feature image, comprising:

[0079] identify the feature weight of the fifth level feature image; in the fifth level feature image, the fifth level feature image and the feature weight are fused through an adaptive spatial fusion module to obtain a sixth level feature image; then the sixth level feature image and the fifth level feature image are fused to obtain a final fifth level feature image; the fifth level feature image is the second fusion feature image; through the above steps, the high-level feature image is merged into the low-level feature image layer by layer, and then the context information is transmitted from the high layer to the low layer;

[0080] As an embodiment of the present application, the dimension reduction processing is performed on the plurality of feature images to be adjusted to obtain a plurality of feature images to be predicted, comprising:

[0081] The plurality of feature images to be adjusted are respectively transmitted into a convolution filter for dimension reduction processing;

[0082] During the dimension reduction processing, the second image parameters of the plurality of feature images to be adjusted are respectively adjusted to match the first image parameters of the small target detection image by using deconvolution, so that the size of the plurality of feature images to be adjusted is adjusted to the same size as the small target detection image, and finally the plurality of feature images to be predicted are obtained.

[0083] As an embodiment of the present application, the target prediction is performed based on the plurality of feature images to be predicted to obtain the small target prediction result, comprising:

[0084] The plurality of feature images to be predicted are respectively introduced into a hierarchical boosting network (Hierarchical Boosting Network) of an FPHBN model (FPHBN, Aug-FPHBN, Augment-Feature Pyramid and Hierarchical Boosting Network), to generate a plurality of target object prediction maps corresponding to the plurality of feature images to be predicted respectively; a plurality of loss functions (sigmoid cross-entropy loss functions) corresponding to the plurality of target object prediction maps are calculated; finally, the final target prediction map is confirmed in the plurality of target object prediction maps based on the plurality of loss functions, and the final target prediction map is taken as the small target prediction result. Embodiment 2

[0085] Based on the same inventive concept as the small target detection method based on residual enhancement described in embodiment 1, a small target detection device based on residual enhancement is provided, as shown in Figure 3 , comprising:

[0086] The initialization unit 711 is configured to acquire a small target detection image and configure a residual feature enhancement algorithm.

[0087] The noise removal unit 712 is configured to perform noise removal processing on the small target detection image by using a noise removal network to obtain a first to-be-processed image.

[0088] The target prediction unit 713 is configured to call the residual feature enhancement algorithm to perform layer-by-layer target prediction processing on the first to-be-processed image to obtain a small target prediction result.

[0089] As an embodiment of the present application, the residual feature enhancement algorithm called by the target prediction unit 713 is as follows: the target prediction unit 713 performs hierarchical feature extraction on the first to-be-processed image to obtain a plurality of initial feature images; the target prediction unit 713 performs feature merging processing on the plurality of initial feature images respectively to obtain a plurality of to-be-adjusted feature images; the target prediction unit 713 performs dimension reduction processing on the plurality of to-be-adjusted feature images to obtain a plurality of to-be-predicted feature images; and the target prediction unit 713 performs target prediction based on the plurality of to-be-predicted feature images to obtain the small target prediction result.

[0090] As an embodiment of the present application, the target prediction unit 713 performs hierarchical feature extraction on the first to-be-processed image to obtain a plurality of initial feature images, including: the target prediction unit 713 extracts a plurality of the initial feature images about the first to-be-processed image by using a plurality of convolution networks from bottom to top; and the feature levels of the plurality of initial feature images are matched with the plurality of convolution networks respectively.

[0091] As an embodiment of the present application, the target prediction unit 713 performs feature merging processing on the plurality of initial feature images respectively to obtain a plurality of to-be-adjusted feature images, including:

[0092] The target prediction unit 713 divides the plurality of initial feature images into non-adaptive merging layer images and adaptive merging layer images according to the feature levels;

[0093] The target prediction unit 713 fuses the initial feature image corresponding to the non-adaptive merging layer image and the high-level layer feature image of the initial feature image to obtain a plurality of first fusion feature images;

[0094] The target prediction unit 713 performs adaptive fusion processing on the initial feature image corresponding to the adaptive merging layer image to obtain a second fusion feature image;

[0095] The target prediction unit 713 collates the second fusion feature image and the plurality of first fusion feature images to obtain the plurality of to-be-adjusted feature images.

[0096] As an embodiment of the present application, the initial feature images include a first-level feature image, a second-level feature image, a third-level feature image, a fourth-level feature image, and a fifth-level feature image.

[0097] The initial feature images corresponding to the non-adaptive merging layer image include a first-level feature image, a second-level feature image, a third-level feature image, and a fourth-level feature image.

[0098] The initial feature image corresponding to the adaptive merging layer image includes a fifth-level feature image.

[0099] The target prediction unit 713 fuses the initial feature image corresponding to the non-adaptive merging layer image with the high-level feature image of the initial feature image, to obtain a plurality of first fused feature images, including:

[0100] The target prediction unit 713 fuses the first-level feature image with the second-level feature image, to obtain a first fused image.

[0101] The target prediction unit 713 fuses the second-level feature image with the third-level feature image, to obtain a second fused image.

[0102] The target prediction unit 713 fuses the third-level feature image with the fourth-level feature image, to obtain a third fused image.

[0103] The target prediction unit 713 fuses the fourth-level feature image with the fifth-level feature image, to obtain a fourth fused image.

[0104] The target prediction unit 713 causes the first fused image, the second fused image, the third fused image, and the fourth fused image to be the first fused feature images.

[0105] The target prediction unit 713 performs adaptive fusion processing on the initial feature image corresponding to the adaptive merging layer image, to obtain a second fused feature image, including:

[0106] The target prediction unit 713 identifies a feature weight of the fifth-level feature image.

[0107] The target prediction unit 713 fuses the fifth-level feature image with the feature weight, to obtain a sixth-level feature image.

[0108] The target prediction unit 713 fuses the sixth-level feature image with the fifth-level feature image, to obtain a final fifth-level feature image.

[0109] The target prediction unit 713 takes the fifth-level feature image as the second fusion feature image.

[0110] As an embodiment of the present application, the target prediction unit 713 performs dimension reduction processing on the plurality of to-be-adjusted feature images to obtain a plurality of to-be-predicted feature images, including:

[0111] The target prediction unit 713 respectively inputs the plurality of to-be-adjusted feature images into a convolution filter for dimension reduction processing.

[0112] When performing the dimension reduction processing, the target prediction unit 713 adjusts the second image parameters of the plurality of to-be-adjusted feature images to match the first image parameters of the small target detection image by using deconvolution, to obtain the plurality of to-be-predicted feature images.

[0113] As an embodiment of the present application, the target prediction unit 713 performs target prediction based on the plurality of to-be-predicted feature images to obtain the small target prediction result, including:

[0114] The target prediction unit 713 respectively introduces the plurality of to-be-predicted feature images into a hierarchical boosting network of an FPHBN model to generate a plurality of target object prediction maps respectively corresponding to the plurality of to-be-predicted feature images.

[0115] The target prediction unit 713 calculates a plurality of loss functions respectively corresponding to the plurality of target object prediction maps.

[0116] The target prediction unit 713 confirms a final target prediction map in the plurality of target object prediction maps based on the plurality of loss functions, and takes the final target prediction map as the small target prediction result.

[0117] As an embodiment of the present application, before the noise removal unit 712 performs noise removal processing on the small target detection image by using a noise removal network, the noise removal unit 712 initializes the noise removal network based on feature extraction layer parameters in the noise removal network and a VGG network trained on ImageNet. Embodiment 3

[0118] The present embodiment provides a computer-readable storage medium, including:

[0119] The storage medium is used to store computer software instructions for implementing the small target detection method based on residual enhancement described in Embodiment 1, which includes a program for executing the small target detection method based on residual enhancement described above; specifically, the executable program can be built into the small target detection device based on residual enhancement described in Embodiment 2, so that the small target detection device based on residual enhancement can implement the small target detection method based on residual enhancement described in Embodiment 1 by executing the built-in executable program.

[0120] In addition, the computer-readable storage medium of the present embodiment can adopt any combination of one or more readable storage media, wherein the readable storage medium includes an electrical, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any combination thereof.

[0121] Unlike the prior art, the small target detection method, device and medium based on residual enhancement can add a residual feature enhancement algorithm to the fifth layer of the bottom-up module in the feature pyramid and the hierarchical boosting network. When applied, the high-level feature map is merged into the low-level feature map layer by layer, so as to transfer the context information from the high layer to the low layer, and perform deep supervision learning to reduce the dimension, and then use deconvolution to adjust the size of the input image, use the hierarchical boosting network to reduce the weight of the easily recognizable sample, directly enhance the representation ability of the target detection object and the background, and finally make the detection model pay more attention to the more difficult to recognize detection sample, improve the applicability of the detection model, avoid feature extraction confusion, improve the detection efficiency and detection accuracy, and have very high application value.

[0122] It should be understood that in various embodiments herein, the size of the sequence number of each process described above does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments herein.

[0123] It should also be understood that in the embodiments herein, the term "and / or" is only a description of the association relationship between the associated objects, which means that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are in an "or" relationship.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0126] In several embodiments provided herein, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can also be electrical, mechanical or other form of connection.

[0127] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0128] In addition, each functional unit in each embodiment herein can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0129] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions herein or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments herein. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0130] The above description is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made by using the content of the specification and drawings of the present application, is also included in the patent protection scope of the present application.

Claims

1. A small target detection method based on residual enhancement, characterized in that, include: Acquire small target detection images; configure residual feature enhancement algorithms; A noise removal network is used to remove noise from the small target detection image to obtain a first image to be processed. The residual feature enhancement algorithm is invoked to perform layer-by-layer target prediction processing on the first image to be processed, and the small target prediction result is obtained. The residual feature enhancement algorithm is as follows: hierarchical feature extraction is performed on the first image to be processed to obtain several initial feature images; feature merging processing is performed on the several initial feature images to obtain several feature images to be adjusted; dimensionality reduction processing is performed on the several feature images to be adjusted to obtain several feature images to be predicted; target prediction is performed based on the several feature images to be predicted to obtain the small target prediction result. The step of performing hierarchical feature extraction on the first image to be processed to obtain several initial feature images includes: extracting several initial feature images about the first image to be processed through several convolutional networks from bottom to top; and matching the feature levels of the several initial feature images with the several convolutional networks respectively. The step of performing feature merging processing on several initial feature images to obtain several feature images to be adjusted includes: dividing the several initial feature images into non-adaptive merging layer images and adaptive merging layer images according to the feature level; fusing the initial feature image corresponding to the non-adaptive merging layer image with the higher-level feature image of the initial feature image to obtain several first fused feature images; performing adaptive fusion processing on the initial feature image corresponding to the adaptive merging layer image to obtain a second fused feature image; and combining the second fused feature image and several first fused feature images to obtain several feature images to be adjusted. The step of predicting the target based on several feature images to be predicted to obtain the small target prediction result includes: introducing several feature images to be predicted into the hierarchical boosting network of the FPHBN model to generate several target object prediction maps corresponding to the several feature images to be predicted; calculating several loss functions corresponding to the several target object prediction maps; confirming the final target prediction map based on the several loss functions in the several target object prediction maps, and taking the final target prediction map as the small target prediction result.

2. The small target detection method based on residual enhancement according to claim 1, characterized in that: The initial feature images are respectively: a first-level feature image, a second-level feature image, a third-level feature image, a fourth-level feature image, and a fifth-level feature image; The initial feature images corresponding to the non-adaptive merging layer images include: a first-level feature image, a second-level feature image, a third-level feature image, and a fourth-level feature image; The initial feature image corresponding to the adaptive merging layer image includes: the fifth-level feature image; The step of fusing the initial feature image corresponding to the non-adaptive merging layer image with the high-level feature image of the initial feature image to obtain several first fused feature images includes: The first-level feature image and the second-level feature image are fused to obtain a first fused image; The second-level feature image and the third-level feature image are fused to obtain a second fused image; The third-level feature image and the fourth-level feature image are fused to obtain a third fused image; The fourth-level feature image and the fifth-level feature image are fused to obtain a fourth fused image; Let the first fused image, the second fused image, the third fused image, and the fourth fused image be each of the first fused feature images; The adaptive fusion processing of the initial feature image corresponding to the adaptive merging layer image to obtain the second fused feature image includes: Identify the feature weights of the fifth-level feature image; The fifth-level feature image is fused with the feature weights to obtain the sixth-level feature image; The sixth-level feature image and the fifth-level feature image are fused to obtain the final fifth-level feature image; Let the fifth-level feature image be the second fused feature image.

3. The small target detection method based on residual enhancement according to claim 1, characterized in that: The step of performing dimensionality reduction processing on several feature images to be adjusted to obtain several feature images to be predicted includes: The several feature images to be adjusted are respectively fed into a convolutional filter for dimensionality reduction processing; During the dimensionality reduction process, deconvolution is used to adjust the second image parameters of several feature images to be adjusted to match the first image parameters of the small target detection image, thereby obtaining several feature images to be predicted.

4. The small target detection method based on residual enhancement according to claim 1, characterized in that: Before performing noise removal processing on the small target detection image using the noise removal network, the noise removal network is initialized based on the feature extraction layer parameters in the noise removal network and the VGG network trained on ImageNet.

5. A small target detection device based on residual enhancement, based on the small target detection method based on residual enhancement as described in claim 4, characterized in that, include: An initialization unit is used to acquire small target detection images and configure residual feature enhancement algorithms; The noise removal unit is used to perform noise removal processing on the small target detection image using a noise removal network to obtain a first image to be processed. The target prediction unit is used to call the residual feature enhancement algorithm to perform layer-by-layer target prediction processing on the first image to be processed, and obtain the small target prediction result.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the small target detection method based on residual enhancement as described in any one of claims 1 to 4.

Images