Small target image processing method, small target image processing model, small target image processing system and storage medium

Abstract

The invention relates to the technical field of depth image processing, in particular to a small target image processing method, a small target image processing model, a small target image processing system and a storage medium. On one hand, a high-resolution feature network is introduced for keeping high spatial resolution, and excessive down-sampling is avoided or multi-scale feature fusion is combined; on the other hand, cross-stage partial connection integrated with a local context semantic layer is introduced to be combined with a high-resolution feature network, and calculation redundancy is reduced and feature fusion is enhanced through partial gradient shunting; on the other hand, a stack mechanism is introduced for feature representation, gradient disappearance of a deep network is relieved, and it is ensured that weak signals of a small target can be effectively and reversely propagated; and on the other hand, background semantic information in the output feature map is extracted in a local window range, large-range background interference is reduced, and effective decoupling of small target features and background semantics is realized. The objective of the invention is to solve the problem of how to accurately capture a small target in an image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of depth image processing, in particular to a small target image processing method, a small target image processing model, a system and a storage medium. BACKGROUND

[0002] A small target refers to a target in an image whose pixel area is usually less than a certain proportion of the total pixels or a certain pixel value. Due to weak feature information and insignificant semantics, a small target is easily submerged in noise and difficult to accurately capture when the image background is complex.

[0003] At present, for the capture of small targets in an image, a larger scale large kernel convolution is mainly used in a deep network to enhance the semantic information extraction capability of the network, to capture deep semantic information in the image, and then a multi-scale fusion method is used to gradually fuse the deep semantic information with shallow features, so as to significantly serve as the semantic part of the small target, thereby more easily capturing the small target features in a complex background image.

[0004] However, the applicant found in the process of conceiving and implementing the present application that the significant semantics would make the network dependent, eventually leading to overfitting, and that a large amount of redundant information is usually accompanied by deep features, and the features of the small target itself are few or have disappeared, so direct extraction of deep features would increase the difficulty of capturing the small target.

[0005] In view of this, the present application aims to propose a new small target image processing method to overcome the above-mentioned problems. SUMMARY

[0006] The main purpose of the present application is to provide a small target image processing method, aiming to solve the problem of how to accurately capture small targets in an image.

[0007] To achieve the above-mentioned purpose, the present application provides a small target image processing method, which comprises:

[0008] S10, collecting small target feature information in an image data set containing multiple small targets through a high-resolution feature network, to obtain an output feature map;

[0009] The high-resolution feature network comprises a cross-stage partial connection module, which divides each input feature map in the image data set into a first sub-feature map and a second sub-feature map in the channel dimension, inputs the second sub-feature map into a bottleneck layer for processing and then embeds a local context semantic layer for multi-scale learning to generate a third sub-feature map, and fuses the first sub-feature map, the second sub-feature map and the third sub-feature map to generate the output feature map.

[0010] The high-resolution feature network performs feature representation through a stacking mechanism.

[0011] S20, extracting background semantic information in the output feature map within a local window range, and outputting a small target feature map image result.

[0012] Optionally, in the S10, the second sub-feature map is input to a bottleneck layer for processing, and then embedded in a local context semantic layer for multi-scale learning to generate a third sub-feature map, and the multi-scale learning specifically includes:

[0013] S11, for each second sub-feature map input to the bottleneck layer, sequentially embedding a local context semantic layer, and setting different down-sampling factors s for each local context semantic module according to the difference of i, and gradually performing multi-scale learning:

[0014]

[0015] In the formula, represents the third feature map processed by the bottleneck layer, represents the i-th bottleneck layer, represents the local context semantic layer, represents a down-sampling factor gradually increasing with the increase of i, represents the output of the previous bottleneck layer.

[0016] Optionally, in the S10, the step of fusing the first sub-feature map, the second sub-feature map and the third sub-feature map to generate the output feature map specifically includes:

[0017] S111, using channel splicing and point-by-point convolution to fuse the first sub-feature map , the second sub-feature map and the output third feature map to obtain an output feature map :

[0018]

[0019] In the formula, represents feature fusion.

[0020] Optionally, the feature representation through the stacking mechanism specifically includes:

[0021] S100, taking to represent the feature map at each stage, wherein i represents that the layer is at the i-th stage, j represents the stacking stage, and when j=0, the network first generates a high-resolution feature map at the initial stage:

[0022] ​

[0023] wherein, , is a residual module for extracting features, is a stride convolution operation with a stride of 2;

[0024] S200, when j>0, the stack calculation formula satisfies:

[0025]

[0026] wherein, , denotes a bilinear interpolation up-sampling operation, denotes a feature concatenation operation in the channel dimension;

[0027] the stack calculation formula satisfies:

[0028]

[0029] wherein .

[0030] Optionally, the S20 specifically comprises:

[0031] S21, obtaining an output feature map of a high-resolution feature network ;

[0032] S22, outputting a reduced dimension feature map after the output feature map is subjected to down-sampling and point-wise convolution operations ;

[0033] S23, up-sampling the reduced dimension feature map through content-aware feature reorganization to obtain a new feature map restored to the original dimension as the small target feature map image result.

[0034] Optionally, the S22 specifically comprises:

[0035] S221, inputting the output feature map to a channel learning module composed of a pixel-unshuffle operation with a down-sampling factor of and a point-wise convolution operation, to obtain a reduced dimension feature map output by the channel learning module:

[0036]

[0037] In the formula, denotes the pixel value of the local region of the feature map, is the weight parameter of the convolution kernel of the point-wise convolution, is an output channel of a bias term;

[0038] S222, set , obtain a reduced dimension feature map .

[0039] Optionally, the S23 specifically includes:

[0040] S231, determining the corresponding target position of each original position in the reduced dimension feature map ;

[0041] In the formula, , ;

[0042] S232, representing a sub-region with a size of centered at the position in the feature map , constructing a recombination kernel for each position through an encoding module , and after recombination, obtaining a new feature map for each target position and the corresponding local matrix centered at the position :

[0043]

[0044] In the formula, .

[0045] In addition, to achieve the above purpose, the present application also provides a small target image processing model, the small target image processing model includes:

[0046] YOLOv8 architecture;

[0047] The high-resolution feature network is used as the backbone network for collecting small target feature information in the image data set containing multiple small targets, and an output feature map is obtained;

[0048] The high-resolution feature network module includes a cross-stage partial connection module, which divides each input feature map in the image data set into a first sub-feature map and a second sub-feature map according to the channel dimension, inputs the second sub-feature map into a bottleneck layer for processing, then embeds a local context semantic layer for multi-scale learning to generate a third sub-feature map, and fuses the first sub-feature map, the second sub-feature map and the third sub-feature map to generate the output feature map;

[0049] The high-resolution feature network performs feature representation through a stacking mechanism.

[0050] The local context semantic module is configured to extract background semantic information in the output feature map within a local window range, and output a small target feature image result.

[0051] In addition, to achieve the above-mentioned purpose, the present application also provides a computer system, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program implements the steps of the processing method of the small target image according to any one of the above-mentioned embodiments when executed by the processor.

[0052] In addition, to achieve the above-mentioned purpose, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the processing method of the small target image according to any one of the above-mentioned embodiments when executed by a processor.

[0053] The present application has at least the following beneficial effects:

[0054] 1. A high-resolution feature network is introduced to maintain high spatial resolution, and over-sampling is avoided or multi-scale feature fusion is combined;

[0055] 2. A cross-stage partial connection integrated with a local context semantic layer is combined with the high-resolution feature network, partial gradient shunting is used to reduce calculation redundancy and enhance feature fusion;

[0056] 3. A stacking mechanism is introduced to perform feature representation, to alleviate gradient vanishing of a deep network and ensure that weak signals of small targets can be effectively back-propagated;

[0057] 4. Background semantic information in the output feature map is extracted within a local window range, to reduce large-scale background interference and effectively decouple small target features and background semantics. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A flowchart of a first embodiment of the processing method of the small target image according to the present application;

[0059] Figure 2 An architecture diagram of the cross-stage partial connection according to the embodiment of the present application;

[0060] Figure 3 An architecture diagram of the high-resolution feature network according to the embodiment of the present application, in which feature representation is performed through a stacking mechanism;

[0061] Figure 4 An architecture diagram of the local context semantic module according to the embodiment of the present application;

[0062] Figure 5 An architecture diagram of a small target image processing model involved in an embodiment of the present application;

[0063] Figure 6 An architecture diagram of a hardware running environment of a computer system involved in an embodiment of the present application.

[0064] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0065] In order to better understand the above technical solutions, the exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0066] First embodiment

[0067] Referring to Figure 1 , the present embodiment provides a small target image processing method, which comprises the following steps:

[0068] S10, collecting small target feature information in an image data set containing multiple small targets through a high-resolution feature network to obtain an output feature map;

[0069] The high-resolution feature network comprises a cross-stage partial connection module, which divides each input feature map in the image data set into a first sub-feature map and a second sub-feature map in the channel dimension, inputs the second sub-feature map into a bottleneck layer for processing and then embeds a local context semantic layer for multi-scale learning to generate a third sub-feature map, and fuses the first sub-feature map, the second sub-feature map and the third sub-feature map to generate the output feature map.

[0070] The high-resolution feature network performs feature representation through a stacking mechanism.

[0071] In the present embodiment, YOLOv8 is used as the basic model of the deep learning network.

[0072] First, a high-resolution network (High-Resolution Feature Networks, HRFNnet) is used as the backbone network to replace the structure of the traditional backbone network combined with the feature pyramid network, and the computing resources are concentrated on the high-resolution feature layer, which is convenient for extracting fine-grained feature information of small targets.

[0073] In the embodiment, the small target refers to a target with a pixel area in an image generally less than a certain proportion of total pixels or a certain pixel value. In some optional manners, a target with a pixel area less than or equal to 16*16 pixels is regarded as a small target.

[0074] It should be noted that in the embodiment, the high-resolution feature network is introduced to maintain high spatial resolution, and the model's ability to capture details is significantly improved by avoiding excessive downsampling or combining multi-scale feature fusion. On this basis, to further improve the model performance, the embodiment creatively introduces the cross-stage partial connection (Clf) integrated with the local context semantic layer combined with the high-resolution feature network, and reduces the calculation redundancy and enhances the feature fusion through partial gradient shunting.

[0075] Specifically, referring to the cross-stage partial connection architecture shown in the figure, the working principle of the cross-stage partial connection module in step S10 is further described in the embodiment. Figure 2

[0076] Let the input feature map be , and the feature map pass through the CLf module first , and then perform a segmentation operation , and divide the feature map into two parts in the channel dimension, that is, a first sub-feature map and a second sub-feature map .

[0077] , and the feature map is transmitted to the bottleneck layer. Different bottleneck layers are represented by , and the feature map processed by the bottleneck layer is represented by . The first stage is represented by . Then, for the output feature map of each bottleneck layer, the local context semantic layer is embedded in sequence, and different down-sampling factors are set for each local context semantic module according to the difference , and multi-scale learning is gradually performed:

[0078]

[0079] , wherein represents a local context semantic module, represents that the down-sampling factor gradually increases with , and the addition operation represents that a residual connection is added to the feature maps before and after the input local context semantic module. The final output feature map is obtained through the fusion of the output feature maps of each stage:

[0080] ​

[0081] In some optional embodiments, the fusion manner is to use point-by-point convolution fusion after channel splicing.

[0082] In some optional embodiments, in order to improve the cross-channel learning ability of feature expression, the CLf module adopts a gradual down-sampling rate, and according to the setting of the local context semantic module, generates feature maps with 4, 16 and 64 times the input channel number, and feature maps of different scales, further enhancing the multi-scale learning ability.

[0083] On the other hand, considering that the deep features of the image dataset with multiple small targets are usually accompanied by a large amount of redundant information, and the features of the small targets themselves are few or have disappeared, directly using deep features for detection is easy to lead to the decline of the detection performance of the network, and cannot accurately capture small targets in the image. Based on this consideration, a stacking mechanism is introduced for feature representation, which alleviates the gradient disappearance of the deep network and ensures that the weak signal of the small target can be effectively back propagated.

[0084] Specifically, referring to Figure 3 the high-resolution feature network shown in the feature representation schematic diagram of the high-resolution feature network, specifically includes;

[0085] S100, as shown in representing the feature map of each stage, wherein i represents the i-th layer, j represents the stacking stage, and when j=0, the network first generates the high-resolution feature map of the initial stage down-sampling layer, j represents the stacking stage, and when j=0, the network first generates the high-resolution feature map of the initial stage :

[0086]

[0087] wherein, , is a residual module for extracting features, is a stride convolution operation with a stride of 2;

[0088] S200, when j>0, the stacking calculation formula satisfies:

[0089]

[0090] wherein, , represents a bilinear interpolation up-sampling operation, represents a feature channel dimension splicing operation;

[0091] the stacking calculation formula satisfies:

[0092]

[0093] wherein .

[0094] S20, extracting background semantic information in the output feature map in a local window range, and outputting a small target feature image result.

[0095] In this embodiment, after extracting small target feature information through a high-resolution network to obtain an output feature map, background semantic information in the output feature map is extracted in a local window range, so as to reduce large-range background interference and realize effective decoupling of small target features and background semantics.

[0096] In some optional embodiments, a module for extracting background semantic information in the output feature map in a local window range is referred to as a local context semantic module.

[0097] Referring to Figure 4 The architecture schematic diagram of the local context semantic module is shown, and the local context semantic module specifically includes a channel learning module in the left half and a feature reorganization module in the right half.

[0098] On the one hand, for the channel learning module, its function is to realize output of a reduced-dimension feature map after the output feature map is subjected to down-sampling and point-wise convolution operations.

[0099] S221, inputting the output feature map to a channel learning module composed of a pixel-unshuffle operation with a down-sampling factor of and a point-wise convolution operation, to obtain a reduced-dimension feature map output by the channel learning module:

[0100]

[0101] In the formula, represents a pixel value of a local region of a feature map, is a weight parameter of a convolution kernel of the point-wise convolution, is a bias term of an output channel .

[0102] S222, assuming , a reduced-dimension feature map is obtained.

[0103] On the other hand, for the feature reorganization module, a content-aware feature reorganization (CARAFE) up-sampling operator is used as a component thereof. After the channel learning module, the network has learned semantic information, but the feature scale has been deformed, so a reorganization module is needed to restore the original shape of the feature. Specifically as follows:

[0104] S231, determining the reduced-dimension feature map each original position corresponding target position ;

[0105] wherein, , ;

[0106] S232, to represent each target position centered on the size of the sub-region, the reorganization core is constructed for each position by the coding module After that, each target position and the corresponding local matrix centered on is reorganized to obtain a new feature map :

[0107]

[0108] wherein, .

[0109] In the technical scheme provided in the embodiment, a high-resolution feature network is introduced to maintain high spatial resolution by avoiding excessive downsampling or combining multi-scale feature fusion; a cross-stage partial connection integrated with a local context semantic layer is introduced in combination with the high-resolution feature network to reduce calculation redundancy and enhance feature fusion through partial gradient shunting; a stack mechanism is introduced for feature representation to alleviate gradient vanishing of a deep network and ensure that weak signals of small targets can be effectively back propagated; background semantic information in the output feature map is extracted within a local window range to reduce large-range background interference and effectively decouple small target features and background semantics.

[0110] Second embodiment

[0111] Based on the small target image processing method proposed in the first embodiment, a small target image processing model is provided in the embodiment, comprising:

[0112] YOLOv8 architecture;

[0113] The high-resolution feature network is used as the backbone network to collect small target feature information in the image dataset containing multiple small targets, and an output feature map is obtained;

[0114] The high-resolution feature network module comprises a cross-stage partial connection module, the cross-stage partial connection module divides each input feature map in the image data set into a first sub-feature map and a second sub-feature map in a channel dimension, inputs the second sub-feature map into a bottleneck layer for processing, embeds a local context semantic layer for multi-scale learning to generate a third sub-feature map, and fuses the first sub-feature map, the second sub-feature map and the third sub-feature map to generate the output feature map.

[0115] The high-resolution feature network performs feature representation through a stacking mechanism.

[0116] The local context semantic module is configured to extract background semantic information in the output feature map in a local window range and output a small target feature map image result.

[0117] It should be noted that the small target image processing model shown in Figure 5 The architecture improvement schematic diagram of the small target image processing model is shown, in the improvement, all deep core convolutional layers in the YOLOv8 model are removed, only the P2 layer is reserved (as shown in (a)), the improved CLf-L module (as shown in (b)) is used, the CLf-L module is modified in a targeted manner, the number of stacking of the local context semantic module is increased and the stacking manner is changed, and the CLf-L module is called CLf-L (as shown in (c)). After such modification, the network becomes more lightweight, and the required computing power is smaller.

[0118] Third embodiment

[0119] The effect of the small target image processing model proposed in the foregoing embodiments is further described in this embodiment.

[0120] (1) Simulation experiment conditions:

[0121] The experiment of this embodiment is performed on a Windows operating system, is realized based on a deep learning framework PyTorch (2.1.1+cu118), and is run on an NVIDIA 4090 GPU workstation.

[0122] (2) Simulation experiment content and result analysis

[0123] On the above-mentioned public data set VisDrone2019, the deep learning network model proposed in this application and the target detection method of the prior art are respectively used for simulation comparison and analysis, the detection frame and the category of the target are obtained, the average precision value (mAP50) is used, and the average precision mean (mAP50-95) calculated in the range of IoU threshold from 0.50 to 0.95 (increasing by 0.05 every time) is used as an evaluation index. The detection result of the simulation experiment obtained by evaluation is shown in Table 2:

[0124] Table 2 Detection results of the present application and prior art in simulation experiments

[0125]

[0126] As can be seen from Table 2, compared with the existing technologies EFC, YOLO and RT-DETR, the present application (ours) achieves significant improvement. In addition, the present application significantly improves the detection accuracy while reducing the parameters and GFLOP. This is sufficient to show that the present application can significantly improve the inference speed while ensuring high accuracy. Especially in the small target detection task, the present application not only has an advantage in computational efficiency, but also achieves higher accuracy in the small target detection task, can effectively reduce the computational cost without sacrificing the detection performance of the model, meets the strict requirements of computational efficiency in practical applications, has good practical application value, and is especially suitable for resource-limited embedded or edge computing devices.

[0127] As an implementation scheme, Figure 6 The present application embodiment relates to the architecture schematic diagram of the hardware running environment of the computer system.

[0128] As Figure 6 shown, the computer system can include a processor 1001 such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen (Display) and an input unit such as a keyboard (Keyboard). The optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory) such as a magnetic disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0129] Those skilled in the art can understand that Figure 6 the computer system architecture shown in the present application does not constitute a limitation on the computer system, and can include more or fewer components than the diagram, or combine certain components, or different component arrangements.

[0130] As Figure 6 shown, the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and a computer program. The operating system is a program that manages and controls the hardware and software resources of the computer system, and the running of the computer program and other software or programs.

[0131] In Figure 6 In the computer system shown, the user interface 1003 is mainly used to connect the terminal and communicate data with the terminal; the network interface 1004 is mainly used to connect the background server and communicate data with the background server; and the processor 1001 can be used to call the computer program stored in the memory 1005.

[0132] In this embodiment, the computer system comprises a memory 1005, a processor 1001 and a computer program stored in the memory and executable on the processor, wherein:

[0133] When the processor 1001 calls the computer program stored in the memory 1005, the following operations are performed:

[0134] S10, acquiring small target feature information in an image data set containing multiple small targets through a high-resolution feature network, to obtain an output feature map;

[0135] The high-resolution feature network comprises a cross-stage partial connection module, which divides each input feature map in the image data set into a first sub-feature map and a second sub-feature map in the channel dimension, inputs the second sub-feature map into a bottleneck layer for processing and then embeds a local context semantic layer for multi-scale learning to generate a third sub-feature map, and fuses the first sub-feature map, the second sub-feature map and the third sub-feature map to generate the output feature map.

[0136] The high-resolution feature network performs feature representation through a stacking mechanism.

[0137] S20, extracting background semantic information in the output feature map within a local window range, and outputting a small target feature image result.

[0138] When the processor 1001 calls the computer program stored in the memory 1005, the following operations are performed:

[0139] S11, for each second sub-feature map input into the bottleneck layer, sequentially embedding a local context semantic layer, and setting different down-sampling factors s for each local context semantic module according to the difference of i, and gradually performing multi-scale learning:

[0140]

[0141] In the formula, represents the third feature map processed through the bottleneck layer, represents the i-th bottleneck layer, represents the local context semantic layer, represents a down-sampling factor that gradually increases with the increase of i, ​Output of the last neck layer.

[0142] When the processor 1001 invokes the computer program stored in the memory 1005, the following operations are performed:

[0143] S111, the first sub-feature map is fused by using channel splicing and point-by-point convolution , the second sub-feature map and the output third feature map to obtain an output feature map :

[0144]

[0145] In the formula, indicates feature fusion.

[0146] When the processor 1001 invokes the computer program stored in the memory 1005, the following operations are performed:

[0147] S100, the feature map of each stage is represented by , wherein i represents that the layer is at the i-th stage, j represents the stack stage, and when j=0, the network first generates a high-resolution feature map of the initial stage :

[0148]

[0149] wherein, , is a residual module for extracting features, is a stride convolution operation with a stride of 2;

[0150] S200, when j>0, the stack calculation formula satisfies:

[0151]

[0152] wherein, , indicates a bilinear interpolation upsampling operation, indicates a channel dimension feature splicing operation;

[0153] the stack calculation formula satisfies:

[0154]

[0155] wherein .

[0156] When the processor 1001 invokes the computer program stored in the memory 1005, the following operations are performed:​

[0157] S21, obtain the output feature map of the high-resolution feature network ;

[0158] S22, outputting a reduced-dimensional feature map after downsampling and point-by-point convolution operations on the output feature map ;

[0159] S23, upsampling the reduced-dimensional feature map through content-aware feature reorganization to obtain a new feature map restored to the original dimension , as the small target feature image result.

[0160] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0161] S221, output the feature map , the input to the downsampling factor is A channel learning module composed of a pixel-unshuffle operation and a point-by-point convolution operation is used to obtain the dimensionality reduction feature map output by the channel learning module. :

[0162]

[0163] Where, Represents the pixel value of the local area of ​​the feature map, is the weight parameter of the convolution kernel of point-by-point convolution, Is the output channel The bias term;

[0164] S222, set , get the dimensionality reduction feature map .

[0165] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0166] S231, determining the dimension reduction feature map Each original position Corresponding target location ;

[0167] Where, , ;

[0168] S232, with express China-Israel Location-centric The sub-region of size is encoded by the module for each Positional construction of recombinant nuclei After that, for each target position and the corresponding local matrix centered at , a new feature map is reorganized:

[0169]

[0170] where .

[0171] In addition, those skilled in the art can understand that all or part of the processes in the method for implementing the above embodiments can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the process steps of the above embodiments of the method.

[0172] Therefore, the present application also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement each step of the processing method of the small target image according to the above embodiments.

[0173] The computer readable storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer readable storage media that can store program codes.

[0174] It should be noted that the storage medium provided by the embodiments of the present application is a storage medium used to implement the method of the embodiments of the present application. Therefore, based on the method introduced in the embodiments of the present application, those skilled in the art can understand the specific structure and modification of the storage medium, and therefore will not be described here. Any storage medium used by the method of the embodiments of the present application belongs to the scope of protection of the present application.

[0175] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0176] The present application is described in reference to the flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or blocks Figure 1 means for carrying out the function specified by the flow or block.

[0177] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or blocks Figure 1 means for carrying out the function specified by the flow or block.

[0178] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or blocks Figure 1 means for carrying out the function specified by the flow or block.

[0179] It should be noted that any references made in the claims to an "apparatus" or "means" should not be construed to cover the corresponding structures only. Rather, such claims should be construed to cover both the corresponding structures and their equivalents. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a programmed computer. In the unitary or integrated claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. The word "comprise", "comprising", and "comprises" and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense.

[0180] Although preferred embodiments of the application have been described, a person of ordinary skill in the art will be able to make additional changes and modifications to these embodiments once armed with the present disclosure. Accordingly, the appended claims are intended to cover all such changes and modifications as fall within the scope of the application.

[0181] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for processing small target images, characterized in that: The method comprises the following steps: S10, collecting small target feature information from an image dataset containing multiple small targets through a high-resolution feature network to obtain an output feature map; The high-resolution feature network includes a cross-stage partial connection module, which divides each input feature map in the image dataset into a first sub-feature map and a second sub-feature map according to the channel dimension, inputs the second sub-feature map into the bottleneck layer for processing, and then embeds it into the local context semantic layer for multi-scale learning to generate a third sub-feature map, and fuses the first sub-feature map, the second sub-feature map and the third sub-feature map to generate the output feature map; Wherein, the high-resolution feature network performs feature representation through a stacking mechanism; S20, extracting background semantic information from the output feature map within a local window range, and outputting a small target feature image result.

2. The method according to claim 1, wherein In S10, inputting the second sub-feature map into the bottleneck layer for processing and then embedding it into the local context semantic layer for multi-scale learning to generate the third sub-feature map specifically includes: S11: For each second sub-feature map input to the bottleneck layer, embed it into the local context semantic layer in sequence, and set a different downsampling factor s for each local context semantic module according to the difference in i, and gradually perform multi-scale learning: ； Where, represents the third feature map processed by the bottleneck layer, represents the bottleneck layer of layer i, Represents the local context semantic layer, Indicates that with The downsampling factor increases gradually with the increase of Represents the output of the previous bottleneck layer.

3. The method according to claim 1 or 2, wherein: In S10, the step of fusing the first sub-feature map, the second sub-feature map, and the third sub-feature map to generate the output feature map specifically includes: S111, using channel splicing and point-by-point convolution to fuse the first sub-feature map , the second sub-feature map And the third feature map of the output , get the output feature map : ； Where, Indicates feature fusion.

4. The method according to claim 1, wherein The feature representation by the stack mechanism specifically includes: S100, Represents the feature map of each stage, where i indicates that the layer is in the Downsampling layer, j represents the stacking stage. When j=0, the network first generates the high-resolution feature map of the initial stage. : ； in, , It is the residual module for extracting features, It is a strided convolution operation with a stride of 2; S200, when j>0, The stack calculation formula satisfies: ； in, , represents the bilinear interpolation upsampling operation, Indicates that features are concatenated according to the channel dimension; The stack calculation formula satisfies: ； in .

5. The method according to claim 1, wherein The S20 specifically includes: S21, obtain the output feature map of the high-resolution feature network ; S22, outputting a reduced-dimensional feature map after downsampling and point-by-point convolution operations on the output feature map ; S23, upsampling the reduced-dimensional feature map through content-aware feature reorganization to obtain a new feature map restored to the original dimension , as the small target feature image result.

6. The method according to claim 5, wherein Said S22 specifically includes: S221, output the feature map , the input downsampling factor is A channel learning module composed of a pixel-unshuffle operation and a point-by-point convolution operation is used to obtain the dimensionality reduction feature map output by the channel learning module. : ； Where, Represents the pixel value of the local area of ​​the feature map, is the weight parameter of the convolution kernel of point-by-point convolution, Is the output channel The bias term; S222, set , get the dimensionality reduction feature map .

7. The method according to claim 5, wherein Said S23 specifically includes: S231, determining the dimension reduction feature map Each original position Corresponding target position ; Where, , ; S232, with express China-Israel Location-centric The sub-region of size is encoded by the module for each Positional construction of recombinant nuclei Then, for each target position And the corresponding The local matrix centered , reorganize to get a new feature map : ； Where, .

8. A small target image processing model, characterized in that: The small target image processing model includes: YOLOv8 architecture; A high-resolution feature network is used as the backbone network to collect feature information of small targets in an image dataset containing multiple small targets and obtain an output feature map. The high-resolution feature network module includes a cross-stage partial connection module, which divides each input feature map in the image dataset into a first sub-feature map and a second sub-feature map according to the channel dimension, inputs the second sub-feature map into the bottleneck layer for processing, and then embeds it into the local context semantic layer for multi-scale learning to generate a third sub-feature map, and fuses the first sub-feature map, the second sub-feature map and the third sub-feature map to generate the output feature map; Wherein, the high-resolution feature network performs feature representation through a stacking mechanism; The local context semantic module is used to extract background semantic information in the output feature map within a local window range and output a small target feature image result.

9. A computer system, characterized in that: The computer system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method for processing small target images according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for processing a small object image according to any one of claims 1 to 7.

Images