Low-image-quality target detection method and system

Abstract

The invention discloses a low-image-quality target detection method and system. A to-be-detected image is acquired; inputting the to-be-detected image into a multi-scale feature extraction module, wherein the multi-scale feature extraction module comprises a coarse feature extraction module, a core feature extraction module, a receptive field enhancement module and an attention module; wherein an image to be detected is input into the coarse feature extraction module to obtain an initial feature; based on a core feature extraction module, a receptive field enhancement module and an attention module, obtaining feature maps of n scales according to the initial features; inputting the image into a feature fusion module for bidirectional fusion to obtain n fused feature maps; and inputting the n fusion feature maps into a target regression module to obtain a target prediction result. Through receptive field enhancement and attention enhancement, the global context perception capability is improved; through full fusion of high-level semantic information and bottom-level spatial information and precise modeling of an uncertain boundary, the positioning precision is improved, and precise detection of a target in a low-quality image is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, in particular to a target detection method and system for low image quality. BACKGROUND

[0002] Target detection aims to accurately identify and locate various targets of interest in images or videos. However, in the real world, image degradation problems caused by various environmental and device factors often need to be faced, such as image blur and low spatial resolution. Therefore, the quality of the actual collected images is often difficult to guarantee, which leads to serious attenuation or even loss of target detection features, causing a significant decrease in detection accuracy and a high rate of missed detection. In extreme cases, the target detection system may even fail completely, which seriously affects the robustness and value of the target detection technology.

[0003] In existing target detection methods for blurred imaging and low spatial resolution images of industrial images, the general architecture of the target detection model usually includes three core parts: a backbone network, which usually adopts a convolutional network to extract feature maps of different levels from the input image through multi-layer convolution and downsampling operations; a neck network that uses structures such as feature pyramids and path aggregation networks to fuse the multi-scale features output by the backbone network to enhance the model's detection ability for targets of different sizes; and a detection head for predicting after feature fusion to output the class, confidence, and bounding box of the object.

[0004] However, since traditional convolutional neural networks mainly learn local features such as edges and textures by stacking small-sized convolutional kernels, they rely too much on local detailed features, making it difficult for them to extract effective features when facing images with a large amount of high-frequency information missing. In addition, existing target detection models lack effective long-distance dependency modeling mechanisms and lack global context awareness, making it difficult for them to judge objects with unclear local features. Moreover, existing technologies usually use simple coordinate regression, which is difficult to accurately describe the boundaries of blurred objects, resulting in insufficient positioning accuracy of the target.

[0005] Therefore, there is a need to design a method that can efficiently and accurately extract target features and locate targets for blurred imaging and low spatial resolution images. SUMMARY

[0006] The main technical problem solved by the present application is how to efficiently and accurately extract target features and locate targets for blurred imaging and low spatial resolution images to achieve accurate detection of targets in low-quality images.

[0007] According to a first aspect, a target detection method for low image quality is provided in an embodiment, comprising:

[0008] acquire an image to be tested;

[0009] input the image to be tested to a multi-scale feature extraction module to obtain n-scale feature maps T i , wherein i=1, 2, …, n, n≥3; the multi-scale feature extraction module comprises a coarse feature extraction module, a core feature extraction module, a receptive field enhancement module, and an attention module; wherein the image to be tested is input to the coarse feature extraction module to obtain initial features; based on the core feature extraction module, the receptive field enhancement module, and the attention module, the initial features are obtained n-scale feature maps T i ;

[0010] input n-scale feature maps T i to a feature fusion module for bidirectional fusion to obtain n fusion feature maps D i ;

[0011] input n fusion feature maps D i to a target regression module for target prediction to obtain a target prediction result.

[0012] According to a second aspect, an embodiment of a low image quality target detection system is provided, comprising:

[0013] an image to be tested acquisition unit for acquiring an image to be tested;

[0014] a multi-scale feature extraction unit for inputting the image to be tested to a multi-scale feature extraction module to obtain n-scale feature maps T i , wherein i=1, 2, …, n, n≥3; the multi-scale feature extraction module comprises a coarse feature extraction module, a core feature extraction module, a receptive field enhancement module, and an attention module; wherein the image to be tested is input to the coarse feature extraction module to obtain initial features; based on the core feature extraction module, the receptive field enhancement module, and the attention module, the initial features are obtained n-scale feature maps T i ;

[0015] a feature fusion unit for inputting n-scale feature maps T i to a feature fusion module for bidirectional fusion to obtain n fusion feature maps D i ;

[0016] a target prediction unit for inputting n fusion feature maps D i to a target regression module for target prediction to obtain a target prediction result.

[0017] According to the low-image-quality target detection method and system of the above embodiment, rich multi-scale contexts are captured through the receptive field enhancement module, and the target detection model has strong global context reasoning capability through the integration of the attention module at the network end of the multi-scale feature extraction module. When local details are unreliable, the target detection model can rely on the overall contour of the object and the scene layout for judgment, fundamentally solving the dependence of existing methods on local detail features in high-quality images, so that the low-quality images collected under poor imaging conditions still have excellent detection robustness. Through the top-down and bottom-up bidirectional fusion of the feature fusion module, the deep fusion and repeated purification of high-level semantic information and low-level spatial information are realized, so that the target detection model has stronger multi-scale target detection capability, ensuring that even small target features can be effectively enhanced and retained in the multi-scale feature map, thereby significantly improving the detection rate. Through the target regression module, the boundary prediction is modeled as a probability distribution, which is more consistent with the physical reality of fuzzy boundaries, so that the uncertain boundaries can be modeled in a more flexible and robust way, thereby greatly improving the target positioning accuracy. Thus, the target features are efficiently and accurately extracted and the target is positioned for the fuzzy imaging and low spatial resolution images, realizing the accurate detection of the target in the low-quality images. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A method flowchart of a low-image-quality target detection method;

[0019] Figure 2 A system block diagram of a low-image-quality target detection system;

[0020] Figure 3 A core feature extraction flowchart of a core feature extraction module;

[0021] Figure 4 A receptive field enhancement flowchart of a receptive field enhancement module;

[0022] Figure 5 An attention enhancement flowchart of an attention module;

[0023] Figure 6 A top-down fusion process schematic diagram;

[0024] Figure 7 A bottom-up fusion process schematic diagram;

[0025] Figure 8 A processing flowchart of a first branch in a target regression module;

[0026] Figure 9 A processing flowchart of a second branch in a target regression module;

[0027] Figure 10 This is a schematic diagram of a low-quality image to be tested.

[0028] Figure 11 This is a schematic diagram of the target location detection results. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0030] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0031] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).

[0032] Blurred imaging is characterized by the following features: First, the fine texture, internal structure, and local details of an object are smoothed or completely erased, making it difficult for deep learning models to extract sufficient features; second, the sharp edges of the target object become blurred, diffused, or broken, directly affecting the accurate prediction ability of the bounding box and leading to inaccurate bounding box localization; third, the overall or local contrast of the image decreases, and the boundary between the target and the background becomes blurred, increasing the difficulty of separating the target from the complex background.

[0033] Low spatial resolution images have the following characteristics: First, the details of the image show obvious blocky structures and jagged edges, making it difficult for target detection models to capture continuous geometric features and high-frequency information; second, small targets are small in size, key feature information is severely lacking, and they are easily swallowed by background noise; third, similar targets that could originally be distinguished by subtle differences become difficult to identify, leading to misclassification.

[0034] In automated inspection systems for industrial products, blurred imaging and low spatial resolution images represent a "degraded" input problem for target detection tasks. For general target detection scenarios, such as a rough assessment of the presence of common parts on a production line, the task can be completed even with some blurriness or low resolution images. However, for high-quality, high-performance, and high-precision industrial products, blurred imaging and low spatial resolution images become fatal obstacles. For example, in the detection and precise assembly positioning of missing microelectronic components on smartphone motherboards, each component is extremely small and requires high positional accuracy. If the high-speed motion blur of the detection image, or the limited space of a low-resolution camera, prevents accurate detection or positioning of defective components, it will directly lead to functional failures, short circuits, and performance instability in the final smartphone motherboard. Therefore, improving the ability to detect targets using blurred imaging and low spatial resolution images is a critical bottleneck that manufacturers of such high-quality industrial products must overcome to ensure product quality.

[0035] Existing computer vision models are adept at capturing the fine textures and edges of objects when processing sharp images. However, when faced with blurry or low-resolution images, the severe degradation of local information makes it difficult to accurately identify target objects.

[0036] To address the shortcomings of existing target detection technologies, such as insufficient robustness, low accuracy, and severe under-detection of small targets when handling blurred images and low spatial resolution images, this invention departs from the traditional paradigm of relying solely on local details. Instead, it simulates the cognitive strategies of humans when visual information is insufficient. Specifically, when details are unclear, the human target detection eye relies more on the overall outline, structural relationships, and contextual information of the object's environment for inference. By improving the core components of the traditional real-time target detection framework, a target detection model Y based on deep feature fusion and enhanced attention mechanism is proposed. This model enables accurate detection of blurred industrial images and low spatial resolution images.

[0037] The target detection model in this embodiment of the invention includes a multi-scale feature extraction module, a feature fusion module, and a target regression module. The multi-scale feature extraction module comprises a coarse feature extraction module, a core feature extraction module, a receptive field enhancement module, and an attention module. It is primarily used to extract multi-layer features from the image input to the target detection model Y, and to aggregate multi-scale contextual information through the receptive field enhancement module. The attention module enhances the global information of the extracted multi-layer features. The feature fusion module, based on the general rules of human visual recognition—namely, the principle of "from whole to part" and "from part to whole"—fully and effectively fuses the input multi-scale features, achieving cross-stage feature fusion and obtaining a multi-scale fused feature map. The target regression module then predicts the target based on the fused feature map output by the feature fusion module, obtaining the final target prediction result.

[0038] During the training of the object detection model Y, a training set consisting of low-quality images is first acquired. Each image has corresponding annotation information, including the bounding box of the target object and its category. Then, each image in the training set is input into the object detection model Y. The multi-scale feature extraction module extracts features from each image to obtain multiple feature maps at different scales. Then, the feature fusion module performs bidirectional fusion of these feature maps at different scales from top to bottom and bottom to top to obtain multiple fused feature maps at different scales. Finally, all the obtained fused feature maps are input into the object regression module. The object regression module uses decoupled branches to handle localization and classification separately to predict the position and category of the target object, and obtain the coordinates of the bounding box and the confidence score of each category.

[0039] A loss function is constructed based on the predicted results and the annotation information corresponding to the input image of the target detection model Y. In this embodiment, the loss function used by the target detection model Y is the total loss function obtained based on the cross-union loss and cross-entropy loss. The target detection model Y is trained based on the total loss function of the target detection model Y until the target detection model Y converges.

[0040] The Intersection over Union (IoU) loss function measures the overlap between the predicted and actual bounding boxes by using the area of ​​intersection and the area of ​​union between the bounding boxes output by the object detection model Y and the bounding boxes in the annotation information. Therefore, the IoU loss function L in this embodiment of the invention... IoU It can be represented as:

[0041]

[0042] Where F(k) is the predicted bounding box corresponding to the k-th image in the training set, and F'(k) is the actual bounding box corresponding to the k-th image in the training set; num represents the number of images contained in the training set.

[0043] The cross-entropy loss function is used to measure the degree of difference between the probability distribution predicted by the object detection model Y and the true probability distribution. Therefore, the cross-entropy loss function L in this embodiment of the invention... CE It can be represented as:

[0044]

[0045] Where num represents the number of images in the training set; N represents the number of categories; y kc The sign function is defined as follows: when the true class of the k-th image in the training set (i.e., the actual class in the corresponding annotation information) is the c-th class, the value is 1; otherwise, the value is 0. kc This represents the predicted probability when the k-th image in the training set belongs to the c-th category.

[0046] The weighted sum of the obtained intersection-over-union loss function and cross-entropy loss function yields the total loss function of the object detection model Y, i.e., L. total =λ1L IoU +λ2L CE λ1 and λ2 are the weight coefficients corresponding to the cross-union loss function and the cross-entropy loss function, respectively, used to balance the contribution of the two branch losses to the total loss. Their values ​​can be adjusted according to the task requirements. For example, λ1 and λ2 can both be set to 1.

[0047] During the inference process, the image to be tested is input into the trained target detection model Y, thereby obtaining the target prediction result in an end-to-end manner.

[0048] Please refer to Figure 1 Some embodiments provide a low-image-quality target detection method, which includes the following steps:

[0049] Step S100: Obtain the image to be tested.

[0050] Images captured by cameras on the production line are used to obtain the image to be tested. The image to be tested is a low-quality image with motion blur due to the high-speed movement of the product being inspected, or a low-quality image captured by a low-resolution camera limited by space.

[0051] Step S110: Input the image to be tested into the multi-scale feature extraction module to obtain feature maps T at n scales. i , where i = 1, 2, ..., n, n ≥ 3.

[0052] The obtained test image is input into the trained object detection model Y. After processing by the coarse feature extraction module, core feature extraction module, receptive field enhancement module, and attention module in the multi-scale feature extraction module, n feature maps with different scales are obtained; where the feature map of the i-th scale is denoted as T. i , where i represents the index value of the feature map, and the larger the value of i, the higher the corresponding feature map T at the i-th scale. i The lower the resolution, the better; in this embodiment, i = 1, 2, ..., n, n ≥ 3, and the value of n can be set according to the actual situation.

[0053] After the image to be tested is input into the multi-scale feature module, the coarse feature extraction module first performs coarse feature extraction on the image to be tested to obtain initial features. In the coarse feature extraction module, the image to be tested is coarsely extracted through two first preset convolutions to obtain initial features. In this embodiment, the first preset convolution Conv is a 3×3 standard convolution block, which is composed of a convolutional layer with a 3×3 kernel, a batch normalization layer BN, and a SiLU activation function.

[0054] Then, based on the core feature extraction module, receptive field enhancement module, and attention module, feature maps of n scales are obtained from the initial features.

[0055] To balance performance and speed, after the initial features are extracted, the core feature extraction module further extracts core features from the obtained initial features. As the number of core feature extractions increases, the detailed information contained in the feature map at the corresponding scale will gradually decrease, while the structural information will gradually increase.

[0056] During this process, when 1≤i≤n-1, the feature map T at the (i-1)th scale is... i-1 The input is fed into the core feature extraction module to obtain the feature map T at the i-th scale. i Where, when i = 1, the feature map T at the (i-1)th scale i-1 That is, T1 is the initial feature output by the coarse feature extraction module; when i = n, the feature map T at the (i-1)th scale is... i-1 The input is fed into the receptive field enhancement module to obtain the receptive field enhancement feature map; then the receptive field enhancement feature map is fed into the attention module to obtain the attention enhancement feature map; finally, the attention enhancement feature map is fed into the core feature extraction module to obtain the feature map T at the nth scale. n .

[0057] It should be noted that the processing flow for core feature extraction through the core feature extraction module is the same, the difference being that the number of channels in the input and output feature maps of the core feature extraction module is different each time.

[0058] The feature map T at the (i-1)th scale i-1 For example, the core feature extraction process for the feature map input to the core feature extraction module is as follows: Figure 3 As shown:

[0059] First, the feature map T at the (i-1)th scale is processed by the first preset convolution Conv. i-1 Channel expansion and resolution reduction are performed, and then the second preset convolution 1×1Conv is used to double the processing result in the channel dimension to obtain the first convolution operation result P1; wherein, in this embodiment, the second preset convolution 1×1Conv is composed of a convolutional layer with a convolution kernel of 1×1 and a stride of 1×1, a batch normalization layer BN, and a SiLU activation function.

[0060] Then, based on the first convolution operation result P1, the first sub-feature map y0 and the second sub-feature map y1 are obtained; wherein, the first convolution operation result is divided into the first sub-feature map y0 and the second sub-feature map y1 on an average basis in the channel dimension by the split operation;

[0061] Next, the first sub-feature map y0 is processed according to the first preset convolution Conv, and the processed result is added to the first sub-feature map y0 to obtain the first addition result Add1; the first addition result Add1 is concatenated with the second sub-feature map y1 to obtain the first concatenation result; the first concatenation result is processed by the second preset convolution to obtain the feature map T of the i-th scale. i .

[0062] This embodiment first reduces the dimensionality of the first sub-feature map y0 using a first preset convolution Conv to reduce computational cost. Then, it increases the dimensionality of the reduced feature space using the first preset convolution Conv to bring the number of channels of the first sub-feature map y0 back to its original number of channels. By performing dimensionality reduction and then dimensionality increase on the first sub-feature map y0, the feature extraction capability is improved. The processing result obtained at this point is then vector-added with the initial first sub-feature map y0 using a skip connection to prevent gradient vanishing and gradient explosion. The result of this addition is the first addition result Add1.

[0063] Then, the first summation result Add1 is concatenated with the second sub-feature map y1 along the channel dimension to integrate information from different feature maps. The resulting concatenation is the first concatenation result. Finally, the first concatenation result is processed by a second preset 1×1 Conv convolution to complete the fusion and interaction of information between feature map channels. At the same time, the channel dimension is changed to effectively control the amount of computation and parameters, keeping the overall object detection model highly efficient. The output at this point is the feature map T at the i-th scale. i .

[0064] Because local details in blurred imaging and low spatial resolution images become unreliable, the importance of global and contextual information increases dramatically. The receptive field enhancement module in this embodiment, through an efficient implementation, greatly enhances the feature map at the network's end (i.e., the feature map T at the (n-1)th scale). n-1 The module expands the receptive field of the object detection model, enabling it to fuse contextual information at different scales. Through multiple equivalent large-scale pooling kernels, it provides a "bird's-eye view," allowing the object detection model to move beyond analyzing a small, blurry local area and simultaneously observe a broad region, performing multi-scale aggregation on this region.

[0065] In the receptive field enhancement module, the flowchart for enhancing the receptive field of the input feature map is as follows: Figure 4 As shown, then:

[0066] First, the feature map T at the (n-1)th scale input to the receptive field enhancement module is processed by a second preset 1×1 convolutional layer (Conv). n-1 The process involves processing to obtain the second convolution result P2. Then, multiple max-pooling operations are applied to this second convolution result P2, resulting in a feature vector after each max-pooling operation. The max-pooling kernel is 5×5, and the number of max-pooling operations can be set according to actual needs; here, three max-pooling operations are used as an example, resulting in three feature vectors (feature tensors) with receptive fields of 5×5, 9×9, and 13×13 respectively. Then, all feature vectors and the second convolution result P2 are concatenated along the channel dimension, and this concatenated result is denoted as the second concatenated result. Finally, a second pre-defined 1×1 Conv convolution is applied to fuse the second concatenated result, and the number of channels in the output result is adjusted to match the initial input feature map T. n-1 The number of channels is consistent, thus obtaining the receptive field enhancement feature map S.

[0067] The attention module further enhances the fused features (i.e., the receptive field enhanced feature map S) output by the receptive field enhancement module, thereby endowing the object detection model with a true global perspective and content inference capability. This ensures long-range modeling of the object detection model and overcomes the fatal flaws of blurred imaging and the lack of local detail information in low-resolution images. It allows the object detection model to move beyond pixel-level local observation and instead analyze long-range dependencies and semantic context information between any regions in the image. Thus, even when the object's own features are unclear, it can be accurately identified and located based on its overall outline, structure, or its relationship with the surrounding environment.

[0068] In this embodiment, the process of the attention module enhancing the receptive field feature map is as follows: Figure 5 As shown, the channel information of the receptive field enhancement feature map S is first fused by a second preset convolution 1×1 Conv. The processing result at this time is recorded as the result of the third convolution operation P3. The number of channels of the result of the third convolution operation P3 is the same as the number of channels of the receptive field enhancement feature map S.

[0069] Then, based on the result P3 of the third convolution operation, the third sub-feature map a and the fourth sub-feature map b are obtained. In this embodiment, the result P3 of the third convolution operation is divided into the third sub-feature map a and the fourth sub-feature map b in the channel dimension.

[0070] Based on the fourth sub-feature map b, the second summation result Add2 is obtained. In this embodiment, the fourth sub-feature map b is processed by the second preset convolution 1×1 Conv to obtain the query matrix Q, the key matrix K, and the value matrix V. The attention score matrix is ​​obtained based on the query matrix Q and the key matrix K. The product of the query matrix Q and the key matrix K is calculated by matrix multiplication (Matmul) to obtain the attention score matrix. The obtained attention score matrix is ​​normalized (Softmax) and then the result of the normalization operation is multiplied by the value matrix V (Matmul) to obtain the multiplication result M.

[0071] The first preset convolution Conv is used to perform feature fusion on the value matrix V to obtain the fourth convolution operation result P4; then the fourth convolution operation result P4 is added to the obtained multiplication result M to obtain the second addition result Add2;

[0072] The fourth addition result Add4 is obtained based on the fourth sub-feature map b and the second addition result Add2. The second addition result Add2 is then fused using a second pre-defined convolution 1×1 Conv to obtain the fifth convolution result P5. The fifth convolution result P5 is then added to the fourth sub-feature map b to obtain the third addition result Add3. The third addition result Add3 is then processed using two second pre-defined convolution 1×1 Convs to obtain the sixth convolution result P6. The sixth convolution result P6 is then added to the third addition result Add3 to obtain the fourth addition result Add4.

[0073] The fourth summation result Add4 is concatenated with the third sub-feature map a along the channel dimension to obtain the third concatenation result; finally, the third concatenation result is processed by a second preset convolution 1×1 Conv to obtain the attention-enhanced feature map A.

[0074] In the attention enhancement module, the second preset convolution 1×1Conv can integrate and refine features, and at the same time, it performs non-linear and independent transformation and refinement on the features at each position, which can enhance the expressive power of the object detection model.

[0075] For example, when n=3, the core feature extraction module can obtain feature maps at three scales. Since the feature map T1 at the first scale contains the most detailed information, it can also be called a detail feature map. The feature map T2 at the second scale contains less detailed information but more structural information, so it can also be called a structural feature map. The feature map T3 at the third scale is obtained by processing the feature map T2 at the second scale through the receptive field enhancement module and the attention enhancement module, and then inputting it into the core feature extraction module. It integrates the semantic context information from the feature maps at different scales, so it can also be called a semantic feature map.

[0076] Step S120: Convert the feature maps T at n scales i The inputs are fed into the feature fusion module for bidirectional fusion, resulting in n fused feature maps D. i .

[0077] The feature fusion module mainly performs bidirectional fusion of feature maps at multiple scales output by the multi-scale feature extraction module. It is based on the general rules of human visual recognition of things, namely the principle of "from the whole to the part" and "from the part to the whole", to fully and effectively fuse the input features.

[0078] The bidirectional fusion in this embodiment includes top-down fusion and bottom-up fusion; wherein, when performing top-down fusion, as... Figure 6 As shown, let the two feature maps for top-down fusion be the first input input1 and the second input input2. Then, for the first input input1 and the second input input2 for top-down fusion: Upsample the second input input2 so that the resolution of the obtained upsampled result is the same as that of the first input; concatenate the obtained upsampled result with the first input input1, process the obtained concatenation result through the second preset convolution, and output the fusion result of the first input input1 and the second input input2.

[0079] When performing bottom-up fusion, such as Figure 7As shown, let the two feature maps for bottom-up fusion be the third input (input3) and the fourth input (input4). For the third input (input3) and the fourth input (input4) for bottom-up fusion: the fourth input (input4) is downsampled by the first preset convolution (Conv) so that the downsampled result has the same resolution as the third input (input3); the downsampled result is concatenated with the third input (input3), and the concatenated result is processed by the second preset convolution to output the fusion result (output) of the third input (input3) and the fourth input (input4).

[0080] By performing an upsampling operation or a downsampling Conv operation (without changing its number of channels) on the second input input2 (or the fourth input input4), the resolution is changed to the size of the first input input1 (or the third input input3). Then, the upsampled result (or downsampled result) and the first input input1 (or the third input input3) are concatenated in the channel dimension to complete the information integration of different feature maps. Finally, the number of channels of the concatenated result is adjusted to the required size using a second preset convolution 1×1 Conv, completing the interaction and integration of feature map channel information, while achieving channel dimensionality reduction and improving the detection efficiency of the target detection model. This yields intermediate fusion results (or fused feature maps) corresponding to different scales.

[0081] In the feature fusion module, the multi-scale feature maps are first fused from top to bottom; where, when 1≤i≤n-1, for the feature map T at the i-th scale... i For the feature map T at the i-th scale i The intermediate fusion result Z corresponding to the (i+1)th scale i+1 Perform top-down fusion to obtain the intermediate fusion result Z corresponding to the i-th scale. i In this embodiment, the intermediate fusion result Z corresponding to the (i+1)th scale is used. i+1 The second input for top-down fusion is the feature map T at the i-th scale. i The first input for top-down fusion is used to obtain the intermediate fusion result Z corresponding to the i-th scale. i When i = n-1, the intermediate fusion result Z corresponding to the (i+1)th scale is... i+1 For the feature map T at the nth scale n ;

[0082] Then, bottom-up fusion is performed on the multi-scale feature maps; where, when 2≤i≤n, for the feature map T at the i-th scale... i For the feature map T at the i-th scale iThe fused feature map D corresponding to the (i-1)th scale i-1 Perform bottom-up fusion to obtain the fused feature map D corresponding to the i-th scale. i In this embodiment, the fused feature map D corresponding to the (i-1)th scale is used. i-1 The fourth input for bottom-up fusion is the feature map T at the i-th scale. i The third input for bottom-up fusion is used to obtain the fused feature map D corresponding to the i-th scale. i When i = 2, the fused feature map D corresponding to the (i-1)th scale i-1 Z1 is the intermediate fusion result corresponding to the first scale, meaning that the fusion feature map D1 corresponding to the first scale is the intermediate fusion result Z1 for that scale. Here, the larger the value of i, the higher the value of the fusion feature map D1 at the i-th scale. i The resolution will gradually decrease.

[0083] For example, when n=3, the feature map T1 (detail feature map) at the first scale, the feature map T2 (structural feature map) at the second scale, and the feature map T3 (semantic feature map) at the third scale are first fused from top to bottom. Specifically, the semantic feature map T3 is used as the intermediate fusion result Z3 corresponding to the third scale. The structural feature map T2 is then fused with the intermediate fusion result Z3 corresponding to the previous scale from top to bottom to obtain the intermediate fusion result Z2 corresponding to the second scale. The detail feature map T1 is then fused with the intermediate fusion result Z3 corresponding to the second scale from top to bottom to obtain the intermediate fusion result Z1 corresponding to the first scale. At this point, the intermediate fusion result Z1 is also the fused feature map D1 corresponding to the first scale.

[0084] Then, a bottom-up fusion is performed on the feature map T1 (detail feature map) at the first scale, the feature map T2 (structural feature map) at the second scale, and the feature map T3 (semantic feature map) at the third scale. Specifically, a bottom-up fusion is performed on the structural feature map T2 and the fusion feature map D1 corresponding to the first scale to obtain the fusion feature map D2 corresponding to the second scale; then a bottom-up fusion is performed on the semantic feature map T3 and the fusion feature map D2 corresponding to the second scale to obtain the fusion feature map D3 corresponding to the third scale. Thus, three fusion feature maps at different scales are obtained.

[0085] It should be noted that in the feature fusion module, the top-down fusion process is the same each time, and the bottom-up fusion process is also the same each time. The difference is that the number of channels corresponding to the input and output fused feature maps of the feature fusion module is different each time.

[0086] Because high-level feature maps (feature maps corresponding to larger values ​​of i) have smaller resolutions, larger receptive fields, stronger overall semantic information, and richer global contextual information, while low-level feature maps (feature maps corresponding to smaller values ​​of i) have larger resolutions, smaller receptive fields, and capture more pixel details and precise spatial location information, the feature fusion module performs feature fusion layer by layer from high to low levels. This reflects the human visual recognition pattern of "from the whole to the part," thus using a holistic and reliable context to interpret blurred local details. Furthermore, the feature fusion module performs feature fusion layer by layer from low to high levels, reflecting the human visual cognition pattern of "from the part to the whole." High-precision positioning is achieved through the precise information of low-level features, effectively addressing the problem of inaccurate positioning in "blurry and low-resolution" images.

[0087] Step S130: Combine the n fused feature maps D i The data is input into the target regression module for target prediction, and the target prediction results are obtained.

[0088] The target regression module obtains the final prediction results of the target detection model for the image under test. Decoupled branches are used to process localization and classification separately, allowing each branch to learn features that are advantageous to itself, thereby improving the accuracy of localization. At the same time, the predicted discrete points are transformed into a probability distribution through normalization (Softmax) operation, which makes the localization of targets in blurred imaging and low-resolution images more accurate.

[0089] In this embodiment, the target regression module contains two branches, referred to as the first branch and the second branch. The first branch is used to output the bounding box corresponding to the target object in each fused feature map, and the second branch is used to output the confidence score of the target object in each fused feature map belonging to each category.

[0090] The n fused feature maps are input into the target regression module for target prediction, resulting in each fused feature map D. i The corresponding initial prediction results; where each initial prediction result includes a bounding box and a confidence score for each category;

[0091] Among them, in the i-th fusion feature map D i After being input into the target regression module, the i-th fused feature map D is obtained through the first branch of the target regression module. i The corresponding bounding box; the i-th fused feature map D is obtained through the second branch in the target regression module. i The confidence score for each corresponding category;

[0092] The first branch processes the fused feature map as follows: Figure 8 As shown, in the first branch, the i-th fused feature map D is processed by two first preset convolutions Conv and one third preset convolution Conv2d.i In this embodiment, the third preset convolution Conv2d is a standard convolution with a kernel size of 1×1. Then, the obtained processing result is normalized (Softmax), and the normalized result is further processed by the second preset convolution 1×1 Conv to obtain the i-th fused feature map D. i The corresponding bounding box (outbox).

[0093] The second branch's processing flow for the fused feature map is as follows: Figure 9 As shown, in the second branch, the i-th fused feature map D is processed by two first preset convolutions Conv and one third preset convolution Conv2d. i The i-th fused feature map D is obtained through processing. i The corresponding confidence score (outcls) for each category.

[0094] The target prediction result is obtained based on all initial prediction results, wherein the target category prediction result is obtained based on the confidence score of each category in all initial prediction results; for example, the category corresponding to the largest confidence score in each initial prediction result is taken as the target category corresponding to that initial prediction result; and then the target category corresponding to the largest confidence score in all initial prediction results is taken as the target category prediction result.

[0095] The target location prediction result is obtained based on the bounding boxes in all initial prediction results. In this embodiment, the low-quality image to be tested is as follows: Figure 10 As shown, the obtained target location prediction results are as follows: Figure 11 As shown.

[0096] For example, the confidence level of each initial prediction result when the category is the target category prediction result is obtained, and the obtained confidence level is used as the confidence level of each pixel in the bounding box corresponding to the initial detection result. The sum of the confidence levels of each pixel in all bounding boxes is obtained to obtain the comprehensive confidence level of the pixel. The pixels belonging to the target object are obtained according to the comprehensive confidence level of each pixel and a preset threshold, i.e., the target pixels. The minimum bounding box of all target pixels is obtained, and the obtained bounding box is the final prediction result of the target object, i.e., the target position prediction result.

[0097] This embodiment integrates an attention module at the network end of the multi-scale feature extraction module, enabling the object detection model to possess powerful global contextual reasoning capabilities. When local details are unreliable, the object detection model can instead rely on the overall outline of the object and the scene layout for judgment. This fundamentally solves the problem of existing methods' dependence on local detail features in high-quality images, and makes it still have excellent detection robustness for low-quality images acquired under adverse imaging conditions, thereby improving the adaptability of the object detection model to low-quality images.

[0098] Traditional methods are prone to losing scarce information in low-resolution images during downsampling, leading to missed detection of targets, especially small targets. This embodiment captures rich multi-scale context through a receptive field enhancement module and performs bidirectional fusion from top to bottom and bottom to top through a feature fusion module. This achieves deep fusion and repeated purification of high-level semantic information and low-level spatial information, giving the target detection model stronger multi-scale target detection capabilities. It ensures that even blurry small target features can be effectively enhanced and preserved in the multi-scale feature map. Through targeted network module design for low-quality images, efficient detection of small targets is achieved, thereby significantly improving the detection rate.

[0099] Furthermore, since traditional methods typically employ simple coordinate regression, they struggle to accurately describe the boundaries of ambiguous objects, leading to positioning drift. In this embodiment, the target regression module used in the detection head no longer predicts the boundary as a single, definite value, but instead models it as a probability distribution. This better reflects the physical reality of ambiguous boundaries and allows for more flexible and robust modeling of uncertain boundaries, especially ambiguous ones. This significantly improves target positioning accuracy in challenging scenarios, enabling efficient and accurate target feature extraction and target positioning for blurred imaging and low spatial resolution images, thus achieving precise target detection in low-quality images.

[0100] Please refer to Figure 2 Some embodiments provide a target detection system with low image quality, which includes the following system units:

[0101] The image acquisition unit 200 is used to acquire the image to be tested.

[0102] The multi-scale feature extraction unit 210 is used to input the image to be tested into the multi-scale feature extraction module to obtain feature maps T at n scales. iWhere i = 1, 2, ..., n, n ≥ 3; the multi-scale feature extraction module in this embodiment includes a coarse feature extraction module, a core feature extraction module, a receptive field enhancement module, and an attention module; wherein, the image to be tested is input into the coarse feature extraction module to obtain initial features; based on the core feature extraction module, the receptive field enhancement module, and the attention module, feature maps T at n scales are obtained according to the obtained initial features. i ;

[0103] Feature fusion unit 220 is used to fuse feature maps T at n scales. i The inputs are fed into the feature fusion module for bidirectional fusion, resulting in n fused feature maps D. i ;

[0104] Target prediction unit 230 is used to combine n fused feature maps D i The data is input into the target regression module for target prediction, and the target prediction results are obtained.

[0105] It should be noted that each system unit in this embodiment corresponds to the method steps in the above-described low-image-quality target detection method, and its specific implementation has been described in detail in the above embodiments, and will not be repeated here.

[0106] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0107] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A target detection method with low image quality, characterized in that, include: Acquire the image to be tested; The image to be tested is input into a multi-scale feature extraction module to obtain feature maps T at n scales. i Where i = 1, 2, ..., n, n ≥ 3; the multi-scale feature extraction module includes a coarse feature extraction module, a core feature extraction module, a receptive field enhancement module, and an attention module; wherein, the image to be tested is input into the coarse feature extraction module to obtain initial features; based on the core feature extraction module, the receptive field enhancement module, and the attention module, feature maps T at n scales are obtained according to the initial features. i ; Feature maps T at n scales i The inputs are fed into the feature fusion module for bidirectional fusion, resulting in n fused feature maps D. i ; The n fused feature maps D i The data is input into the target regression module for target prediction, and the target prediction results are obtained.

2. The target detection method as described in claim 1, characterized in that, The step of inputting the image to be tested into the coarse feature extraction module to obtain initial features includes: performing coarse feature extraction on the image to be tested through a first preset convolution to obtain initial features.

3. The target detection method as described in claim 1, characterized in that, The feature map T at n scales is obtained based on the core feature extraction module, the receptive field enhancement module, and the attention module, according to the initial features. i ,include: When 1≤i≤n-1, the feature map T at the (i-1)th scale... i-1 The input is fed into the core feature extraction module to obtain the feature map T at the i-th scale. i ; Wherein, when i = 1, the feature map T at the (i-1)th scale i-1 The initial feature; When i = n, the feature map T at the (n-1)th scale... n-1 The input is given to the receptive field enhancement module to obtain a receptive field enhancement feature map; the receptive field enhancement feature map is then given to the attention module to obtain an attention enhancement feature map; the attention enhancement feature map is then given to the core feature extraction module to obtain a feature map T at the nth scale. n .

4. The target detection method as described in claim 3, characterized in that, The feature map T at the i-1th scale i-1 The input is fed into the core feature extraction module to obtain the feature map T at the i-th scale. i ,include: The feature map T at the i-1 scale is processed by the first preset convolution and the second preset convolution. i-1 The result of the first convolution operation is obtained by processing the data. The first sub-feature map and the second sub-feature map are obtained based on the result of the first convolution operation; The first sub-feature map is processed according to the first preset convolution, and the processing result is added to the first sub-feature map to obtain the first addition result; The first summation result is concatenated with the second sub-feature map to obtain the first concatenation result; The first concatenation result is processed by a second preset convolution to obtain the feature map T at the i-th scale. i .

5. The target detection method as described in claim 3, characterized in that, The feature map T at the (n-1)th scale n-1 The input is fed into the receptive field enhancement module to obtain a receptive field enhancement feature map, including: The feature map T at the n-1th scale is processed by a second preset convolution. n-1 The result of the second convolution operation is obtained by processing the data. The result of the second convolution operation is processed by multiple max pooling operations, and each max pooling operation yields a feature vector; The second concatenation result is obtained by concatenating all the aforementioned feature vectors and the result of the second convolution operation. The second concatenation result is processed by a second preset convolution to obtain a receptive field enhanced feature map.

6. The target detection method as described in claim 3, characterized in that, The step of inputting the receptive field enhancement feature map into the attention module to obtain the attention enhancement feature map includes: The receptive field enhancement feature map is processed by a second preset convolution to obtain the result of the third convolution operation; The third sub-feature map and the fourth sub-feature map are obtained based on the result of the third convolution operation. The second addition result is obtained based on the fourth sub-feature map; The fourth addition result is obtained based on the fourth sub-feature map and the second addition result; The fourth summation result is concatenated with the third sub-feature map to obtain the third concatenation result; The third concatenation result is processed by a second preset convolution to obtain an attention-enhanced feature map.

7. The target detection method as described in claim 6, characterized in that, The step of obtaining the second addition result based on the fourth sub-feature map includes: The fourth sub-feature map is processed by a second preset convolution to obtain a query matrix, a key matrix, and a value matrix; The attention score matrix is ​​obtained based on the query matrix and the key matrix; The attention score matrix is ​​normalized, and the normalization result is multiplied by the value matrix to obtain the multiplication result. The value matrix is ​​processed by the first preset convolution to obtain the result of the fourth convolution operation; The result of the fourth convolution operation is added to the result of the multiplication to obtain the second summation result.

8. The target detection method as described in claim 6, characterized in that, The step of obtaining the fourth addition result based on the fourth sub-feature map and the second addition result includes: The second summation result is fused using a second preset convolution to obtain the result of the fifth convolution operation; The result of the fifth convolution operation is added to the fourth sub-feature map to obtain the third addition result; The third addition result is processed by the second preset convolution to obtain the sixth convolution operation result; The result of the sixth convolution operation is added to the result of the third addition to obtain the result of the fourth addition.

9. The target detection method as described in claim 3, characterized in that, The feature maps T of n scales i The inputs are fed into the feature fusion module for bidirectional fusion, resulting in n fused feature maps D. i ,include: The multi-scale feature maps are subjected to bidirectional fusion, which includes top-down fusion and bottom-up fusion. The multi-scale feature maps are fused from top to bottom; where, when 1≤i≤n-1, for the feature map T at the i-th scale... i For the feature map T at the i-th scale i The intermediate fusion result Z corresponding to the (i+1)th scale i+1 Perform top-down fusion to obtain the intermediate fusion result Z corresponding to the i-th scale. i When i = n-1, the intermediate fusion result Z corresponding to the (i+1)th scale is i+1 For the feature map T at the nth scale n ; The multi-scale feature maps are fused from bottom to top; where, when 2≤i≤n, for the feature map T at the i-th scale... i For the feature map T at the i-th scale i The fused feature map D corresponding to the (i-1)th scale i-1 Perform bottom-up fusion to obtain the fused feature map D corresponding to the i-th scale. i When i = 2, the fused feature map D corresponding to the (i-1)th scale i-1 Z1 is the intermediate fusion result corresponding to the first scale.

10. The target detection method as described in claim 9, characterized in that, The feature map T at the i-th scale i The intermediate fusion result Z corresponding to the (i+1)th scale i+1 Perform top-down fusion to obtain the intermediate fusion result Z corresponding to the i-th scale. i ,include: When performing top-down fusion, for the first input and the second input to be fused: the second input is upsampled so that the resulting upsampled result has the same resolution as the first input; the resulting upsampled result is concatenated with the first input, and the concatenated result is processed by a second preset convolution to obtain the fusion result of the first input and the second input; The intermediate fusion result Z corresponding to the (i+1)th scale i+1 The second input to the top-down fusion is the feature map T at the i-th scale. i The first input to the top-down fusion is used to obtain the intermediate fusion result Z corresponding to the i-th scale. i .

11. The target detection method as described in claim 9, characterized in that, The feature map T at the i-th scale i The fused feature map D corresponding to the (i-1)th scale i-1 Perform bottom-up fusion to obtain the fused feature map D corresponding to the i-th scale. i ,include: When performing bottom-up fusion, for the third and fourth inputs of the bottom-up fusion: the fourth input is downsampled so that the resulting downsampled result has the same resolution as the third input; the resulting downsampled result is concatenated with the third input, and the concatenated result is processed by a second preset convolution to obtain the fusion result of the third input and the fourth input; The fused feature map D corresponding to the (i-1)th scale i-1 The fourth input to the bottom-up fusion is the feature map T at the i-th scale. i The third input to the bottom-up fusion is used to obtain the fused feature map D corresponding to the i-th scale. i .

12. The target detection method as described in claim 1, characterized in that, The fusion of n feature maps D i The data is input into the target regression module for target prediction, and the target prediction results are obtained, including: The n fused feature maps D i The inputs are respectively fed into the target regression module for target prediction, resulting in each fused feature map D. i The corresponding initial prediction results; wherein, the initial prediction results include bounding boxes and confidence scores for each category; The target prediction result is obtained based on all initial prediction results, wherein the target category prediction result is obtained based on the confidence score of each category in all initial prediction results; and the target location prediction result is obtained based on the bounding boxes in all initial prediction results.

13. The target detection method as described in claim 12, characterized in that, The fusion feature map D of the i-th time i After being input into the target regression module, the i-th fused feature map D is obtained through the first branch of the target regression module. i The corresponding bounding box; wherein, in the first branch, the i-th fused feature map D is processed by a first preset convolution and a third preset convolution. i The process involves normalizing the obtained result and then further processing it using a second preset convolution to obtain the i-th fused feature map D. i The corresponding bounding box.

14. The target detection method as described in claim 12, characterized in that, The fusion feature map D of the i-th time i After being input into the target regression module, the i-th fused feature map D is obtained through the second branch of the target regression module. i The confidence score for each corresponding category; wherein, in the second branch, the fused feature map D is processed by a first preset convolution and a third preset convolution. i The process is performed to obtain the i-th fused feature map D. i The confidence score for each category.

15. The target detection method as described in claim 12, characterized in that, The step of obtaining the target category prediction result based on the confidence score of each category in all initial prediction results includes: The category corresponding to the highest confidence score in each initial prediction result is taken as the target category for that initial prediction result; The target category corresponding to the highest confidence score among all initial prediction results is taken as the target category prediction result.

16. A target detection system with low image quality, characterized in that, include: The image acquisition unit is used to acquire the image to be tested. The multi-scale feature extraction unit is used to input the image to be tested into the multi-scale feature extraction module to obtain feature maps T at n scales. i Where i = 1, 2, ..., n, n ≥ 3; the multi-scale feature extraction module includes a coarse feature extraction module, a core feature extraction module, a receptive field enhancement module, and an attention module; wherein, the image to be tested is input into the coarse feature extraction module to obtain initial features; based on the core feature extraction module, the receptive field enhancement module, and the attention module, feature maps T at n scales are obtained according to the initial features. i ; The feature fusion unit is used to fuse feature maps T at n scales. i The inputs are fed into the feature fusion module for bidirectional fusion, resulting in n fused feature maps D. i ; The target prediction unit is used to combine the n fused feature maps D i The data is input into the target regression module for target prediction, and the target prediction results are obtained.

17. A computer-readable storage medium, characterized in that, The medium stores a computer program that can be executed by a processor to implement the method as described in any one of claims 1-15.

Images