Deep learning-based low-quality image target detection method and system

Abstract

The invention relates to a low-quality image target detection method and system based on deep learning, electronic equipment and a storage medium. The method comprises the following steps: acquiring a multi-scale degraded representation; respectively guiding the target detection coding branch and the restoration coding branch to carry out feature extraction on the low-quality image by utilizing multi-scale degradation representation so as to obtain a target detection feature and a restoration feature; inputting the target detection feature and the restoration feature into a multi-scale bidirectional feature fusion module for processing to obtain a double-branch fusion feature, and inputting the double-branch fusion feature into a target detection regression module and a restoration decoder for processing to obtain a detection result and a restoration image; according to the method, multi-scale degradation representation is used for guiding target detection and feature extraction of a restoration double-branch structure network, degradation information can be well captured, the adaptability of the model to different degradation types and degrees can be improved while the accuracy of target detection of a low-quality image by the model is improved, and the method is suitable for popularization and application. Therefore, different models do not need to be trained for different degradation types.

Description

Technical Field

[0001] The present application relates to the field of image recognition and detection, and in particular to a method, system, computer device and computer-readable storage medium for low-quality image target detection based on deep learning. Background Art

[0002] Object detection is one of the advanced visual tasks in the field of computer vision. Its main purpose is to capture and locate objects in a given image or video data, and classify the objects to determine their categories. Therefore, object detection is different from simple image classification. In addition to classifying the object, it is also necessary to locate the exact position of the object in the image, which requires high feature extraction and modeling capabilities of the network model. Object detection technology is widely used in real life, such as intelligent driving, automatic driving, intelligent monitoring, fault detection, medical auxiliary diagnosis, and road traffic violation detection. With the further maturity of related technologies, the application demand and prospects of object detection technology are becoming more and more extensive, which has strongly promoted the process of social intelligence development.

[0003] In the related technologies, most target detection technologies are based on relatively ideal high-quality image or video data; however, due to environmental factors such as bad weather, lens aging and dirtiness, the acquired image or video data usually has varying degrees of degradation (referred to as low-quality images in this application), for example, images or videos taken on rainy days usually have interference from rain streaks or raindrops; images or videos acquired on foggy days have interference from fog. The impact of these degradation factors on the image causes its domain to change, further increasing the difficulty of extracting image features; target detection models trained with high-quality image data usually cannot effectively process such images or videos, resulting in a significant decrease in detection performance, which greatly affects the stability and reliability of the actual application deployment of related algorithms.

[0004] In addition, for actual scenes, the image degradation obtained at different times and spaces varies, and the characteristics and image domains of different degradations are different. Models trained with image data of a single degradation type and domain cannot efficiently handle target detection with different degradations. The usual practice is to train different models for different degradations, and switch models according to the image degradation type when using them. However, this approach is inefficient and requires the introduction of an image degradation type discrimination algorithm, which further increases the complexity of practical applications. Summary of the invention

[0005] The embodiments of the present application provide a method, system, computer device, computer-readable storage medium and computer program product for low-quality image target detection based on deep learning, so as to at least solve the problem of poor target detection accuracy in low-quality images in the related art.

[0006] In a first aspect, an embodiment of the present application provides a low-quality image target detection method based on deep learning, which is used to detect targets in low-quality images through a target detection model, and the method includes:

[0007] Obtain multi-scale degraded representation of low-quality images through a degraded encoder;

[0008] Through the multi-scale degraded representation, guiding the target detection coding branch and the restoration coding branch to respectively extract features from the low-quality image, and obtaining target detection features and restoration features respectively;

[0009] Inputting the target detection feature and the restoration feature into a multi-scale bidirectional feature fusion module at the same time, performing multi-scale feature fusion processing, and obtaining a dual-branch fusion feature;

[0010] The dual-branch fusion features are respectively input into the target detection regression module and the restoration decoder for processing to obtain the detection result and the restoration image respectively;

[0011] Among them, the target detection model training process constrains the detection result according to the detection loss, constrains the restoration result according to the restoration loss, and constrains the overall parameter optimization of the target detection model based on the total training loss obtained based on the detection loss and the restoration loss.

[0012] In some embodiments, obtaining a multi-scale degraded representation of a low-quality image by a degraded encoder includes:

[0013] Extracting shallow low-quality features of the low-quality image through a first convolutional layer, and performing downsampling feature extraction on the shallow low-quality features through a plurality of residual modules, to obtain degraded representations of a plurality of different scales;

[0014] The minimum-scale degraded representation is globally averaged pooled to obtain a one-dimensional vector, and the one-dimensional vector is channel-transformed through a fully connected layer to obtain a one-dimensional degraded representation vector for calculating the contrast loss.

[0015] In some embodiments, during the training of the degraded encoder, the degraded encoder is constrained by a contrast loss to obtain the multi-scale degraded representation, wherein the contrast loss is determined by:

[0016] Taking any target low-quality image as a query sample, randomly flipping and randomly rotating the target low-quality image to obtain a positive sample, and taking other images in the training data set except the target low-quality image as negative samples;

[0017] The query samples are respectively encoded by the degradation encoder to obtain a multi-scale degradation representation and a one-dimensional degradation representation vector;

[0018] Encoding the positive sample and the negative sample to obtain corresponding one-dimensional degraded representation vectors respectively;

[0019] The contrast loss is constructed based on the one-dimensional degraded representation vectors corresponding to the query sample, the positive sample and the negative sample.

[0020] In some embodiments, guiding the target detection encoding branch to perform feature extraction through the multi-scale degraded representation includes:

[0021] Through a Focus operation, shallow features are extracted from the low-quality image and the shallow features are scaled down to obtain first shallow image features;

[0022] Through the backbone network, feature extraction of different scales is performed on the first shallow image features respectively to obtain features of the target to be detected at multiple scales, wherein, in the process of extracting the features of the target to be detected at any scale, feature transformation is performed through a collection module composed of multiple cross-stage local modules, and a downsampling operation is performed through a multi-core convolution with a preset step size;

[0023] The dynamic Transformer feature enhancement module is used to extract the features of the target to be detected that are finally output, and the extracted features are guided by the multi-scale degradation representation to obtain the target detection features.

[0024] In some embodiments, the restoration coding branch is constructed based on a channel attention module, and guiding the restoration coding branch to perform feature extraction through the degraded representation includes:

[0025] Through a Focus operation, shallow features are extracted from the low-quality image and the shallow features are downsampled to obtain second shallow image features;

[0026] Through multiple serially connected CAB modules, deep features of the second shallow image are extracted at different scales respectively, and the extracted features are guided by the multi-scale degradation representation to obtain the restored features, wherein the CAB module includes multiple convolution modules and a channel attention module.

[0027] In some embodiments, the multiple scales of the degraded representation correspond one-to-one to multiple scales of the target detection feature and the image restoration feature, respectively, and guiding the extracted features through the multi-scale degraded representation includes:

[0028] The features extracted by the target detection branch and the features at multiple different scales extracted by the restoration coding branch are respectively added and fused with the multi-scale degraded representation pixel by pixel;

[0029] The pixel-by-pixel addition and fusion results are processed by the second convolution layer and the sigmoid function to obtain the weight value of each pixel;

[0030] The weight value is multiplied pixel by pixel with the features extracted by the target detection branch and the features extracted by the restoration coding branch to obtain the target detection features and the restoration features respectively.

[0031] In some embodiments, target detection features and restoration features at multiple preset scales are respectively input into a multi-scale bidirectional feature fusion module, and multi-scale feature fusion processing is performed to obtain dual-branch fusion features, including:

[0032] Through the multi-scale bidirectional feature fusion module, multi-scale feature fusion processing is performed on the target detection features and restoration features of the three preset scales respectively, and the dual-branch fusion features of the three preset scales are output, wherein the feature fusion processing at any preset scale includes:

[0033] Through the learnable weights, pixel-by-pixel multiplication is performed with the target detection features and restoration features at any preset scale;

[0034] Performing pixel-by-pixel addition and fusion on the pixel-by-pixel multiplication results corresponding to the target detection feature and the restoration feature, and processing the pixel-by-pixel addition and fusion results through a CAB module to obtain a fusion feature at any preset scale;

[0035] The fusion feature is upsampled to obtain a dual-branch fusion feature at any preset scale.

[0036] In some embodiments, the target detection regression module includes a progressive feature pyramid network and a detection head, and processing the fused features by the target detection regression module includes:

[0037] Through the progressive feature pyramid network, multi-scale feature transformation is performed on fusion features of different scales to obtain features after multi-scale transformation;

[0038] The detection head performs target detection on the features after the multi-scale transformation to obtain the detection result.

[0039] In some embodiments, processing the fused features by restoring the decoder includes:

[0040] The restoration decoder decodes the dual-branch fusion feature of a preset scale, and performs residual fusion on the decoding result and the feature of the corresponding scale of the restoration encoding branch to obtain a three-channel feature;

[0041] The three-channel features are added to the low-quality image pixel by pixel to obtain the restored image.

[0042] In some of the embodiments, during the process of training the target detection model, the image restoration part includes the restoration encoder and the restoration decoder, and the method further includes:

[0043] Using an L1 loss between a clean image paired with the low-quality image and the restored image, in the process of training the restored decoder, to provide a constraint on the restored result;

[0044] Based on the restoration loss weight decay strategy, the weight of the restoration loss in the overall loss is dynamically adjusted according to the number of training iterations.

[0045] In a second aspect, an embodiment of the present application provides a low-quality image target detection system based on deep learning, wherein the system is used to detect targets in low-quality images through a target detection model, including: a guidance information extraction module, a feature extraction module, a feature fusion module and a detection and restoration module, wherein:

[0046] A guidance information extraction module for obtaining a multi-scale degraded representation of a low-quality image through a degraded encoder;

[0047] A feature extraction module, used to guide the target detection coding branch and the restoration coding branch to extract features from the low-quality image through the multi-scale degraded representation, and obtain target detection features and restoration features respectively;

[0048] A feature fusion module, used for inputting the target detection feature and the restoration feature into a multi-scale bidirectional feature fusion module, performing multi-scale feature fusion processing, and obtaining a dual-branch fusion feature;

[0049] The detection and restoration module is used to input the dual-branch fusion features into the target detection regression module and the restoration decoder for processing, respectively, to obtain the detection result and the restored image;

[0050] Among them, the target detection model training process constrains the detection result according to the detection loss, constrains the restoration result according to the restoration loss, and constrains the overall parameter optimization of the target detection model based on the total training loss obtained based on the detection loss and the restoration loss.

[0051] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect above when executing the computer program.

[0052] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0053] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0054] Compared with the related art, the method for detecting targets in low-quality images based on deep learning provided in the embodiment of the present application obtains a multi-scale degraded representation of the low-quality image through a degraded encoder; further, through the degraded representation, the target detection encoding branch and the restoration encoding branch are guided to perform feature extraction on the low-quality image, and target detection features and restoration features are obtained respectively; then, the target detection features and restoration features are fused through a multi-scale bidirectional feature fusion module to obtain dual-branch fusion features of multiple different scales; finally, the above-mentioned dual-branch fusion features are respectively input into the target detection regression module and the restoration decoder for processing, and the detection results and the restored images are obtained respectively; since the present application performs targeted training based on low-quality images and guides the dual-branch feature extraction process with degraded representation, it can better capture degradation information and improve the accuracy of target detection in low-quality images; in addition, it can also better improve the adaptability of the model to different degrees of degradation, so that there is no need to train different models for different types of degradation, thereby enhancing the applicability of the model to different types of low-quality images. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0056] Figure 1 is a flow chart of a method for low-quality image target detection based on deep learning according to an embodiment of the present application;

[0057] Figure 2 is a schematic diagram of a target detection model according to an embodiment of the present application;

[0058] Figure 3 is a schematic diagram of guiding feature extraction through multi-scale degradation representation according to an embodiment of the present application;

[0059] Figure 4 is a schematic diagram of multi-scale bidirectional feature fusion according to an embodiment of the present application;

[0060] Figure 5 A comparison diagram of visualization effects of the low-quality image target detection method based on deep learning according to this embodiment;

[0061] Figure 6 is a structural block diagram of a low-quality image target detection system based on deep learning according to an embodiment of the present application;

[0062] Figure 7 It is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0064] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0065] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0066] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0067] The present application embodiment provides a low-quality image target detection method based on deep learning, which is used to detect targets in low-quality images through a target detection model. Figure 1 is a flowchart of a method for low-quality image target detection based on deep learning according to an embodiment of the present application, such as Figure 1 As shown, the process includes the following steps:

[0068] S101, obtaining a multi-scale degraded representation of a low-quality image through a degraded encoder;

[0069] In this embodiment, the low-quality image refers to an image whose quality is degraded due to various factors. Specifically, the quality degradation may be manifested as reduced clarity, color distortion, increased noise, etc.

[0070] The reasons and situations for low-quality images include but are not limited to:

[0071] 1. Environmental factors that increase image noise in bad weather such as rainy days, foggy days, and snowy days. For example, raindrops on rainy days will form blurred spots on the lens, affecting image clarity; fog will cause image contrast to decrease, and distant objects will become blurred; on snowy days, snowflakes may block the lens.

[0072] 2. Image quality is reduced due to human factors such as compression distortion and motion blur. For example, over-compressing images (such as JPEG high compression) will cause block artifacts and loss of details, and camera shake during shooting will cause image blur.

[0073] Furthermore, the degradation representation is a mathematical or numerical description of the degradation features contained in the low-quality image. In this embodiment, it is extracted from the low-quality image by a degradation encoder to simulate the features of the low-quality image. Optionally, the degradation representation may include information such as noise patterns, blur characteristics, color distortion, etc.

[0074] Specifically, through the following steps S1011 to S1013, a degraded encoder is used to obtain a multi-scale degraded representation of a low-quality image:

[0075] S1011, extracting shallow low-quality features of the low-quality image through the first convolutional layer, and performing downsampling feature extraction on the shallow low-quality features through multiple residual modules to obtain degraded representations of multiple different scales;

[0076] Figure 2 is a schematic diagram of a target detection model according to an embodiment of the present application, such as Figure 2 As shown, Figure 2 The middle dotted line without shading is the degraded encoder.

[0077] Specifically, the downgraded encoder first extracts the shallow low-quality features F in the low-quality image through a first convolutional layer (Conv3) with a convolution kernel size of 3×3. low , the process can be expressed as:

[0078] F low =T conv3 (X)

[0079] Among them, T conv3 (·) is a 3×3 convolution, and X is the input low-quality image.

[0080] Furthermore, the shallow low-quality features F low The input is further extracted into 5 residual modules, and each residual module performs a downsampling operation on the features to obtain degraded representations of multiple scales. The processing process of the residual module can be expressed as:

[0081] F rb =δ(T bn (T conv1 (δ(T bn (T conv3 (F low ))))))+T bn (T conv1 (Flow )))

[0082] Among them, F rb is the output feature of the residual module, δ(.) is the activation function; in this embodiment, the SiLU activation function is used, T bn (·) is the BatchNorm normalization operation, T conv1 (·) is a 1×1 convolution. After this step, the output of the last residual module has a degraded representation with a scale of 1 / 32 of the original low-quality image.

[0083] Through the above step S1011, rich degradation features can be more effectively extracted from the low-quality image, providing better input for subsequent image processing tasks.

[0084] S1012, performing global average pooling processing on the minimum scale degraded representation to obtain a one-dimensional vector, and performing channel transformation on the one-dimensional vector through a fully connected layer to obtain a one-dimensional degraded representation vector.

[0085] After step S1012, the obtained degraded representation is a one-dimensional degraded representation vector r with 256 channels. The process can be expressed by the formula:

[0086]

[0087] in, is the degraded representation of the output of the last residual module, T linear (·) is the fully connected layer, T GAP (·) is the global average pooling operation.

[0088] The above steps S1011 to S1012 describe the execution flow of the encoder to obtain the degraded representation in practical applications; it should also be noted that during the model training process, the obtained representation vector r will participate in the calculation of the contrast loss, thereby realizing the learning of the degradation information in the image. In this embodiment, the contrast learning method is used to model the degradation, and the degradation encoder learns the degradation information of the image through the contrast loss constraint.

[0089] Specifically:

[0090] Given an original low-quality image X and use it as a query sample Q, the image obtained by randomly flipping and rotating the low-quality image is the positive sample P. It can be understood that the degradation type of the positive sample P is the same as that of the query sample Q, and other low-quality images in the dataset are negative samples N.

[0091] The above samples are encoded by the degradation encoder, and the multi-scale degradation representations corresponding to different samples are learned. Finally, a one-dimensional representation vector is obtained to construct the contrast loss, which are r Q 、rP and r N It should be noted that since contrastive learning requires a high number of negative samples, a negative sample queue r is generally constructed. Queue As a large negative sample set, based on the above sample representation vector, the degradation modeling method based on contrastive learning can be expressed as:

[0092]

[0093] Wherein, K is the length of the negative sample queue, and τ is the temperature coefficient constant, which is optionally set to 0.07 in this embodiment.

[0094] Through the above step S101, the degradation is modeled and the corresponding representation is learned based on the contrastive learning method, so that the degradation encoder can better distinguish different degradation features and capture more diverse degradation features, thereby improving the discrimination ability of feature representation. In addition, the multi-scale feature extraction capability of the degradation encoder can capture degradation information at different scales, thereby improving the ability to handle complex image tasks. The degradation representation obtained through the above steps can better guide subsequent feature transformations to achieve more efficient low-quality image target detection.

[0095] S102, guiding the target detection coding branch and the restoration coding branch to extract features of the low-quality image through multi-scale degradation representation, and obtaining target detection features and restoration features respectively;

[0096] like Figure 2 As shown in the figure, the target detection encoding branch is an encoder branch of the target detection model, which is mainly used for the extraction and downsampling of target detection features and is mainly constructed based on the YOLOXs structure:

[0097] Specifically, in step S102a, the target detection encoding branch performs target detection feature extraction including the following steps Step 1-Step 4:

[0098] Step 1: Through the Focus operation, shallow features are extracted from the low-quality image and the shallow features are scaled down to obtain the first shallow image features;

[0099] Among them, the Focus operation is a specific image processing technology, which is used to reduce the image scale while reducing information loss, and extract shallow features through convolution operations. The specific operation is to sample every other point in the H and W directions to reduce the feature or image scale by half;

[0100] In step 1, by downscaling the shallow features, the computational complexity of subsequent network layers can be significantly reduced, processing efficiency can be improved, and key information can be retained.

[0101] Step 2, extracting features of different scales from the first shallow image features through the backbone network to obtain multiple features of the target to be detected at different scales;

[0102] The backbone network contains feature extraction processes at four scales. The feature transformation at each scale is composed of several cross-stage partial modules (CSP), which can effectively extract rich features while maintaining computational efficiency.

[0103] In addition, the downsampling operation is performed by a 3×3 convolution with a step size of 2. This convolution operation can effectively reduce the spatial size of the feature map, which not only simplifies the calculation process but also retains important feature information.

[0104] Step 2: Through the Dynamic Transformer Feature Enhancement module (DTFE), the feature extraction of the target to be detected outputted at last is performed to obtain the extraction result of the target detection feature.

[0105] Among them, DTFE consists of two deformable convolution DCN and Transformer modules. Among them, the deformable convolution can adaptively deform the convolution kernel according to the characteristics of the features and the needs of the network, which improves the flexibility of convolution and enables the network to better extract the key feature information in the image;

[0106] Furthermore, since the entire network is composed of ordinary convolutions, the local information of the features is extracted more fully, but the global information is relatively lacking. Therefore, the Transformer module is introduced to establish long-distance dependencies of features, capture the global information of features, and improve the feature extraction capability for large targets.

[0107] like Figure 2 As shown, the restoration coding branch is another encoder branch of the target detection model. Different from the target detection coding branch, the restoration coding branch is mainly used to extract restoration-related features. In this embodiment, the restoration coding branch is constructed with the channel attention block (CAB) as the basic module;

[0108] Specifically, in step S102b, the restoration coding branch performs restoration feature extraction including the following steps Step 1-Step 2:

[0109] Step 1: Through the Focus operation, shallow features are extracted from the low-quality image and downsampled to obtain the second shallow image features;

[0110] The specific process of extracting features in this step is the same as step 1 in the above step S102a, so the specific implementation method here will not be repeated in this embodiment.

[0111] Step 2: Use multiple serially connected CAB modules to perform deep feature extraction on shallow images at different scales to obtain the extraction results of restored features.

[0112] Optionally, the number of CAB modules can be set to 5, and deep features can be extracted at 5 scales; in addition, the downsampling process can be achieved by bilinear interpolation, where bilinear interpolation is a smooth downsampling method that helps to maintain the continuity and detail information of the feature map.

[0113] In this embodiment, CAB is mainly composed of two 3×3 convolutions and a channel attention module. The channel attention module is used to weight the features of each channel so that the network can focus on important features and reduce the influence of unimportant features, thereby better helping the network to screen out important features.

[0114] After the above steps S102a and S102b, the target detection branch can efficiently extract target detection features that are rich in information and contain global and local information, and extract features with restoration characteristics through the image complex branch, which complement the information with the target detection branch features and enhance the representation ability of the model features. ;

[0115] On this basis, in order to better utilize the image degradation information to assist in the feature extraction of the detection and restoration branches, in this embodiment, the multi-scale degradation representation of the image obtained by learning the degradation encoder is used to guide the target detection and restoration coding branches respectively. Specifically:

[0116] like Figure 2 As shown, multiple scales of the degraded representation correspond one-to-one to multiple scales of the target detection features and the restoration features, respectively. In this embodiment, the degraded representation at five scales is learned by the degraded encoder, and the degraded representation can be represented as 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image space scale, which correspond to the five feature scales of the target detection branch and the restoration coding branch, respectively.

[0117] Further, Figure 3 is a schematic diagram of guiding feature extraction through degraded representation according to an embodiment of the present application, such as Figure 3 As shown, the boot process includes the following steps Step1-Step3:

[0118] Step 1: The features at multiple different scales extracted by the target detection branch and the restoration coding branch are respectively added and fused with the degraded representations at different scales pixel by pixel;

[0119] Step 2, through the second convolution layer and sigmoid function, the pixel-by-pixel addition and fusion results are processed to obtain the weight value of each pixel;

[0120] Step 3: Multiply the weight value pixel by pixel with the features extracted by the target detection branch, the features extracted by the image encoding branch, and the input fusion features to obtain target detection features and restoration features respectively.

[0121] In an exemplary embodiment, the above steps Step 1-Step 3 are specifically exemplified as follows:

[0122] For feature F in and the degraded representation of the input F DE First, the two are added and fused pixel by pixel; further, the weight of each pixel is generated after a convolution and sigmoid function processing; finally, the weight and the input feature are multiplied pixel by pixel to complete the degradation guidance operation to obtain the output feature F out ;

[0123] The above process can be expressed as:

[0124]

[0125] Among them, σ(·) is the sigmoid operation, Multiply pixel by pixel.

[0126] It should be noted that the target detection branch and the restoration coding branch are symmetrical structures. In this embodiment, the degradation guidance methods of the target detection branch and the restoration coding branch are consistent.

[0127] The degraded dual-guide structure proposed in this embodiment is modeled at multiple scales, corresponding to the feature scales of the target detection and restoration coding branches, so that the model can adapt to degraded features of different scales. In addition, it can also improve the adaptability of the model to different degrees of degradation and enhance the model's feature extraction capabilities for different types of low-quality images.

[0128] S103, inputting the target detection features and the restoration features into a multi-scale bidirectional feature fusion module, performing multi-scale feature fusion processing, and obtaining a dual-branch fusion feature;

[0129] After steps S101 and S102, the target detection branch and the restoration coding branch have processed the input image respectively to generate multi-scale features; further, the target detection features and restoration features at multiple preset scales are respectively input into the multi-scale bidirectional feature fusion module for processing to obtain fusion features of multiple different scales;

[0130] The specific steps of feature fusion are as follows:

[0131] Through the multi-scale bidirectional feature fusion module, multi-scale feature fusion processing is performed on the target detection features and restoration features of the three preset scales respectively, and the dual-branch fusion features of the three preset scales are output, wherein the feature fusion steps at any preset scale include:

[0132] Step 1, multiply pixel by pixel with the target detection features and restoration features at any preset scale through the learnable weights;

[0133] Step 2, performing pixel-by-pixel addition and fusion on the pixel-by-pixel multiplication results corresponding to the target detection feature and the restoration feature, and processing the pixel-by-pixel addition and fusion results through the CAB module to obtain a fusion feature at any preset scale;

[0134] Step 3, up-sample the fused features to obtain dual-branch fused features at any preset scale. In an exemplary embodiment, a specific example of the feature fusion step is as follows:

[0135] Figure 4 is a schematic diagram of a multi-scale bidirectional feature fusion module according to an embodiment of the present application, such as Figure 4 As shown, for target detection encoding branch features at 1 / 8, 1 / 16 and 1 / 32 scales as well as and restore the encoding branch features as well as The feature fusion process is:

[0136]

[0137] in, and is the output feature of the three scales after feature fusion, T up (·) is the upsampling operation, B (·) is the CAB module, and α1, α2, α3, β1, β2 and β3 are the learnable weights for controlling the feature proportions of the object detection and restoration encoding branches, respectively.

[0138] It should be noted that since target detection usually performs regression prediction on features at scales of 1 / 8, 1 / 16, and 1 / 32, in this embodiment, the target detection and restoration coding branch features of these three scales are fused to obtain features with stronger representation capabilities, which can not only promote the improvement of target detection performance, but also restore the semantic expression capabilities of some features through the assistance of target detection features.

[0139] The feature fusion step provided in this embodiment is different from the common target detection algorithm based on high-definition image data sets that directly regresses and predicts the extracted features. In this embodiment, a restoration collaborative model is adopted. In addition to the target detection coding branch, a restoration coding branch is added to extract the degradation information, which can obtain more concrete features, and further fuse this feature with the feature of the target detection branch, thereby further enhancing the model's modeling ability for low-quality images.

[0140] S104, inputting the dual-branch fusion features into the target detection regression module and the restoration decoder for processing, respectively, to obtain the detection result and the restoration image;

[0141] This step includes a target detection step and an image restoration step, specifically:

[0142] 1. Target detection steps

[0143] In this embodiment, the target detection regression module is mainly responsible for processing the multi-scale fused features to obtain the final target detection result. The module consists of an Asymptotic Feature Pyramid Network (AFPN) and a decoupled detection head.

[0144] In an exemplary embodiment, processing the fused features by the object detection regression module includes:

[0145] Through the progressive feature pyramid network, multi-scale feature transformation is performed on the fused features of different scales to obtain processed features; further, target detection is performed on the processed features through the detection head to obtain detection results.

[0146] Input Features and After AFPN processing, the output features can be obtained and

[0147]

[0148] Among them, T AFPN (·) is the AFPN module. Furthermore, the features of the three scales output above are used by three decoupled detection heads to generate detection results, including the location information and category information of the target.

[0149] In the target detection step provided in this embodiment, AFPN can fully integrate deep and shallow features at multiple scales, effectively avoiding information loss or degradation in multi-level transmission of features; in addition, the decoupled detection head can independently process position information and category information, thereby improving detection accuracy.

[0150] 2. Image restoration steps

[0151] Different from the target detection regression module, the dual-branch fusion features of the preset scale are decoded by the restoration decoder, and the decoding results are residually fused with the features of the corresponding scale of the restoration encoding branch to obtain three-channel features; finally, the three-channel features are added pixel by pixel to the low-quality image to obtain the restored image.

[0152] In an exemplary embodiment, processing the dual-branch fusion features by the restoration decoder includes:

[0153] The restoration decoder directly uses the features of the multi-scale bidirectional feature fusion module output at 1 / 8 the input image scale. Decode it, and perform residual fusion with the features of the corresponding scale of the restoration coding branch to reduce information loss. Finally, add the output three-channel features to the input low-quality image pixel by pixel to obtain the restored image Y. The above process is expressed as follows:

[0154]

[0155] Where T Decoder (·) is the recovery decoder.

[0156] The image restoration step provided in this embodiment has the advantages of reducing information loss, enhancing restoration effect, adapting to restoration requirements of different scales, simplifying the restoration process, and realizing end-to-end image restoration in the process of decoding and residual fusion using the features output by the multi-scale fusion module. It can effectively improve the quality and accuracy of the restored image.

[0157] The above steps S101-S103 describe the structure of the model as a whole and the specific process of the model detecting objects from low-quality images;

[0158] In addition, it should be noted that, in the target detection model training process in this embodiment, the detection result is constrained according to the detection loss, the restoration result is generated according to the restoration loss, and the overall parameter optimization of the target detection model is constrained by the total training loss obtained based on the detection loss and the restoration loss.

[0159] Specifically, in this embodiment, the image restoration method is integrated with the target detection method, and a dual-branch network for target detection and restoration is designed. In addition to the detection result, the final network output also includes a restored image; the restored image is used to obtain the L1 loss in combination with a clean image paired with a low-quality image, and the L1 loss is used to constrain the restoration result. The L1 loss is used as part of the loss function, and the parameters of the restoration decoder and the first few layers of the feature extraction network are updated through the back propagation algorithm; and the total training loss is obtained based on the detection loss and the restoration loss. The overall parameter optimization of the constrained target detection model will gradually optimize the model, making it perform better and better in restoring images.

[0160] Furthermore, for object detection in low-quality images, object detection is the main line of the model, and restoration is an auxiliary means. The proportion of restoration loss should be different in different training stages.

[0161] Specifically, in the early stage of model training, it is hoped that the restoration loss will account for a larger proportion, so that the model can effectively use the features of the restoration coding branch to complement the features of the target detection coding branch, enhance the expressiveness of the features, and overcome the negative impact of degradation on the features; in the later stage of training, the model training has been close to completion. In order to prevent the restoration loss from affecting target detection too much, its weight should be appropriately reduced.

[0162] Therefore, in this embodiment, a restoration loss weight decay strategy is proposed. In an exemplary embodiment, in the set 100 rounds of training, the weight is decayed every 20 rounds, the initial value of the restoration loss weight is set to 1.0, and then the restoration loss weight is decayed to [0.9, 0.8, 0.7, 0.6] at rounds [20, 40, 60, 80] respectively. It can be understood that by setting the loss weight decay strategy, the influence of the restoration branch on target detection is better controlled, so that image restoration can achieve the best promotion effect on target detection, thereby improving the robustness of the model.

[0163] In this embodiment, the overall structure is a dual-branch structure, which mainly includes six parts: degradation encoder, target detection encoding branch, restoration encoding branch, multi-scale bidirectional feature fusion module, target detection regression module and restoration decoder. The following example is an exemplary description of a specific detection target of the detection model:

[0164] Among them, the degradation encoder is responsible for learning the degradation representation F of the low-quality image DE , the process can be expressed as:

[0165] F DE =T DE (X)

[0166] Among them, T DE(·) represents the degradation encoder, X is the input low-quality image. Then the corresponding degradation represents the feature transformation used to guide the target detection and restoration coding branches, that is:

[0167] F DetE =T DetE (F De , F DE )

[0168] F ResE =T ResE (F Re , F DE )

[0169] Among them, F DetE and F ResE are the features obtained by the target detection and restoration coding branches after degradation-guided transformation, T DetE (·) and T ResE (·) are the target detection and restoration coding branches respectively, and F De and F Re They are the features of the target detection and restoration coding branches before degradation guidance. In order to make the features of the target detection and restoration coding branches interact, the multi-scale features of the above two branches are fused through the multi-scale bidirectional feature fusion module. The process can be expressed as:

[0170] F BtFF =T BiFF (F DetE , F ResE )

[0171] Among them, T BiFF (·) and F BiFF They are the multi-scale bidirectional feature fusion module and its output features. The fused features have the information required for target detection and restoration, and are then processed by the target detection regression module and the restoration decoder to obtain the corresponding target detection results R det And the recovery result Y, that is:

[0172] R det =T DetD (F BiFF )

[0173] Y=T ResD (F BiFF )

[0174] Among them, T DetD (·) and T ResD (·) are the target detection regression module and the restoration decoder respectively. Finally, the detection result and the restoration result Y are combined with the label to calculate the detection loss and restore losses They are:

[0175]

[0176] Wherein, λ is the loss weight, which is set to 5 in the present invention. is the regression loss, is the classification loss, Z is the high-quality image corresponding to the low-quality image, and ||.||1 is the regularization of L1 loss. Based on the above definition, the total loss of the training method of the present invention is Defined as:

[0177]

[0178] in, To reduce the contrast loss, α, β and γ are weights for controlling the proportion of detection loss, restoration loss and contrast loss. The present invention sets their initial values ​​to 0.2, 0.8 and 0.1 respectively based on experience.

[0179] Through the above steps S101 to S103, compared with the target detection method in the low-quality image in the related art, the method provided in this embodiment obtains the degraded representation of the low-quality image through the degraded encoder; further, through the degraded representation, the target detection coding branch and the restoration coding branch are respectively guided to extract features of the low-quality image, and the target detection features and the restoration features are respectively obtained; then the target detection features and the restoration features are fused through the multi-scale bidirectional feature fusion module to obtain a plurality of dual-branch fusion features of different scales; finally, the dual-branch fusion features are respectively input into the target detection regression module and the restoration decoder for processing to obtain the detection result and the restored image respectively; since the method can be targeted based on the low-quality image for training, and the dual-branch feature extraction process is guided by the degraded representation, the degradation information can be better captured, and the accuracy of target detection in the low-quality image can be improved; in addition, the adaptability of the model to different degrees of degradation can be better improved, so that there is no need to train different models for different degradation types, and the applicability of the model to different types of low-quality images is enhanced.

[0180] In a second aspect, this embodiment further provides a low-quality image target detection system based on deep learning, the system is used to detect targets in low-quality images through a target detection model. Figure 6 is a structural block diagram of a low-quality image target detection system based on deep learning according to an embodiment of the present application, such as Figure 6 As shown, the system includes: a guidance information extraction module 60, a feature extraction module 61, a feature fusion module 62 and a detection and restoration module 63, wherein:

[0181] A guide information extraction module 60, configured to obtain a multi-scale degraded representation of a low-quality image through a degraded encoder;

[0182] A feature extraction module 61 is used to guide the target detection coding branch and the restoration coding branch to extract features from the low-quality image through multi-scale degradation representation, and obtain target detection features and restoration features respectively;

[0183] A feature fusion module 62 is used to input the target detection feature and the restoration feature into a multi-scale bidirectional feature fusion module to perform multi-scale feature fusion processing to obtain a dual-branch fusion feature;

[0184] The detection and restoration module 63 is used to input the dual-branch fusion features into the target detection regression module and the restoration decoder for processing, respectively, to obtain the detection result and the restored image;

[0185] Among them, the target detection model training process constrains the detection results according to the detection loss, constrains the restoration results according to the restoration loss, and constrains the overall parameter optimization of the target detection model based on the total training loss obtained based on the detection loss and the restoration loss.

[0186] Through the above system, a multi-scale degraded representation of a low-quality image is obtained through a degraded encoder; further, the multi-scale degraded representation is used to guide the target detection encoding branch and the restoration encoding branch to extract features from the low-quality image, and obtain target detection features and restoration features respectively; then the target detection features and restoration features are simultaneously input into a multi-scale bidirectional feature fusion module for processing to obtain fused features; finally, the fused features are respectively input into a target detection regression module and a restoration decoder for processing to obtain detection results and restored images respectively; since the method can perform targeted training based on low-quality images and guide the dual-branch feature extraction process with degraded representation, it can better capture degradation information and improve the accuracy of target detection in low-quality images; in addition, it can also better improve the adaptability of the model to different degrees of degradation, so that there is no need to train different models for different degradation types, thereby enhancing the applicability of the model to different types of low-quality images.

[0187] Finally, in order to verify the effectiveness of the low-quality image object detection method based on deep learning in this embodiment, a detailed comparison will be made through experiments below:

[0188] The experimental environment is: operating system Ubuntu20.04, deep learning framework Pytorch 1.10, Python version 3.6. Due to the large number of severe weather types, the present invention takes the most common rainy and foggy scenes as examples, and conducts experiments on the synthetic rainy and foggy scene image mixed dataset VOC-Mix. Specifically, the proposed method is trained using the VOC-Mix-train training set, and then the mAP (mean Average Precision) indicator of the model is tested on the VOC-Mix-test test set. The mainstream target detection (YOLOXs), restoration + target detection (AirNet-YOLOXs, RestorNet-YOLOXs), and low-quality target detection methods (TogetherNet) in recent years were selected for comparison, which are:

[0189] YOLOXs: The method proposed by Ge et al., reference "Ge Z, Liu S, Wang F, et a1.Yolox: Exceeding yolo series in 2021[EB / OL].arXiv preprint arXiv:2107.08430, 2021."

[0190] AirNet: The method proposed by Li et al., reference "Li B, Liu X, HuP, et al. All-in-one image restoration for unknown corruption [C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition. 2022: 17452-17462."

[0191] RestorNet: The method proposed by Wang et al., reference "Wang X, Chen H, Gou H, et al. RestorNet: An efficient network for multiple degradation image restoration[J]. Knowledge-Based Systems, 2023, 282: 111116."

[0192] TogetherNet: The method proposed by Wang et al., reference "Wang Y, Yan X, Zhang K, et al.Togethemet: bridging image restoration and object detection together via dynamic enhancement learning[C] / / Computer Graphics Forum.2022, 41(7): 465-476."

[0193] The test results are shown in Table 1. Compared with other methods, it can be found that the method of the present invention has better performance in detecting targets in rainy and foggy images. Figure 5 FIG. 4 is a schematic diagram showing a comparison of visualization effects of the low-quality image target detection method based on deep learning according to the present embodiment. Figure 5 As shown, the visualization visual effect comparison of the present application scheme in rainy and foggy scene images is demonstrated. The images from the 1st to the 2nd row are foggy images and rainy images respectively. It can be found that the method provided in this embodiment can detect more targets, and the target confidence is more advantageous.

[0194] Table 1 Comparison of object detection results of the VOC-Mix-test dataset for mixed scenes in rainy and foggy weather. * indicates that the model is trained with high-definition images and its performance is tested on low-quality images.

[0195]

[0196] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0197] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0198] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0199] S1, obtains a multi-scale degraded representation of a low-quality image through a degraded encoder;

[0200] S2, guiding the target detection coding branch and the restoration coding branch to extract features of the low-quality image through multi-scale degradation representation, and obtaining target detection features and restoration features respectively;

[0201] S3, inputting the target detection features and the restoration features into the multi-scale bidirectional feature fusion module at the same time, performing multi-scale feature fusion processing, and obtaining dual-branch fusion features;

[0202] S4, input the dual-branch fusion features into the target detection regression module and the restoration decoder for processing, and obtain the detection result and the restored image respectively;

[0203] Among them, the target detection model training process constrains the detection result according to the detection loss, constrains the restoration result according to the restoration loss, and constrains the overall parameter optimization of the target detection model based on the total training loss obtained based on the detection loss and the restoration loss.

[0204] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0205] In one embodiment, Figure 7 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 7 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 7 As shown. The electronic device includes a processor, a network interface, an internal memory and a non-volatile memory connected through an internal bus, wherein the non-volatile memory stores an operating system, a computer program and a database. The processor is used to provide computing and control capabilities, the network interface is used to communicate with an external terminal through a network connection, the internal memory is used to provide an environment for the operation of the operating system, the computer program is executed by the processor to implement a low-quality image target detection method based on deep learning, and the database is used to store data.

[0206] Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0207] In addition, in combination with the low-quality image target detection method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the low-quality image target detection methods in the above embodiment is implemented.

[0208] In one embodiment, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, any one of the low-quality image object detection methods in the above embodiments is implemented.

[0209] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0210] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0211] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for low-quality image object detection based on deep learning, characterized in that: The method is used to detect an object in a low-quality image by using an object detection model, and the method comprises: Obtain multi-scale degraded representation of low-quality images through a degraded encoder; Through the multi-scale degraded representation, guiding the target detection coding branch and the restoration coding branch to respectively extract features from the low-quality image, and obtaining target detection features and restoration features respectively; Inputting the target detection feature and the restoration feature into a multi-scale bidirectional feature fusion module to perform multi-scale feature fusion processing to obtain a dual-branch fusion feature; The dual-branch fusion features are respectively input into the target detection regression module and the restoration decoder for processing to obtain the detection result and the restoration image respectively; Among them, the target detection model training process constrains the detection result according to the detection loss, constrains the restoration result according to the restoration loss, and constrains the overall parameter optimization of the target detection model based on the total training loss obtained based on the detection loss and the restoration loss.

2. The method according to claim 1, characterized in that The multi-scale degraded representation of the low-quality image obtained by the degraded encoder includes: Extracting shallow low-quality features of the low-quality image through a first convolutional layer, and performing downsampling feature extraction on the shallow low-quality features through a plurality of residual modules, to obtain degraded representations of a plurality of different scales; The minimum-scale degraded representation is globally average pooled to obtain a one-dimensional vector, and the one-dimensional vector is channel-transformed through a fully connected layer to obtain a one-dimensional degraded representation vector for calculating the contrast loss.

3. The method according to claim 2, characterized in that In the process of training the degraded encoder, the degraded encoder is constrained by a contrast loss to obtain the multi-scale degraded representation, wherein the contrast loss is determined by: Taking any target low-quality image as a query sample, randomly flipping and randomly rotating the target low-quality image to obtain a positive sample corresponding to the query sample, and taking other images in the training data set except the target low-quality image as negative samples; Encoding the query sample by the degradation encoder to obtain a multi-scale degradation representation and a one-dimensional degradation representation vector; The positive sample and the negative sample are encoded by the degradation encoder to obtain corresponding one-dimensional degradation representation vectors respectively; The contrast loss is constructed based on the one-dimensional degraded representation vectors corresponding to the query sample, the positive sample and the negative sample.

4. The method according to claim 2, characterized in that: Through the multi-scale degraded representation, the target detection encoding branch is guided to perform feature extraction, including: Through a Focus operation, shallow features are extracted from the low-quality image and the shallow features are scaled down to obtain first shallow image features; Through the backbone network, feature extraction of different scales is performed on the first shallow image features respectively to obtain features of the target to be detected at multiple scales, wherein, in the process of extracting the features of the target to be detected at any scale, feature transformation is performed through a collection module composed of multiple cross-stage local modules, and a downsampling operation is performed through a multi-core convolution with a preset step size; The dynamic Transformer feature enhancement module is used to extract the features of the target to be detected that are finally output, and the extracted features are guided by the multi-scale degradation representation to obtain the target detection features.

5. The method according to claim 1, characterized in that: The restoration coding branch is constructed based on the channel attention module, and the restoration coding branch is guided by the multi-scale degradation representation to perform feature extraction, including: Through a Focus operation, shallow features are extracted from the low-quality image and the shallow features are downsampled to obtain second shallow image features; Through multiple serially connected CAB modules, deep features of the second shallow image are extracted at different scales respectively, and the extracted features are guided by the multi-scale degradation representation to obtain the restored features, wherein the CAB module includes multiple convolution modules and a channel attention module.

6. The method according to any one of claims 4 and 5, characterized in that The multiple scales of the degraded representation correspond one-to-one to the multiple scales of the target detection feature and the restoration feature, respectively, and guiding the extracted features through the multi-scale degraded representation includes: The features extracted by the target detection branch and the features at multiple different scales extracted by the restoration coding branch are respectively added and fused with the multi-scale degraded representation pixel by pixel; The pixel-by-pixel addition and fusion results are processed by the second convolution layer and the sigmoid function to obtain the weight value of each pixel; The weight value is multiplied pixel by pixel with the features extracted by the target detection branch and the features extracted by the restoration coding branch to obtain the target detection features and the restoration features respectively.

7. The method according to claim 1, characterized in that The target detection features and restoration features at multiple preset scales are respectively input into the multi-scale bidirectional feature fusion module to perform multi-scale feature fusion processing to obtain dual-branch fusion features, including: Through the multi-scale bidirectional feature fusion module, multi-scale feature fusion processing is performed on the target detection features and restoration features of the three preset scales respectively, and the dual-branch fusion features of the three preset scales are output, wherein the feature fusion processing at any preset scale includes: Through the learnable weights, pixel-by-pixel multiplication is performed with the target detection features and restoration features at any preset scale; Performing pixel-by-pixel addition and fusion on the pixel-by-pixel multiplication results corresponding to the target detection feature and the restoration feature, and processing the pixel-by-pixel addition and fusion results through a CAB module to obtain a fusion feature at any preset scale; The fusion feature is upsampled to obtain a dual-branch fusion feature at any preset scale.

8. The method according to claim 1, characterized in that The target detection regression module includes a progressive feature pyramid network and a detection head, and processing the fusion feature by the target detection regression module includes: Through the progressive feature pyramid network, multi-scale feature transformation is performed on fusion features of different scales to obtain features after multi-scale transformation; The detection head performs target detection on the features after the multi-scale transformation to obtain the detection result.

9. The method according to claim 1, characterized in that: Processing the fused features by restoring the decoder includes: The restoration decoder decodes the dual-branch fusion feature of a preset scale, and performs residual fusion on the decoding result and the feature of the corresponding scale of the restoration encoding branch to obtain a three-channel feature; The three-channel features are added to the low-quality image pixel by pixel to obtain the restored image.

10. The method according to claim 9, characterized in that In the process of training the target detection model, the image restoration part includes the restoration encoder and the restoration decoder, and the method further includes: Using an L1 loss between a clean image paired with the low-quality image and the restored image, in the process of training the restored decoder, to provide a constraint on the restored result; Based on the restoration loss weight decay strategy, the weight of the restoration loss in the overall loss is dynamically adjusted according to the number of training iterations.

11. A low-quality image object detection system based on deep learning, characterized in that: The system is used to detect targets in low-quality images through a target detection model, including: a guidance information extraction module, a feature extraction module, a feature fusion module and a detection and restoration module, wherein: A guidance information extraction module for obtaining a multi-scale degraded representation of a low-quality image through a degraded encoder; A feature extraction module, used to guide the target detection coding branch and the restoration coding branch to extract features from the low-quality image through the multi-scale degraded representation, and obtain target detection features and restoration features respectively; A feature fusion module, used for inputting the target detection feature and the restoration feature into a multi-scale bidirectional feature fusion module, performing multi-scale feature fusion processing, and obtaining a dual-branch fusion feature; The detection and restoration module is used to input the dual-branch fusion features into the target detection regression module and the restoration decoder for processing, respectively, to obtain the detection result and the restored image; Among them, the target detection model training process constrains the detection result according to the detection loss, constrains the restoration result according to the restoration loss, and constrains the overall parameter optimization of the target detection model based on the total training loss obtained based on the detection loss and the restoration loss.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images