Training method for object detection model for low-quality image, object detection method for low-quality image, and related device

Abstract

Provided in the present disclosure are a training method for an object detection model for a low-quality image, an object detection method for a low-quality image, and a related device. The training method comprises: acquiring a low-quality sample image for a target region; inputting the low-quality sample image into a first encoder to obtain an initial feature of the low-quality sample image; inputting the initial feature into a transfer convolutional network, and on the basis of a first target feature, training the transfer convolutional network to obtain a target convolutional network, wherein the first target feature is obtained on the basis of the initial feature; on the basis of the target convolutional network, obtaining a second target feature for the low-quality sample image; inputting the second target feature into a first detection head, and training the first detection head to obtain an object detection head, wherein the first detection head is obtained on the basis of the first target feature; and on the basis of the first encoder, the target convolutional network and the object detection head, obtaining an object detection model, wherein the object detection model is used for performing object detection on a low-quality image to be subjected to detection. The present disclosure provides technical support for realizing efficient object detection.

Description

Low-quality image target detection model training and detection method and related device

[0001] Cross-reference to Related Applications

[0002] The present disclosure is based on and claims priority to Chinese Patent Application No. 2024110763629, filed on August 7, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of data processing, and in particular, to a low-quality image target detection model training and detection method and related device. BACKGROUND

[0004] In scenarios such as offshore operations, visual target detection technology can provide a lot of help for operation decision-making. However, in scenarios such as offshore operations, the quality of the pictures taken is often not up to standard due to bad weather, complex sea conditions and other problems, thereby leading to low accuracy and efficiency of target detection. Therefore, how to achieve efficient target detection has become a technical problem to be solved. SUMMARY

[0005] The present disclosure provides a low-quality image target detection model training and detection method and related device to at least solve the above technical problems existing in the prior art.

[0006] The present disclosure provides a low-quality image target detection model training method, which comprises:

[0007] Obtaining a low-quality sample image for a target region;

[0008] Inputting the low-quality sample image into a first encoder to obtain initial features of the low-quality sample image;

[0009] Inputting the initial features into a transfer convolutional network, training the transfer convolutional network based on first target features to obtain a target convolutional network; wherein the first target features are obtained based on the initial features of the low-quality sample image; the feature clarity of the first target features is stronger than that of the initial features;

[0010] Obtaining second target features for the low-quality sample image based on the target convolutional network;

[0011] Inputting the second target features into a first detection head, training the first detection head to obtain a target detection head; the first detection head is obtained based on the first target features;

[0012] Obtain a target detection model based on the first encoder, the target convolutional network and the target detection head; the target detection model is used for target detection on the low-quality image to be detected.

[0013] In the scheme, the initial feature is input into the transfer convolutional network, the transfer convolutional network is trained based on the first target feature, and a target convolutional network is obtained.

[0014] The initial feature is input into the transfer convolutional network, and the transfer convolutional network is trained with the first target feature as the feature constraint output by the transfer convolutional network to obtain the target convolutional network.

[0015] In the scheme, the first target feature is obtained based on the initial feature of the low-quality sample image, and includes:

[0016] The initial feature of the low-quality sample image is input into a pre-trained image type prediction model to obtain a low-quality type of the low-quality sample image.

[0017] Based on the low-quality type of the low-quality sample image, a first target feature for the low-quality sample image is obtained.

[0018] In the scheme, the first target feature for the low-quality sample image is obtained based on the low-quality type of the low-quality sample image, and includes:

[0019] The low-quality type of the low-quality sample image is input into a pre-trained content mapping network to obtain mapping encoding information for the low-quality type.

[0020] The mapping encoding information and the initial feature of the low-quality sample image are input into a first decoder to obtain a restored sample image; the clarity of the restored sample image is higher than that of the low-quality sample image.

[0021] Based on the restored sample image, a first target feature for the low-quality sample image is obtained.

[0022] In the scheme, the first target feature for the low-quality sample image is obtained based on the restored sample image, and includes:

[0023] The restored sample image and the low-quality sample image are input into a second encoder to obtain the first target feature.

[0024] The first detection head is obtained based on the first target feature, and includes:

[0025] The first target feature is input into a second detection head to be trained, and the second detection head to be trained is trained to obtain the first detection head.

[0026] The disclosure provides a low-quality image target detection method, which comprises the following steps:

[0027] obtaining a low-quality to-be-detected image for a to-be-detected region;

[0028] inputting the low-quality to-be-detected image into a target detection model to obtain a target detection result for the to-be-detected region; wherein the target detection result is obtained by performing feature extraction on the low-quality to-be-detected image by a first encoder in the target detection model, performing migration on the extracted features by a target convolutional network in the target detection model, and performing target detection based on the migrated features by a target detection head in the target detection model; and the target detection result is used to represent whether a target object exists in the to-be-detected region.

[0029] The disclosure provides a training device of a low-quality image target detection model, which comprises the following steps:

[0030] a first obtaining unit configured to obtain a low-quality sample image for a target region;

[0031] a second obtaining unit configured to input the low-quality sample image into a first encoder to obtain initial features of the low-quality sample image;

[0032] a third obtaining unit configured to input the initial features into a migration convolutional network, train the migration convolutional network based on first target features, and obtain a target convolutional network; wherein the first target features are obtained based on the initial features of the low-quality sample image; and the feature clarity of the first target features is stronger than that of the initial features;

[0033] a fourth obtaining unit configured to obtain second target features for the low-quality sample image based on the target convolutional network;

[0034] a fifth obtaining unit configured to input the second target features into a first detection head, train the first detection head, and obtain a target detection head; the first detection head is obtained based on the first target features;

[0035] a sixth obtaining unit configured to obtain a target detection model based on the first encoder, the target convolutional network, and the target detection head; the target detection model is used for target detection on a low-quality to-be-detected image.

[0036] The disclosure provides a low-quality image target detection device, which comprises the following steps:

[0037] a seventh obtaining unit configured to obtain a low-quality to-be-detected image for a to-be-detected region;

[0038] The detection unit is configured to input the low-quality image to be detected into a target detection model to obtain a target detection result for the region to be detected. The target detection result is obtained by performing feature extraction on the low-quality image to be detected by a first encoder in the target detection model, performing migration on the extracted features by a target convolution network in the target detection model, and performing target detection based on the migrated features by a target detection head in the target detection model. The target detection result is used to indicate whether a target object exists in the region to be detected.

[0039] The present disclosure provides an electronic device, comprising:

[0040] at least one processor; and

[0041] a memory connected to the at least one processor in communication; wherein

[0042] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0043] The present disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of the present disclosure.

[0044] In the present disclosure, a low-quality sample image for a target region is obtained; the low-quality sample image is input into a first encoder to obtain initial features of the low-quality sample image; the initial features are input into a migration convolution network, and the migration convolution network is trained based on first target features to obtain a target convolution network; the first target features are obtained based on the initial features of the low-quality sample image; the feature clarity of the first target features is stronger than that of the initial features; second target features for the low-quality sample image are obtained based on the target convolution network; the second target features are input into a first detection head, and the first detection head is trained to obtain a target detection head; the first detection head is obtained based on the first target features; a target detection model is obtained based on the first encoder, the target convolution network and the target detection head; and the target detection model is used for target detection on a low-quality image to be detected. Efficient target detection can be achieved.

[0045] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0046] The above and other objects, features and advantages of the present disclosure exemplary embodiments will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which:

[0047] In the drawings, the same or similar numerals indicate the same or similar parts.

[0048] Fig. 1 shows a schematic diagram of the implementation process of the training method of the low-quality image target detection model according to an embodiment of the present disclosure;

[0049] Fig. 2 shows a schematic diagram of the acquisition process of the target convolution network according to an embodiment of the present disclosure;

[0050] Fig. 3 shows a schematic diagram of the acquisition process of the first detection head according to an embodiment of the present disclosure;

[0051] Fig. 4 shows a schematic diagram of the component structure of the image type prediction model according to an embodiment of the present disclosure;

[0052] Fig. 5 shows a schematic diagram of the processing process of the content mapping network according to an embodiment of the present disclosure;

[0053] Fig. 6 shows a schematic diagram of the structure comparison between the target detection head and the first detection head according to an embodiment of the present disclosure;

[0054] Fig. 7 shows a schematic diagram of the implementation process of the low-quality image target detection method according to an embodiment of the present disclosure;

[0055] Fig. 8 shows a schematic diagram of the component structure of the target detection model according to an embodiment of the present disclosure;

[0056] Fig. 9 shows a schematic diagram of the component structure of the training device of the low-quality image target detection model according to an embodiment of the present disclosure;

[0057] Fig. 10 shows a schematic diagram of the component structure of the low-quality image target detection device according to an embodiment of the present disclosure;

[0058] Fig. 11 shows a schematic diagram of the component structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0059] In order to make the purpose, features and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.

[0060] In the embodiments of the present disclosure, a low-quality image refers to a distorted image with a clarity less than a preset (or standard) threshold, which cannot be subjected to (accurate) target detection. The low-quality sample image and the low-quality image to be detected mentioned in the embodiments of the present disclosure are both distorted images with a clarity less than a preset (or standard) threshold.

[0061] The technical solutions of the embodiments of the present disclosure relate to a scheme of training a low-quality image target detection model, and a scheme of performing target detection on a low-quality image to be detected by using the trained target detection model. The technical solutions of the present disclosure can realize efficient acquisition of a target detection model, and further realize efficient detection of a target.

[0062] The embodiments of the present disclosure provide a training method of a low-quality image target detection model, as shown in FIG. 1, the method comprises:

[0063] S101: Obtain a low-quality sample image for a target region.

[0064] In this step, the target region is a region where a known target object is located, and the low-quality sample image refers to an image with poor image quality (low clarity), that is, the low-quality sample image is a low-quality image including the target object.

[0065] In a marine operation scenario, due to high humidity conditions such as fog and rain commonly seen at sea, or due to problems such as fast movement of a ship, severe sea conditions, complex reflection on the water surface, cluttered environment, and strong interference of a dynamic background, the image obtained by photographing the target region is a low-quality image.

[0066] S102: Input the low-quality sample image into a first encoder to obtain initial features of the low-quality sample image.

[0067] In this step, the first encoder is an encoder for image restoration. After image restoration, the low-quality sample image can be restored to a relatively clear image, thereby improving the accuracy of subsequent target detection model training.

[0068] The low-quality sample image is input into the first encoder, which can extract features of the low-quality sample image to obtain initial features of the low-quality sample image. The first encoder can adopt a convolution module, a residual module, or a Swin Transformer (shift window multi-level vision model) module, etc., and usually adopts a layer-by-layer downsampling manner to collect 3-4 scale convolution features of the image (low-quality sample image), the purpose being to select deep semantic information of the image. For specific application of the first encoder, please refer to the detailed description in the relevant part below, which will not be repeated here.

[0069] S103: input the initial feature into the transfer convolution network, train the transfer convolution network based on the first target feature, and obtain a target convolution network; wherein the first target feature is obtained based on the initial feature of the low-quality sample image; the feature clarity of the first target feature is stronger than that of the initial feature.

[0070] In this step, the first target feature is a feature that can be used for target detection and is obtained based on the initial feature of the low-quality sample image. The feature clarity of the first target feature is stronger than that of the initial feature. For details of the process of obtaining the first target feature, please refer to the detailed description of the related part below, which will not be repeated here.

[0071] After obtaining the initial feature of the low-quality sample image in step S102, the initial feature is input into the transfer convolution network (a convolution network to be trained), the transfer convolution network is trained based on the first target feature, and a trained transfer convolution network, i.e., a target convolution network, is obtained. The target convolution network is used for feature transfer, i.e., transferring the initial feature of the low-quality sample image to a feature that can be used for target detection (i.e., the second target feature described below).

[0072] S104: obtaining a second target feature for the low-quality sample image based on the target convolution network.

[0073] In this step, based on the target convolution network obtained by training, the initial feature can be directly transferred to a feature that can be used for target detection, i.e., the second target feature. The feature clarity of the second target feature is stronger than that of the initial feature. Since the second target feature is obtained by training the target convolution network based on the first target feature, and considering that deep learning training cannot be 100% accurate, the feature clarity of the second target feature is less than or equal to that of the first target feature. The reason for using the target convolution network to transfer the initial feature is that, compared with the scheme of using multiple models in series to obtain a feature for target detection, directly using the trained target convolution network to transfer the initial feature to obtain a feature for target detection can greatly reduce the amount of data calculation, reduce the processing time, ensure the accuracy of target detection, realize lightweight target detection, and improve the efficiency of target detection.

[0074] S105: input the second target feature into the first detection head, train the first detection head, and obtain a target detection head; the first detection head is obtained based on the first target feature.

[0075] In this step, the first detection head is a detection head obtained in advance based on the first target feature, and is used to perform target detection. After obtaining the feature used for target detection, i.e., the second target feature, in order to ensure that the first detection head can better adapt to the second target feature, thereby realizing the training consistency of the target convolutional network and the first detection head, the second target feature is input to the first detection head for training, and a target detection head is obtained.

[0076] S106: Obtain a target detection model based on the first encoder, the target convolutional network, and the target detection head; the target detection model is used for target detection on low-quality to-be-detected images.

[0077] In this step, the first encoder, the target convolutional network, and the target detection head obtained through the foregoing steps jointly constitute a target detection model. The target detection model can be applied to an actual scene to perform target detection on low-quality to-be-detected images.

[0078] In the scheme shown in steps S101-S106, a low-quality sample image for a target region is obtained; the low-quality sample image is input to the first encoder to obtain initial features of the low-quality sample image; the initial features are input to the transfer convolutional network, and the transfer convolutional network is trained based on the first target features to obtain a target convolutional network; wherein the first target features are obtained based on the initial features of the low-quality sample image; the feature clarity of the first target features is stronger than that of the initial features; based on the target convolutional network, second target features for the low-quality sample image are obtained; the second target features are input to the first detection head to train the first detection head to obtain a target detection head; the first detection head is obtained based on the first target features; based on the first encoder, the target convolutional network, and the target detection head, a target detection model is obtained; the target detection model is used for target detection on low-quality to-be-detected images. Efficient target detection can be realized.

[0079] In an optional scheme, in step S103: the initial features are input to the transfer convolutional network, the transfer convolutional network is trained based on the first target features to obtain a target convolutional network, including:

[0080] The initial features are input to the transfer convolutional network, the transfer convolutional network is trained based on the first target features to obtain a target convolutional network, including:

[0081] In the present disclosure, referring to FIG. 2, the initial feature obtained after the low-quality sample image is input into the first encoder is input into the transfer convolutional network, the first target feature is taken as the output feature of the transfer convolutional network, the transfer convolutional network is trained to obtain the target convolutional network. The initial feature is a feature extracted by the first encoder for image restoration, and the first target feature is a feature used for target detection. Due to the difference in the domains, the initial feature cannot be directly used for target detection, so the initial feature needs to be transferred before target detection. Specifically, the initial feature is input into the transfer convolutional network, the first target feature is taken as the output feature of the transfer convolutional network, and the transfer convolutional network is trained. The loss function of the transfer convolutional network is shown in formulas (1)-(4):

[0082] L = L p (f u ,k u ) + L r (f u ,k u ) + L t (f u ,k u ) Formula (1)

[0083] wherein,

[0084] In formulas (1)-(4), L is the loss function value of the transfer convolutional network. f is the initial feature, and k is the first target feature. u is the number of features. L p (f u ,k u ) is the point feature difference between the initial feature and the first target feature. L r (f u ,k u ) is the region feature difference between the initial feature and the first target feature. L t (f u ,k u ) is the channel feature difference between the initial feature and the first target feature. c, h, and w are three dimensions of the feature, c represents the number of channels of the feature, h represents the height of the feature, and w represents the width of the feature. f u,c,w,h represents the initial feature composed of u features with c channels, h height, and w width. k u,c,w,h represents the first target feature composed of u features with c channels, h height, and w width.

[0085] When the loss of the migration convolutional network is minimum, i.e., the value of L in the aforementioned formula (1) is minimum, the training of the migration convolutional network ends, and a trained target convolutional network is obtained. The target convolutional network can directly migrate the extracted initial features of the low-quality (sample) image to features that can be used for target detection, and can realize lightweight design of the model while not affecting the accuracy of the model.

[0086] In an optional solution, the first target feature is obtained based on the initial feature of the low-quality sample image, and includes:

[0087] The initial feature of the low-quality sample image is input into a pre-trained image type prediction model to obtain a low-quality type of the low-quality sample image.

[0088] The first target feature for the low-quality sample image is obtained based on the low-quality type of the low-quality sample image.

[0089] In the present disclosure, referring to FIG. 3, the initial feature of the low-quality sample image obtained through the first encoder is input into an image type prediction model. The image type prediction model is a pre-trained model used to predict the low-quality type of the low-quality sample image. The image type prediction model is trained by learning the features of images of different low-quality types in advance. The structure of the image type prediction model is shown in FIG. 4, including a convolutional layer, a pooling layer, and a fully connected layer. The convolutional layer and the pooling layer are used to extract features, and the fully connected layer is used to output the low-quality type (i.e., the cause of the low-quality of the image). For example, in a marine operation scenario, the low-quality types output by the fully connected layer include fog, rain, motion blur, sea clutter, etc., and this layer is used to analyze what kind of harsh conditions the current (low-quality sample) image has experienced, so as to use it as a high-level semantic prior to guide the first decoder (see the related description below) to complete clear image reconstruction (restoration) on the encoded feature (initial feature). The low-quality type of the same low-quality sample image can be one or multiple. For example, assuming that there is a low-quality sample image A, the initial feature of the image A is input into the image type prediction model, and the low-quality type (and its probability) of the image A can be obtained, such as the causes of the low-quality of the image A including sea wave shaking (20%) and water surface reflection (80%). Based on the low-quality type of the low-quality sample image, the first target feature for the low-quality sample image can be obtained. The specific obtaining process of the first target feature is described in detail in the related description below, which is not repeated here.

[0090] By predicting the low-quality type of the low-quality sample image, a data basis is provided for subsequent targeted restoration of the low-quality sample image to a clear image, and technical support is provided for realizing accurate target detection.

[0091] In an optional solution, the first target feature for the low-quality sample image is obtained based on the low-quality type of the low-quality sample image, and includes:

[0092] inputting the low-quality type of the low-quality sample image into the pre-trained content mapping network to obtain mapping encoding information for the low-quality type;

[0093] inputting the mapping encoding information and the initial feature of the low-quality sample image into the first decoder to obtain a restored sample image; the definition of the restored sample image is higher than that of the low-quality sample image;

[0094] based on the restored sample image, obtaining a first target feature for the low-quality sample image.

[0095] In the present disclosure, referring to FIG. 3, the output (low-quality type) of the image type prediction model is taken as the input of the content mapping network, and the output (mapping encoding information) of the content mapping network is taken as the input of the first decoder.

[0096] The structure of the content mapping network is shown in FIG. 5. The content mapping network takes the output of the image type prediction model as the input, expands the degradation cause probability (the probability of each low-quality type causing image degradation, such as the aforementioned 80%, 20%) to a two-dimensional tensor consistent with the corresponding feature map scale, then performs a unified convolution, and then performs two different convolutions respectively to generate γ and β which have the same number of channels and the same size as the processed features (the features of the low-quality type). Then γ is multiplied by the processed features, and β is added. This is equivalent to individually performing a regularization on each pixel point of each channel of the low-quality type feature, thereby fusing the prior information, encoding the prior information, and embedding it into the feature space of the first decoding layer for feature transformation calculation.

[0097] The first decoder has the same structure as the first encoder and can also use convolution modules, residual modules or Swin Transformer modules, etc. The difference is that the first decoder uses a layer-by-layer upsampling method to gradually reconstruct the features extracted by the first encoder to the spatial resolution of the low-quality sample image, and finally outputs a clear image (restored sample image) with the same spatial and channel size as the low-quality sample image. It can be understood that the first decoder is a decoder for image restoration.

[0098] The pre-trained content mapping network is used to convert the output of the image type prediction model into input recognizable by the first decoder, that is, to convert the low-quality type of the low-quality sample image into mapping encoding information recognizable by the first decoder. The mapping encoding information and the initial features of the low-quality sample image are jointly used as the input of the first decoder, and the first decoder reconstructs the initial features based on the mapping encoding information to obtain the restored sample image (a clear image). The data basis is provided for accurate target detection. The first target feature of the low-quality sample image is obtained based on the restored sample image, and the specific process is described in detail below.

[0099] In an optional solution, the first target feature of the low-quality sample image is obtained based on the restored sample image, including: inputting the restored sample image and the low-quality sample image into the second encoder to obtain the first target feature.

[0100] In the present disclosure, the second encoder is an encoder for target detection. Referring to FIG. 3, the restored sample image and the low-quality sample image are input into the second encoder, and the second encoder extracts features therefrom, which can further ensure the accuracy of the feature extraction of the second encoder and obtain a clearer and more accurate first target feature.

[0101] In an optional solution, the first detection head is obtained based on the first target feature, including: inputting the first target feature into the second detection head to be trained, training the second detection head to be trained to obtain the first detection head.

[0102] The trained first detection head can perform target detection. As shown in FIG. 6, the target detection head is obtained by training the first detection head, and is a structure obtained by equivalently combining the convolution modules of the first detection head. It can be understood that 1x1 convolution is equivalent to a special 3x3 convolution (with many 0s in the convolution kernel), and identity mapping is a special 1x1 convolution (with a unit matrix as the convolution kernel), so it can also be regarded as a special 3x3 convolution. Therefore, the 1x1 convolution in the first detection head is equivalently converted into a 3x3 convolution, and part of the feature space calculation is combined by using a mathematical combination operation. The capacity and effective feature space of the target detection head can be improved, and the forward calculation operation can be reduced when the target detection head is used for target detection, thereby improving the calculation efficiency of target detection. It should be noted that FIG. 6 only shows a structure of the target detection head, and the target detection head in the present disclosure is not limited to the structure shown in FIG. 6, and the present disclosure does not make a specific limitation in this regard.

[0103] The present disclosure provides an application method of a low-quality image target detection model (i.e., a low-quality image target detection method), as shown in FIG. 7, including:

[0104] S701: Obtain a low-quality to-be-detected image for a to-be-detected region.

[0105] In this step, the to-be-detected region is a region to be detected, and the to-be-detected region may or may not contain a target object. The low-quality to-be-detected image is an image with poor image quality (low definition). For example, in a marine operation scenario, the low-quality to-be-detected image can be a low-quality image caused by high humidity conditions such as fog and rain on the sea, or strong interference caused by rapid movement of a ship, severe sea conditions, complex reflection on the water surface, cluttered environment, and dynamic background.

[0106] S702: input the low-quality to-be-detected image into the target detection model to obtain a target detection result for the to-be-detected region; wherein the target detection result is obtained by performing feature extraction on the low-quality to-be-detected image by a first encoder in the target detection model, performing migration on the extracted features by a target convolutional network in the target detection model, and performing target detection based on the migrated features by a target detection head in the target detection model; and the target detection result is used to represent whether there is a target object in the to-be-detected region.

[0107] In this step, the low-quality to-be-detected image is input into the target detection model as shown in FIG. 8 to obtain a target detection result for the to-be-detected region. The target detection result can represent whether there is a target object in the to-be-detected region, and the position information of the target object if there is a target object. In a marine operation scenario, the target object can be an obstacle such as a rock. By detecting whether there is a rock in the to-be-detected region and the position information of the rock, the ship in the marine operation can timely avoid the obstacle or make other decisions (such as returning, etc.) for the obstacle. Specifically, the target detection result is obtained by performing feature extraction on the low-quality to-be-detected image by a first encoder in the target detection model, performing migration on the extracted features by a target convolutional network in the target detection model, and performing target detection based on the migrated features by a target detection head in the target detection model. The specific obtaining process of the first encoder, the target convolutional network, and the target detection head is described above, and is not repeated here.

[0108] The scheme shown in steps S701-S702 can be regarded as an application scheme of the low-quality image target detection model. The target detection model obtained through the scheme shown in steps S101-S106 can detect targets in the to-be-detected region. The disclosed scheme is a lightweight scheme that can not only improve the accuracy of target detection, but also improve the efficiency of target detection.

[0109] An embodiment of the present disclosure provides a training device for a low-quality image target detection model, as shown in FIG. 9, the device comprises:

[0110] The first acquisition unit 901 is configured to acquire a low-quality sample image for a target region.

[0111] The second acquisition unit 902 is configured to input the low-quality sample image into a first encoder to obtain initial features of the low-quality sample image.

[0112] The third acquisition unit 903 is configured to input the initial features into a transfer convolutional network, train the transfer convolutional network based on first target features, and obtain a target convolutional network; the first target features are obtained based on the initial features of the low-quality sample image; the feature clarity of the first target features is stronger than that of the initial features.

[0113] The fourth acquisition unit 904 is configured to obtain second target features for the low-quality sample image based on the target convolutional network.

[0114] The fifth acquisition unit 905 is configured to input the second target features into a first detection head, train the first detection head, and obtain a target detection head; the first detection head is obtained based on the first target features.

[0115] The sixth acquisition unit 906 is configured to obtain a target detection model based on the first encoder, the target convolutional network, and the target detection head; the target detection model is used for target detection on a low-quality to-be-detected image.

[0116] In an optional implementation, the third acquisition unit 903 is configured to input the initial features into the transfer convolutional network, train the transfer convolutional network by taking the first target features as features output by the transfer convolutional network, and obtain the target convolutional network.

[0117] In an optional implementation, the third acquisition unit 903 is configured to input the initial features of the low-quality sample image into a pre-trained image type prediction model to obtain a low-quality type of the low-quality sample image; and obtain the first target features for the low-quality sample image based on the low-quality type of the low-quality sample image.

[0118] In an optional implementation, the third acquisition unit 903 is configured to input the low-quality type of the low-quality sample image into a pre-trained content mapping network to obtain mapping encoding information for the low-quality type; input the mapping encoding information and the initial features of the low-quality sample image into a first decoder to obtain a restored sample image; the clarity of the restored sample image is stronger than that of the low-quality sample image; and obtain the first target features for the low-quality sample image based on the restored sample image.

[0119] In an optional solution, the third acquisition unit 903 is configured to input the recovered sample image and the low-quality sample image into a second encoder to obtain a first target feature.

[0120] The fifth acquisition unit 905 is configured to input the first target feature into the second detection head to be trained, train the second detection head to be trained, and obtain the first detection head.

[0121] Embodiments of the present disclosure provide a low-quality image target detection device, as shown in FIG. 10, which comprises:

[0122] A seventh acquisition unit 1001 is configured to acquire a low-quality to-be-detected image for a to-be-detected region.

[0123] A detection unit 1002 is configured to input the low-quality to-be-detected image into a target detection model to obtain a target detection result for the to-be-detected region; wherein the target detection result is obtained by performing feature extraction on the low-quality to-be-detected image by a first encoder in the target detection model, performing migration on the extracted features by a target convolutional network in the target detection model, and performing target detection based on the migrated features by a target detection head in the target detection model; and the target detection result is used to represent whether a target object exists in the to-be-detected region.

[0124] It should be noted that the training device of the low-quality image target detection model and the low-quality image target detection device of the embodiments of the present disclosure have similar problem solving principles to the training method of the low-quality image target detection model and the low-quality image target detection method described above, and therefore the implementation process and implementation principles, and the beneficial effects of the training device of the low-quality image target detection model and the low-quality image target detection device can be referred to the descriptions of the implementation process and implementation principles, and the beneficial effects of the aforementioned method, and the repeated parts will not be described herein.

[0125] According to embodiments of the present disclosure, the present disclosure further provides an electronic device and a readable storage medium.

[0126] FIG. 11 shows a schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0127] As shown in FIG. 11, the electronic device 1100 includes a computing unit 1101 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded into a random access memory (RAM) 1103 from a storage unit 1108. Various programs and data required for the operation of the electronic device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0128] Various components in the electronic device 1100 are connected to the I / O interface 1105, including an input unit 1106 such as a keyboard, a mouse, and the like, an output unit 1107 such as various types of displays, a speaker, and the like, a storage unit 1108 such as a magnetic disk, an optical disk, and the like, and a communication unit 1109 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0129] The computing unit 1101 can be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 1101 performs various methods and processes described above, such as the training and detection methods of the low-quality image object detection model. For example, in some embodiments, the training and detection methods of the low-quality image object detection model can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the training and detection methods of the low-quality image object detection model described above can be performed. Alternatively, in other embodiments, the computing unit 1101 can be configured to perform the training and detection methods of the low-quality image object detection model by any other appropriate means, such as by means of firmware.

[0130] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0131] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0132] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0133] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0134] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0135] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is generally established by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0136] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without departing from the desired results of the technology disclosed in the present disclosure, which are not limited herein.

[0137] In addition, the terms "first", "second", etc., are used herein only to describe different instances, and do not imply or suggest relative importance or imply the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0138] The above merely provides the specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present disclosure, which should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for training a low-quality image target detection model, the method comprising: obtaining a low-quality sample image for a target region; inputting the low-quality sample image into a first encoder to obtain initial features of the low-quality sample image; inputting the initial features into a transfer convolutional network, training the transfer convolutional network based on first target features to obtain a target convolutional network, wherein the first target features are obtained based on the initial features of the low-quality sample image, and a feature clarity of the first target features is stronger than that of the initial features; obtaining second target features for the low-quality sample image based on the target convolutional network; inputting the second target features into a first detection head, training the first detection head to obtain a target detection head, wherein the first detection head is obtained based on the first target features; obtaining a target detection model based on the first encoder, the target convolutional network and the target detection head, wherein the target detection model is used for target detection on a low-quality image to be detected.

2. The method of claim 1, wherein the initial features are inputted into the transfer convolutional network, the transfer convolutional network is trained based on the first target features to obtain the target convolutional network, and the method comprises: inputting the initial features into the transfer convolutional network, training the transfer convolutional network based on the first target features as features outputted by the transfer convolutional network to obtain the target convolutional network.

3. The method of claim 1 or 2, wherein the first target features are obtained based on the initial features of the low-quality sample image, and the method comprises: inputting the initial features of the low-quality sample image into a pre-trained image type prediction model to obtain a low-quality type of the low-quality sample image; obtaining the first target features for the low-quality sample image based on the low-quality type of the low-quality sample image.

4. The method of claim 3, wherein the first target features for the low-quality sample image are obtained based on the low-quality type of the low-quality sample image, and the method comprises: inputting the low-quality type of the low-quality sample image into a pre-trained content mapping network to obtain mapping encoding information for the low-quality type; inputting the mapping encoding information and the initial features of the low-quality sample image into a first decoder to obtain a restored sample image, wherein a clarity of the restored sample image is stronger than that of the low-quality sample image; obtaining the first target features for the low-quality sample image based on the restored sample image.

5. The method of claim 4, wherein the first target features for the low-quality sample image are obtained based on the restored sample image, and the method comprises: inputting the restored sample image and the low-quality sample image into a second encoder to obtain the first target features; the first detection head is obtained based on the first target features, and the method comprises: inputting the first target features into a second detection head to be trained to train the second detection head to be trained to obtain the first detection head.

6. A low-quality image target detection method, the method comprising: obtaining a low-quality image to be detected for a detection region; The low-quality image to be detected is input into the target detection model to obtain a target detection result for the to-be-detected region; wherein the target detection result is obtained by performing feature extraction on the low-quality image to be detected by a first encoder in the target detection model, performing migration on the extracted features by a target convolution network in the target detection model, and performing target detection based on the migrated features by a target detection head in the target detection model. The target detection result is used to represent whether a target object exists in the to-be-detected region.

7. A device for training a low-quality image target detection model, the device comprising: a first obtaining unit configured to obtain a low-quality sample image for a target region; a second obtaining unit configured to input the low-quality sample image into a first encoder to obtain initial features of the low-quality sample image; a third obtaining unit configured to input the initial features into a migration convolution network, train the migration convolution network based on first target features, and obtain a target convolution network; wherein the first target features are obtained based on the initial features of the low-quality sample image; and the feature clarity of the first target features is stronger than that of the initial features; a fourth obtaining unit configured to obtain second target features for the low-quality sample image based on the target convolution network; a fifth obtaining unit configured to input the second target features into a first detection head, train the first detection head, and obtain a target detection head; the first detection head is obtained based on the first target features; a sixth obtaining unit configured to obtain a target detection model based on the first encoder, the target convolution network, and the target detection head; the target detection model is used for target detection on a low-quality image to be detected.

8. A device for low-quality image target detection, the device comprising: a seventh obtaining unit configured to obtain a low-quality image to be detected for a to-be-detected region; a detection unit configured to input the low-quality image to be detected into a target detection model to obtain a target detection result for the to-be-detected region; wherein the target detection result is obtained by performing feature extraction on the low-quality image to be detected by a first encoder in the target detection model, performing migration on the extracted features by a target convolution network in the target detection model, and performing target detection based on the migrated features by a target detection head in the target detection model; the target detection result is used to represent whether a target object exists in the to-be-detected region.

9. An electronic device comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5 or 6.

10. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-5 or 6.

Images