Method, device, processor and electronic device for determining the quality of steel bar binding

Abstract

The present invention discloses a method, device, processor and electronic device for determining the quality of steel bar binding. The present invention relates to the field of image processing technology. The method comprises: obtaining an initial image to be processed of steel bars at a steel bar binding site; inputting the initial image into an image processing model for deblurring to obtain a target image corresponding to the initial image, wherein the image quality of the target image is higher than the image quality of the initial image, and the image processing model is obtained by training a deep convolutional neural network model using the initial image samples and the target image samples, and the image quality of the target image samples is higher than the image quality of the corresponding initial image samples; based on the target image, determining the binding quality of steel bars at the steel bar binding site. The present invention solves the technical problem of low accuracy in determining the quality of steel bar binding.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, processor and electronic device for determining the quality of steel bar binding. Background Art

[0002] At present, when using drones to conduct quality inspections on steel bar binding sites, the video shot at the steel bar binding site can be intercepted frame by frame to obtain images of the steel bar binding site. The steel bar binding points can be identified from the images to determine whether the binding quality meets the standards.

[0003] However, the above method is susceptible to wind during the flight shooting process, or the drone itself may be in the process of flying, and the captured images may have motion jitter and blur, resulting in the loss of detailed information of the steel bar binding points in the image. Therefore, there is still a technical problem of low accuracy in determining the quality of steel bar binding.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] Embodiments of the present invention provide a method, device, processor and electronic device for determining the quality of steel bar binding, so as to at least solve the technical problem of low accuracy in determining the quality of steel bar binding.

[0006] According to one aspect of an embodiment of the present invention, a method for determining the quality of steel bar binding is provided, comprising: obtaining an initial image to be processed of steel bars at a steel bar binding site; inputting the initial image into an image processing model for deblurring processing to obtain a target image corresponding to the initial image, wherein the image quality of the target image is higher than the image quality of the initial image, the image processing model is obtained by training a deep convolutional neural network model using initial image samples and target image samples, and the image quality of the target image samples is higher than the image quality of the corresponding initial image samples; based on the target image, determining the binding quality of the steel bars at the steel bar binding site.

[0007] Optionally, the image processing model is a multi-scale parallel residual network model, wherein, before the initial image is input into the multi-scale parallel residual network model for deblurring to obtain a target image corresponding to the initial image, the method also includes: blurring the target image sample to obtain an initial image sample, and constructing a data set of the steel bar binding site based on the target image sample and the initial image sample; preprocessing the data set to obtain a preprocessed data set, wherein the amount of image information in the preprocessed data set is higher than the amount of image information in the data set before preprocessing; building an initial multi-scale parallel residual network model based on a deep convolutional neural network model; training the initial multi-scale parallel residual network model using a training data set in the preprocessed data set, and verifying the trained initial multi-scale parallel residual network model using a verification data set in the preprocessed data set to obtain a multi-scale parallel residual network model, wherein the data set includes a training data set and a verification data set.

[0008] Optionally, blurring the target image sample to obtain an initial image sample includes: performing a convolution operation in a computer vision library on the target image sample to obtain the corresponding initial image sample, wherein the convolution operation is used to simulate the relative movement between the image acquisition device and the plane of the steel bar binding site; the method also includes: determining the target image sample as label data of the initial image sample, wherein the label data is used to compare the target image sample with the corresponding target image obtained after the initial image sample is deblurred.

[0009] Optionally, constructing a training data set and a verification data set based on the target image samples and the initial image samples includes: determining the training data set and the verification data set by dividing the target image samples and the initial image samples in the data set according to a target ratio.

[0010] Optionally, the data set is preprocessed to obtain a preprocessed data set, including: downsampling the initial image samples in the data set to obtain downsampled initial image samples, wherein the image size of the downsampled initial image samples is smaller than the image size of the initial image samples before downsampling; in response to obtaining the downsampled initial image samples, processing the initial image samples using a superior gradient shock filter to obtain processed initial image samples, wherein the superior gradient shock filter is used to increase the amount of image information in the initial image samples.

[0011] Optionally, the optimal gradient shock filter includes Gaussian filter convolution, wherein the initial image sample is processed using the optimal gradient shock filter to obtain the processed initial image sample, including: determining the directional derivative of the initial image sample in the gradient direction; using the directional derivative, performing edge detection on the initial image sample to obtain a detection result, wherein the detection result is used to indicate whether the edge of the image content in the initial image sample is a steel bar; performing Gaussian filter convolution on the initial image sample to obtain gradient data of the initial image sample at each pixel point, wherein the Gaussian filter convolution is used to determine the texture edge of the initial image sample; and determining the processed initial image sample based on the gradient data and the detection result.

[0012] Optionally, the initial multi-scale parallel residual network model includes an upper network and a lower network, and the upper network and the lower network respectively include a single-path feature fusion residual module, a dual-path multi-scale feature fusion module, an ordinary convolution module and a transposed convolution module, wherein, based on the deep convolutional neural network model, an initial multi-scale parallel residual network model is constructed, including: in the upper network and the lower network, ordinary convolution modules and transposed convolution modules are deployed, wherein the ordinary convolution modules are used to extract the image information in the initial image sample and to reduce the image size of the initial image sample, and the transposed convolution module is used to increase the image size and to increase the image information; in the upper network and the lower network, a single-path feature fusion residual module is deployed, wherein the first ordinary convolution module, the second ordinary convolution module and the third ordinary convolution module are deployed in sequence in the single-path feature fusion residual module, the convolution kernel and the step size of the first ordinary convolution module and the second ordinary convolution module are the same, and the first ordinary convolution module and the second ordinary convolution module are the same. It is used to connect with the third ordinary convolution module after the channel splicing operation, and the convolution kernel of the third ordinary convolution module is smaller than the convolution kernel of the first ordinary convolution; in the upper network and the lower network, a dual-path multi-scale feature fusion module is deployed, wherein the dual-path multi-scale feature fusion module is used to perceive the jitter motion trend characteristics of the initial image sample at multiple scales and restore the texture information of the steel bar; in the upper network, two single-path feature fusion residual modules, one dual-path multi-scale feature fusion module, two single-path feature fusion residual modules, one dual-path multi-scale feature fusion module, one transposed convolution module, two single-path feature fusion residual modules and an ordinary convolution module are deployed in sequence, and in the lower network, two single-path feature fusion residual modules, one dual-path multi-scale feature fusion module, two single-path feature fusion residual modules, one transposed convolution module, two single-path feature fusion residual modules and an ordinary convolution module are deployed in sequence to obtain an initial multi-scale parallel residual network model.

[0013] Optionally, the initial multi-scale parallel residual network model is trained using a training data set in the preprocessed data set, including: initializing parameters of the training initial multi-scale parallel residual network model to obtain initialized parameters; in response to completion of parameter initialization, inputting an initial image sample of a first image size in the training data set into an upper network, and inputting an initial image sample of a second image size in the training data set into a lower network, wherein the first image size is larger than the second image size; in response to the upper network and the lower network receiving corresponding initial image samples, training the initial multi-scale parallel residual network model; the method further includes: in training the initial multi-scale parallel residual network model, In the process of constructing an initial multi-scale parallel residual network model, a target image determined based on an initial image sample is obtained; initial loss data of the initial multi-scale parallel residual network model is determined based on Fourier changes corresponding to the target image and the label data of the initial image sample; target loss data of the initial multi-scale parallel residual network model is determined based on the initial loss data and the perceptual loss data of the initial multi-scale parallel residual network model; in response to the target loss data reaching a convergence state, the initial multi-scale parallel residual network model is determined as the target multi-scale parallel residual network model, wherein the convergence state is used to characterize that the similarity between the label data and the corresponding target image reaches a similarity threshold.

[0014] Optionally, the trained initial multi-scale parallel residual network model is verified using a verification data set in the preprocessed data set to obtain a multi-scale parallel residual network model, including: inputting the initial image samples in the verification data set into the target multi-scale parallel residual network model, and using the target multi-scale parallel residual network model to deblur the initial image samples to obtain a target image corresponding to the initial image samples; in response to the similarity between the label data of the target image and the corresponding initial image sample reaching a similarity threshold, determining the target multi-scale parallel residual network model as a multi-scale parallel residual network model.

[0015] According to another aspect of an embodiment of the present invention, a device for determining the quality of steel bar binding is also provided, including: an acquisition unit, used to acquire an initial image to be processed of steel bars at a steel bar binding site; a processing unit, used to input the initial image into an image processing model for deblurring processing to obtain a target image corresponding to the initial image, wherein the image quality of the target image is higher than the image quality of the initial image, and the image processing model is obtained by training a deep convolutional neural network model using the initial image samples and the target image samples, and the image quality of the target image samples is higher than the image quality of the corresponding initial image samples; a determination unit, used to determine the binding quality of steel bars at the steel bar binding site based on the target image.

[0016] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing any one of the above-mentioned methods for determining the quality of steel bar binding.

[0017] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any one of the above-mentioned methods for determining the quality of steel bar binding.

[0018] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a computer program. When the computer program is executed by a processor, it can be used to execute any one of the above-mentioned methods for determining the quality of steel bar binding.

[0019] In an embodiment of the present invention, if it is necessary to detect the binding quality of the steel bar binding points at the steel bar binding site, an initial image with poor image quality taken from the steel bar binding site can be obtained. In order to ensure the accuracy of the binding quality detection, the initial image can be input into the image processing model for deblurring, so that the image quality is improved and the corresponding target image is obtained. Thus, the target image with higher image quality can be used to determine the quality of the steel bar binding at the steel bar binding site. Since the embodiment of the present invention takes into account the low accuracy of determining the binding quality by directly collecting the image, an image processing model can be pre-trained to deblur the acquired image, that is, the relatively blurred initial image can be adjusted to a clearer target image. Thereby, the purpose of realizing the deblurring and restoration project of the steel bar binding site image is achieved, and then the technical effect of improving the accuracy of determining the steel bar binding quality is achieved, and the technical problem of low accuracy in determining the steel bar binding quality is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 is a flow chart of a method for determining the quality of steel bar binding according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a method for deblurring a UAV image at a steel bar binding site according to an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a multi-scale parallel residual network structure according to an embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of a single-channel feature fusion residual module according to an embodiment of the present invention;

[0025] Figure 5 is a schematic diagram of a dual-path multi-scale feature fusion module of an upper network according to an embodiment of the present invention;

[0026] Figure 6 is a schematic diagram of a dual-path multi-scale feature fusion module of a lower layer network according to an embodiment of the present invention;

[0027] Figure 7 is a structural schematic diagram of a device for determining the quality of steel bar binding according to an embodiment of the present invention;

[0028] Figure 8 is a schematic diagram of an electronic device for determining a method for steel bar binding quality according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Example 1

[0032] According to an embodiment of the present invention, an embodiment of a method for determining the quality of steel bar binding is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 1 is a flow chart of a method for determining the quality of steel bar binding according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0034] Step S102, obtaining an initial image of the steel bars to be processed at the steel bar binding site.

[0035] In the technical solution provided in the above step S102 of the embodiment of the present invention, the initial image may be a jittery and blurred image of the steel bar binding site taken by a drone.

[0036] In this embodiment, an initial image to be processed at the steel bar binding site may be acquired.

[0037] Optionally, if the quality of the steel bar binding at the steel bar binding site needs to be determined, the drone can be controlled to shoot a video of the steel bar binding site. A frame screenshot can be taken from the captured video to obtain a corresponding initial image.

[0038] Step S104, inputting the initial image into the image processing model for deblurring, and obtaining a target image corresponding to the initial image.

[0039] In the technical solution provided in the above step S104 of the embodiment of the present invention, the image processing model can be used to deblur the input image to obtain an image with higher image quality corresponding to the image, which can be a multi-scale parallel residual network model. The image processing model can be obtained by training a deep convolutional neural network model using initial image samples and target image samples. The image quality of the target image is higher than the image quality of the initial image. The image quality of the target image sample is higher than the image quality of the corresponding initial image sample. The target image can be a clear, quality-enhanced image of the steel bar binding site output by the image processing model. Image quality can be used to represent the clarity of the image and the amount of detail information displayed in the image.

[0040] It should be noted that the above-mentioned image quality is only for illustrative purposes and is not specifically limited here. As long as the multi-scale parallel residual network model can be used to improve the image quality, the method and process are within the protection scope of the embodiments of the present application.

[0041] In this embodiment, after obtaining the initial image to be processed of the steel bars at the steel bar binding site, the initial image can be input into the image processing model, and the image initial model can be used to deblur the received initial image to obtain a target image with higher image quality.

[0042] Optionally, during the deblurring application, the jittery and blurred steel bar binding scene image taken by the drone can be input into a pre-trained multi-scale parallel residual network. After calculation and processing by the multi-scale parallel residual network model, the corresponding clear and quality-enhanced steel bar binding scene image can be output.

[0043] Since the images of the steel bar binding site taken by the drone may be affected by many factors, such as wind, drone shaking and insufficient light, the images taken may be blurred. Therefore, if the binding quality inspection is performed based on such blurred images, there will be a technical problem of low accuracy of the binding quality inspection. However, in the embodiment of the present application, the deblurring preprocessing technology of the image processing model can be used to restore the clarity and details of the image as much as possible, making it easier to analyze and identify the steel bar binding points, thereby achieving the purpose of improving the image quality of the steel bar binding site, and then achieving the technical effect of improving the accuracy of the binding quality inspection.

[0044] Step S106, determining the binding quality of the steel bars at the steel bar binding site based on the target image.

[0045] In the technical solution provided in the above step S106 of the embodiment of the present invention, the binding quality can be used to indicate the degree of compliance of the binding quality of the steel bar binding points at the binding site.

[0046] In this embodiment, after the initial image is input into the image processing model for deblurring and the target image corresponding to the initial image is output, the target image can be used to determine the binding quality of the steel bars.

[0047] Optionally, after acquiring the target image, the steel bar binding point can be identified and located from the target image, and it can be determined whether the binding quality of the steel bar binding point meets the standard.

[0048] In the technical solution provided by the above steps S102 to S106 of the embodiment of the present invention, if it is necessary to detect the binding quality of the steel bar binding points at the steel bar binding site, an initial image with poor image quality taken from the steel bar binding site can be obtained. In order to ensure the accuracy of the binding quality detection, the initial image can be input into the image processing model for deblurring, so that the image quality is improved and the corresponding target image is obtained. Thus, the target image with higher image quality can be used to determine the quality of the steel bar binding at the steel bar binding site. Since the embodiment of the present invention takes into account the low accuracy of determining the binding quality by directly collecting the image, an image processing model can be pre-trained to deblur the acquired image, that is, the relatively blurred initial image can be adjusted to a clearer target image. Thereby, the purpose of realizing the deblurring restoration project of the steel bar binding site image is achieved, and then the technical effect of improving the accuracy of determining the steel bar binding quality is achieved, and the technical problem of low accuracy in determining the steel bar binding quality is solved.

[0049] The embodiment of the present invention is described in detail below in combination with the above steps.

[0050] As an optional embodiment, the image processing model is a multi-scale parallel residual network model, wherein, before the initial image is input into the image processing model for deblurring processing to obtain the target image corresponding to the initial image, the method also includes: blurring the target image sample to obtain the initial image sample, and constructing a data set of the steel bar binding site based on the target image sample and the initial image sample; preprocessing the data set to obtain a preprocessed data set, wherein the amount of image information in the preprocessed data set is higher than the amount of image information in the data set before preprocessing; building an initial multi-scale parallel residual network model based on a deep convolutional neural network model; using the training data set in the preprocessed data set to train the initial multi-scale parallel residual network model, and using the verification data set in the preprocessed data set to verify the trained initial multi-scale parallel residual network model to obtain a multi-scale parallel residual network model, wherein the data set includes a training data set and a verification data set.

[0051] In this embodiment, before using the image processing model to deblur the initial image, the target image sample can be blurred to obtain the initial image sample. And based on the initial image sample and the target image sample, a data set of the steel bar binding site can be constructed. The data set can be preprocessed to obtain a preprocessed data set. An initial multi-scale parallel residual network model can be built based on a deep convolutional neural network model. The initial multi-scale parallel residual network model can be trained using a training data set in the preprocessed data set. The trained initial multi-scale parallel residual network model can also be verified using a verification data set in the preprocessed data set to obtain a multi-scale parallel residual network model that is finally trained and verified.

[0052] Optionally, the image processing model can be a multi-scale parallel residual network model. The target image sample can be a clear image of the steel bar binding site taken by a drone, or a set of pre-prepared clear images, or a data set D1. The initial image sample can be an image after the target image sample is blurred, or a set of blurred images obtained by blurring the clear image, that is, a blurred steel bar binding site image data set, or a data set D2. The data set of the steel bar binding site can also be called a steel bar binding site image data set D0 for training a deep convolutional neural network model, wherein the data set D0 can include not only the steel bar binding site image data taken by the drone, but also the processed steel bar binding site image data. It should be noted that the image data contained in the above-mentioned data set D0 is only for example, and no specific limitation is made here. As long as the image data can be used to train the deep convolutional neural network model to obtain the final multi-scale parallel residual network model, it is within the protection scope of the embodiment of the present invention.

[0053] Optionally, the amount of image information may be image details. The image details of the preprocessed data set are greater than those of the data set before preprocessing, that is, preprocessing can be used to enhance the details of the image. The initial multi-scale parallel residual model may be a multi-scale parallel residual network constructed to deblur the images of the steel bar binding site taken by the untrained drone, and may also be referred to as a network model. The data set may include a training data set and a verification data set. The training data set may be a data set required for training the network model, and may also be referred to as a training data set train. The verification data set may be a data set required for verifying the trained network model, and may also be referred to as a verification data set val.

[0054] Optionally, if it is necessary to pre-process the steel bar binding scene taken by the drone to remove blur, a multi-scale parallel residual network model can be prepared in advance. In the process of preparing the model, the process of data set construction, image pre-processing, initial multi-scale parallel residual network construction, and multi-scale parallel residual network training can be performed.

[0055] Optionally, the clear image of the steel bar binding site taken by the drone is blurred, and the blurred image and its corresponding clear image together constitute a data set. That is, the pre-prepared target image sample can be blurred to obtain the corresponding initial image sample, so that the target image sample and the initial image sample can together constitute a data set.

[0056] Optionally, the input blurred image may be preprocessed to enhance the details of the blurred image, that is, the initial image sample obtained by the blurring may be preprocessed to increase the image details therein.

[0057] Optionally, a multi-scale parallel residual network can be constructed according to the deep convolutional neural network, that is, an initial multi-scale parallel residual network model can be established based on the deep convolutional neural network model.

[0058] Optionally, training parameters (network parameters) are given to the initial multi-scale parallel residual network model, and the constructed training set can be input into the initial multi-scale parallel residual network model for model training. The initial multi-scale parallel residual network model is continuously fed back and corrected by the constructed loss function, and the network parameters can be continuously updated. After the training is completed, the test set can be input into the trained network model to verify the effect of the model. If the effect of the model meets the standard, the final trained and verified multi-scale parallel residual network model can be determined.

[0059] As an optional embodiment, blurring the target image sample to obtain the initial image sample includes: performing a convolution operation in a computer vision library on the target image sample to obtain the corresponding initial image sample, wherein the convolution operation is used to simulate the relative movement between the image acquisition device and the plane of the steel bar binding site; the method also includes: determining the target image sample as the label data of the initial image sample, wherein the label data is used to compare the target image sample with the corresponding target image obtained after the initial image sample is deblurred.

[0060] In this embodiment, in the process of blurring the target image sample to obtain the initial image sample, the convolution operation in the computer vision library can be used to process the target image sample to obtain the corresponding initial image sample. And the target image sample can be set as the label data of the initial image sample.

[0061] Optionally, the computer vision library may be an open source computer vision library (OpenCV), which provides a wealth of image processing and computer vision algorithms. The convolution operation may be used to convolve the target image sample to simulate the relative motion between the image acquisition device and the plane where the steel bar binding site is located, for example, the cv2.filter2D() method.

[0062] It should be noted that the convolution operation performed above is only an example and is not specifically limited here. The image acquisition device may be a drone camera, which may also be referred to as a drone. The label data may be used to compare the target image sample with the target image obtained after the initial image sample is deblurred, and may also be referred to as a label value label, that is, it may be used to compare with the image result after the blurred image is deblurred.

[0063] Optionally, clear images of the steel bar binding site taken by a drone are manually screened out as the data set D1.

[0064] Optionally, the clear image in D1 is processed using the cv2.filter2D() method in the OpenCV library to simulate and generate a steel bar binding site image dataset D2 that is blurred by relative motion noise between the drone and the steel bar binding site plane. The clear image in D1 and its corresponding blurred image in D2 form a set of reference data pairs, which together constitute the dataset D0 required to determine the multi-scale parallel residual network model. The clear image in D1 can be used as the label value label of the blurred image in D2 to compare with the image result after the blurred image is deblurred.

[0065] Optionally, the cv2.filter2D() method can perform a custom convolution operation on the image, and by applying a specific convolution kernel (filter) to each pixel of the image, it can achieve image smoothing, sharpening, edge detection and other functions. Users can customize the value of the convolution kernel as needed to achieve different image processing effects. This method is widely used in simulating image motion blur, image enhancement, edge detection, etc. Use the cv2.filter2D() method to blur the clear image. By passing in the motion blur kernel matrix parameters of any angle, it simulates the blur effect of the drone image caused by flight or wind shaking, which provides convenience for constructing a blurred rebar binding site image dataset.

[0066] As an optional embodiment, constructing a training data set and a verification data set based on target image samples and initial image samples includes: determining the training data set and the verification data set according to a target ratio of the target image samples and the initial image samples in the data set.

[0067] In this embodiment, in the process of constructing a training data set and a verification data set based on the target image sample and the initial image sample, the data set can be split into a training data set and a verification data set according to a certain target ratio, wherein the target ratio can be a split ratio determined based on a large number of experiments and experience, or can be set according to actual training needs, for example, it can be pre-set to 8:2. It should be noted that the above target ratio is only for example and is not specifically limited here.

[0068] Optionally, the constructed rebar binding site image dataset D0 photographed by a drone can be divided into a training dataset train and a validation dataset val according to a ratio of 8:2.

[0069] For example, 1600 clear images of steel bar binding sites taken by drones at different heights and angles are selected as data set D1, with a resolution of 1280x720 pixels. The cv2.filter2D() method in the OpenCV library is used to perform convolution calculations on the clear images in D1, and 1600 blurred images simulating the motion jitter blur of drones are obtained as D2. D1 is used as the label of D2 to form D0, so D0 has 1600 pairs (3200 images) of data sets, the training data set train has 1280 pairs of data sets, and the validation data set val has 320 pairs of data sets.

[0070] As an optional embodiment, a data set is preprocessed to obtain a preprocessed data set, including: downsampling the initial image samples in the data set to obtain downsampled initial image samples, wherein the image size of the downsampled initial image samples is smaller than the image size of the initial image samples before downsampling; in response to obtaining the downsampled initial image samples, the initial image samples are processed using a superior gradient shock filter to obtain processed initial image samples, wherein the superior gradient shock filter is used to increase the amount of image information in the initial image samples.

[0071] In this embodiment, in the process of preprocessing the data set to obtain the preprocessed data set, the initial image samples and the target image samples in the data set can be downsampled to obtain the downsampled initial image samples and the target image samples. After downsampling, the initial image samples and the target image samples can be processed using a superior gradient shock filter to increase the image details in the initial image samples and the target image samples, and obtain the target image samples and the initial image samples with increased image details after processing, wherein the image scale of the downsampled initial image samples and the target image samples is smaller than the image size of the initial image samples and the target image samples before downsampling. The superior gradient shock filter can be used to increase the image details in the initial image samples and the target image samples.

[0072] Optionally, since the multi-scale parallel residual network model in the embodiment of the present invention needs to receive two sizes of images as input, namely 512×512 and 256×256, it is considered that the steel bar binding site images in the dataset D0 are taken by a drone, and the actual drone resolution is 1280×720, 1920×1080, etc.

[0073] Optionally, based on the above analysis, the bilinear interpolation method can be used to downsample the blurred image u in the data set D0. After downsampling, the blurred image u can have two different sizes of 512×512 and 256×256. Considering that the blurred image u will lose too much detail information after downsampling, the optimal gradient shock filter method can be used to enhance the texture details of the blurred image u after downsampling.

[0074] Optionally, downsampling refers to the process of reducing the resolution of an image in digital image processing. This process usually involves reducing the number of pixels in the image, thereby reducing the size of the image. Typically, downsampling can be achieved by simply discarding some pixels or merging pixels. In image processing, downsampling can be used to reduce the size of an image, thereby reducing the storage space and computing resources occupied by the image. In addition, downsampling can also be used to reduce the resolution of an image, thereby reducing the details in the image, which makes sense in some cases, such as in some image processing tasks, where the complexity of the image needs to be reduced in order for the algorithm to process the image faster. However, it should be noted that downsampling can cause information loss in the image, especially for some details and textures, so careful consideration should be given when downsampling. In some cases, some interpolation techniques can be used to preserve the details of the image as much as possible, such as bilinear interpolation or bicubic interpolation.

[0075] Optionally, when bilinear interpolation downsampling is performed on the blurred image u, a point of the image before downsampling is set as point P, and the coordinates of point P are expressed as P = (x, y). The downsampled value f(x, y) at point P = (x, y) is required. The four points Q closest to point P with known values ​​are obtained. 11 , Q 12 , Q 21 , Q 22 The coordinates of the four points are represented by Q 11 =(x1, y1), Q 12 =(x1, y2), Q 21 =(x2, y1), Q 22 =(x2,y2).

[0076] Optionally, linear interpolation is performed in the x direction, and the calculation process is shown in the following formula:

[0077]

[0078]

[0079] Among them, R1=(x, y1), R2=(x, y2).

[0080] Optionally, linear interpolation is performed in the y direction, and the calculation process is shown in the following formula:

[0081]

[0082] Alternatively, the value f(x, y) at the point P = (x, y) calculated by bilinear interpolation is as shown below:

[0083]

[0084] As an optional embodiment, the optimal gradient shock filter includes Gaussian filter convolution, wherein the initial image sample is processed using the optimal gradient shock filter to obtain the processed initial image sample, including: determining the directional derivative of the initial image sample in the gradient direction; using the directional derivative, performing edge detection on the initial image sample to obtain a detection result, wherein the detection result is used to indicate whether the edge of the image content in the initial image sample is a steel bar; performing Gaussian filter convolution on the initial image sample to obtain gradient data of the initial image sample at each pixel point, wherein the Gaussian filter convolution is used to determine the texture edge of the initial image sample; and determining the processed initial image sample based on the gradient data and the detection result.

[0085] In this embodiment, in the process of processing the initial image sample by using the optimal gradient shock filter, the directional derivative of the initial image sample in the gradient direction can be determined, and the directional derivative can be used to perform edge detection on the initial image sample to obtain a detection result. The initial image sample can be processed by Gaussian filter convolution to obtain the gradient data of the initial image sample at each pixel point. Based on the gradient data and the detection result, the initial image sample processed by the optimal gradient shock filter can be determined.

[0086] Optionally, the detection result can be used to indicate whether the edge of the image content in the initial image sample is a steel bar. Gaussian filter convolution can be used to determine the texture edge of the initial image sample. The gradient data can be a gradient estimate. The directional derivative can be a second-order directional derivative of the blurred image u along the gradient direction. The gradient can be used to indicate that the directional derivative of a function at this point achieves a maximum value along this direction, that is, the function changes fastest along the direction (the direction of the gradient) at this point.

[0087] In the embodiment of the present invention, the optimal gradient shock filtering method can be used to first apply Gaussian filter convolution to process the image, and then calculate the gradient of the image after the Gaussian filter convolution process.

[0088] Alternatively, the equation for the original shock filtering method is shown below:

[0089]

[0090] Among them, u t It can be used to represent the blurred image obtained by the impact filtering method; u ηη It can be used to represent the second-order directional derivative of the blurred image u along the gradient direction; sign(u ηη ) can be used to represent edge detection items.

[0091] Optionally, u ηη It can be calculated by the following formula:

[0092]

[0093] Among them, u x and u y Can be used to represent the first-order variance in the x and y directions respectively; u xx 、u yy and u xy It can be used to represent the second-order difference and mixed difference in the x and y directions respectively.

[0094] Optionally, calculating the gradient of the image after Gaussian filtering can eliminate the noise caused by optical problems in some drone photography, which helps to restore higher quality images of the steel bar binding site.

[0095] Optionally, based on the equation of the original impact filtering method, the equation of the optimal gradient product filtering method shown below can be improved. The initial image sample after the optimal gradient impact filtering method is processed can be determined by the following formula:

[0096]

[0097] Among them, u t It can be used to represent the blurred image obtained by the impact filtering method, that is, the image of the steel bar binding site after the steel bar and wire are partially enhanced by the optimal gradient impact filtering; G σ It can be used to represent a Gaussian function with a standard deviation of σ; * can be used to represent a convolution operation; It can be used to represent the gradient estimation of the image after Gaussian filter convolution at the current pixel point. Its function is to estimate the texture edge of the image and judge the difference between noise; u ηη It can be used to represent the second-order directional derivative of the blurred image u along the gradient direction; sign(u ηη ) is an edge detection item, for example, the background output such as the sky or cement floor is negative, and the edge output such as steel bars and iron coils is positive; η can be used to represent the direction with the largest contrast in the image of the steel bar binding site, ensuring that the filter produces an impact orthogonal to the flow direction of the image texture. In the steel bar binding site, this direction is often orthogonal to the direction of the steel bars or the binding wires. t can be used to represent the calculation time; The u in can be u0, which can be the initial image, that is, the initial image sample before processing.

[0098] Optionally, after the data set preprocessing is completed, each image in D0 is processed into two images of size 512×512 and 256×256, wherein all images in D1 are preprocessed by a gradient shock filter.

[0099] As an optional embodiment, the initial multi-scale parallel residual network model includes an upper network and a lower network, and the upper network and the lower network respectively include a single-path feature fusion residual module, a dual-path multi-scale feature fusion module, a normal convolution module and a transposed convolution module, wherein, based on the deep convolutional neural network model, the initial multi-scale parallel residual network model is constructed, including: in the upper network and the lower network, ordinary convolution modules and transposed convolution modules are deployed, wherein the ordinary convolution modules are used to extract the image information in the initial image sample and to reduce the image size of the initial image sample, and the transposed convolution module is used to increase the image size and to increase the image information; in the upper network and the lower network, a single-path feature fusion residual module is deployed, wherein the first ordinary convolution module, the second ordinary convolution module and the third ordinary convolution module are sequentially deployed in the single-path feature fusion residual module, the convolution kernel and the step size of the first ordinary convolution module and the second ordinary convolution module are the same, and the first ordinary convolution module and the second ordinary convolution module have the same convolution kernel and step size. The convolution module is used to connect with the third ordinary convolution module after performing the channel splicing operation, and the convolution kernel of the third ordinary convolution module is smaller than the convolution kernel of the first ordinary convolution; in the upper network and the lower network, a dual-path multi-scale feature fusion module is deployed, wherein the dual-path multi-scale feature fusion module is used to perceive the jitter motion trend characteristics of the initial image sample at multiple scales and restore the texture information of the steel bar; in the upper network, two single-path feature fusion residual modules, one dual-path multi-scale feature fusion module, two single-path feature fusion residual modules, one dual-path multi-scale feature fusion module, one transposed convolution module, two single-path feature fusion residual modules and an ordinary convolution module are deployed in sequence, and in the lower network, two single-path feature fusion residual modules, one dual-path multi-scale feature fusion module, two single-path feature fusion residual modules, one transposed convolution module, two single-path feature fusion residual modules and an ordinary convolution module are deployed in sequence to obtain an initial multi-scale parallel residual network model.

[0100] In this embodiment, in the upper network and the lower network of the initial multi-scale parallel residual network model, ordinary convolution modules and transposed convolution modules can be deployed respectively. In the upper network and the lower network, single-channel feature fusion residual networks can be deployed respectively. In the upper network and the lower network, dual-channel multi-scale feature fusion modules can be deployed respectively. In the upper network, two single-channel feature fusion residual modules, one dual-channel multi-scale feature fusion module, two single-channel feature fusion residual modules, one dual-channel multi-scale feature fusion module, a transposed convolution module, two single-channel feature fusion residual modules and an ordinary convolution module can be deployed in sequence. In the lower network, two single-channel feature fusion residual modules, one dual-channel multi-scale feature fusion module, two single-channel feature fusion residual modules, a transposed convolution module, two single-channel feature fusion residual modules and an ordinary convolution module can be deployed in sequence. After the above-mentioned upper network and the lower network are deployed, the constructed initial multi-scale parallel residual network model can be obtained.

[0101] Optionally, the initial multi-scale parallel residual network model may include an upper network and a lower network. The upper network and the lower network may include a single-channel feature fusion residual module, a dual-channel multi-scale feature fusion module, a normal convolution module, and a transposed convolution module, respectively. The normal convolution module (Convolution, referred to as CONV) can be used to extract image details in the initial image sample, and to reduce the image scale of the initial image sample, that is, CONV can be used to extract the features of the steel bars and the tied wires in the steel bar binding site image, and can reduce the size of the steel bar binding site feature map. The transposed convolution module (Transposed Convolution, referred to as TRAN_CONV) can be used to increase the image size and to increase the image details, that is, TRAN_CONV is mainly used to increase the size of the steel bar binding site feature map, so that the network model can finally output a high-resolution clear steel bar binding site feature map, and enhance the binding status detail information of the steel bar intersection. The single-channel feature fusion residual module may be deployed with a first common convolution module, a second common convolution module, and a third common convolution module, and the convolution kernel and step size of the first common convolution module and the second common convolution module are the same. The first common convolution module and the second common convolution module may be used to perform a channel concatenation operation (CONCAT) and then be connected to the third common convolution module. The convolution kernel of the third common convolution module is smaller than the convolution kernel of the first common convolution.

[0102] Optionally, the first common convolution module may be a common convolution operation with a step size of 1, a convolution kernel size of 3*3, and a padding value of 1. The second common convolution module may be a common convolution operation with a step size of 1, a convolution kernel size of 3*3, and a padding value of 1 connected to the first common convolution module. The third common convolution module may be a convolution operation with a step size of 1 and a convolution kernel of 1*1.

[0103] Optionally, a dual-channel multi-scale feature fusion block (Multi-scale Feature Fusion Block, MFF_Block for short) can be used to perceive the jitter motion trend characteristics of the initial image sample at multiple scales and restore the texture information of the steel bar, that is, it can perceive the jitter motion trend characteristics of the steel bar binding site feature map taken by the drone at multiple scales and learn how to restore the texture details of the steel bar and wire. A single-channel feature fusion residual block (Feature Fusion Residual Block, FFR_Block for short) can be used to deepen the texture characteristics of the steel bar and wire.

[0104] Optionally, the embodiment of the present invention constructs a motion blurred steel bar binding scene image u t The multi-scale parallel residual network for clear restoration is divided into two layers of network paths, upper and lower, which need to receive u of size 512×512 and 256×256 respectively. t The network model is composed of ordinary convolution, single-channel feature fusion residual module FFR_Block, dual-channel multi-scale feature fusion module MFF_Block, and transposed convolution stacking. Finally, the upper and lower layers of the network pathway output clear steel bar binding site images u with resolutions of 512×512 and 256×256 respectively. t ′.

[0105] Optionally, in the upper network path, the image u t First, it passes through a normal convolution layer, then passes through 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, and 1 transposed convolution layer TRAN_CONV, and finally outputs a clear quality steel bar binding site image u after a convolution operation. t ′. This description points out the specific structure of the upper network pathway, where FFR_Block and MFF_Block represent the single-path feature fusion residual module and the dual-path multi-scale feature fusion module respectively. The combination of these modules constitutes the structure of the entire network, which aims to achieve image deblurring through multiple feature fusion and convolution operations, thereby outputting a higher quality clear image u t ′.

[0106] Optionally, in the lower network path, the image u t First, it passes through a normal convolution layer, then passes through 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, and 1 transposed convolution layer, and finally outputs a clear quality steel bar binding site image u after a convolution operation.t ′. This description points out the specific structure of the lower network path, where FFR_Block and MFF_Block represent the single-path feature fusion residual module and the dual-path multi-scale feature fusion module respectively. The combination of these modules constitutes the structure of the entire network, which aims to achieve image deblurring through multiple feature fusion and convolution operations, thereby outputting a higher quality clear image u t ′.

[0107] Optionally, u t It is processed into a feature map, whose number of channels can be 3, 16 or 32. The width and height of the feature map of the upper network is twice that of the feature map of the lower network, which means that in the multi-scale parallel residual network, the upper network processes a coarser feature representation, while the lower network processes a finer feature representation. Such a setting can effectively capture image features at different scales, which helps to improve the network's understanding and processing capabilities of images.

[0108] Optionally, ordinary convolution CONV and transposed convolution TRAN_CONV are used in the multi-scale parallel residual network. The embodiment of the present invention uses ordinary convolution CONV mainly to extract the features of the steel bars and the tied wires in the steel bar binding site image, which can reduce the size of the steel bar binding site feature map, change the number of channels, reduce the parameters of the network, and reduce the amount of calculation during training. The embodiment of the present invention uses transposed convolution TRAN_CONV mainly to increase the size of the steel bar binding site feature map, so that the network model can finally output a high-resolution clear steel bar binding site feature map, enhance the binding status details of the steel bar intersection, and reduce the number of channels at the same time, to prevent the problem of large amount of calculation caused by too many network parameters.

[0109] Optionally, in order to avoid the problem of vanishing gradients caused by the back propagation of information flow during model training, and to ensure that the fused feature map has a stronger ability to express the characteristics of steel bars and wire textures as the network deepens, the present invention designs an FFR_Block using a residual connection method, so that the multi-scale parallel residual network can more easily learn how to deblur and restore images of steel bar binding sites. The purpose of this module is to avoid the problem of vanishing gradients caused by the back propagation of information flow during model training, and to ensure that the fused feature map has a stronger ability to express the characteristics of steel bars and wire textures as the network deepens. By using residual connections, this module helps to make it easier for the multi-scale parallel residual network to learn how to deblur and restore images of steel bar binding sites. The residual connection method helps to alleviate the problem of vanishing gradients, and also helps to improve the training effect and learning ability of the network. This design helps to improve the performance and training effect of the network.

[0110] Optionally, after receiving the rebar binding site feature map output by the previous module as input, the FFR_Block module undergoes two ordinary convolution operations with a step size of 1, a convolution kernel size of 3*3, and a padding value of 1, to obtain a rebar binding site feature map with unchanged size and number of channels. The results of the first convolution operation and the second convolution operation are spliced ​​in the channel dimension to obtain a rebar binding site feature map with unchanged size and doubled number of channels. The spliced ​​feature map undergoes a convolution operation with a step size of 1 and a convolution kernel size of 1*1 to obtain a rebar binding site feature map with restored number of channels. The feature map initially input to this module and the feature map output after 1*1 convolution are residually connected to obtain the feature map result after bitwise addition of the values ​​of the two feature map matrices.

[0111] In an embodiment of the present invention, the FFR_Block module first receives the steel bar binding site feature map output by the previous module as input, and then obtains the steel bar binding site feature map with unchanged size and number of channels after two ordinary convolution operations with a step size of 1, a convolution kernel size of 3*3, and a padding value of 1. Next, the result of the first convolution operation and the result of the second convolution operation are spliced ​​in the channel dimension to obtain a steel bar binding site feature map with unchanged size and doubled number of channels. This setting helps to improve the network's ability to extract features of steel bar binding site images, and also helps to keep the size and number of channels of the feature map unchanged. Such operation steps help to ensure that the network can better express the steel bar and wire grain features when learning and extracting features. The spliced ​​feature map undergoes a convolution operation with a step size of 1 and a convolution kernel size of 1*1 to obtain a steel bar binding site feature map with restored number of channels. Then, the feature map initially input to this module and the feature map output after 1*1 convolution are residually connected to obtain the feature map result after the values ​​of the two feature map matrices are bitwise added. This setting helps ensure that the network can better maintain the size and number of channels of the feature map when learning and extracting features, and improve the training effect and learning ability of the network through residual connection. Such operation steps help the network better express the characteristics of steel bars and wire textures, and also help to alleviate the gradient disappearance problem and improve the performance of the network.

[0112] For example, suppose the input feature map received by the FFR_Block module is x, and the output feature map after 1*1 convolution is The result of the residual connection is The result after residual connection is processed by Gaussian Error Linear Unit (GELU) activation function to become the final output of FFR_Block. Since the number of convolution kernels is equal to the number of channels of the input feature map during convolution, the size and number of channels of the feature map of the steel bar binding site taken by the drone after FFR_Block processing remain unchanged. That is, the FFR_Block module receives the input feature map x, and the output feature map after 1*1 convolution is The result of the residual connection is Then, this result is processed by the GELU activation function and becomes the final output of the FFR_Block module. Since the number of convolution kernels is equal to the number of channels of the input feature map during convolution, the size and number of channels of the feature map of the steel bar binding site photographed by the drone after FFR_Block processing remain unchanged. This setting helps to ensure that the network can better maintain the size and number of channels of the feature map when learning and extracting features, and improve the training effect and learning ability of the network through residual connection. Such operation steps help the network better express the characteristics of steel bars and wire textures, and also help to alleviate the gradient disappearance problem and improve the performance of the network.

[0113] Optionally, due to the use of single-scale image training, the robustness of the deblurring model to scale changes of the images of the steel bar binding site taken by the drone cannot be enhanced, and the problem that the images taken by the drone at different heights and distances show different characteristics of the steel bars and the tied wires is not considered. Therefore, the present invention sets MFF_Block to allow the multi-scale parallel residual network to fuse the steel bar binding site feature maps of different scales from the upper and lower networks during the forward propagation process, and can feel the jitter motion trend characteristics of the steel bar binding site feature maps taken by the drone at multiple scales, and learn how to restore the texture details of the steel bars and wires.

[0114] In an embodiment of the present invention, the setting of MFF_Block is intended to solve the problems caused by single-scale image training, so as to enhance the robustness of the deblurring model for scale changes of the steel bar binding scene images taken by drones. This module allows the multi-scale parallel residual network to fuse the steel bar binding scene feature maps of different scales from the upper and lower networks during the forward propagation process, so that it can perceive the jitter motion trend characteristics of the steel bar binding scene feature map taken by drones at multiple scales, and learn how to restore the texture details of steel bars and wires. The MFF_Block structure of the upper network needs to receive two inputs, one of which is the steel bar binding scene feature map output by the previous module, and the other is the steel bar binding scene feature map output by the fourth FFR_Block of the lower network after the first convolution. This design helps to enhance the robustness of the model for images of different scales, so that it can better adapt to image features at different scales, thereby improving the performance and generalization ability of the model.

[0115] Optionally, the setting of MFF_Block is very important because it solves the problems caused by single-scale image training and enhances the robustness of the deblurring model for scale changes in the rebar binding site images taken by drones. This module allows the multi-scale parallel residual network to fuse the rebar binding site feature maps of different scales from the upper and lower networks during the forward propagation process, so that it can perceive the jitter motion trend characteristics of the rebar binding site feature maps taken by drones at multiple scales and learn how to restore the texture details of the rebar and wire. This setting helps to enhance the robustness of the model for images of different scales, enabling it to better adapt to image features at different scales, thereby improving the performance and generalization ability of the model. Such a design is very important for solving the learning problems of scale changes and jitter motion trend characteristics, and helps to improve the model's understanding and processing capabilities for rebar binding site images.

[0116] Optionally, the MFF_Block structure of the upper network needs to receive two inputs. One of the inputs u1 is the rebar binding site feature map output from the previous module, and the other input u2 is the rebar binding site feature map output from the first convolution of the lower network or the fourth FFR_Block of the lower network. This setting allows the network model to perceive the jitter motion trend characteristics of the rebar binding site feature map taken by the drone at different scales, and learn how to restore the texture details of the rebar and wire. By fusing feature maps of different scales from the upper and lower networks, the model can better adapt to image features at different scales, thereby improving the performance and generalization ability of the model. Such a setting is very important for solving the learning problems of scale changes and jitter motion trend characteristics, and helps to improve the model's understanding and processing capabilities for rebar binding site images.

[0117] Optionally, the MFF_Block structure of the lower network also needs to receive two inputs. One of the inputs u1 is the rebar binding site feature map output from the previous module, and the other input u2 is the rebar binding site feature map output from the first convolution of the upper network. This setting allows the model to perceive the jitter motion trend characteristics of the rebar binding site feature map taken by the drone at different scales, and learn how to restore the texture details of the rebar and wire. By fusing feature maps of different scales from the upper and lower networks, the model can better adapt to image features at different scales, thereby improving the performance and generalization ability of the model. Such a setting is very important for solving the learning problems of scale changes and jitter motion trend characteristics, and helps to improve the model's understanding and processing capabilities for rebar binding site images.

[0118] Optionally, in the MFF_Block of the upper network, the input u2 is subjected to a TRAN_CONV operation with a step size of 2 and a convolution kernel size of 3*3, and an upsampling operation is performed to obtain u2′ whose size is consistent with u1. In the MFF_Block of the lower network, u2 is subjected to a downsampling operation using CONV to obtain u2′ whose size is consistent with u1. This setting helps to keep the feature maps from the upper and lower networks consistent in size for subsequent feature fusion operations. Such a design helps the model better fuse feature maps from different scales and improves the model's ability to process images of different scales.

[0119] Optionally, in the MFF_Block of the upper network, after obtaining u2′, u1 and u2′ are respectively subjected to a common convolution process with a step size of 1, a convolution kernel size of 3*3, and a padding value of 1. Then, the feature maps output by the two common convolutions are spliced ​​in the channel dimension to obtain a feature map of the steel bar binding site with the same size and doubled number of channels. Finally, in order to reduce the amount of calculation and fuse the feature maps of different scales from the upper and lower networks, the spliced ​​feature map is processed using a convolution operation with a step size of 1 and a convolution kernel of 1*1. This setting helps to keep the feature maps from the upper and lower networks consistent in size, and through feature fusion operations, improves the model's processing capabilities for images of different scales. Such a design helps to improve the performance and generalization ability of the model, enabling it to better adapt to image features at different scales.

[0120] Optionally, in the constructed MFF_Block module, the multi-scale feature fusion result is obtained by processing with the GELU activation function, and the feature map of the steel bar binding site taken by the drone with the same size and number of channels as the input u1 is output. In this module, the number of convolution kernels is kept at 32, so that the number of channels of the feature map of the steel bar binding site remains unchanged after the convolution operation. This setting helps to ensure that the model can better maintain the size and number of channels of the feature map when learning and extracting features, and improve the training effect and learning ability of the network through feature fusion operations. Such operation steps help the network better express the characteristics of steel bars and wire textures, and also help to improve the model's processing capabilities for images of different scales.

[0121] Optionally, after constructing FFR_Block and MFF_B lock respectively, a complete multi-scale parallel residual network model can be built.

[0122] Optionally, both the upper network and the lower network first build a CONV as the network entrance, and input the pre-processed fuzzy steel bar binding scene image u t , after a convolution operation with a step size of 2, a convolution kernel size of 3*3, and a number of convolution kernels of 32, the output of this convolution is obtained respectively - the rebar binding site feature map u with half the size and 32 channels. t1 .

[0123] Optionally, for the above-mentioned pre-constructed FFR_Block and MFF_Block, in the upper-layer network path, 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, 1 MFF_Block, 1 TRAN_CONV, 2 FFR_Blocks, and 1 CONV may be constructed in sequence.

[0124] Optionally, for the above-mentioned pre-built FFR_Block and MFF_Block, in the lower network path, 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, 1 TRAN_CONV, 2 FFR_Blocks, and 1 CONV can be built in sequence. Among them, in the built TRAN_CONV module, the steel bar binding site feature map output by the previous module is received, and the feature map will undergo a transposed convolution operation with a step size of 2, a convolution kernel size of 2*2, and a number of convolution kernels of 16, to obtain a steel bar binding site feature map with a size doubled and a channel number of 16.

[0125] Optionally, the last CONV of the upper and lower networks receives the rebar binding site feature map output after the last FFR_Block operation as input. The input feature map will undergo a convolution operation with a convolution kernel size of 3*3, a step size of 2, a number of convolution kernels of 3, and a padding value P of 1, and finally be processed by the GELU activation function to obtain the blurred image u t A 3-channel drone shot of the steel bar binding site with the same resolution and size and clear blur. t ′.

[0126] For example, the change in image size after a convolution operation is calculated as follows: suppose the input feature map size is W×W, the convolution kernel size is K×K, the step size is S, and the padding value is P. According to the calculation formula of the feature map size after convolution, B=(W-K+2P) / S+1, the feature map size after convolution is B×B. According to the calculation formula of the feature map size after transposed convolution, B=(W-1)×S+K-2P, the feature map size after transposed convolution is B×B. Transposed convolution is the reverse process of convolution. The image size before convolution can be restored according to the convolution kernel size and the output size. Therefore, transposed convolution is used to achieve upsampling.

[0127] As an optional embodiment, the initial multi-scale parallel residual network model is trained using a training data set in a preprocessed data set, including: initializing the parameters of the training initial multi-scale parallel residual network model to obtain the initialized parameters; in response to the completion of the parameter initialization, the initial image samples of the first image size in the training data set are input into the upper network, and the initial image samples of the second image size in the training data set are input into the lower network, wherein the first image size is larger than the second image size; in response to the upper network and the lower network receiving the corresponding initial image samples, the initial multi-scale parallel residual network model is trained.

[0128] In this embodiment, in the process of training the initial multi-scale parallel residual network model using the training data set in the preprocessed data set, the parameters of the training initial multi-scale parallel residual network model can be initialized to obtain the initialized parameters. If the parameter initialization is completed, the initial image samples of the first image size in the training data set can be input into the upper network, and the initial image samples of the second image size in the training data set can be input into the lower network. After the upper network and the lower network receive the corresponding initial image samples, the initial multi-scale parallel residual network model can be trained, wherein the first image size is larger than the second image size, the first image size can be 512×512, and the second image size can be 256×256. The parameters that need to be initialized can be training parameters, which can also be called network parameters.

[0129] Optionally, after building a multi-scale parallel residual network, the training data set in the preprocessed data set D0 is used to train the multi-scale parallel residual network, and the model parameters are optimized through the back propagation algorithm. The present invention designs a suitable loss function according to the requirements of steel bar binding quality inspection and the characteristics of steel bar binding site images. The loss function includes multiple parts, which can take into account the characteristic information of the steel bar binding image and the characteristics of the deblurring task. The hyperparameters in the network are adjusted according to the results of the loss function to optimize the overall performance of the multi-scale parallel residual network. The hyperparameters include the weight of the loss function, the learning rate, etc.

[0130] Optionally, initialize the parameters required for the training model, such as the batch size of the dataset DO, the number of training rounds, the learning rate, etc. After the preprocessing operation, two sizes of motion blurred steel bar binding site images u t , and set u to 512×512 t As the input of the upper network, u with a size of 256×256 t As the input of the lower network.

[0131] As an optional embodiment, during the training of the initial multi-scale parallel residual network model, a target image determined based on an initial image sample is obtained; initial loss data of the initial multi-scale parallel residual network model is determined based on Fourier changes corresponding to the label data of the target image and the initial image sample respectively; target loss data of the initial multi-scale parallel residual network model is determined based on the initial loss data and the perceptual loss data of the initial multi-scale parallel residual network model; in response to the target loss data reaching a convergence state, the initial multi-scale parallel residual network model is determined as the target multi-scale parallel residual network model, wherein the convergence state is used to characterize that the similarity between the label data and the corresponding target image reaches a similarity threshold.

[0132] In this embodiment, in the process of training the initial multi-scale parallel residual network model, the target image determined by the initial image sample can be obtained, and the initial loss data of the initial multi-scale parallel residual network model can be determined based on the Fourier changes corresponding to the label data of the target image and the initial image sample. Based on the initial loss data and the perceptual loss data of the initial multi-scale parallel residual network model, the target loss data of the initial multi-scale parallel residual network model is determined. When the target loss data reaches a convergence state, the initial multi-scale parallel residual network model can be determined as the target multi-scale parallel residual network model, wherein the convergence state can be used to characterize that the similarity between the label data and the target image reaches a similarity threshold. The initial loss data can be the loss function of the initial multi-scale parallel residual network using the Euclidean norm as the loss function in the model training process. The perceptual loss data can be a perceptual loss function. The target loss data can be a loss function finally determined.

[0133] Optionally, during the training process, the output deblurred image result is compared with the label clear image of the blurred image input to the network, that is, the network can be optimized in this way. After each round of training data is fed into the model, the restored steel bar binding scene image is output through forward propagation, and the difference between the deblurred image and the label clear image, i.e., the loss value, is calculated using the loss function. After obtaining the loss value, the model updates the various parameters of the module through back propagation to reduce the loss between the deblurred prediction value and the label true value during subsequent training, so that the model's deblurring result for the input image is closer and closer to the clear and true steel bar binding scene image, thereby achieving the application purpose of the multi-scale parallel residual network being able to deblur the steel bar binding scene image taken by the drone.

[0134] Optionally, the loss function of the multi-scale parallel residual network of the embodiment of the present invention uses the Euclidean norm as the loss function in the model training process. In the deblurring and enhancement task of the steel bar binding site image, the high-frequency components lost in the blurred image are mainly restored. Therefore, the steel bar binding site image signal is first converted to the frequency domain using the fast Fourier transform, and finally the calculation results of different scales are added.

[0135] For example, the initial loss data can be determined by the following formula:

[0136]

[0137] in, It can be used to represent the initial loss data; i can be used to represent the scale factor. This model contains two upper and lower networks and is trained at two scales, so i can be 1 and 2; S i and G iIt can be used to represent network-generated images and real clear images respectively; t i It can be used to represent the number of losses in each round of calculation; F can be used to represent fast Fourier transform.

[0138] For another example, the perceptual loss function is also used in the deblurring model of the steel bar binding site image taken by the drone. The final loss function is shown as follows:

[0139]

[0140] in, Can be used to represent target loss data.

[0141] As an optional example, the perceptual loss function can be determined by the following formula:

[0142]

[0143] Among them, x can be used to represent the input image; y can be used to represent the target image; F i (x) and F i (y) can be used to represent their feature representations at the i-th layer in the pre-trained neural network, respectively; N can be used to represent the number of feature layers.

[0144] Optionally, the multi-scale parallel residual network model undergoes multiple iterations during training, and when the loss function continues to decrease and reaches a convergence state, it indicates that the deblurred and restored steel bar binding site image output by the model has clear quality.

[0145] For example, the dataset of steel bar binding scene images taken by drones is input into the multi-scale parallel residual network framework in batches. The appropriate batch size is selected according to the hardware and memory conditions. The batch size is set to 16, and the hyperparameters of the network framework are set. The initial learning rate is 0.001, the number of iterations epochs is set to 120, and the learning rate is changed to 0.1 times the previous one every 40 epochs. One iteration of the model is equivalent to learning how to restore all the images in the network training set to clear images of the steel bar binding scene. The ADAM optimizer is used for training, and the loss function The weight ω is set to 0.4.

[0146] Alternatively, the Euclidean norm mainly represents the square of the distance between the predicted value and the actual value, that is, the L2 norm. Its advantage is that the curve of the function is relatively smooth when it is close to the true value. When using this function, you can use this feature to make it converge slowly when it is close to the true value.

[0147] Alternatively, the Euclidean norm can be determined by the following formula:

[0148]

[0149] in, It can be used to represent the Euclidean norm; F(x) can be used to represent the image of the input model, and F(y) can be used to represent the image of the output model.

[0150] As an optional embodiment, a verification data set in a preprocessed data set is used to verify the trained initial multi-scale parallel residual network model to obtain a multi-scale parallel residual network model, including: inputting the initial image samples in the verification data set into the target multi-scale parallel residual network model, and using the target multi-scale parallel residual network model to deblur the initial image samples to obtain a target image corresponding to the initial image samples; in response to the similarity between the label data of the target image and the corresponding initial image sample reaching a similarity threshold, determining the target multi-scale parallel residual network model as a multi-scale parallel residual network model.

[0151] In this embodiment, in the process of using the verification data set in the preprocessed data set to verify the trained initial multi-scale parallel residual network model to obtain the multi-scale parallel residual network model, the initial image sample in the verification data set can be input into the target multi-scale parallel residual network model, and the target multi-scale parallel residual network model can be used to deblur the initial image sample to obtain the target image corresponding to the initial image sample. The target image and the corresponding label data can be compared for similarity, and if the similarity reaches a similarity threshold, the target multi-scale parallel residual network can be determined as a multi-scale parallel residual network model.

[0152] Optionally, after the initial multi-scale parallel residual network model is trained to obtain the target multi-scale parallel residual network model, the target multi-scale parallel residual network model can be verified using a pre-prepared verification data set. The initial image sample in the verification data set can be input into the target multi-scale parallel residual network model for deblurring to obtain the corresponding target image. If the similarity between the target image and the corresponding label value reaches a similarity threshold, the target multi-scale parallel residual network model at this time can be determined as a trained and successfully verified multi-scale parallel residual network model.

[0153] In the embodiment of the present invention, considering that the accuracy of determining the binding quality by directly collecting images is low, an image processing model can be pre-trained to perform deblurring on the acquired images, that is, the relatively blurred initial image can be adjusted to a relatively clear target image. Thus, the purpose of realizing the deblurring and restoration project of the steel bar binding site image is achieved, and the technical effect of improving the accuracy of determining the steel bar binding quality is achieved, and the technical problem of low accuracy in determining the steel bar binding quality is solved.

[0154] Example 2

[0155] The above technical solution is described in detail below in conjunction with another optional specific implementation manner.

[0156] At present, when using drones to inspect the quality of steel bar binding sites, we first capture the frames of the captured video to obtain image data, then identify and locate the steel bar binding points in the image, and finally determine the quality of the binding. Since drones are easily affected by wind during flight or may be in flight, the captured images will have motion jitter and blur, resulting in the loss of detailed information of the wire tied to the steel bars in the image. In order to make the drone's quality inspection of steel bar binding more accurate and smooth, we need to perform deblurring and enhancement preprocessing on the images of the steel bar binding site taken by the drone.

[0157] Image deblurring is a process from coarse to fine, which can be divided into traditional methods and methods based on deep learning. Traditional methods were proposed earlier, for example, the Lucy-Richardson iterative algorithm using Bayesian estimation theory and maximum likelihood estimation theory; image restoration filtering algorithms such as Wiener filtering and inverse filtering based on statistical signal processing. However, the irregular vibration of the drone and the relative motion between the drone and the photographed target are unknown, and the blur kernel function cannot be accurately known. Therefore, the traditional image restoration algorithm cannot solve the engineering problem of deblurring and restoring the images of the steel bar binding site taken by the actual drone.

[0158] In recent years, with the continuous maturity of deep learning technology, image deblurring methods based on convolutional neural networks have emerged. The neural network method is an end-to-end method. When used, the blurred image is used as the network input, and the network outputs the deblurred image. Among them, the neural network model based on conditional generative adversarial network and multivariate content loss, that is, the deblurring generative adversarial network (DeblurGAN for short) is widely known and used in deblurring applications, and has a good image deblurring and restoration effect. Using the deep learning method, as long as it is possible to collect and produce a sufficiently high-quality and large dataset of rebar binding site images taken by drones, and the designed network structure is excellent, the effect of restoring motion-blurred rebar binding site images will be better than the traditional method of estimating the blur kernel. However, the above method still has the technical problem of low accuracy in determining the quality of rebar binding.

[0159] However, the embodiment of the present invention proposes a method for preprocessing the deblurring of the steel bar binding scene image taken by a drone. The constructed and trained multi-scale parallel residual network model can be used to deblur the acquired steel bar binding scene image, so that the image quality is improved, and the detailed information of a large amount of steel bar binding in the image is not easily lost. If such high-quality images are used to determine the binding quality, the deblurring and restoration of the steel bar binding scene image actually taken can be solved, thereby achieving the technical effect of improving the accuracy of the quality inspection of the steel bar binding scene, and solving the technical problem of low accuracy of the quality inspection of the steel bar binding scene.

[0160] The method is further described below.

[0161] In this embodiment, Figure 2 : is a flowchart of a method for deblurring a UAV image at a steel bar binding site according to an embodiment of the present invention. The method may include the following steps:

[0162] Step S201, creating a training data set and a test data set of a steel bar binding site photographed by a drone.

[0163] In this embodiment, a clear image of the steel bar binding site taken by a drone is blurred, and the blurred image and its corresponding clear image together constitute a data set.

[0164] Optionally, clear images of the steel bar binding site taken by a drone are manually screened out as the data set D1.

[0165] Optionally, the cv2.filter2D() method in the OpenCV library is used to process the clear image in D1, and simulate and generate a steel bar binding site image dataset D2 that is blurred due to the relative motion between the drone camera and the steel bar binding site plane. The clear image in D1 and its corresponding blurred image in D2 form a set of reference data pairs, which together constitute the dataset D0 required for training the network model. The clear image in D1 is used as the label value label of the blurred image in D2, and is used to compare with the image result after the blurred image is deblurred.

[0166] Optionally, when using the cv2.filter2D() method in the OpenCV library, passing in motion blur kernel matrix parameters of arbitrary angles to the method can well simulate the blurring effect of drone images due to flight or wind shaking.

[0167] Optionally, the constructed rebar binding site image dataset D0 photographed by a drone is divided into a training dataset train and a validation dataset val according to a ratio of 8:2.

[0168] For example, 1600 clear images of steel bar binding sites taken by drones at different heights and angles are selected as data set D1, with a resolution of 1280x720 pixels. The cv2.filter2D() method in the OpenCV library is used to perform convolution calculations on the clear images in D1, and 1600 blurred images simulating the motion jitter blur of drones are obtained as D2. D1 is used as the label of D2 to form D0, so D0 has 1600 pairs (3200 images) of data sets, the training data set train has 1280 pairs of data sets, and the validation data set val has 320 pairs of data sets.

[0169] Step S202, preprocessing the input image using downsampling and optimal gradient shock filtering technology.

[0170] In this embodiment, the blurred image to be input is preprocessed, including downsampling and using a gradient impact filter to enhance the details of the blurred image.

[0171] Optionally, the multi-scale parallel residual network model of the present invention needs to receive two sizes of images as input, namely 512×512 and 256×256. The steel bar binding site images of the dataset D0 are taken by drones, and the actual drone resolution is 1280×720, 1920×1080, etc.

[0172] Optionally, the image u in the data set D0 is first downsampled using a bilinear interpolation method. After downsampling, the image u has two different sizes: 512×512 and 256×256. Considering that the blurred image u will lose too much detail information after downsampling, the texture detail enhancement preprocessing of the blurred image u is performed using an optimal gradient shock filter method after downsampling.

[0173] Optionally, when bilinear interpolation downsampling is performed on the blurred image u, a point of the image before downsampling is set as point P, and the coordinates of point P are expressed as P = (x, y). The downsampled value f(x, y) at point P = (x, y) is required. The four points Q closest to point P with known values ​​are obtained. 11 , Q 12 , Q 21 , Q 22 The coordinates of the four points are represented by Q 11 =(x1, y1), Q 12 =(x1, y2), Q 21 =(x2, y1), Q 22 =(x2,y2).

[0174] Optionally, linear interpolation is performed in the x direction, and the calculation process is shown in the following formula:

[0175]

[0176]

[0177] Among them, R1=(x, y1), R2=(x, y2).

[0178] Optionally, linear interpolation is performed in the y direction, and the calculation process is shown in the following formula:

[0179]

[0180] Alternatively, the value f(x, y) at the point P = (x, y) calculated by bilinear interpolation is as shown below:

[0181]

[0182] Optionally, in an embodiment of the present invention, a superior gradient impact filtering method may be used to first apply Gaussian filter convolution to process the image, and then calculate the gradient of the image after the Gaussian filter convolution process.

[0183] Alternatively, the equation for the original shock filtering method is shown below:

[0184]

[0185] Among them, ut It can be used to represent the blurred image obtained by the impact filtering method; u ηη It can be used to represent the second-order directional derivative of the blurred image u along the gradient direction; sign(u ηη ) can be used to represent edge detection items.

[0186] Optionally, u ηη It can be calculated by the following formula:

[0187]

[0188] Among them, u x and u y Can be used to represent the first-order variance in the x and y directions respectively; u xx 、u yy and u xy It can be used to represent the second-order difference and mixed difference in the x and y directions respectively.

[0189] Optionally, calculating the gradient of the image after Gaussian filtering can eliminate the noise caused by optical problems in some drone photography, which helps to restore higher quality images of the steel bar binding site.

[0190] Optionally, based on the equation of the original impact filtering method, the equation of the optimal gradient product filtering method shown below can be improved. The initial image sample after the optimal gradient impact filtering method is processed can be determined by the following formula:

[0191]

[0192] Among them, u t It can be used to represent the blurred image obtained by the impact filtering method, that is, the image of the steel bar binding site after the steel bar and wire are partially enhanced by the optimal gradient impact filtering; G σ It can be used to represent a Gaussian function with a standard deviation of σ; * can be used to represent a convolution operation; It can be used to represent the gradient estimation of the image after Gaussian filter convolution at the current pixel point. Its function is to estimate the texture edge of the image and judge the difference between noise; u ηη It can be used to represent the second-order directional derivative of the blurred image u along the gradient direction; sign(u ηη ) is an edge detection item, for example, the background output such as the sky or cement floor is negative, and the edge output such as steel bars and iron coils is positive; η can be used to represent the direction with the largest contrast in the image of the steel bar binding site, ensuring that the filter produces an impact orthogonal to the flow direction of the image texture. In the steel bar binding site, this direction is often orthogonal to the direction of the steel bars or the binding wires. t can be used to represent the calculation time; The u in can be u0, which can be the initial image, that is, the initial image sample before processing.

[0193] Optionally, after the data set preprocessing is completed, each image in D0 is processed into two images of sizes 512×512 and 256×256, wherein all images in D1 are preprocessed by a gradient shock filter.

[0194] For example, all images in D0 are downsampled from 1280×720 pixels to 512×512 pixels and 256×256 pixels using bilinear interpolation. Gaussian convolution is performed on the blurred 512×512 pixel and 256×256 pixel images, and then the gradient value of the convolved image is calculated and finally used as the gradient value of the impact filter for the blurred image, and the image resolution remains unchanged.

[0195] Step S203: building a multi-scale parallel residual network.

[0196] In this embodiment, a multi-scale parallel residual network is constructed, which is mainly composed of a single-path feature fusion residual module and a dual-path multi-scale feature fusion module stacked.

[0197] Optionally, the present invention constructs a motion blurred steel bar binding scene image u t The multi-scale parallel residual network for clear restoration is divided into two layers of network pathways, upper and lower, which need to receive 512×512 and 256×256 size u as input respectively. The network model consists of ordinary convolution, single-channel feature fusion residual module FFR_Block, dual-channel multi-scale feature fusion module MFF_Block, and transposed convolution stacking. Finally, the upper and lower layers of network pathways output clear rebar binding site images u with resolutions of 512×512 and 256×256 respectively. t ′.

[0198] Figure 3 is a schematic diagram of a multi-scale parallel residual network structure according to an embodiment of the present invention. Figure 3 As shown, in the upper network path, the image u t First, it enters a normal convolution layer, and then passes through 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, 1 transposed convolution layer TRAN_CONV, 2 FFR_Blocks, and finally outputs a clear quality steel bar binding site image u after a convolution operation. t ′. In the lower network path, image u tFirst, it enters a normal convolution layer, then passes through 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, 1 transposed convolution layer, 2 FFR_Blocks, and finally outputs a clear quality steel bar binding site image u after a convolution CONV operation. t ′.u t When input into the multi-scale parallel residual network, the operation is processed into the feature map of the steel bar binding site taken by the drone. The number of feature map channels is 3, 16 or 32, and the width and height of the feature map of the upper network is twice as large as that of the feature map of the lower network.

[0199] Optionally, use normal convolution CONV and transposed convolution TRAN_CONV.

[0200] Optionally, the present invention uses ordinary convolution CONV and transposed convolution TRAN_CONV in the multi-scale parallel residual network. Ordinary convolution CONV is mainly used to extract the features of the steel bars and the tied wires in the steel bar binding site image, which can reduce the size of the steel bar binding site feature map, change the number of channels, reduce the parameters of the network, and reduce the amount of calculation during training. Transposed convolution TRAN_CONV is mainly used to increase the size of the steel bar binding site feature map, so that the network model can finally output a high-resolution clear steel bar binding site feature map, enhance the binding status details of the steel bar intersection, and reduce the number of channels at the same time, to prevent the problem of excessive network parameters leading to large amount of calculation.

[0201] Optionally, a single-channel feature fusion residual module is constructed.

[0202] Optionally, in order to avoid the problem of gradient vanishing caused by the back propagation of information flow during model training and to ensure that the fused feature map has a stronger ability to express the characteristics of steel bars and wire textures as the network deepens, the present invention sets up an FFR_Block using a residual connection method, so that the multi-scale parallel residual network can more easily learn how to deblur and restore the images of the steel bar binding site.

[0203] Figure 4 is a schematic diagram of a single-channel feature fusion residual module according to an embodiment of the present invention, such as Figure 4As shown in the figure, after receiving the steel bar binding site feature map output by the previous module as input, the module undergoes two ordinary convolution operations with a step size of 1, a convolution kernel size of 3*3, and a padding value of 1 to obtain a steel bar binding site feature map with unchanged size and number of channels. The results of the first convolution operation and the second convolution operation are spliced ​​in the channel dimension to obtain a steel bar binding site feature map with unchanged size and doubled number of channels. Then, the spliced ​​feature map undergoes a convolution operation with a step size of 1 and a convolution kernel size of 1*1 to obtain a steel bar binding site feature map with restored number of channels. The feature map initially input to this module and the feature map output after 1*1 convolution are residually connected to obtain the feature map result after the values ​​of the two feature map matrices are bitwise added.

[0204] For example, suppose the input feature map received by the FFR_Block module is x, and the output feature map after 1*1 convolution is The result of the residual connection is The result after residual connection is processed by GELU activation function and becomes the final output of FFR_Block. Since the number of convolution kernels is equal to the number of channels of the input feature map during convolution, the size and number of channels of the feature map of the steel bar binding site taken by the drone after FFR_Block processing remain unchanged.

[0205] Optionally, due to the use of single-scale image training, the robustness of the deblurring model to scale changes of the images of the steel bar binding site taken by the drone cannot be enhanced, and the problem that the images taken by the drone at different heights and distances show different characteristics of the steel bars and the tied wires is not considered. Therefore, the present invention sets MFF_Block to allow the multi-scale parallel residual network to fuse the steel bar binding site feature maps of different scales from the upper and lower networks during the forward propagation process, and can feel the jitter motion trend characteristics of the steel bar binding site feature maps taken by the drone at multiple scales, and learn how to restore the texture details of the steel bars and wires.

[0206] Optionally, Figure 5 is a schematic diagram of a dual-path multi-scale feature fusion module of an upper network according to an embodiment of the present invention, such as Figure 5 As shown in the figure, the MFF_Block structure of the upper network needs to receive two inputs. The input u1 is the steel bar binding site feature map output from the previous module, and the other input u2 is the steel bar binding site feature map after the first convolution of the lower network or the output of the fourth FFR_Block of the lower network.

[0207] Optionally, in the MFF_Block of the upper network, u2 is upsampled using a transposed convolution TRAN_CONV with a step size of 2 and a convolution kernel size of 3*3 to obtain u2′ whose size is consistent with u1; in the MFF_Block of the lower network, u2 is downsampled using a normal convolution CONV to obtain u2′ whose size is consistent with u1.

[0208] Optionally, after obtaining u2′, u1 and u2′ are respectively subjected to a common convolution process with a step size of 1, a convolution kernel size of 3*3, and a padding value of 1. The feature maps output after the two common convolutions are spliced ​​in the channel dimension to obtain a rebar binding site feature map with the same size and doubled number of channels. In order to reduce the amount of calculation and to fuse the feature maps of different scales from the upper and lower layers of the network, the convolution operation with a step size of 1 and a convolution kernel of 1*1 is used to process the spliced ​​feature map.

[0209] Optionally, Figure 6 is a schematic diagram of a dual-path multi-scale feature fusion module of a lower layer network according to an embodiment of the present invention, such as Figure 6 As shown in the figure, the MFF_Block structure of the lower network also needs to receive two inputs. The input u1 is the steel bar binding site feature map output from the previous module, and the other input u2 is the steel bar binding site feature map output from the first convolution of the upper network.

[0210] Optionally, the multi-scale feature fusion result of MFF_Block is obtained by processing with the GELU activation function, and the feature map of the steel bar binding site taken by the drone with the same size and number of channels as the input u1 is output. In the constructed MFF_Block module, the number of convolution kernels is kept at 32, so that the number of channels of the feature map of the steel bar binding site remains unchanged after the convolution operation.

[0211] Optionally, build a multi-scale parallel residual network.

[0212] Optionally, after constructing the single-path feature fusion residual module FFR_Block and the dual-path multi-scale feature fusion module MFF_Block respectively, a complete multi-scale parallel residual network can be built.

[0213] Optionally, both the upper network and the lower network first build a common convolution module CONV as the network entrance, and input the pre-processed blurred steel bar binding site image u t , after a convolution operation with a step size of 2, a convolution kernel size of 3*3, and a number of convolution kernels of 32, the output of this convolution is obtained respectively - the rebar binding site feature map u with half the size and 32 channels. t1. In the upper network path, 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, 1 MFF_Block, 1 transposed convolution TRAN_CONV, 2 FFR_Blocks, and 1 ordinary convolution CONV are constructed in sequence. In the lower network path, 2 FFR_Blocks, 1 MFF_Block, 2 FFR_Blocks, 1 transposed convolution TRAN_CONV, 2 FFR_Blocks, and 1 ordinary convolution CONV are constructed in sequence. Among them, the constructed TRAN_CONV module accepts the steel bar binding site feature map output by the previous module. The feature map will undergo a transposed convolution operation with a step size of 2, a convolution kernel size of 2*2, and a number of convolution kernels of 16, and a steel bar binding site feature map with a size doubled and a number of channels of 16 is obtained.

[0214] Optionally, the last common convolution module CONV of the upper and lower networks accepts the rebar binding site feature map output after the last FFR_Block operation as input. The input feature map will undergo a convolution operation with a convolution kernel size of 3*3, a step size of 2, a number of convolution kernels of 3, and a padding value P of 1, and finally be processed by the GELU activation function to obtain the blurred image u t A 3-channel drone shot of the steel bar binding site with the same resolution and size and clear blur. t ′.

[0215] For example, the change in image size after a convolution operation is calculated as follows: suppose the input feature map size is W×W, the convolution kernel size is K×K, the step size is S, and the padding value is P. According to the calculation formula of the feature map size after convolution, B=(W-K+2P) / S+1, the feature map size after convolution is B×B. According to the calculation formula of the feature map size after transposed convolution, B=(W-1)×S+K-2P, the feature map size after transposed convolution is B×B. Transposed convolution is the reverse process of convolution. The image size before convolution can be restored according to the convolution kernel size and the output size. Therefore, transposed convolution is used to achieve upsampling.

[0216] Step S204, training the multi-scale parallel residual network to obtain a final multi-scale parallel residual network model for deblurring the steel bar binding scene photographed by the drone.

[0217] In this embodiment, given training parameters, the constructed training set is input into the model for training, the network model is continuously fed back and corrected through the constructed loss function, the network parameters are updated, and after the training is completed, the test set is input to verify the effect of the model.

[0218] Optionally, after building a multi-scale parallel residual network, the multi-scale parallel residual network is trained using a training data set in a preprocessed data set D0, and the model parameters are optimized through a back propagation algorithm. The present invention designs a suitable loss function based on the requirements for steel bar binding quality inspection and the characteristics of steel bar binding site images. The loss function includes multiple parts, which can take into account the characteristic information of the steel bar binding image and the characteristics of the deblurring task. The hyperparameters in the network are adjusted according to the results of the loss function to optimize the overall performance of the multi-scale parallel residual network, and the hyperparameters include the weight of the loss function, the learning rate, etc.

[0219] Optionally, initialize the parameters required for training the model, such as the batch size of the dataset DO, the number of training rounds, the learning rate, etc. Start training the model. After preprocessing operations, two sizes of motion blurred steel bar binding site images u are obtained. t , and set u to 512×512 t As the input of the upper network, u with a size of 256×256 t As the input of the lower network.

[0220] Optionally, during training, the output deblurred image result is compared with the label clear image of the blurred image input to the network to optimize the network. After each round of training data is fed into the model, the restored steel bar binding scene image is output through forward propagation, and the difference between the deblurred image and the label clear image, i.e., the loss value, is calculated using the loss function. After obtaining the loss value, the model updates the various parameters of the module through back propagation to reduce the loss between the deblurred prediction value and the label true value during subsequent training, so that the model's deblurring result for the input image is closer and closer to the clear and true steel bar binding scene image, thereby achieving the application purpose of the multi-scale parallel residual network being able to deblur the steel bar binding scene image taken by the drone.

[0221] Optionally, the loss function of the multi-scale parallel residual network of the embodiment of the present invention uses the Euclidean norm as the loss function in the model training process. In the deblurring and enhancement task of the steel bar binding site image, the high-frequency components lost in the blurred image are mainly restored. Therefore, the steel bar binding site image signal is first converted to the frequency domain using the fast Fourier transform, and finally the calculation results of different scales are added.

[0222] For example, the initial loss data can be determined by the following formula:

[0223]

[0224] in, It can be used to represent the initial loss data; i can be used to represent the scale factor. This model contains two upper and lower networks and is trained at two scales, so i can be 1 and 2; S i and G i It can be used to represent network-generated images and real clear images respectively; t i It can be used to represent the number of losses in each round of calculation; F can be used to represent fast Fourier transform.

[0225] For another example, the perceptual loss function is also used in the deblurring model of the steel bar binding site image taken by the drone. The final loss function is shown as follows:

[0226]

[0227] in, Can be used to represent target loss data.

[0228] As an optional example, the perceptual loss function can be determined by the following formula:

[0229]

[0230] Among them, x can be used to represent the input image; y can be used to represent the target image; F i (x) and F i (y) can be used to represent their feature representations at the i-th layer in the pre-trained neural network, respectively; N can be used to represent the number of feature layers.

[0231] Optionally, the multi-scale parallel residual network model undergoes multiple iterations during training, and when the loss function continues to decrease and reaches a convergence state, it indicates that the deblurred and restored steel bar binding site image output by the model has clear quality.

[0232] Alternatively, the Euclidean norm mainly represents the square of the distance between the predicted value and the actual value, that is, the L2 norm. Its advantage is that the curve of the function is relatively smooth when it is close to the true value. When using this function, you can use this feature to make it converge slowly when it is close to the true value.

[0233] Alternatively, the Euclidean norm can be determined by the following formula:

[0234]

[0235] in, It can be used to represent the Euclidean norm; F(x) can be used to represent the image of the input model, and F(y) can be used to represent the image of the output model.

[0236] For example, the dataset of steel bar binding scene images taken by drones is input into the built multi-scale parallel residual network framework in batches. The appropriate batch size is selected according to the hardware and memory conditions, and the batch size is set to 16. The hyperparameters of the network framework are set, the initial learning rate is 0.001, the number of iterations epochs is set to 120, and the learning rate is changed to 0.1 times the previous one every 40 epochs. One iteration of the model is equivalent to learning how to restore all the images in the network training set to clear images of the steel bar binding scene. The ADAM optimizer is used for training, and the loss function The weight ω is set to 0.4.

[0237] Step S205, applying a deblurring technique for images of the steel bar binding site taken by a drone.

[0238] In this embodiment, a technical method is finally obtained that can clearly restore the blurred and shaken image of the steel bar binding site taken by the drone.

[0239] Optionally, when starting the deblurring application, an image of the steel bar binding site that is blurred by the shaking taken by the drone is input into the trained multi-scale parallel residual network. The image is first pre-processed by downsampling and optimal gradient shock filtering, and then calculated and processed by the multi-scale parallel residual network. The 512×512 resolution image finally output by the upper network will be used as the final output of the model, and a clear and quality-enhanced image of the steel bar binding site will be obtained.

[0240] For example, taking the lower layer network as an example, a forward propagation process in the application is provided: the lower layer input is a 256×256 size motion blurred steel bar binding image u taken by a drone after bilinear interpolation and improved shock filtering preprocessing t ′, the number of channels is 3, after a convolution operation with a convolution kernel size of 3*3, a stride of 2, and a number of convolution kernels of 32, the output feature map size is 128×128, and the number of channels is 32. After two single-channel feature fusion residual modules FFR_BLOCK, the input feature map size is 128×128, the number of channels is 32, and the output feature map size is 128×128, and the number of channels is 32. After a dual-channel multi-scale feature fusion module MFF_BLOCK, input one is the output feature map, and input two is the feature map from the upper network. After a convolution operation with a convolution kernel size of 3*3, a stride of 2, and a number of convolution kernels of 32, the output size is 256×256, and the number of channels is 32.

[0241] For another example, input 1 undergoes a second convolution operation with a convolution kernel size of 3*3, a step size of 1, and a number of convolution kernels of 32; input 2 first undergoes a convolution operation with a convolution kernel size of 3*3, a step size of 2, and a number of convolution kernels of 32, and then undergoes a convolution operation with a convolution kernel size of 3*3, a step size of 1, and a number of convolution kernels of 32; after the above operations, they are spliced ​​into a feature map of size 128×128 and 64 channels in the channel dimension. Then a convolution operation with a convolution kernel size of 1*1, a step size of 1, and a number of convolution kernels of 32 is performed. Finally, after being processed by the GELU activation function, the output feature map size is 128×128 and the number of channels is 32. After two single-channel feature fusion residual modules FFR_BLOCK, the input feature map size is 128×128 and the number of channels is 32, and the output feature map size is 128×128 and the number of channels is 32. After a transposed convolution layer with a convolution kernel size of 3*3, a stride of 2, and a number of convolution kernels of 16, the input feature map size is 128×128, the number of channels is 32, and the output feature map size is 256×256, and the number of channels is reduced to 16. After two single-channel feature fusion residual modules FFR_BLOCK, the input feature map size is 256×256, the number of channels is 16, and the output feature map size and number of channels are the same as the input size and number of channels. After a convolution layer with a convolution kernel size of 3*3, a stride of 1, and a padding value of 1, the input feature map size is 256×256, the number of channels is 16, and the final output size is 256×256, the number of channels is 3, and the clear steel bar binding site image u t ′.

[0242] In the embodiment of the present invention, considering that the accuracy of determining the binding quality by directly collecting images is low, an image processing model can be pre-trained to perform deblurring on the acquired images, that is, the relatively blurred initial image can be adjusted to a relatively clear target image. Thus, the purpose of realizing the deblurring and restoration project of the steel bar binding site image is achieved, and the technical effect of improving the accuracy of determining the steel bar binding quality is achieved, and the technical problem of low accuracy in determining the steel bar binding quality is solved.

[0243] Example 3

[0244] The embodiment of the present invention provides a device for determining the quality of steel bar binding. It should be noted that the document generation device of the embodiment of the present invention can be used to execute Figure 1 The method for determining the quality of steel bar binding provided by the embodiment of the present invention is described below.

[0245] Figure 7is a schematic diagram of a device for determining the quality of steel bar binding according to an embodiment of the present invention, such as Figure 7 As shown, the device may include: an acquisition unit 702 , a processing unit 704 and a determination unit 706 .

[0246] The acquisition unit 702 is used to acquire the initial image to be processed of the steel bars at the steel bar binding site.

[0247] The processing unit 704 is used to input the initial image into the image processing model for deblurring processing to obtain a target image corresponding to the initial image, wherein the image quality of the target image is higher than the image quality of the initial image. The image processing model is obtained by training a deep convolutional neural network model using the initial image samples and the target image samples, and the image quality of the target image samples is higher than the image quality of the corresponding initial image samples.

[0248] The determination unit 706 is used to determine the binding quality of the steel bars at the steel bar binding site based on the target image.

[0249] The device for determining the steel bar binding quality provided in the embodiment of the present invention obtains the initial image to be processed of the steel bars at the steel bar binding site through the acquisition unit 702; inputs the initial image into the image processing model for deblurring processing through the processing unit 704 to obtain the target image corresponding to the initial image; and determines the binding quality of the steel bars at the steel bar binding site based on the target image through the determination unit 706, thereby solving the technical problem of low accuracy in determining the steel bar binding quality and achieving the technical effect of improving the accuracy of determining the steel bar binding quality.

[0250] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set to control the same device type of devices to be shut down to perform graceful shutdown by adjusting kernel parameters.

[0251] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0252] The processor contains a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the work efficiency of traders can be improved by adjusting the kernel parameters.

[0253] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0254] Example 4

[0255] According to an embodiment of the present invention, there is further provided a computer-readable storage medium having a program stored thereon, which implements a method for determining the quality of steel bar binding when the program is executed by a processor.

[0256] Example 5

[0257] According to an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein the method for determining the quality of steel bar binding is executed when the program is run.

[0258] Example 6

[0259] Figure 8 is a schematic diagram of an electronic device for determining a method for steel bar binding quality according to an embodiment of the present invention, such as Figure 8 As shown, an embodiment of the present invention further provides an electronic device 800, which includes a processor 801, a memory 802, and a program stored in the memory and executable on the processor. When the processor executes the program, the above method of the embodiment of the present invention is implemented.

[0260] The device in this article can be a server, a personal computer (PC), a tablet computer (PAD), a mobile phone, etc.

[0261] Example 7

[0262] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing the above method of initializing an embodiment of the present invention.

[0263] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0264] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0265] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0266] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0267] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, referred to as CPU), input / output interface, network interface and memory.

[0268] The memory may include non-permanent memory, random access memory and / or non-volatile memory in a computer-readable medium, such as read-only memory or flash RAM. The memory is an example of a computer-readable medium.

[0269] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory, read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data information and carrier waves.

[0270] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0271] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0272] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for determining the quality of steel bar binding, characterized in that: include: Acquire an initial image of the steel bars to be processed at the steel bar binding site; Inputting the initial image into an image processing model for deblurring to obtain a target image corresponding to the initial image, wherein the image quality of the target image is higher than the image quality of the initial image, the image processing model is obtained by training a deep convolutional neural network model using the initial image samples and the target image samples, and the image quality of the target image samples is higher than the image quality of the corresponding initial image samples; Based on the target image, determining the binding quality of the steel bars at the steel bar binding site; Among them, the image processing model is a multi-scale parallel residual network model, which includes an upper network and a lower network. The upper network and the lower network are used to take the initial image of different image sizes as input. The upper network and the lower network respectively include a single-channel feature fusion residual module, a dual-channel multi-scale feature fusion module, an ordinary convolution module and a transposed convolution module. The ordinary convolution module is used to extract the image information in the initial image and to reduce the image size. The transposed convolution module is used to increase the image size and to increase the image information. The dual-channel multi-scale feature fusion module is used to perceive the jitter motion trend characteristics of the initial image sample at multiple scales and restore the texture information of the steel bar.

2. The determination method according to claim 1, characterized in that: Before inputting the initial image into the image processing model for deblurring to obtain a target image corresponding to the initial image, the method further includes: Performing fuzzy processing on the target image sample to obtain the initial image sample, and constructing a data set of the steel bar binding site based on the target image sample and the initial image sample; Preprocessing the data set to obtain a preprocessed data set, wherein the amount of image information in the preprocessed data set is higher than the amount of image information in the data set before the preprocessing; Based on the deep convolutional neural network model, an initial multi-scale parallel residual network model is built; The initial multi-scale parallel residual network model is trained using a training data set in the preprocessed data set, and the trained initial multi-scale parallel residual network model is verified using a verification data set in the preprocessed data set to obtain the multi-scale parallel residual network model, wherein the data set includes the training data set and the verification data set.

3. The determination method according to claim 2, characterized in that: Fuzzifying the target image sample to obtain the initial image sample includes: Performing a convolution operation in a computer vision library on the target image sample to obtain the corresponding initial image sample, wherein the convolution operation is used to simulate the relative movement between an image acquisition device and a plane at the steel bar binding site; The method further includes: determining the target image sample as label data of the initial image sample, wherein the label data is used to compare the target image sample with the corresponding target image obtained after the initial image sample is deblurred.

4. The determination method according to claim 2, characterized in that: Constructing the training data set and the verification data set based on the target image sample and the initial image sample, including: The target image samples and the initial image samples in the data set are divided into target proportions to determine the training data set and the verification data set.

5. The determination method according to claim 2, characterized in that: Preprocessing the data set to obtain a preprocessed data set includes: Downsampling the initial image samples in the data set to obtain downsampled initial image samples, wherein an image size of the downsampled initial image samples is smaller than an image size of the initial image samples before downsampling; In response to obtaining the downsampled initial image samples, the initial image samples are processed using a superior gradient shock filter to obtain processed initial image samples, wherein the superior gradient shock filter is used to increase the amount of image information in the initial image samples.

6. The determination method according to claim 5, characterized in that: The optimal gradient shock filter includes Gaussian filter convolution, wherein the initial image sample is processed by the optimal gradient shock filter to obtain the processed initial image sample, including: Determining a directional derivative of the initial image sample in a gradient direction; Using the directional derivative, edge detection is performed on the initial image sample to obtain a detection result, wherein the detection result is used to indicate whether the edge of the image content in the initial image sample is the steel bar; Performing the Gaussian filter convolution on the initial image sample to obtain gradient data of the initial image sample at each pixel point, wherein the Gaussian filter convolution is used to determine the texture edge of the initial image sample; Based on the gradient data and the detection result, the processed initial image sample is determined.

7. The determination method according to claim 2, characterized in that: Based on the deep convolutional neural network model, an initial multi-scale parallel residual network model is built, including: In the upper network and the lower network, the ordinary convolution module and the transposed convolution module are deployed; In the upper network and the lower network, the single-channel feature fusion residual module is deployed, wherein a first common convolution module, a second common convolution module and a third common convolution module are sequentially deployed in the single-channel feature fusion residual module, the convolution kernel and the step size of the first common convolution module and the second common convolution module are the same, the first common convolution module and the second common convolution module are used to perform a channel splicing operation and then be connected to the third common convolution module, and the convolution kernel of the third common convolution module is smaller than the convolution kernel of the first common convolution; In the upper network and the lower network, deploying the dual-path multi-scale feature fusion module; In the upper network, two of the single-path feature fusion residual modules, one of the dual-path multi-scale feature fusion modules, two of the single-path feature fusion residual modules, one of the dual-path multi-scale feature fusion modules, one of the transposed convolution modules, two of the single-path feature fusion residual modules and one of the ordinary convolution modules are sequentially deployed, and in the lower network, two of the single-path feature fusion residual modules, one of the dual-path multi-scale feature fusion modules, two of the single-path feature fusion residual modules, one of the transposed convolution modules, two of the single-path feature fusion residual modules and one of the ordinary convolution modules are sequentially deployed to obtain the initial multi-scale parallel residual network model.

8. The determination method according to claim 7, characterized in that: The initial multi-scale parallel residual network model is trained using a training data set in the preprocessed data set, including: Initializing parameters of the training initial multi-scale parallel residual network model to obtain initialized parameters; In response to completion of the parameter initialization, inputting the initial image sample of a first image size in the training data set into the upper network, and inputting the initial image sample of a second image size in the training data set into the lower network, wherein the first image size is larger than the second image size; In response to the upper network and the lower network receiving the corresponding initial image samples, training the initial multi-scale parallel residual network model; The method further comprises: In the process of training the initial multi-scale parallel residual network model, obtaining a target image determined based on the initial image sample; Determining initial loss data of the initial multi-scale parallel residual network model based on the Fourier transforms corresponding to the label data of the target image and the initial image sample respectively; Determining target loss data of the initial multi-scale parallel residual network model based on the initial loss data and the perceived loss data of the initial multi-scale parallel residual network model; In response to the target loss data reaching a convergence state, the initial multi-scale parallel residual network model is determined as a target multi-scale parallel residual network model, wherein the convergence state is used to characterize that the similarity between the label data and the corresponding target image reaches a similarity threshold.

9. The determination method according to claim 8, characterized in that: Using the validation data set in the preprocessed data set to validate the trained initial multi-scale parallel residual network model, to obtain the multi-scale parallel residual network model, including: Inputting the initial image samples in the verification data set into the target multi-scale parallel residual network model, and using the target multi-scale parallel residual network model to perform deblurring processing on the initial image samples to obtain a target image corresponding to the initial image samples; In response to the similarity between the target image and the label data of the corresponding initial image sample reaching the similarity threshold, the target multi-scale parallel residual network model is determined as the multi-scale parallel residual network model.

10. A processor, characterized in that: The processor is used to run a program, wherein the program, when run by the processor, executes the method for determining the steel bar binding quality according to any one of claims 1 to 9.

Images