Inspection blurred image recognition method and device based on regional target detection algorithm

Abstract

The invention relates to the technical field of image processing, and relates to an inspection fuzzy image recognition method and device based on a region target detection algorithm. The method comprises the following steps: acquiring an original image shot by a to-be-identified device; carrying out image preprocessing, and inputting the image into the trained region target detection network for processing; and carrying out deblurring processing by adopting a multi-Wiener filtering algorithm to obtain a reconstructed clear image, and finally carrying out image recognition. According to the invention, a region target detection network based on YOLO network modification is adopted, so that the to-be-deblurred target region in the equipment image can be accurately identified while the model is lightened; according to the method, a multi-Wiener filtering algorithm is adopted, so that the to-be-deblurred target area in the equipment image can be effectively deblurred; according to the method, the clear image can still be generated under the condition that the inspection robot moves, the definition and detail performance of the image are greatly improved, the recognition accuracy is further improved, and misjudgment caused by blurring is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically to: 1. a patrol fuzzy image recognition method based on a regional target detection algorithm; 2. a patrol fuzzy image recognition device based on a regional target detection algorithm. Background Art

[0002] Inspection robots are increasingly used in factories, power facilities and other scenarios. These robots need to capture and identify the status of various equipment during movement, among which the monitoring of thermal power generation relay cabinets is particularly important. However, due to the movement of the inspection robots and the working environment, the images captured often show motion blur, which seriously affects the subsequent image recognition and analysis work.

[0003] At present, the common deblurring methods for these images with motion blur are: image restoration-based methods, deep learning-based methods, and multi-frame image-based methods. Among them, although the image restoration-based methods have fast calculation speed, they are not effective for complex motion blur; deep learning-based methods such as convolutional neural networks (CNN) and generative adversarial networks (GAN) are effective in large-scale data scenarios, but require a large amount of training data and computing resources; multi-frame image-based methods such as optical flow estimation and sub-pixel registration are not suitable for deblurring single-frame images. However, inspection robots usually need to process a large number of images in real time, and many deblurring algorithms have high computational complexity. Inspection tasks usually only need to focus on specific areas (such as dashboards and indicator lights) rather than the entire image. Due to the mobile characteristics of robots and cameras, the blur degree in the image may be uneven, and there are limitations such as limited computing resources, special recognition requirements, and uneven blur degree.

[0004] Therefore, it is necessary to design a method that can quickly and effectively remove motion blur from images taken by a patrol robot in the relay room of a thermal power plant, so as to achieve high-accuracy image recognition. Summary of the invention

[0005] Based on this, it is necessary to provide a patrol fuzzy image recognition method and device based on a regional target detection algorithm to address the problem that images taken by existing patrol robots during movement are prone to motion blur and affect subsequent image recognition.

[0006] The present invention is implemented by the following technical solutions:

[0007] In a first aspect, the present invention discloses a patrol inspection fuzzy image recognition method based on a regional target detection algorithm, comprising:

[0008] Step 1: Obtain the original image g taken by the inspection robot equipped with a camera of the device to be identified0 (x, y), and perform image preprocessing to obtain a higher quality device image g 1 (x,y).

[0009] Among them, x represents the horizontal pixel coordinate of the image, and y represents the vertical pixel coordinate of the image.

[0010] Step 2: Set the device image g 1 (x,y) inputs the trained regional object detection network to detect the captured image g 1 The target area in (x,y) to be deblurred.

[0011] Among them, the construction method of the regional target detection network includes:

[0012] Replace the backbone network of the YOLO network with the MobileNetv3 structure network to obtain a regional object detection network.

[0013] Step 3: Use the multi-Wiener filtering algorithm to filter the device image g 1 The target area to be deblurred in (x, y) is deblurred to obtain a reconstructed clear image g 2 (x,y).

[0014] Step 4: Reconstruct the clear image g 2 (x, y) performs image recognition to obtain the status of the device to be identified.

[0015] This inspection fuzzy image recognition method based on the regional target detection algorithm implements the method or process according to the embodiment of the present disclosure.

[0016] In a second aspect, the present invention discloses a patrol fuzzy image recognition device based on a regional target detection algorithm, which uses the patrol fuzzy image recognition method based on a regional target detection algorithm disclosed in the first aspect.

[0017] The inspection fuzzy image recognition device based on the regional target detection algorithm includes: an image acquisition and preprocessing module, a regional target detection module, a defuzzification processing module, and an image recognition module.

[0018] The image acquisition and preprocessing module is used to obtain the original image g taken by the inspection robot equipped with a camera on the device to be identified. 0 (x, y), and perform image preprocessing to obtain a higher quality device image g 1 (x,y).

[0019] The regional object detection module is used to transform the device image g 1 (x,y) inputs the trained regional object detection network to detect the captured image g 1The target area in (x,y) to be deblurred.

[0020] The deblurring module is used to use the multi-Wiener filtering algorithm to deblur the device image g 1 The target area to be deblurred in (x, y) is deblurred to obtain a reconstructed clear image g 2 (x,y).

[0021] The image recognition module is used to reconstruct the clear image g 2 (x, y) performs image recognition to obtain the status of the device to be identified.

[0022] The inspection fuzzy image recognition device based on the regional target detection algorithm implements the method or process according to the embodiment of the present disclosure.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. The present invention adopts a regional target detection network modified based on the YOLO network, which can accurately identify the target area to be deblurred in the device image while lightweighting the model; the present invention adopts a multi-Wiener filtering algorithm, which can effectively deblur the target area to be deblurred in the device image; the present invention can still generate a clear image when the inspection robot is moving, greatly improving the clarity and detail expression of the image, thereby improving the recognition accuracy and reducing the misjudgment caused by blur.

[0025] 2. The regional target detection network adopted by the present invention adopts a variety of optimization training methods in the training stage, such as: relevant preprocessing of the training data set, using non-maximum suppression algorithm and multi-level optimization strategy for parameter adjustment, etc., which can effectively balance the universality and specificity of the regional target detection network and ensure the use effect of the regional target detection network.

[0026] 3. The device of the present invention can realize modular design, so that the recognition method of the present invention can be easily integrated into the inspection robot, which not only reduces the dependence on high-performance hardware, but also can realize clear real-time image processing and accurate recognition of equipment status. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0028] Figure 1A flowchart of a patrol fuzzy image recognition method based on a regional target detection algorithm provided in Example 1 of the present invention;

[0029] Figure 2 for Figure 1 The structure diagram of the target detection network in the middle area;

[0030] Figure 3 To get Figure 1 Process diagram of the trained regional object detection network;

[0031] Figure 4 for Figure 1 Specific flow chart of step three in . DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] It should be noted that when a component is referred to as being "mounted on" another component, it may be directly on the other component or there may be a central component. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a central component at the same time. When a component is considered to be "fixed to" another component, it may be directly fixed on the other component or there may be a central component at the same time.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "or / and" used herein includes any and all combinations of one or more of the related listed items.

[0035] Example 1

[0036] See also Figure 1 , Figure 1 A flowchart of a patrol fuzzy image recognition method based on a regional target detection algorithm is shown, which includes:

[0037] Step 1: Obtain the original image g taken by the inspection robot equipped with a camera of the device to be identified 0 (x, y), and perform image preprocessing to obtain a higher quality device image g 1 (x,y).

[0038] Among them, x represents the horizontal pixel coordinate of the image, and y represents the vertical pixel coordinate of the image.

[0039] Specifically, the inspection robot uses a camera to take real-time photos of the device to be identified during movement to obtain the original image g 0 (x,y);

[0040] For the original image g 0 The preprocessing of (x, y) is intended to improve its image quality. In this embodiment 1, the image preprocessing method may be: one or more of real-time denoising, color correction, and distortion correction.

[0041] Specific:

[0042] (1) Real-time denoising: The median filter algorithm is applied to reduce the impact of environmental noise on image quality. By replacing the value of each pixel with the median value of its neighboring pixels, the impact of sharp noise is suppressed.

[0043] (2) Color correction: Perform white balance correction on the image color according to the ambient lighting conditions. By adjusting the ratio of different color channels in the image, ensure that the image truly reflects the color of the object.

[0044] (3) Distortion correction: Based on the camera’s intrinsic and extrinsic calibration data, the geometric distortion (including radial distortion and tangential distortion) in the image is corrected.

[0045] Of course, other image preprocessing methods may also be used as long as they can improve the image quality.

[0046] It should be noted that the original image g 0 (x, y) is preferably accompanied by detailed metadata (e.g., timestamp, location information, environmental parameters, etc.) to provide comprehensive context information for the trained regional object detection network in the subsequent step 2.

[0047] Step 2: Set the device image g 1 (x,y) inputs the trained area object detection network to detect the device image g 1 The target area in (x,y) to be deblurred.

[0048] It should be noted that the present invention requires the use of a trained regional object detection network whose network parameters are in an optimal state.

[0049] One of the cores of the present invention lies in the structural design of the regional object detection network, and the other core lies in the training method of the regional object detection network; the combination of the two can balance the deblurring effect and computing resource consumption.

[0050] (I) First, the structural design of the regional target detection network is introduced, that is, the construction method of the regional target detection network, which includes:

[0051] Replace the backbone network of the YOLO network with the MobileNetv3 structure network to obtain a regional object detection network.

[0052] The YOLO network is a popular target detection algorithm that directly predicts the target bounding box and category from the input image through a single neural network to achieve end-to-end target detection. However, for inspection robots, their hardware configuration is generally not suitable for loading the YOLO network. Therefore, the inventor uses the YOLO network as the basic network for the design of the regional target detection network, and introduces the MobileNetv3 structure network to replace the backbone of the basic network to achieve lightweight architecture optimization of the YOLO network, which can greatly reduce the number of parameters and calculations, and improve processing efficiency while maintaining target detection accuracy, which is especially suitable for situations where resources are limited such as inspection robots.

[0053] In other words, see Figure 2 ,The regional object detection network includes: input module, backbone network module (MobileNetV3), feature pyramid module, detection head module, and network optimization module.

[0054] Among them, the input module is used to normalize and standardize the input image and output the processed image; the backbone network module is used to perform multi-level features on the processed image output by the input module and output feature maps of different resolutions; the feature pyramid module performs feature fusion on the multi-scale feature map output by the backbone network module to form a high-semantic, high-resolution multi-level feature map; the detection head module is used to generate the final detection result based on the multi-level feature map output by the feature pyramid module, directly predict the target category, bounding box and confidence, and thus obtain the target area; the network optimization module is used to adjust the parameters of the regional target detection network to improve the calibration accuracy of the target area.

[0055] (ii) See Figure 3 , then introduce the training method of regional target detection network, which includes:

[0056] S201, pre-training the YOLO network based on a large-scale general dataset;

[0057] Among them, large-scale general data sets can be existing public data sets, such as CIFAR-10, CIFAR-100, STL-10, ShareGPT4V, etc.

[0058] S201 aims to enable the YOLO network to acquire the basic ability to recognize general objects.

[0059] S202, migrating the network parameters of the pre-trained YOLO network to the untrained regional object detection network to obtain the pre-trained regional object detection network.

[0060] The regional object detection network and the YOLO network are the same except for the backbone part, so the network parameters of the other parts of the YOLO network can be directly copied to the other parts of the regional object detection network to achieve network parameter migration. The network parameters of the backbone part of the regional object detection network are assigned by random initialization.

[0061] S202 makes full use of the knowledge transfer of large-scale general data sets, so that the regional object detection network has considerable general object detection capabilities.

[0062] S203, constructing a dedicated device data set, and training the pre-trained regional object detection network based on the dedicated device data set to fine-tune network parameters;

[0063] It should be noted that the device corresponding to the dedicated device data set is the same as the device to be identified. In this embodiment 1, the dedicated device data set is the data set of the thermal power generation relay cabinet, which covers the key components of the thermal power generation relay cabinet such as the digital instrument panel, pointer, indicator light, etc., so as to ensure that the regional target detection network can detect these key features, thereby balancing the versatility and specificity of the network.

[0064] As mentioned above, images in dedicated equipment datasets should have metadata (e.g. timestamp, location information, environmental parameters, etc.) to provide comprehensive contextual information during subsequent training.

[0065] In S203, training the pre-trained regional object detection network based on the dedicated device data set to fine-tune network parameters includes:

[0066] S2031, unify and normalize the image sizes of the dedicated device data set to obtain a normalized dedicated data set.

[0067] In this embodiment 1, the image size of the dedicated device data set is first unified to a standard size (640x640 pixels), and then normalized, so as to ensure the consistency of the data and the stability of the network processing process.

[0068] S2032, first perform data enhancement processing on the normalized dedicated data set, and then perform multi-task learning to obtain an enhanced dedicated data set.

[0069] In this embodiment 1, data enhancement technologies such as MixUp and Mosaic are introduced so that the subsequent regional target detection network can obtain stronger generalization ability through training; the introduction of multi-task learning can provide more postures for the data set, so that the subsequent regional target detection network can not only identify key features through training, but also further improve the detection accuracy according to the posture information.

[0070] S2033, use the enhanced dedicated dataset to train the pre-trained regional object detection network, and use the non-maximum suppression algorithm and multi-level optimization strategy to adjust the parameters.

[0071] As mentioned above, the non-maximum suppression algorithm and the multi-level optimization strategy are both placed in the network optimization module for operation.

[0072] The non-maximum suppression algorithm obtains the score attenuation function S based on IoU, and uses the soft NMS strategy to eliminate duplicate detection boxes and redundant detection results, thereby retaining more effective information in target-dense scenes.

[0073] Among them, the expression of the attenuation function S is:

[0074]

[0075] In the formula, s i Represents the score of the detection box; IoU represents the intersection over union ratio with the highest scoring detection box; σ represents the adjustable attenuation coefficient.

[0076] The multi-level optimization strategy aims to improve the calibration accuracy of the target area. In this embodiment 1, the multi-level optimization strategy can adopt one or more of: optimizing the dynamic threshold function, adjusting the context weight, and using an edge detection algorithm.

[0077] Specific:

[0078] (1) Optimizing the dynamic threshold function: By constructing a dynamic threshold function T based on the detection confidence C, the threshold is dynamically set according to different confidence levels to reduce the generation of redundant detection boxes and optimize the balance between the recall rate and precision of the model.

[0079] Among them, the dynamic threshold function T can be expressed as:

[0080] T = μ + k * σ;

[0081] In the formula, μ represents the average value of the detection confidence C; σ represents the standard deviation of the detection confidence C; k represents the adjustable coefficient.

[0082] (2) Adjusting context weights: As mentioned above, the metadata of the images in the dedicated device dataset provides context information. By setting the context weight W, the feature learning of the area around the detection box is strengthened, thereby improving the network's recognition accuracy of the target area under complex backgrounds.

[0083] Among them, the context weight W can be expressed as:

[0084]

[0085] Where W(x,y) represents the context weight of the coordinate point (x,y); D(x,y) represents the distance between the coordinate point (x,y) and the expected position; σ represents an adjustable scale parameter.

[0086] (3) Use edge detection algorithm: Combined with the edge detection algorithm, additional training samples are generated near the target boundary. Through multiple iterative adjustments of the boundary features, the precise positioning of the detection box boundary is further refined, while reducing the phenomenon of missed detection.

[0087] Specifically, the edge detection algorithm will first calculate the gradient and edge direction angle to obtain the precise final target position. This process can be expressed as:

[0088]

[0089]

[0090]

[0091] P(x,y)=argmax(C(x,y)*W(x,y)*E(x,y));

[0092] In the formula, |G| represents the gradient amplitude; G x , G y Represent the gradients in the x and y directions respectively; θ represents the edge direction angle; P(x,y) represents the final target position; C(x,y) represents the detection confidence; W(x,y) represents the context weight; E(x,y) represents the edge strength.

[0093] Of course, other multi-level optimization strategies can also be adopted as long as they can improve the calibration accuracy of the target area.

[0094] It should be noted that during the training process, GPU hardware can also be used for acceleration to increase the network processing speed.

[0095] Step 3: Use the multi-Wiener filtering algorithm to filter the device image g 1 The target area to be deblurred in (x, y) is deblurred to obtain a reconstructed clear image g 2(x,y).

[0096] See also Figure 4 , step three is to deblur the target area to be deblurred in the device image, which includes:

[0097] S301, calculating a fuzzy kernel function h(x,y) based on an adaptive fuzzy kernel estimation method.

[0098] Specifically, the fuzzy model is first established as:

[0099] g 1 (x,y)=h(x,y)*f(x,y)+n(x,y);

[0100] Where f(x,y) represents the original clear image corresponding to the device to be identified; n(x,y) represents additive noise.

[0101] It can be seen that h(x,y) is an important bridge connecting the clear image and the blurred image.

[0102] Then, based on the relevant parameters of the inspection robot and its equipped camera, a fuzzy kernel function h(x,y) is constructed.

[0103] The expression of h(x,y) is:

[0104] h(x,y)=δ(xcos(θ)+ysin(θ)-L);

[0105] Wherein, δ(.) represents the Dirac delta function; θ represents the moving direction angle of the inspection robot; L represents the fuzzy length; L = v*T, v represents the moving speed of the inspection robot, and T represents time.

[0106] S302, the device image g 1 The target area to be deblurred in (x, y) and the blur kernel function h(x, y) are converted to the frequency domain to obtain the target area image G in the frequency domain 1 (u,v), fuzzy kernel function H(u,v) in frequency domain;

[0107] Among them, u represents the horizontal frequency component in the frequency domain, and v represents the vertical frequency component in the frequency domain.

[0108] Specifically, the device image g is transformed using fast Fourier transform. 1 The target area to be deblurred in (x, y) and the blur kernel function h(x, y) are converted to the frequency domain.

[0109] S303, first use Wiener filtering to filter the target area image G in the frequency domain. 1 (u,v) performs the initial deblurring operation to obtain the initial deblurred area image F 0(u,v);

[0110] Then, the initial deblurred area image F is filtered using iterative Wiener filtering based on gradient descent. 0 (u,v) is iterated for k rounds to obtain the final deblurred area image F k (u,v).

[0111] The process of S303 can be expressed as:

[0112]

[0113]

[0114] In the formula, represents the complex conjugate of H(u,v); K represents the signal-to-noise ratio; |H(u,v)| 2 represents the square of the modulus of H(u,v); F k-1 (u, v) represents the deblurred area image obtained by the k-1th round of iterative processing; α represents the step size factor; Δσ represents the descent gradient.

[0115] It should be noted that k is dynamically determined by the error convergence condition, and its value satisfies the following conditions:

[0116] k=min{n|||F k (u,v)-F k-1 (u,v)|| 2 <ε};

[0117] Where n represents the total number of pixels in the frequency domain image; ‖·‖ 2 represents the binary norm distance between frequency domain images; ε represents the preset convergence threshold, which is used to control the iteration accuracy.

[0118] S204, the final deblurred area image F k (u,v) is converted back to the spatial domain and replaces the device image g 1 The target area to be deblurred in (x, y) is obtained to obtain the reconstructed clear image g 2 (x,y).

[0119] Specifically, the final deblurred area image F is transformed by inverse Fourier transform. k (u,v) is converted back to the spatial domain, so that the device image g can be replaced accordingly 1 The target area in (x,y) to be deblurred.

[0120] Step 4: Reconstruct the clear image g 2 (x, y) performs image recognition to obtain the status of the device to be identified.

[0121] To reconstruct a clear image g 2There are many ways to perform image recognition on (x, y). Existing image recognition models can be used to reconstruct a clear image g. 2 (x, y) for unified recognition processing, or a multimodal recognition method can be used to reconstruct a clear image g 2 (x,y) for image recognition.

[0122] This embodiment 1 adopts the latter method, and the specific method is as follows:

[0123] S401, reconstructing a clear image g 2 (x, y) is subjected to adaptive binarization processing to obtain a processed reconstructed clear image.

[0124] Specifically, reconstruct the clear image g 2 There are deblurred target regions in (x, y). These deblurred target regions are taken as processing objects, and morphological operations (such as opening and closing operations) are applied to remove noise points and fill small holes to achieve adaptive binarization processing.

[0125] S402, using different recognition methods to perform feature recognition on the processed reconstructed clear image to obtain features of different categories.

[0126] The processed reconstructed clear image contains features of different categories. In order to ensure the recognition effect, different recognition methods are used for recognition.

[0127] In this embodiment 1, the processed reconstructed clear image has different types of features such as digital instrument panel, pointer, indicator light, etc., so:

[0128] ① Use optical character recognition technology combined with deep learning CRNN model to achieve high-precision digital instrument panel recognition.

[0129] ② Combine traditional image processing and advanced computer vision technology to achieve high-precision pointer recognition.

[0130] ③ Combine color segmentation algorithm and advanced image analysis technology to achieve accurate and reliable indicator light identification.

[0131] So far, the reconstruction of the clear image g has been achieved. 2 (x, y) is effectively recognized, and the state of the device to be recognized is obtained. In this embodiment 1, the state of the thermal power generation relay cabinet can be represented by the state of the digital instrument panel, pointer, and indicator light.

[0132] In addition, multimodal information fusion technology can be used to process the combined identified features of different categories to make a comprehensive judgment to improve the reliability of the results: for example, if the digital instrument panel and pointer status are both identified as abnormal, but the indicator light is identified as normal, then it is still judged that the thermal power generation relay cabinet is abnormal, which can ensure and improve safety.

[0133] Example 2

[0134] This embodiment 2 provides a patrol fuzzy image recognition device based on a regional target detection algorithm, which uses the patrol fuzzy image recognition method based on a regional target detection algorithm disclosed in embodiment 1.

[0135] The inspection fuzzy image recognition device based on the regional target detection algorithm includes: an image acquisition and preprocessing module, a regional target detection module, a defuzzification processing module, and an image recognition module.

[0136] The image acquisition and preprocessing module is used to obtain the original image g taken by the inspection robot equipped with a camera on the device to be identified. 0 (x, y), and perform image preprocessing to obtain a higher quality device image g 1 (x,y).

[0137] The regional object detection module is used to transform the device image g 1 (x,y) inputs the trained regional object detection network to detect the captured image g 1 The target area in (x,y) to be deblurred.

[0138] The deblurring module is used to use the multi-Wiener filtering algorithm to deblur the device image g 1 The target area to be deblurred in (x, y) is deblurred to obtain a reconstructed clear image g 2 (x,y).

[0139] The image recognition module is used to reconstruct the clear image g 2 (x, y) performs image recognition to obtain the status of the device to be identified.

[0140] Since the present device uses the inspection fuzzy image recognition method based on the regional target detection algorithm in Example 1, it also has the same effect and will not be repeated here.

[0141] Example 3

[0142] This embodiment 3 discloses a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the inspection fuzzy image recognition method based on the regional target detection algorithm disclosed in embodiment 1 are implemented.

[0143] This embodiment 3 also discloses a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps of the inspection fuzzy image recognition method based on the regional target detection algorithm disclosed in embodiment 1 are executed.

[0144] This embodiment 3 also discloses a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the inspection fuzzy image recognition method based on the regional target detection algorithm disclosed in embodiment 1 are implemented.

[0145] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A patrol inspection fuzzy image recognition method based on a regional target detection algorithm, characterized in that: include: Step 1: obtain the original image g0(x, y) of the device to be identified taken by the inspection robot equipped with a camera, and perform image preprocessing to obtain a higher quality device image g1(x, y); Among them, x represents the horizontal pixel coordinate of the image, and y represents the vertical pixel coordinate of the image; Step 2: Input the device image g1(x, y) into the trained area target detection network to detect the target area to be deblurred in the device image g1(x, y); Among them, the construction method of the regional target detection network includes: Replace the backbone network of the YOLO network with the MobileNetv3 structure network to obtain a regional object detection network; Step 3: Use a multi-Wiener filtering algorithm to deblur the target area to be deblurred in the device image g1(x, y) to obtain a reconstructed clear image g2(x, y); Step 4: Perform image recognition on the reconstructed clear image g2(x, y) to obtain the state of the device to be identified.

2. The inspection fuzzy image recognition method based on regional target detection algorithm according to claim 1 is characterized in that: In step 1, the image preprocessing method includes: Real-time denoising, color correction, and / or distortion correction.

3. The inspection fuzzy image recognition method based on regional target detection algorithm according to claim 1 is characterized in that: In step 2, the training method of the regional object detection network includes: S201, pre-training the YOLO network based on a large-scale general dataset; S202, migrating the network parameters of the pre-trained YOLO network to the untrained regional object detection network to obtain the pre-trained regional object detection network; S203, constructing a dedicated device data set, and training the pre-trained regional object detection network based on the dedicated device data set to fine-tune network parameters; The device corresponding to the dedicated device data set is the same as the device to be identified.

4. The inspection fuzzy image recognition method based on regional target detection algorithm according to claim 3 is characterized in that: In S203, training the pre-trained regional object detection network based on the dedicated device data set to fine-tune network parameters includes: S2031, unifying and normalizing the image sizes of the dedicated device data set to obtain a normalized dedicated data set; S2032, performing data enhancement processing on the normalized dedicated data set and then performing multi-task learning to obtain an enhanced dedicated data set; S2033, use the enhanced dedicated dataset to train the pre-trained regional object detection network, and use the non-maximum suppression algorithm and multi-level optimization strategy to adjust the parameters.

5. The inspection fuzzy image recognition method based on regional target detection algorithm according to claim 4 is characterized in that: S2033's multi-level optimization strategy includes: Optimize dynamic threshold function; or / and, adjust context weight; Or / and, use edge detection algorithm.

6. The inspection fuzzy image recognition method based on regional target detection algorithm according to claim 1 is characterized in that: Step three includes: S301, calculating a fuzzy kernel function h(x,y) based on an adaptive fuzzy kernel estimation method; S302, converting the target area to be deblurred in the device image g1(x, y) and the fuzzy kernel function h(x, y) into the frequency domain to obtain the target area image G1(u, v) in the frequency domain and the fuzzy kernel function H(u, v) in the frequency domain; Wherein, u represents the horizontal frequency component in the frequency domain, and v represents the vertical frequency component in the frequency domain; S303, first use Wiener filtering to perform a primary deblurring operation on the target region image G1(u,v) in the frequency domain to obtain a preliminary deblurred region image F 0 (u,v); Then, the initial deblurred area image F is filtered using iterative Wiener filtering based on gradient descent. 0 (u,v) is iterated for k rounds to obtain the final deblurred area image F k (u,v); S204, the final deblurred area image F k (u,v) is converted back to the spatial domain and replaces the target area to be deblurred in the device image g1(x,y) to obtain the reconstructed clear image g2(x,y).

7. The inspection fuzzy image recognition method based on regional target detection algorithm according to claim 6 is characterized in that: In S301, The expression of h(x,y) is: h(x,y)=δ(xcos(θ)+ysin(θ)-L); Wherein, δ(.) represents the Dirac delta function; θ represents the moving direction angle of the inspection robot; L represents the fuzzy length; L = v*T, v represents the moving speed of the inspection robot, and T represents time.

8. The inspection fuzzy image recognition method based on regional target detection algorithm according to claim 1 is characterized in that: In S303, F 0 The expression of (u,v) is: In the formula, represents the complex conjugate of H(u,v); K represents the signal-to-noise ratio; |H(u,v)| 2 represents the square of the modulus of H(u,v); F k The expression of (u,v) is: F k (u,v)=F k-1 (u,v)+α▽σ(F k-1 (u,v)); In the formula, F k-1 (u, v) represents the deblurred image obtained by the k-1th round of iterative processing; α represents the step size factor; Δσ represents the descent gradient.

9. The inspection fuzzy image recognition method based on regional target detection algorithm according to claim 1 is characterized in that: In step 4, a multimodal recognition method is used to perform image recognition on the reconstructed clear image g2(x, y).

10. A patrol inspection fuzzy image recognition device based on a regional target detection algorithm, characterized in that: It uses the inspection fuzzy image recognition method based on the regional target detection algorithm as described in any one of claims 1 to 9; The inspection fuzzy image recognition device based on the regional target detection algorithm includes: An image acquisition and preprocessing module is used to acquire the original image g0(x, y) taken by the inspection robot equipped with a camera of the device to be identified, and perform image preprocessing to obtain a higher quality device image g1(x, y); A regional object detection module is used to input the device image g1(x, y) into the trained regional object detection network to detect the target area to be deblurred in the captured image g1(x, y); A deblurring processing module, which is used to perform deblurring processing on the target area to be deblurred in the device image g1(x, y) by using a multi-Wiener filtering algorithm to obtain a reconstructed clear image g2(x, y); as well as The image recognition module is used to perform image recognition on the reconstructed clear image g2(x, y) to obtain the state of the device to be recognized.

Images