Machine vision image restoration and enhancement method based on deep learning

Abstract

The invention discloses a machine vision image restoration and enhancement method based on deep learning, and the method comprises the steps: S1, detecting an image abnormal region through a dual-channel full-convolution discriminator network, and generating an abnormal region position map and a confidence map; s2, performing morphological closed operation on the abnormal region position map to aggregate regions, screening abnormal regions in combination with a dynamic threshold, and generating a dynamic repair mask; s3, selectively repairing the dynamic repair mask coverage area by adopting a gating repair generator, and outputting a preliminary repair image; s4, calculating a characteristic difference quality index of the restoration area, and determining whether local super-resolution enhancement is triggered or not according to a threshold value; and S5, feeding back the final image to S1 for closed-loop verification, and if the residual abnormal region exceeds the tolerance, iteratively executing S2-S4. According to the method, breakthrough is realized in the dimensions of calculation efficiency, repair quality, scene adaptability and the like through a collaborative mechanism of abnormal region intelligent screening, quality dynamic evaluation and closed-loop iterative optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a machine vision image restoration and enhancement method based on deep learning. Background Art

[0002] With the rapid development of computer vision and deep learning technologies, machine vision image restoration and enhancement techniques have been widely applied in fields such as industrial inspection, medical image analysis, and cultural heritage preservation. Traditional methods rely primarily on global restoration algorithms or fixed-threshold region segmentation strategies to remove image anomalies such as noise, scratches, and occlusions. However, the diversity and dynamic nature of image anomalies in complex scenarios place higher demands on existing technologies.

[0003] Existing global repair models process the entire image indiscriminately, resulting in a large amount of computing resources being consumed in normal areas. For example, in industrial inspection scenarios, invalid calculations account for up to 70%; secondly, traditional methods rely on fixed thresholds or manual intervention to judge the repair effect, making it difficult to achieve adaptive enhanced decision-making in complex scenarios; at the same time, most solutions use a single repair process, which has limited effect on deep anomalies or multiple damages and lacks closed-loop verification capabilities. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a method for machine vision image restoration and enhancement based on deep learning, comprising:

[0007] S1. Detect abnormal image regions through a dual-channel fully convolutional discriminator network and generate abnormal region location maps and confidence maps.

[0008] S2. Perform morphological closing operation on the abnormal area location map to aggregate the areas, combine the dynamic threshold to filter the abnormal areas, and generate a dynamic repair mask;

[0009] S3, using a gated restoration generator to selectively restore the area covered by the dynamic restoration mask and output a preliminary restoration image;

[0010] S4, calculating the feature difference quality index of the repaired area, and deciding whether to trigger local super-resolution enhancement based on the threshold;

[0011] S5: Feed the final image back to S1 for closed-loop verification. If the residual abnormal area exceeds the tolerance, iteratively execute S2-S4.

[0012] As a preferred solution of the deep learning-based machine vision image restoration and enhancement method of the present invention, wherein: the dual-channel fully convolutional discriminator network D in S1 detect The build includes:

[0013] Input layer: accepts RGB images of any size;

[0014] Feature extraction layer: uses dilated convolution stacking to extract global abnormal features;

[0015] Output layer: Generates two channels of probability maps in parallel, including the abnormal area location map M loc and confidence map M conf ;

[0016] Through the joint loss function L detect =α·BCE(M loc , M gt )+β·SSIM(I raw ⊙M loc , I clean ) optimizes the network, where BCE is the binary cross entropy loss; M gt is the binary mask of the real abnormal area; SSIM is the structural similarity loss; ⊙ is the pixel-by-pixel multiplication; α and β are weight coefficients; I clean is the area corresponding to the reference image without abnormality; I raw ⊙M loc is the mask coverage area in the original image.

[0017] As a preferred solution of the deep learning-based machine vision image restoration and enhancement method of the present invention, the morphological closing operation in S2 adopts a rectangular structure element, whose size is dynamically adjusted according to the input image resolution, and the calculation formula is: Where: H, W are the image height and width;

[0018] The dynamic threshold in S2 is set to T high and T low , and T high >T low , while M conf The filtering rules include:

[0019] When the maximum confidence of the region max(M conf )>T high When the area R is marked as needing to be repaired fix ;

[0020] When T low<max(M conf )≤T high When the area is marked as the area to be verified R verify .

[0021] As a preferred solution of the machine vision image restoration and enhancement method based on deep learning of the present invention, wherein: the dynamic restoration mask M in S2 mask =I(M loc ∈R fix ∪R verify ).

[0022] As a preferred solution of the deep learning-based machine vision image restoration and enhancement method of the present invention, the operation of the gated restoration generator in S3 satisfies:

[0023] G out =G main (I raw )·M mask +I raw ·(1-M mask );

[0024] Among them: G out is the final output image of the gated restoration generator; G main The main restoration network adopts the U-Net structure to replace pixels in the mask area only; raw is the original damaged image.

[0025] As a preferred solution of the deep learning-based machine vision image restoration and enhancement method of the present invention, the training of the gated restoration generator adopts an adversarial training strategy, and the adversarial training strategy includes the following steps:

[0026] a. Build a discriminator D based on the PatchGAN architecture adv , whose input is the repaired image G out With mask M mask The concatenated tensor [G out ⊙M mask ;M mask ], outputs a region-level authenticity probability map;

[0027] b. Define the hybrid loss function L total =L adv +λL percepltual ,in:

[0028]

[0029] VGG l Represents the lth feature layer of the VGG19 network;

[0030] c. Use Adam optimizer for training, and set the initial learning rate to 2×10 -4 , momentum parameters β1=0.5, β2=0.999.

[0031] As a preferred solution of the deep learning-based machine vision image restoration and enhancement method of the present invention, the quality index in S4 is calculated as follows:

[0032]

[0033] Among them: VGG is the pre-trained VGG-19 network; I repair is the preliminary repaired image output in S3; I clean The reference image is normal.

[0034] As a preferred solution of the deep learning-based machine vision image restoration and enhancement method of the present invention, the threshold decision in S4 is as follows:

[0035] Q>γ: trigger local super-resolution enhancement, and output I after enhancement final ;

[0036] Q≤γ:Directly output I repair →I final .

[0037] As a preferred solution of the machine vision image restoration and enhancement method based on deep learning of the present invention, the iterative condition of the closed-loop verification in S5 is: the area ratio of the residual abnormal area|M residual | / |M mask |>δ, update the repair mask M mask =M mask ∪M residual , jump to S3 for secondary repair; |M residual | / |M mask |≤δ, output the final image I output ; δ is the preset tolerance threshold.

[0038] In a second aspect, the present invention provides a machine vision image restoration and enhancement system based on deep learning, comprising:

[0039] Abnormal region detection module: detects abnormal regions in the input image and generates abnormal location maps and confidence maps;

[0040] Dynamic decision module: Screens abnormal areas based on the confidence map and generates dynamic repair masks;

[0041] Selective restoration module: only repairs the mask-covered area, retaining the original pixels in the normal area;

[0042] Enhancement decision module: evaluates the restoration quality and triggers local enhancement or direct output;

[0043] Closed-loop feedback module: Verifies the repair results and iteratively optimizes until the residual anomalies meet the standards.

[0044] Beneficial effects of the present invention:

[0045] By using a dynamic repair mask to precisely locate abnormal areas, repair and enhancement operations are performed only on abnormal areas that account for 5%-30% of the total image area. Compared with traditional global processing methods, this method significantly reduces the amount of computation required to meet the needs of industrial real-time detection. Secondly, a quality indicator based on deep features, combined with a dynamic threshold, is introduced to achieve quantitative evaluation of repair results and enhance triggering decisions. At the same time, a threshold for the residual abnormal area ratio is used to control iteration termination, combined with a maximum iteration limit to ensure that deep abnormalities are thoroughly repaired while avoiding infinite loops. In industrial parts inspection, this closed-loop mechanism reduces the missed detection rate from 8% with traditional single-shot repair to 0.5%. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0047] Figure 1 This is a flowchart of the method for machine vision image restoration and enhancement based on deep learning proposed by the present invention;

[0048] Figure 2 This is a flowchart of the method flow of the deep learning-based machine vision image restoration and enhancement method proposed in the present invention;

[0049] Figure 3 This is a system architecture diagram of the deep learning-based machine vision image restoration and enhancement system proposed in this invention. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0052] Reference Figure 1-3 The present invention provides a machine vision image restoration and enhancement method based on deep learning, comprising:

[0053] S1. Detect abnormal image regions through a dual-channel fully convolutional discriminator network and generate abnormal region location maps and confidence maps.

[0054] The dual-channel fully convolutional discriminator network D in S1 detect The build includes:

[0055] Input layer: accepts RGB images of any size;

[0056] Feature extraction layer: uses dilated convolution stacking to extract global abnormal features;

[0057] The dilated convolution configuration includes:

[0058] Number of layers and parameters: Contains 4 levels of dilated convolutional layers, which gradually expand the receptive field. The specific configuration is as follows:

[0059] First layer: convolution kernel size 3×3, dilation rate = 2, output channels 64;

[0060] Second layer: convolution kernel size 3×3, dilation rate = 4, output channels 128;

[0061] The third layer: convolution kernel size 3×3, dilation rate = 6, number of output channels 256;

[0062] Fourth layer: convolution kernel size 3×3, dilation rate = 8, number of output channels 512;

[0063] Feature fusion: The output of each layer is superimposed with the features of the subsequent layers through skip connections to form a multi-scale feature pyramid;

[0064] Output layer: Generates two channels of probability maps in parallel, including the abnormal area location map M loc and confidence map M conf After the last layer of dilated convolution, two branches are used for independent processing: the abnormal area location branch, 1×1 convolution compression channel to 1, and the abnormal area location map M is output through the Sigmoid activation function. loc ∈[0,1] H×W ; Confidence evaluation branch, 1×1 convolution compresses the channel to 1, and outputs the abnormal confidence map M through the Softmax activation function conf ∈[0,1] H×W ; The two branches share the underlying features extracted by the first four layers of dilated convolution;

[0065] Through the joint loss function L detect =α·BCE(M loc , M gt)+β·SSIM(I raw ⊙M loc , I clean ) optimizes the network, where BCE is the binary cross entropy loss, which optimizes the positioning accuracy; M gt is the binary mask of the real abnormal area; SSIM is the structural similarity loss, which maintains the structural consistency of the normal area; ⊙ is the pixel-by-pixel multiplication; α and β are weight coefficients; I clean is the area corresponding to the reference image without abnormality; I raw ⊙M loc The mask coverage area in the original image is dynamically combined with BCE and SSIM through weight coefficients α and β. BCE provides accurate abnormal area positioning capability, and the SSIM constraint model only acts on the abnormal area I raw ⊙M loc Rather than a global image, this avoids the blurring tendency of traditional SSIM;

[0066] S2. Perform morphological closing operation on the abnormal area location map to aggregate the areas, combine the dynamic threshold to filter the abnormal areas, and generate a dynamic repair mask;

[0067] The morphological closing operation in S2 uses a rectangular structure element, whose size is dynamically adjusted according to the input image resolution. The calculation formula is: Where: H, W are the image height and width, ensuring the minimum size is 5×5 pixels;

[0068] Operation process: loc Perform dilation to fill small gaps between regions; perform erosion on the dilation result to restore the approximate shape of the original region;

[0069] Confidence adaptive rule: If the regional confidence max(M conf ) <T low , expand the structural element to 1.5 times the original size; for high confidence areas (M conf >T high ) retain the original boundary and do not perform closing operation;

[0070] Through parameter formulation and dynamic adjustment, it adapts to the abnormal distribution characteristics of different scenarios; at the same time, it limits the maximum size of structural elements to avoid excessive calculations;

[0071] The dynamic threshold in S2 is set to T high and T low , and T high >T low , while M conf The filtering rules include:

[0072] When the maximum confidence of the region max(M conf )>Thigh When the area R is marked as needing to be repaired fix ;

[0073] When T low <max(M conf )≤T high When the area is marked as the area to be verified R verify ;

[0074] T high / T low It can be adjusted dynamically according to the scene. For example, when medical imaging requires high sensitivity, T low =0.2;

[0075] The dynamic repair mask M mask =I(M loc ∈R fix ∪R verify ); the mask generation logic is as follows: Only generate repair masks for high-confidence anomalies and areas to be verified to avoid wasting resources;

[0076] S3, using a gated restoration generator to selectively restore the area covered by the dynamic restoration mask and output a preliminary restoration image;

[0077] The operation of the gated repair generator in S3 satisfies:

[0078] G out =G main (I raw )·M mask +I raw ·(1-M mask );

[0079] Among them: G out is the final output image of the gated restoration generator; G main The main restoration network adopts the U-Net structure to replace pixels in the mask area only; raw is the original damaged image; for M mask =1 area: use G main The generated repair pixels cover the original image; mask = 0: retain the original image pixels to avoid unnecessary modifications; only replace pixels in the mask-covered area to achieve local precise repair;

[0080] The training of the gated inpainting generator adopts an adversarial training strategy, which includes the following steps:

[0081] a. Build a discriminator D based on the PatchGAN architecture adv , whose input is the repaired image G out With mask Mmask The concatenated tensor [G out ⊙M mask ;M mask ], outputs a region-level authenticity probability map;

[0082] b. Define the hybrid loss function L total =L adv +λL perceptual ,in:

[0083]

[0084] VGG l Represents the lth feature layer of the VGG19 network;

[0085] c. Use Adam optimizer for training, and set the initial learning rate to 2×10 -4 , momentum parameters β1 = 0.5, β2 = 0.999;

[0086] Discriminator architecture:

[0087] Discriminator D adv The PatchGAN network is a 5-layer convolutional network with the following configuration:

[0088]

[0089] Local discrimination mechanism: only the mask coverage area (M mask =1) Calculate the adversarial loss, and the non-masked area does not participate in the gradient backpropagation;

[0090] Hybrid loss function:

[0091] Adversarial Loss (L adv ): The distribution of the driving repair area is close to the real abnormality-free area;

[0092] Perceptual loss (L perceptual ): Based on the feature differences of the relu1-2, relu2-2, and relu3-3 layers of the VGG19 network, the calculation formula is:

[0093]

[0094] Among them C l , H l , W l is the number of channels, height and width of the feature map of the lth layer;

[0095] Balance coefficient: λ = 10 was determined through grid search. Experiments show that this value has the highest naturalness score for the inpainted edge.

[0096] Training parameter settings:

[0097] Optimizer: Adam optimizer, initial learning rate is set to 2×10 -4 , momentum parameters β1 = 0.5, β2 = 0.999;

[0098] Training cycle: total number of iterations is 100,000, the learning rate remains unchanged for the first 50,000 iterations, and then linearly decays to 0 for the next 50,000 iterations;

[0099] Batch size: set to 4-16 (recommended value 8) based on CPU memory;

[0100] S4. For the area to be verified R verify Calculate the feature difference quality index of the repaired area and decide whether to trigger local super-resolution enhancement based on the threshold; the area R must be repaired fix Directly trigger local super-resolution enhancement;

[0101] The quality indicators in S4 are calculated as follows:

[0102]

[0103] Among them: VGG is the pre-trained VGG-19 network; I repair is the preliminary repaired image output in S3; I clean It is a reference image without abnormalities; based on the features extracted by the pre-trained VGG-19 network, it captures texture, edge, semantics and differences. Due to the traditional pixel-level indicators, only M is calculated at the same time. mask Differences in coverage areas to avoid invalid calculations in normal areas;

[0104] The threshold decision in S4 is as follows:

[0105] Q>γ (preset quality threshold): trigger local super-resolution enhancement, and output I after enhancement final ;

[0106] When local super-resolution enhancement is triggered, the ESPCN network is used to mask The area is upsampled by 2 times;

[0107] Q≤γ:Directly output I repair →I final ;

[0108] S5: Feed the final image back to S1 for closed-loop verification. If the residual abnormal area exceeds the tolerance, S2-S4 are iteratively executed. The maximum number of iterations can be set by the staff as needed to avoid infinite loops.

[0109] The iteration condition of the closed-loop verification in S5 is: the area ratio of the residual abnormal area|M residual | / |M mask|>δ, update the repair mask M mask =M mask ∪M residual , jump to S3 for secondary repair; |M residual | / |M mask |≤δ, output the final image I output ; δ is the preset tolerance threshold; the final output I in S4 final Re-input the discriminator in S1 to detect the abnormal areas that have not been repaired. If they meet the requirements, the repaired image will be directly output. If they do not meet the requirements, the repair will be performed again until they meet the requirements.

[0110] This embodiment also provides a machine vision image restoration and enhancement system based on deep learning, including:

[0111] Abnormal region detection module: detects abnormal regions in the input image and generates abnormal location maps and confidence maps;

[0112] Dynamic decision module: Screens abnormal areas based on the confidence map and generates dynamic repair masks;

[0113] Selective restoration module: only repairs the mask-covered area, retaining the original pixels in the normal area;

[0114] Enhancement decision module: evaluates the restoration quality and triggers local enhancement or direct output;

[0115] Closed-loop feedback module: Verifies the repair results and iteratively optimizes until the residual anomalies meet the standards.

[0116] In summary, the present invention achieves breakthroughs in computational efficiency, repair quality, and scene adaptability through a collaborative mechanism of intelligent screening of abnormal areas, dynamic quality assessment, and closed-loop iterative optimization, providing an efficient and reliable solution for high-precision image processing tasks such as industrial inspection.

[0117] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A machine vision image restoration and enhancement method based on deep learning, characterized by: include: S1. Detect abnormal image regions through a dual-channel fully convolutional discriminator network and generate abnormal region location maps and confidence maps. S2. Perform morphological closing operation on the abnormal area location map to aggregate the areas, combine the dynamic threshold to filter the abnormal areas, and generate a dynamic repair mask; S3, using a gated restoration generator to selectively restore the area covered by the dynamic restoration mask and output a preliminary restoration image; S4, calculating the feature difference quality index of the repaired area, and deciding whether to trigger local super-resolution enhancement based on the threshold; S5: Feed the final image back to S1 for closed-loop verification. If the residual abnormal area exceeds the tolerance, iteratively execute S2-S4.

2. The method for machine vision image restoration and enhancement based on deep learning according to claim 1, characterized in that: The dual-channel fully convolutional discriminator network D in S1 detect The build includes: Input layer: accepts RGB images of any size; Feature extraction layer: uses dilated convolution stacking to extract global abnormal features; Output layer: Generates two channels of probability maps in parallel, including the abnormal area location map M loc and confidence map M conf ; Through the joint loss function L detect =α·BCE(M loc , M gt )+β·SSIM(I raw ⊙M loc , I clean ) optimizes the network, where BCE is the binary cross entropy loss; M gt is the binary mask of the real abnormal area; SSIM is the structural similarity loss; ⊙ is the pixel-by-pixel multiplication; α and β are weight coefficients; I clean is the area corresponding to the reference image without abnormality; I raw ⊙M loc is the mask coverage area in the original image.

3. The method for machine vision image restoration and enhancement based on deep learning according to claim 2, characterized in that: The morphological closing operation in S2 uses a rectangular structure element, whose size is dynamically adjusted according to the input image resolution. The calculation formula is: Where: H, W are the image height and width; The dynamic threshold in S2 is set to T high and T low , and T high >T low , while M conf The filtering rules include: When the maximum confidence of the region max(M conf )>T high When the area R is marked as needing to be repaired fix ; When T low <max(M conf )≤T high When the area is marked as the area to be verified R verify .

4. The method for machine vision image restoration and enhancement based on deep learning according to claim 3, characterized in that: The dynamic repair mask M in S2 mask =I(M loc ∈R fix ∪R verify ).

5. The method for machine vision image restoration and enhancement based on deep learning according to claim 4, characterized in that: The operation of the gated repair generator in S3 satisfies: G out =G main (I raw )·M mask +I raw ·(1-M mask ); Among them: G out is the final output image of the gated restoration generator; G main The main restoration network adopts the U-Net structure to replace pixels in the mask area only; raw is the original damaged image.

6. The method for machine vision image restoration and enhancement based on deep learning according to claim 5, characterized in that: The training of the gated repair generator adopts an adversarial training strategy, which includes the following steps: a. Build a discriminator D based on the PatchGAN architecture adv , whose input is the repaired image G out With mask M mask The concatenated tensor [G out ⊙M mask ;M mask ], outputs a region-level authenticity probability map; b. Define the hybrid loss function L total =L adv +λL perceptual ,in: VGG l Represents the lth feature layer of the VGG19 network; c. Use Adam optimizer for training, and set the initial learning rate to 2×10 -4 , momentum parameters β1=0.5, β2=0.

999.

7. The method for machine vision image restoration and enhancement based on deep learning according to claim 6, characterized in that: The quality index in S4 is calculated as follows: Among them: VGG is the pre-trained VGG-19 network; I repair is the preliminary repaired image output in S3; I clean The reference image is normal.

8. The method for machine vision image restoration and enhancement based on deep learning according to claim 7, characterized in that: The threshold decision in S4 is as follows: Q>γ: trigger local super-resolution enhancement, and output I after enhancement final ; Q≤γ:Directly output I repair →I final .

9. The method for machine vision image restoration and enhancement based on deep learning according to claim 8, characterized in that: The iteration condition of the closed-loop verification in S5 is: the area ratio of the residual abnormal area|M residua l| / |M mask |>δ, update the repair mask M mask =M mask ∪M residual , jump to S3 for secondary repair; |M residual | / |M mask |≤δ, output the final image I output ; δ is the preset tolerance threshold.

10. A deep learning-based machine vision image restoration and enhancement system, based on the deep learning-based machine vision image restoration and enhancement method according to claims 1-9, characterized in that: include: Abnormal region detection module: detects abnormal regions in the input image and generates abnormal location maps and confidence maps; Dynamic decision module: Screens abnormal areas based on the confidence map and generates dynamic repair masks; Selective restoration module: only repairs the mask-covered area, retaining the original pixels in the normal area; Enhancement decision module: evaluates the restoration quality and triggers local enhancement or direct output; Closed-loop feedback module: Verifies the repair results and iteratively optimizes until the residual anomalies meet the standards.

Images