High dynamic image enhancement method, system, device and medium based on industrial scene

Through multiple rounds of training and weighted fusion processing of the full convolutional network model, the problem of uneven lighting in automated visual detection of workpieces is solved, the dynamic range of images is improved and the quality control of production lines is realized, equipment demand is simplified and costs are reduced.

CN118761947BActive Publication Date: 2024-11-12SHANDONG UNIV OF SCI & TECH +1
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411237183.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-11-12
Estimated Expiration
2044-09-05

AI Technical Summary

Technical Problem

In the automated visual inspection of workpieces, strong reflection on the surface of smooth objects leads to uneven image lighting, reducing dynamic range, affecting the QR code identification and production line quality control efficiency. The existing methods are costly and complex in operation, making it difficult to respond to production line changes in real time.

Method used

Using a high dynamic image enhancement method based on a full convolutional network, through multiple rounds of training models, different exposure images are used for weighted fusion processing, multi-dimensional features are extracted, the average gradient of illumination components is controlled, and high dynamic enhancement images are generated to reduce the impact of uneven light , improve the dynamic range of the image.

Benefits of technology

It realizes that in industrial scenarios, it can effectively remove uneven image lighting without complex equipment, improve image dynamic range, improve production line quality control efficiency, and reduce costs. It is suitable for automated implementation of complex industrial scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118761947B_ABST
    Figure CN118761947B_ABST
Patent Text Reader

Abstract

The present application relates to the field of image enhancement technology, specifically to a high-dynamic image enhancement method, system, device and medium based on industrial scenes; in order to solve the technical problems of uneven illumination distribution and low dynamic range of industrial scene images in the prior art, the present application designs a full convolutional network model, and trains the full convolutional network model with images of the same workpiece with different exposures. During the training process, the previous fusion results are fully utilized to improve the quality of subsequent fusion, and the training quality is improved based on the memory loss values ​​of the previous and next two rounds to obtain a trained full convolutional network model; finally, several images of the workpiece to be inspected with different exposures are obtained and processed by the trained full convolutional network model, and the uniformity of the illumination of the generated image is controlled to obtain a high-dynamic image enhancement result with uniform illumination and improved image dynamic range. The result can be applied to the process of automated visual inspection of workpieces to improve the quality control efficiency of the production line.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and in particular to a high-dynamic image enhancement method, system, device and medium based on industrial scenarios. Background Art

[0002] During the automated visual inspection of workpieces, the strong reflection from the smooth surface of objects often leads to uneven illumination distribution in the image, which results in uneven brightness of the image and reduces the dynamic range. This uneven illumination will produce overexposed or underexposed areas during image capture, seriously affecting the image details and making it difficult for the recognition algorithm to accurately distinguish the QR code or key characters on the workpiece, thus affecting the quality control efficiency of the entire production line.

[0003] In industrial applications, in order to solve the problem of uneven lighting in images, multiple types of light sources or polarizers are usually used. However, these methods can reduce the uneven brightness and light interference caused by smooth surface reflection to a certain extent, thereby improving the usability of the image. However, these devices are not only expensive, but also cumbersome to install and maintain, especially in production environments where equipment needs to be frequently adjusted or replaced. In addition, the use of complex lighting systems and polarizers may increase the operational complexity of the system, and for those industrial scenarios that require high real-time performance, it may not be possible to respond to rapid changes on the production line in real time. Summary of the invention

[0004] The purpose of the present invention is to provide a high-dynamic image enhancement method, system, device and medium based on industrial scenarios, which can reduce the impact of uneven illumination on images and enhance the dynamic range of images.

[0005] The technical solution of the present invention is as follows:

[0006] A high dynamic image enhancement method based on industrial scenes includes the following operations:

[0007] S1. Acquire several different exposure images of the same workpiece to form an initial different exposure image set; use the initial different exposure image set to perform multiple rounds of training on the full convolutional network model. If in the continuous first numerical rounds, the memory loss value of each round output and the corresponding previous round output is less than the memory loss threshold, output the full convolutional network model of the last round in the continuous first numerical rounds to obtain the trained full convolutional network model; the process of processing the full convolutional network model is as follows: in the initial different exposure image set, each exposure image is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the first round output; the first round output and the initial different exposure image set form a first enhanced image set; the first enhanced image set is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the second round output; the second round output and the first enhanced image set form a second enhanced image set, which is used to perform the next round of multi-dimensional feature extraction and weighted fusion processing;

[0008] S2. Acquire several images of the workpiece to be inspected with different exposures to form a set of images to be inspected with different exposures; the set of images to be inspected with different exposures is processed by a multi-round training full convolutional network model, and the output of each round is acquired in real time as a high dynamic enhanced image of each round; if the average gradient of the illumination component of the high dynamic enhanced image of the current round is not greater than the average gradient threshold, then the high dynamic enhanced image of the current round is output as a high dynamic image enhancement result; the method for acquiring the average gradient of the illumination component of the high dynamic enhanced image is as follows: the high dynamic enhanced image is subjected to grayscale processing to obtain a high dynamic enhanced grayscale image; based on a preset Gaussian function and the high dynamic enhanced grayscale image, a high dynamic enhanced illumination component image is obtained; the high dynamic enhanced illumination component image is segmented to obtain several segmented illumination component areas; the illumination component pixel difference between each segmented illumination component area and other segmented illumination component areas within the corresponding neighborhood is obtained; the ratio of the sum of all illumination component pixel differences to the size of the high dynamic enhanced image is taken as the average gradient of the illumination component.

[0009] The specific operations of multidimensional feature extraction in S1 are as follows: in the initial set of images with different exposures, each exposure image is subjected to downsampling, key channel feature learning and spatial feature learning respectively, to obtain all spatial feature learning graphs, thus forming a spatial feature learning atlas; in the initial set of images with different exposures, each exposure image guides the corresponding spatial feature learning graph to perform filtering and upsampling processing, to obtain all weight feature graphs, thus forming a weight feature atlas set, which is used to perform weighted fusion processing.

[0010] The specific operation of spatial feature learning is as follows: the channel feature learning graph obtained after key channel feature learning is segmented to obtain several channel feature block graphs; the current channel feature block graph is processed by dilated convolution to obtain the current dilated convolution channel feature block; the current dilated convolution channel feature block is processed by maximum pooling and average pooling respectively, and then concatenated and convolved to obtain the current spatial feature block; the current spatial feature block is element-wise multiplied with the current dilated convolution channel feature block to obtain the current spatial feature graph; all spatial feature graphs are concatenated and convolved to obtain a spatial feature learning graph; different channel feature block graphs have different dilated convolution kernel sizes when processed by dilated convolution.

[0011] The specific operation of key channel feature learning is as follows: the ratio of the pixel value of each position point in the current channel to the size of the downsampled exposure image obtained after downsampling is used as the first current channel learning image; the first current channel learning image is processed by the ReLU activation function and the Sigmoid activation function in turn, and then added element by element to the downsampled exposure image to obtain the feature learning image of the current channel; the feature learning images of all pre-selected channels are fused to obtain the key channel feature learning image.

[0012] The memory loss value in S1 is obtained by the following formula:

[0013] ,

[0014] ,

[0015] ,

[0016] L is the memory loss value, L=L 1 +L 2 +L 3 , L 1 is the first fusion loss, O is the output of the current round, O pre is the output of the previous round, L 2 is the second fusion loss, L 3 is the third fusion loss, SSIM(O,O pre ) is the structural similarity between the current round output and the previous round output, ‖ ‖ F For paradigm processing.

[0017] The structural similarity is obtained based on the pixel standard deviation and pixel covariance of the current round output and the pixel standard deviation of the previous round output.

[0018] The operation of obtaining the high dynamic enhanced illumination component image in S2 is specifically as follows: performing convolution processing on the high dynamic enhanced grayscale image using a preset Gaussian function to obtain a Gaussian component map; performing weighted sum processing on the Gaussian component map and the high dynamic enhanced grayscale image at corresponding positions to obtain a high dynamic enhanced illumination component image.

[0019] A high dynamic image enhancement system based on industrial scenes, used to implement the above-mentioned high dynamic image enhancement method based on industrial scenes, comprising:

[0020] A training full convolutional network model generation module is used to obtain several different exposure images of the same workpiece to form an initial different exposure image set; the full convolutional network model is trained for multiple rounds using the initial different exposure image set. If in the continuous first numerical rounds, the memory loss value of each round output and the corresponding previous round output is less than the memory loss threshold, the full convolutional network model of the last round in the continuous first numerical round is output to obtain the trained full convolutional network model; the process of processing the full convolutional network model is as follows: in the initial different exposure image set, each exposure image is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the first round output; the first round output and the initial different exposure image set form a first enhanced image set; the first enhanced image set is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the second round output; the second round output and the first enhanced image set form a second enhanced image set for performing the next round of multi-dimensional feature extraction and weighted fusion processing;

[0021] The high dynamic image enhancement result generation module is used to obtain several different exposure images of the workpiece to be inspected, forming a set of images to be inspected with different exposures; the set of images to be inspected with different exposures is processed by a multi-round training full convolutional network model, and the output of each round is obtained in real time as the high dynamic enhanced image of each round; if the average gradient of the illumination component of the high dynamic enhanced image of the current round is not greater than the average gradient threshold, the high dynamic enhanced image of the current round is output as the high dynamic image enhancement result; the method for obtaining the average gradient of the illumination component of the high dynamic enhanced image is: the high dynamic enhanced image is subjected to grayscale processing to obtain a high dynamic enhanced grayscale image; based on a preset Gaussian function and the high dynamic enhanced grayscale image, a high dynamic enhanced illumination component image is obtained; the high dynamic enhanced illumination component image is segmented to obtain several segmented illumination component areas; the illumination component pixel difference between each segmented illumination component area and other segmented illumination component areas within the corresponding neighborhood is obtained; the ratio of the sum of all illumination component pixel differences to the size of the high dynamic enhanced image is used as the average gradient of the illumination component.

[0022] A high-dynamic image enhancement device based on industrial scenes comprises a processor and a memory, wherein the processor implements the above-mentioned high-dynamic image enhancement method based on industrial scenes when executing a computer program stored in the memory.

[0023] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned high-dynamic image enhancement method based on industrial scenes when executed by a processor.

[0024] The beneficial effects of the present invention are:

[0025] The invention provides a high dynamic image enhancement method based on industrial scenes. A full convolutional network model capable of enhancing the ability to express image details is designed, and the full convolutional network model is trained with images of the same workpiece with different exposures. During the training process, the current round output containing important information required for the fusion task is retained and combined with the initial different exposure image set to guide the next round output, so that the previous fusion results are fully utilized to improve the subsequent fusion quality. In addition, the memory loss values ​​of the two previous and subsequent rounds are obtained in real time during the training process, and the model fusion training quality is controlled according to the size comparison result, so as to obtain a trained full convolutional network model capable of effectively removing uneven illumination in industrial images and improving the image dynamic range. A plurality of different exposure images of the workpiece to be inspected are obtained, and the training full convolutional network model is processed, and at the same time, the average gradient of the illumination component of the generated result is controlled to be not greater than the average gradient threshold, so as to control the uniformity of the generated image illumination, so as to obtain a high dynamic image enhancement result with uniform illumination and increased image dynamic range. The method is applied to an automated visual inspection production line for workpieces, does not require complex and expensive equipment, and can be automatically and stably implemented in complex industrial scenes, which is beneficial to reducing production costs while improving the quality control efficiency of the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] By reading the detailed description of the preferred embodiment below, the scheme and advantages of the present application will become clear to those skilled in the art. The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0027] In the attached picture:

[0028] Figure 1 Schematic diagram of the processing process of comparing memory loss values ​​during the processing of the full convolutional network model in this embodiment;

[0029] Figure 2 Graphs 2 and 3 are processing results of different exposure images of industrial scenes by this embodiment and five prior arts. DETAILED DESCRIPTION

[0030] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0031] This embodiment provides a high dynamic image enhancement method based on an industrial scene, including the following operations:

[0032] S1. Acquire several different exposure images of the same workpiece to form an initial different exposure image set; use the initial different exposure image set to perform multiple rounds of training on the full convolutional network model. If in the continuous first numerical rounds, the memory loss value of each round output and the corresponding previous round output is less than the memory loss threshold, output the full convolutional network model of the last round in the continuous first numerical rounds to obtain the trained full convolutional network model; the process of processing the full convolutional network model is as follows: in the initial different exposure image set, each exposure image is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the first round output; the first round output and the initial different exposure image set form a first enhanced image set; the first enhanced image set is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the second round output; the second round output and the first enhanced image set form a second enhanced image set, which is used to perform the next round of multi-dimensional feature extraction and weighted fusion processing;

[0033] S2. Acquire several images of the workpiece to be inspected with different exposures to form a set of images to be inspected with different exposures; the set of images to be inspected with different exposures is processed by a multi-round training full convolutional network model, and the output of each round is acquired in real time as a high dynamic enhanced image of each round; if the average gradient of the illumination component of the high dynamic enhanced image of the current round is not greater than the average gradient threshold, then the high dynamic enhanced image of the current round is output as a high dynamic image enhancement result; the method for acquiring the average gradient of the illumination component of the high dynamic enhanced image is as follows: the high dynamic enhanced image is subjected to grayscale processing to obtain a high dynamic enhanced grayscale image; based on a preset Gaussian function and the high dynamic enhanced grayscale image, a high dynamic enhanced illumination component image is obtained; the high dynamic enhanced illumination component image is segmented to obtain several segmented illumination component areas; the illumination component pixel difference between each segmented illumination component area and other segmented illumination component areas within the corresponding neighborhood is obtained; the ratio of the sum of all illumination component pixel differences to the size of the high dynamic enhanced image is taken as the average gradient of the illumination component.

[0034] S1. Acquire several different exposure images of the same workpiece to form an initial different exposure image set; use the initial different exposure image set to perform multiple rounds of training on the full convolutional network model. If in the first consecutive numerical rounds, the memory loss value of each round output and the corresponding previous round output is less than the memory loss threshold, output the last round of the full convolutional network model in the first consecutive numerical rounds to obtain the trained full convolutional network model.

[0035] A fully convolutional network model that can enhance the ability to express image details is designed, and the fully convolutional network model is trained using images of the same position with different exposure levels. During the training process, the current output containing important information required for the fusion task is retained to guide the generation of images in the next round of training, and the historical fusion results are fully utilized to improve the subsequent fusion quality. During the training process, the memory loss values ​​of the previous and next rounds are obtained in real time. According to the size comparison results, the quality of model fusion training is controlled, and a trained fully convolutional network model is obtained that can effectively remove uneven illumination in industrial images and improve the dynamic range of images.

[0036] First, several different exposure images of the same workpiece are acquired to form an initial different exposure image set.

[0037] Then, the full convolutional network model is trained for multiple rounds using the initial different exposure image sets. If in the first consecutive numerical rounds, the memory loss value of each round output and the corresponding previous round output are both less than the memory loss threshold, it proves that each round output is at least not worse than the previous round output, and the processing capacity of the full convolutional network model is stable. The full convolutional network model of the last round in the first consecutive numerical rounds is output to obtain a trained full convolutional network model. To facilitate technical personnel in this field to understand the above scheme, an example is given as follows: If in 20 consecutive rounds, the memory loss values ​​of the 20 round outputs and the corresponding previous round outputs are both less than the memory loss threshold, the full convolutional network model of the 20th round is output to obtain a trained full convolutional network model.

[0038] The processing process of the full convolutional network model is as follows: in the initial different exposure image set, each exposure image is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the first round of output; the first round of output and the initial different exposure image set form the first enhanced image set; the first enhanced image set is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the second round of output; the second round of output and the first enhanced image set form the second enhanced image set, which is used to perform the next round of multi-dimensional feature extraction and weighted fusion processing. The above process can be seen in Figure 1 By making full use of the previous fusion results to guide the next model training, the fusion quality of the model in the next training is improved, and the continuous evolution of the full convolutional network model is achieved.

[0039] The specific operation of multi-dimensional feature extraction is as follows: in the initial different exposure image set, each exposure image is respectively subjected to downsampling, key channel feature learning and spatial feature learning, and all spatial feature learning graphs are obtained to form a spatial feature learning atlas; in the initial different exposure image set, each exposure image guides the corresponding spatial feature learning graph to perform filtering upsampling processing, improves the resolution and details of the spatial feature learning graph, obtains all weight feature graphs, and forms a weight feature atlas for performing weighted fusion processing. Multi-dimensional feature extraction processing can enhance image details, facilitate the acquisition of similarity between images, and facilitate the subsequent acquisition of more accurate memory loss values, thereby improving the ability to eliminate uneven illumination and enhance the high dynamic range of images in training the fully convolutional network model.

[0040] The specific operation of key channel feature learning is as follows: the ratio of the pixel value of each position point in the current channel to the size of the downsampled exposure image obtained after downsampling is used as the first current channel learning image; the first current channel learning image is processed by the ReLU activation function and the Sigmoid activation function in turn, and then element-by-element addition is performed with the downsampled exposure image to obtain the feature learning image of the current channel; the feature learning images of all pre-selected channels are fused (merged / joined) to obtain the key channel feature learning image. Key channel feature learning can learn the importance of each channel based on the feature information of each channel, and adjust the information of different channels based on their importance, enriching the contextual information expression ability of the key channel feature learning image, making the final image more visually attractive and information-rich.

[0041] Key channel feature learning can be achieved through the following formula:

[0042] ,

[0043] ,

[0044] ,

[0045] Y c is the first current channel learning graph, H is the height of the downsampled exposure map, W is the width of the downsampled exposure image, p c (i,j) The first i Line j Column location point c The pixel value on the channel, C is the total number of pre-selected channels, W 1,c , W 2,care the first current channel weight and the second current channel weight, respectively. δ is the ReLU activation function, σ is the Sigmoid activation function, Y is the downsampled exposure map, ⊙ is an element-by-element multiplication operation, Zc is the c The feature learning graph of channels, S Learning graphs for key channel features.

[0046] The operation of spatial feature learning is as follows: the channel feature learning graph obtained after key channel feature learning is segmented to obtain several channel feature block graphs; the current channel feature block graph is processed by dilated convolution to obtain the current dilated convolution channel feature block; the current dilated convolution channel feature block is processed by maximum pooling and average pooling respectively, and then spliced ​​and convolved to obtain the current spatial feature block; the current spatial feature block is multiplied element by element with the current dilated convolution channel feature block to obtain the current spatial feature graph; all spatial feature graphs are spliced ​​and convolved to obtain the spatial feature learning graph; different channel feature block graphs have different dilated convolution kernel sizes when processed by dilated convolution. By performing different dilated convolution processes, contextual information of different scales can be obtained, and then spliced ​​and convolved after maximum pooling and average pooling respectively, which can suppress image artifacts on the basis of retaining image details, and facilitate the subsequent fusion (splicing and convolution) to improve the image detail expression characteristics without losing resolution.

[0047] The memory loss value is obtained by the following formula:

[0048] ,

[0049] ,

[0050] ,

[0051] L is the memory loss value, L=L 1 +L 2 +L 3 , L 1 is the first fusion loss, O is the output of the current round, O pre is the output of the previous round, L 2 is the second fusion loss, L 3 is the third fusion loss, SSIM(O,O pre) is the structural similarity between the current round output and the previous round output, ‖ ‖ F For paradigm processing.

[0052] The structural similarity is based on the pixel standard deviation and pixel covariance of the current round output, as well as the pixel standard deviation of the previous round output. The addition of the memory loss value allows the model to effectively utilize the information contained in the previous output to improve the fusion results of the current period. This design enables the intermediate results to play an important role in the training process and provide a stronger supervision signal for the image fusion of the current period.

[0053] S2. Acquire several images of the workpiece to be inspected with different exposures to form a set of images to be inspected with different exposures; the set of images to be inspected with different exposures is processed by a multi-round trained full convolutional network model, and the output of each round is acquired in real time as a high dynamic enhanced image of each round; if the average gradient of the illumination component of the high dynamic enhanced image of the current round is not greater than the average gradient threshold, then the high dynamic enhanced image of the current round is output as the high dynamic image enhancement result.

[0054] The acquired images of the workpiece to be inspected with different exposures are processed by the trained full convolutional network model. In the process of forming a fusion result by processing the images with different exposures through the trained full convolutional network model, the uneven illumination in the image will be eliminated, the dynamic range of the image will be improved, and the average gradient of the illumination component of the high dynamic enhanced image will be controlled not to be greater than the average gradient threshold, thereby controlling the uniformity of the generated image and obtaining a high dynamic image enhancement result.

[0055] The method for obtaining the average gradient of the illumination component of a high dynamic enhanced image is as follows: the high dynamic enhanced image is subjected to grayscale processing to obtain a high dynamic enhanced grayscale image; based on a preset Gaussian function and the high dynamic enhanced grayscale image, a high dynamic enhanced illumination component image is obtained; the high dynamic enhanced illumination component image is segmented to obtain a number of segmented illumination component areas; the illumination component pixel difference between each segmented illumination component area and other segmented illumination component areas within the corresponding neighborhood is obtained; the ratio of the sum of all illumination component pixel differences to the size of the high dynamic enhanced image is used as the average gradient of the illumination component.

[0056] Among them, the operation of the high dynamic enhanced illumination component image can be: using a preset Gaussian function to perform convolution processing on the high dynamic enhanced grayscale image to obtain a Gaussian component map; performing weighted summation processing on the Gaussian component map and the high dynamic enhanced grayscale image at corresponding positions to obtain a high dynamic enhanced illumination component image.

[0057] The formula for the Gaussian function is as follows:

[0058] ,

[0059] G(x,y) is the Gaussian function at the point (x,y) The value at x and y is the coordinate of the pixel relative to the center of the convolution kernel, b is the scale factor, λ is the normalization constant.

[0060] To verify the processing effect of this implementation method, three different exposure images were obtained from industrial scenes, see Figure 2 The exposure images ①, ② and ③ in the figure are processed using the method of this embodiment and advanced existing technologies (Mertens method, Li20 method, DeepFuse method, MEFNet method and IFCNN method). The processing results are shown in Figure 2 It is obvious that, compared with the five prior arts, the method of this embodiment has better ability to eliminate uneven illumination and enhance image details.

[0061] This embodiment further provides a high dynamic image enhancement system based on industrial scenes, which is used to implement the above-mentioned high dynamic image enhancement method based on industrial scenes, including:

[0062] A training full convolutional network model generation module is used to obtain several different exposure images of the same workpiece to form an initial different exposure image set; the full convolutional network model is trained for multiple rounds using the initial different exposure image set. If in the continuous first numerical rounds, the memory loss value of each round output and the corresponding previous round output is less than the memory loss threshold, the full convolutional network model of the last round in the continuous first numerical round is output to obtain the trained full convolutional network model; the process of processing the full convolutional network model is as follows: in the initial different exposure image set, each exposure image is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the first round output; the first round output and the initial different exposure image set form a first enhanced image set; the first enhanced image set is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain the second round output; the second round output and the first enhanced image set form a second enhanced image set for performing the next round of multi-dimensional feature extraction and weighted fusion processing;

[0063] The high dynamic image enhancement result generation module is used to obtain several different exposure images of the workpiece to be inspected, forming a set of images to be inspected with different exposures; the set of images to be inspected with different exposures is processed by a multi-round training full convolutional network model, and the output of each round is obtained in real time as the high dynamic enhanced image of each round; if the average gradient of the illumination component of the high dynamic enhanced image of the current round is not greater than the average gradient threshold, the high dynamic enhanced image of the current round is output as the high dynamic image enhancement result; the method for obtaining the average gradient of the illumination component of the high dynamic enhanced image is: the high dynamic enhanced image is subjected to grayscale processing to obtain a high dynamic enhanced grayscale image; based on a preset Gaussian function and the high dynamic enhanced grayscale image, a high dynamic enhanced illumination component image is obtained; the high dynamic enhanced illumination component image is segmented to obtain several segmented illumination component areas; the illumination component pixel difference between each segmented illumination component area and other segmented illumination component areas within the corresponding neighborhood is obtained; the ratio of the sum of all illumination component pixel differences to the size of the high dynamic enhanced image is used as the average gradient of the illumination component.

[0064] This embodiment also provides a high-dynamic image enhancement device based on industrial scenes, including a processor and a memory, wherein the processor implements the above-mentioned high-dynamic image enhancement method based on industrial scenes when executing a computer program stored in the memory.

[0065] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned high-dynamic image enhancement method based on industrial scenarios.

[0066] The present embodiment provides a high dynamic image enhancement method based on industrial scenes, designs a full convolutional network model that can enhance the ability to express image details, and uses images of the same workpiece with different exposures to train the full convolutional network model. During the training process, the current round output containing important information required for the fusion task is retained and combined with the initial different exposure image set to guide the next round output, so as to make full use of the previous fusion results to improve the subsequent fusion quality. In addition, during the training process, the memory loss values ​​of the previous and next rounds are obtained in real time, and the model fusion training quality is controlled according to the size comparison result, so as to obtain a trained full convolutional network model that can effectively remove uneven illumination in industrial images and improve the dynamic range of the image; a plurality of different exposure images of the workpiece to be inspected are obtained, and the training full convolutional network model is processed, and at the same time, the average gradient of the illumination component of the generated result is controlled to be not greater than the average gradient threshold, so as to control the uniformity of the generated image illumination, so as to obtain a high dynamic image enhancement result with uniform illumination and increased image dynamic range; the method is applied to the workpiece automatic visual inspection production line, does not require complex and expensive equipment, and can be automatically and stably implemented in complex industrial scenes, which is conducive to reducing production costs while improving the quality control efficiency of the production line.

Claims

1. A high dynamic image enhancement method based on industrial scenes, characterized in that: The following operations are included: S1. Acquire several different exposure images of the same workpiece to form an initial different exposure image set; use the initial different exposure image set to train the full convolutional network model for multiple rounds. If in the continuous first numerical rounds, the memory loss value of each round output and the corresponding previous round output are both less than the memory loss threshold, output the full convolutional network model of the last round in the continuous first numerical rounds to obtain the trained full convolutional network model; The processing process of the full convolutional network model is as follows: in the initial different exposure image set, each exposure image is subjected to multi-dimensional feature extraction and then weighted fusion processing to obtain a first round of output; the first round of output and the initial different exposure image set form a first enhanced image set; After multi-dimensional feature extraction, the first enhanced image set is subjected to weighted fusion processing to obtain a second round of output; the second round of output and the first enhanced image set form a second enhanced image set for performing the next round of multi-dimensional feature extraction and weighted fusion processing; S2, obtaining a plurality of images of the workpiece to be inspected with different exposures to form a set of images of the workpiece to be inspected with different exposures; the set of images of the workpiece to be inspected with different exposures is processed by a multi-round training full convolutional network model, and the output of each round is obtained in real time as a high dynamic enhanced image of each round; If the average gradient of the illumination component of the current round of high dynamic enhanced image is not greater than the average gradient threshold, the current round of high dynamic enhanced image is output as the high dynamic image enhancement result; The method for acquiring the average gradient of the illumination component of the high dynamic enhanced image is as follows: the high dynamic enhanced image is subjected to grayscale processing to obtain a high dynamic enhanced grayscale image; Based on a preset Gaussian function and the high dynamic enhanced grayscale image, a high dynamic enhanced illumination component image is obtained; the high dynamic enhanced illumination component image is segmented to obtain a plurality of segmented illumination component regions; Obtain the pixel difference of the illumination component between each segmented illumination component area and other segmented illumination component areas within the corresponding neighborhood; The ratio of the sum of the pixel differences of all illumination components to the size of the high dynamic range enhanced image is taken as the average gradient of the illumination component.

2. The high dynamic image enhancement method based on industrial scenes according to claim 1 is characterized in that: The operation of multidimensional feature extraction in S1 is specifically as follows: In the initial different exposure image set, each exposure image is respectively subjected to downsampling, key channel feature learning and spatial feature learning to obtain all spatial feature learning graphs, thereby forming a spatial feature learning graph set; In the initial set of different exposure images, each exposure image guides the corresponding spatial feature learning map to perform filtering and upsampling processing to obtain all weight feature maps, forming a weight feature map set for performing the weighted fusion processing operation.

3. The high dynamic image enhancement method based on industrial scenes according to claim 2 is characterized in that: The operation of the spatial feature learning is specifically as follows: The channel feature learning graph obtained after the key channel feature learning process is segmented to obtain several channel feature block graphs; The current channel feature block map is processed by dilated convolution to obtain a current dilated convolution channel feature block; the current dilated convolution channel feature block is processed by maximum pooling and average pooling respectively, and then concatenated and convolved to obtain a current spatial feature block; the current spatial feature block is multiplied element by element with the current dilated convolution channel feature block to obtain a current spatial feature map; All spatial feature maps are concatenated and convolved to obtain a spatial feature learning map; The sizes of the dilated convolution kernels of feature block images of different channels are different when they are processed by dilated convolution.

4. The high dynamic image enhancement method based on industrial scenes according to claim 2 is characterized in that: The operation of key channel feature learning is specifically as follows: The ratio of the pixel value of each position point in the current channel to the size of the downsampled exposure image in the downsampled exposure image obtained after the downsampling process is used as the first current channel learning image; the first current channel learning image is processed by the ReLU activation function and the Sigmoid activation function in sequence, and then element-by-element addition is performed with the downsampled exposure image to obtain a feature learning image of the current channel; The feature learning graphs of all pre-selected channels are fused to obtain the key channel feature learning graph.

5. The high dynamic image enhancement method based on industrial scenes according to claim 1 is characterized in that: The memory loss value in S1 is obtained by the following formula: , , , L is the memory loss value, L=L 1 +L 2 +L 3 , L 1 is the first fusion loss, O is the output of the current round, O pre is the output of the previous round, L 2 is the second fusion loss, L 3 is the third fusion loss, SSIM(O,O pre ) is the structural similarity between the current round output and the previous round output, ‖ ‖ F For paradigm processing.

6. The high dynamic image enhancement method based on industrial scenes according to claim 5 is characterized in that: The structural similarity is obtained based on the pixel standard deviation and pixel covariance output in the current round, and the pixel standard deviation output in the previous round.

7. The high dynamic image enhancement method based on industrial scenes according to claim 1 is characterized in that: In S2, the operation of obtaining the high dynamic enhanced illumination component image is specifically as follows: The high dynamic enhanced grayscale image is convolved using a preset Gaussian function to obtain a Gaussian component map; the Gaussian component map and the high dynamic enhanced grayscale image are weighted summed at corresponding positions to obtain the high dynamic enhanced illumination component image.

8. A high dynamic image enhancement system based on industrial scenes, used to implement the high dynamic image enhancement method based on industrial scenes as claimed in claim 1, characterized in that: include: A training full convolutional network model generation module is used to obtain a number of different exposure images of the same workpiece to form an initial different exposure image set; the full convolutional network model is trained for multiple rounds using the initial different exposure image set, and if in the continuous first numerical rounds, the memory loss value of each round output and the corresponding previous round output is less than the memory loss threshold, the full convolutional network model of the last round in the continuous first numerical rounds is output to obtain the trained full convolutional network model; the process of the full convolutional network model processing is: in the initial different exposure image set, each exposure image is subjected to multi-dimensional feature extraction and then weighted fusion processing is performed to obtain the first round output; the first round output and the initial different exposure image set form a first enhanced image set; After multi-dimensional feature extraction, the first enhanced image set is subjected to weighted fusion processing to obtain a second round of output; the second round of output and the first enhanced image set form a second enhanced image set for performing the next round of multi-dimensional feature extraction and weighted fusion processing; The high dynamic image enhancement result generation module is used to obtain a plurality of different exposure images of the workpiece to be inspected, forming a set of images to be inspected with different exposures; the set of images to be inspected with different exposures is processed by a multi-round training full convolutional network model, and the output of each round is obtained in real time as a high dynamic enhanced image of each round; If the average gradient of the illumination component of the current round of high dynamic enhanced image is not greater than the average gradient threshold, the current round of high dynamic enhanced image is output as the high dynamic image enhancement result; The method for acquiring the average gradient of the illumination component of the high dynamic enhanced image is as follows: the high dynamic enhanced image is subjected to grayscale processing to obtain a high dynamic enhanced grayscale image; Based on a preset Gaussian function and the high dynamic enhanced grayscale image, a high dynamic enhanced illumination component image is obtained; the high dynamic enhanced illumination component image is segmented to obtain a plurality of segmented illumination component regions; Obtain the pixel difference of the illumination component between each segmented illumination component area and other segmented illumination component areas within the corresponding neighborhood; The ratio of the sum of the pixel differences of all illumination components to the size of the high dynamic range enhanced image is taken as the average gradient of the illumination component.

9. A high dynamic image enhancement device based on industrial scenes, characterized in that: It comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the high dynamic image enhancement method based on industrial scenes as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the high dynamic image enhancement method based on an industrial scene as described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Dynamic illumination face image quality enhancement method based on multi-scale attention mechanism

    CN115880225A

  • Low-illumination image enhancement method based on convolutional neural network

    CN116523794A

  • Lightweight low-illumination image enhancement method based on convolutional neural network

    CN118351041A

  • Methods and apparatus for computer vision based on multi-stream feature-domain fusion

    US20240127584A1

  • KR20230146974A