A high dynamic range image reconstruction method and system based on deep learning

Through deep learning technology, high dynamic range image reconstruction method is adopted on the high-reflection surface of the automobile sunroof guide rail profile, which solves the problem of defect identification caused by overexposure and light angle in machine vision detection, and achieves efficient and accurate defects 检测,支持自动化检测,降低了人工成本。

CN118172254BActive Publication Date: 2024-11-12INTERNATIONAL INSTITUTE FOR INNOVATIVE DESIGN & INTELLIGENT MANUFACTURING OF TIANJIN UNIVERSITY-ZHEJIANG +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410208933.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-26
Publication Date
2024-11-12
Estimated Expiration
2044-02-26

AI Technical Summary

Technical Problem

When detecting the highly reflective surface of the automotive sunroof guide rail profile, due to the influence of overexposure and light angle, small defects are difficult to identify, and the defect recognition accuracy is low and the efficiency is low, making it difficult to meet the needs of high-precision detection.

Method used

Using a high dynamic range image reconstruction method based on deep learning, low dynamic range images are collected in one exposure, and guide filtering and cascaded structural neural network model processing are performed, and multiple exposure fusions of Gaussian decomposition and Laplace decomposition are generated. High dynamic range images solve the problems of light changes and details loss, and improve image details retention and detection efficiency.

Benefits of technology

It realizes accurate identification of surface defects of high-reflective metals, improves detection efficiency and accuracy, reduces labor costs, supports online inspection, and promotes the development of automated inspection of high-reflective metal products.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118172254B_ABST
    Figure CN118172254B_ABST
Patent Text Reader

Abstract

The invention discloses a high dynamic range image reconstruction method and system based on deep learning, which belongs to the field of image processing technology. The method includes: collecting a low dynamic range image of a target object; converting the low dynamic range image to a YUV space to obtain a low dynamic range image; performing a guided filtering algorithm on a brightness layer image to obtain a basic layer image of the low dynamic range image, taking the difference between the brightness layer image and the basic layer image as a detail layer image of the low dynamic range image; generating a first low dynamic range image set and a second low dynamic range image set; performing multi-exposure fusion on the image set to obtain brightness layer high dynamic range images and detail layer high dynamic range images; fusing them, combining the obtained new brightness layer high dynamic range image with the image corresponding to the chroma layer; converting the obtained high dynamic range fused image to an RGB space and then performing denoising to obtain a target high dynamic range image. The image light and dark details are enriched, the running time of the detection algorithm is shortened, and the accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and specifically relates to a high dynamic range image reconstruction method and system based on deep learning. Background Art

[0002] With the continuous upgrading of family car configuration, sunroofs have become standard for most models, so the demand for guide rail profiles required for car sunroof manufacturing has increased year by year. The guide rails for car sunroofs are thin-walled aluminum alloy profiles with complex structures. They are made of aluminum alloy bars through multiple processes such as sawing, extrusion, stretching, and cutting. After forming, the profiles need to be cut and inspected according to customer requirements to eliminate defective products. Since the appearance requirements of car sunroof guide rails are high, minor defects also need to be detected and eliminated. At present, the main inspection methods are manual visual inspection, hand touch, pencil scratching, supplemented by inspection fixtures, calipers, etc., with low inspection efficiency, limited accuracy, and high labor costs.

[0003] In order to improve the efficiency and accuracy of detection and reduce labor costs, it is necessary to adopt automated detection. Common automated detection methods include machine vision detection, laser flaw detection, ultrasonic scanning detection and other detection methods. Among them, laser flaw detection, ultrasonic scanning detection and other methods are mainly used for internal defect detection, which are costly and inefficient, and are mostly limited to random inspections of some high-precision parts. For surface defect detection, machine vision methods are currently the main method. Machine vision detection methods are low-cost, with high efficiency and accuracy, and will not be affected by long-term operation. Machine vision detection equipment is relatively mature and can work reliably in environments that are not suitable for long-term continuous work by humans.

[0004] However, when it comes to inspecting highly reflective surfaces such as automotive sunroof guide profiles, machine vision methods still face problems such as overexposure making it difficult to identify tiny defects, low defect identification accuracy due to the influence of lighting angle, and multiple exposures that take a long time and are inefficient. Summary of the invention

[0005] In order to solve the technical problems existing in the prior art of detecting highly reflective surfaces such as automobile sunroof guide profiles, the machine vision method still faces the problem of overexposure causing small defects to be difficult to identify, low defect identification accuracy due to the influence of lighting angle, and multiple exposures taking a long time and low efficiency, the present invention provides a high dynamic range image reconstruction method and system based on deep learning.

[0006] First aspect

[0007] The present invention provides a high dynamic range image reconstruction method based on deep learning, comprising:

[0008] S101: acquiring a low dynamic range image of a target object in one exposure;

[0009] S102: converting the low dynamic range image from the RGB space to the YUV space to obtain a brightness layer image and a chrominance layer image of the low dynamic range image;

[0010] S103: performing a guided filtering algorithm on the brightness layer image to obtain a base layer image of the low dynamic range image, and taking the difference between the brightness layer image and the base layer image as a detail layer image of the low dynamic range image;

[0011] S104: using a cascade structured neural network model to progressively generate a first low dynamic range image set and a second low dynamic range image set, wherein the first low dynamic range image set includes a plurality of brightness layer images at different exposure levels, and the second low dynamic range image set includes a plurality of detail layer images at different exposure levels;

[0012] S105: performing multi-exposure fusion based on Gaussian decomposition and Laplace decomposition on the first low dynamic range image set and the second low dynamic range image set respectively to obtain a brightness layer high dynamic range image and a detail layer high dynamic range image;

[0013] S106: Fusing the brightness layer high dynamic range image and the detail layer high dynamic range image to obtain a new brightness layer high dynamic range image, and combining the new brightness layer high dynamic range image with the image corresponding to the chroma layer to obtain a high dynamic range fused image;

[0014] S107: converting the high dynamic range fused image from the YUV space to the RGB space;

[0015] S108: De-noising the high dynamic range fused image converted to the RGB space to obtain a target high dynamic range image.

[0016] Second aspect

[0017] The present invention provides a high dynamic range image reconstruction system based on deep learning, comprising a processor and a memory for storing processor executable instructions; the processor is configured to call the instructions stored in the memory to execute the high dynamic range image reconstruction method based on deep learning in the first aspect.

[0018] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0019] In the present invention, a detail layer image is separated from a brightness layer image of a low dynamic range image through a guided filtering algorithm, and a neural network model with a cascade structure is introduced to generate a low dynamic range image set of the collected low dynamic range image at different exposure levels. Then, the generated image set is subjected to multi-exposure fusion based on Gaussian decomposition and Laplace decomposition to generate a corresponding high dynamic range image. While shortening the algorithm running time, the image details are retained, and the problem of small defects being difficult to identify due to multiple exposures during the image acquisition process is avoided, thereby mining more detailed information of the image, obtaining an image with rich light and dark details and high contrast, solving the problem of local brightness oversaturation on the surface of highly reflective metals, improving the accuracy of defect detection, supporting online detection, improving detection efficiency, reducing labor costs, and promoting the development of automated detection of highly reflective metal products. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The preferred implementation modes will be described below in a clear and understandable manner with reference to the accompanying drawings to further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of the present invention.

[0021] Figure 1 It is a flowchart of a high dynamic range image reconstruction method based on deep learning provided by the present invention;

[0022] Figure 2 The present invention provides a low dynamic range image of a highly reflective metal surface;

[0023] Figure 3 It is an image detail layer obtained by separation of detail layers provided by the present invention;

[0024] Figure 4 It is a flowchart of deep learning provided by the present invention;

[0025] Figure 5 The present invention provides a target high dynamic range image obtained by dynamic range enhancement based on deep learning. DETAILED DESCRIPTION

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other specific implementations can be obtained based on these drawings without creative work.

[0027] In order to simplify the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0028] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0029] In this document, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0030] In addition, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. Specific embodiment 1

[0032] In one embodiment, the reference specification Figure 1 , which shows a flow chart of a high dynamic range image reconstruction method based on deep learning provided by the present invention. Figure 2 , showing a low dynamic range image of a highly reflective metal surface collected by the present invention.

[0033] The present invention provides a high dynamic range image reconstruction method based on deep learning, comprising:

[0034] S101: Acquire a low dynamic range image of a target object in one exposure.

[0035] The target object can be a highly reflective surface of a car sunroof rail. A low dynamic range image, also known as an LDR (Low Dynamic Range) image, refers to an image with a relatively small brightness range, which is usually contained within the range of a single exposure of a camera or sensor. The brightness range refers to the difference between the darkest and brightest areas in the image. A low dynamic range image is acquired through a single exposure, which is used as input to the algorithm and then processed through a series of steps to reconstruct a high dynamic range image in order to better preserve brightness changes and details in the scene.

[0036] S102: Convert the low dynamic range image from the RGB space to the YUV space to obtain a brightness layer image and a chrominance layer image of the low dynamic range image.

[0037] Among them, the RGB space usually consists of three channels: red (R), green (G), and blue (B). The YUV space contains a brightness channel (Y) and two chrominance channels (U and V). This color space transformation is an image processing technique that aims to better separate brightness information and chrominance information so that they can be operated more efficiently at different processing stages.

[0038] It should be noted that the specific conversion method is:

[0039] Y=0.299×R+0.587×G+0.114×B

[0040] U=-0.1687×R-0.3313×G+0.5×B

[0041] V=0.5×R-0.4187×G-0.0813×B

[0042] Among them, R, G, and B represent the three components of the low dynamic range image in the RGB space, and Y and UV represent the brightness layer image and the chrominance layer image of the low dynamic range image in the YUV space.

[0043] Reference Manual Attached Figure 3 , showing an image detail layer obtained by detail layer separation provided by the present invention.

[0044] Depend on Figure 3 It can be seen that clearly visible surface textures can be obtained by separating the detail layers, and the same effect can be achieved even on highly reflective surfaces. This method of separating the detail layer from the brightness layer of the low dynamic range image in advance can avoid the problem of detail layer loss caused by fusion after multiple exposures. The separated detail layer can be selectively fused with the high dynamic range image obtained by subsequent fusion to obtain the desired display effect of the highly reflective layer surface.

[0045] S103: Perform a guided filtering algorithm on the brightness layer image to obtain a basic layer image of the low dynamic range image, and take the difference between the brightness layer image and the basic layer image as a detail layer image of the low dynamic range image.

[0046] Among them, the guided filtering algorithm is a method for locally smoothing an image, in which another image is used as a guiding graph to guide the smoothing process. The lightness layer image is input into the guided filtering algorithm, which uses the guiding graph, local window and filter coefficients for calculation and outputs the basic layer image, which represents the basic structure of the low dynamic range image after the guided filtering algorithm. Through the guided filtering algorithm, combined with the ridge regression of the regularization term, the basic layer and detail layer are extracted from the lightness layer image for subsequent processing steps. This method helps to better control the structure and details of the image and improve the overall image quality.

[0047] In a possible implementation manner, S103 specifically includes:

[0048] S1031: Input the brightness layer image into the guided filtering algorithm, and output the base layer image of the low dynamic range image:

[0049]

[0050] Among them, I i represents the guided graph corresponding to the guided filtering algorithm, q i represents the base layer image, i represents the pixel index, k represents the midpoint of the localized window, ω k The window corresponding to the midpoint of the localized window, (a k ,b k ) represents the coefficients of the guided filtering algorithm.

[0051] In one possible implementation, the coefficients of the guided filtering algorithm are solved by a ridge regression equation with a regularization term;

[0052] Ridge regression equation E(a k ,b k ) specifically:

[0053]

[0054] Among them, γ is used to prevent a k The coefficient is too large for the regularization parameter, μ k Indicates that the guided graph is in window ω k The mean of Indicates that the guided graph is in window ω k The variance of |ω| represents the window ω k The number of pixels within Indicates that in window ω k The mean value of the brightness layer image within;

[0055] Take the mean of the coefficients of the guided filtering algorithm under multiple windows to replace the base layer:

[0056]

[0057] in, It represents the mean value of the coefficients of the guided filtering algorithm under multiple windows, i represents the pixel index, and k represents the midpoint of the localized window.

[0058] S1032: Take the difference between the brightness layer image and the base layer image as the detail layer image T of the low dynamic range image i :

[0059] T i =p i -q i

[0060] Among them, p i Represents the brightness layer image, q i Represents the base layer image.

[0061] Specifically, a guided filtering algorithm is used to generate a base layer image in a step-by-step manner, and then calculate the detail layer image. The guided filtering algorithm can better preserve the structural features of the image by considering the local structural information, so that the generated base layer image is closer to the overall structure of the original image. The base layer usually contains the global and large-scale structural information of the image, but does not contain smaller details, which helps to take into account the overall brightness changes of the image, while leaving the smaller details to the detail layer. Through the ridge regression equation with regularization terms, the coefficients of the guided filtering algorithm are adaptively optimized in each window, which can better adapt to the changes in the local structure of the image and make the guided filtering more adaptive. The regularization parameter can control the coefficients of the guided filtering algorithm to prevent it from being too large, which helps to avoid over-reliance on the guided graph and cause unstable filtering effects.

[0062] The difference between the brightness layer image and the base layer image is used to obtain the detail layer image, which represents the detail information in the image. In this way, the detail layer image can more accurately capture the local changes and detail features in the image. This difference operation can flexibly extract details based on the image content without being affected by global brightness changes, which is very helpful for adapting to the dynamic range changes of various scenes.

[0063] In summary, by adopting the guided filtering algorithm to gradually generate the basic layer and detail layer, it helps to better preserve the overall structure of the image, improve the adaptability of the filter, and effectively capture the detail information in the image, laying the foundation for subsequent high dynamic range image reconstruction.

[0064] Reference Manual Attached Figure 4 , showing a flowchart of deep learning provided by the present invention.

[0065] Figure 4The figure shows the process of using a neural network to adjust the exposure of a single LDR image, progressively generating LDR images with different exposure levels through a cascaded network model, and then improving the dynamic range through multi-exposure fusion. Figure 4 The process includes learning and reasoning stages. In the learning stage, the camera response function selected in the camera response function dataset is used to process the images in the high dynamic range image dataset to obtain an equal exposure image sequence, and the exposure change of the image sequence is learned using a convolutional neural network. In the reasoning stage, the input low dynamic range image is inferred, and an equal exposure image sequence is obtained through the upper exposure and lower exposure models. Finally, a high dynamic range image is obtained through image fusion.

[0066] S104: Generate a first low dynamic range image set and a second low dynamic range image set in a progressive manner using a cascade structured neural network model.

[0067] The first low dynamic range image set includes a plurality of brightness layer images at different exposure levels, and the second low dynamic range image set includes a plurality of detail layer images at different exposure levels.

[0068] It should be noted that through deep learning methods, using a neural network model, two sets of low dynamic range images with different exposure levels are gradually generated from the input brightness layer image and detail layer image. This progressive approach can help the model better capture the brightness and detail information of the scene, thereby generating low dynamic range images more effectively.

[0069] S105: Perform multi-exposure fusion based on Gaussian decomposition and Laplace decomposition on the first low dynamic range image set and the second low dynamic range image set respectively to obtain a brightness layer high dynamic range image and a detail layer high dynamic range image.

[0070] Among them, high dynamic range images, also known as HDR (High Dynamic Range) images, are images that can capture and display a larger dynamic range. Dynamic range refers to the brightness difference between the brightest and darkest areas in an image. HDR images provide a richer and more realistic visual experience by capturing a wider range of brightness during the shooting or generation process and displaying this information on the display device. HDR images are closer to the human eye's perception of changes in lighting, making the image more realistic and better reflecting the brightness changes in real scenes. By capturing a wider range of brightness, HDR images are able to present richer and deeper colors and retain more details, especially in shadows and highlights.

[0071] It should be noted that the multi-exposure fusion of the first low dynamic range image set and the second low dynamic range image set finally obtains two important outputs: the brightness layer high dynamic range image and the detail layer high dynamic range image. This process includes Gaussian decomposition and Laplace decomposition, and the pyramid structure is used for image processing. Gaussian decomposition is a method for constructing an image pyramid. Images of different scales are obtained by downsampling the image step by step. The specific process is: Gaussian decomposition is performed on the first low dynamic range image set and the second low dynamic range image set, which involves blurring the image and then reducing the resolution of the image to obtain a series of image pyramid hierarchical structures. Laplace decomposition is an enhancement method for the image pyramid, which is used to capture the detailed information of the image at different scales. The specific process is: for each pyramid level, the image is Gaussian blurred and the blurred image is subtracted to obtain a Laplace pyramid. The Laplace pyramid contains the detailed information of the image at different scales, especially the edge and texture. The purpose of multi-exposure fusion is to fuse the first low dynamic range image set and the second low dynamic range image set at each pyramid level. At each pyramid level, the two image sets of the corresponding level are fused, which involves weighted averaging or other fusion techniques for each pixel. The result of this step is two pyramid structures: one is the brightness layer high dynamic range pyramid, and the other is the detail layer high dynamic range pyramid.

[0072] Through Gaussian decomposition and Laplace decomposition, image sets with different exposure levels are fused into two pyramid structures with high dynamic range, one for the brightness layer and the other for the detail layer, which provides richer and more detailed information for subsequent steps and helps to construct the final high dynamic range image.

[0073] In a possible implementation manner, S105 specifically includes:

[0074] S1051: Determine the number of pyramid decomposition levels n l :

[0075]

[0076] Among them, min(x,y) represents the minimum value of the pixel row and column values ​​of the pyramid.

[0077] S1052: respectively determine the contrast C of the images in the first low dynamic range image set or the second low dynamic range image set according to the contrast C of the images in the first low dynamic range image set or the second low dynamic range image set. n (x,y), saturation S n (x,y) and exposure E n (x,y) is used as the evaluation factor to establish the weight graph W n (x,y):

[0078] Wn (x,y)=C n (x,y)×S n (x,y)×E n (x,y)

[0079] C n (x,y)=|f n (x,y)×H l |

[0080]

[0081] Where n represents the image number, H l represents the Laplacian operator matrix, (x, y) represents the pixel coordinates, f n (x,y) represents the pixel value of the coordinate, R n (x,y),G n (x,y),B n (x, y) represents the pixel value of the coordinate (x, y) of the nth image in the R channel, G channel, and B channel respectively. n (x, y) represents the intermediate variable, σ=0.2 represents the constant, and Respectively represent the exposure of the coordinate (x, y) of the nth image in the R channel, G channel, and B channel.

[0082] S1053: Normalize the weight map to obtain a normalized weight map W n (x,y):

[0083]

[0084] Where N is the number of images, ε = 10 -12 A small non-zero constant to avoid the denominator being zero.

[0085] S1054: Performing Gaussian pyramid decomposition on the normalized weight map according to the number of pyramid decomposition layers to obtain a Gaussian pyramid.

[0086] In a possible implementation manner, S1054 specifically includes:

[0087] S1054A: Set the normalized weight map as the image to be processed;

[0088] S1054B: Perform convolution based on Gaussian kernel on the image to be processed;

[0089] S1054C: Downsample the image to be processed after convolution based on size halving, use the downsampled result as the current layer, and add the current layer to the Gaussian pyramid in sequence:

[0090]

[0091] Among them, x n Indicates the image size in the current layer that is an odd number, x n+1 Represents the size corresponding to the image size that is odd in the current layer;

[0092] S1054D: Use the current layer of the Gaussian pyramid as the image to be processed, and return to S1053B until the number of layers of the Gaussian pyramid reaches the number of layers of the pyramid decomposition, thereby obtaining a Gaussian pyramid.

[0093] It should be noted that the generation process of the Gaussian pyramid mainly consists of the following two steps: Gaussian smoothing: Gaussian smoothing is mainly achieved by convolution of the image. Using the Gaussian kernel for convolution is equivalent to low-pass filtering the image to eliminate high-frequency information. Downsampling: The image size is reduced by half to obtain the next layer of pyramid image. If the width or height of a layer of image is an odd number, the following operation is performed:

[0094]

[0095] Among them, x n The width or height of the current layer image is an odd number. n+1 is the width or height of the image of the next layer. The Gaussian smoothing and downsampling operations are repeated in this way to obtain the Gaussian pyramid image of each layer.

[0096] S1055: Perform Laplace decomposition on the Gaussian pyramid to obtain a Laplace pyramid.

[0097] In a possible implementation manner, S1055 specifically includes:

[0098] S1055A: Upsample each layer of the Gaussian pyramid to obtain multiple upsampled images.

[0099] S1055B: Fill a 1-pixel border for the upsampled image using the nearest neighbor algorithm.

[0100] S1055C: Double the size of the padded upsampled image.

[0101] Specifically, the specific method of expanding the size of the image with a border to twice the original size is to fill a row (column) of pixels with 0 pixels under each row (column) of image pixels, and then expand the original pixel value to 4 times the original. The border is changed from 1 pixel to 2 pixels.

[0102] S1055D: Convolve the dilated upsampled image with a Gaussian kernel.

[0103] S1055E: Remove borders from upsampled images after convolution.

[0104] S1055F: Calculate the difference between the upsampled image after the border is removed and the corresponding original image of the upsampled image to obtain each layer of the Laplacian pyramid, wherein the last layer of the Laplacian pyramid is the same as the last layer of the Gaussian pyramid.

[0105] It should be noted that the Laplacian decomposition of the original image is performed, and the Laplacian pyramid is generated on the basis of the Gaussian pyramid. There are mainly two steps: 1. Upsampling: In order to obtain the current layer of Laplacian pyramid, the Gaussian pyramid of the next layer must be upsampled first. The upsampling process specifically includes: using the nearest neighbor method to fill a 1-pixel border for the image. The size of the image with the border is enlarged to twice the original size. The specific method is to fill a row (column) of pixels with 0 pixels under each row (column) of image pixels, and then enlarge the original pixel value to 4 times the original, and the border changes from 1 pixel to 2 pixels. Then use the Gaussian kernel for convolution to smooth the image. Finally, remove the 2-pixel border. If there is an odd number in the width (height) size of the Gaussian pyramid image of this layer, remove 3 pixels from the right (lower) border to keep it consistent with the size of the Gaussian pyramid image. 2. Calculate the difference. Subtract the Gaussian pyramid image of the current layer from the upsampled image of the Gaussian pyramid of the next layer to obtain the Laplacian pyramid image of the current layer. The last layer of the Laplacian pyramid is the same as the Gaussian pyramid. The upsampling and difference operations are performed in this way in a cycle to obtain the Laplacian pyramid image of each layer.

[0106] S1056: Perform a product operation on the Gaussian pyramid and the Laplacian pyramid according to the spatial positions corresponding to the pixels, and sum the product operation results to obtain a fused pyramid.

[0107] S1057: Perform inverse Laplace transform on the fused pyramid to obtain a corresponding brightness layer high dynamic range image and a detail layer high dynamic range image.

[0108] In a possible implementation manner, S1057 specifically includes:

[0109] S1057A: up-sample the last layer of the fusion pyramid, where the last layer of the fusion pyramid is a top layer of the fusion pyramid;

[0110] S1057B: Add the upsampling result to the previous layer of the last layer as the target layer;

[0111] S1057C: Determine whether the target layer is the first layer of the fusion pyramid. If so, use the image corresponding to the target layer as the brightness layer high dynamic range image or the detail layer high dynamic range image. Otherwise, use the target layer as the last layer and return to S1057A.

[0112] It should be noted that the fused pyramid can be obtained by multiplying the results of Gaussian pyramid decomposition and Laplace pyramid decomposition according to the spatial positions corresponding to the pixels and summing them. The fused pyramid is subjected to inverse Laplace transform to reconstruct the image, so that the final HDR image with good light and dark details to be output is obtained. Unlike Gaussian decomposition and Laplace decomposition, inverse Laplace transform starts from the top of the pyramid, that is, the last layer of the pyramid, and mainly has the following two steps: 1. Upsampling: Upsample the next layer of the current layer reconstruction pyramid, and the last layer of the reconstruction pyramid is the same as the fused pyramid. 2. Summation: Add the upsampled image to the current layer fusion pyramid image to obtain the current layer reconstruction pyramid. The upsampling and summing operations are repeated until the first layer is reached, and the first layer is the HDR image.

[0113] Specifically, first, the number of decomposition layers of the pyramid is determined by S1051 to ensure that the pyramid structure is detailed enough. Then, a weight map is established according to evaluation factors such as contrast, saturation and exposure of the image by S1052, which includes a series of image processing operations, such as convolution and downsampling, to obtain the weight information of the image. S1053 normalizes the weight map to ensure the balance of the influence of each evaluation factor. Then, the Gaussian pyramid is decomposed by S1054 using the Gaussian pyramid. On this basis, Laplace decomposition is performed by S1055 to obtain the Laplace pyramid. This step includes operations such as upsampling, convolution, border removal and difference, and finally forms each layer of the Laplace pyramid. Next, the Gaussian pyramid and the Laplace pyramid are multiplied according to the spatial position corresponding to the pixel by S1056, and the product results are summed to obtain a fused pyramid. Finally, the fused pyramid is subjected to an inverse Laplace transform by S1057 to reconstruct the brightness layer and detail layer of the HDR image. This process involves upsampling, addition and iteration operations, and finally obtains an image with a high dynamic range.

[0114] The advantage of this method is that it makes full use of deep learning and image processing through steps such as pyramid structure, weight map establishment and Laplace decomposition, which can better extract and retain image detail information and generate high-quality HDR images. At the same time, the use of pyramid structure and multi-level processing makes the whole process more robust and efficient.

[0115] S106: Fusing the brightness layer high dynamic range image and the detail layer high dynamic range image to obtain a new brightness layer high dynamic range image, and combining the new brightness layer high dynamic range image with the image corresponding to the chrominance layer to obtain a high dynamic range fused image.

[0116] It should be noted that by fusing the luminance layer and the detail layer, the detail information of the image is fully retained while keeping the overall brightness consistent, so as to obtain a new luminance layer image with a high dynamic range. Next, the new luminance layer high dynamic range image is combined with the image corresponding to the chrominance layer to obtain the final high dynamic range fused image. The purpose of this step is to comprehensively consider the brightness and color information of the image and generate a complete high dynamic range image to show richer visual effects.

[0117] In a possible implementation, the brightness layer high dynamic range image and the detail layer high dynamic range image are fused, specifically as follows:

[0118]

[0119] Among them, L(x,y) represents the new brightness layer high dynamic range image, I(x,y) represents the brightness layer high dynamic range image, and I d (x, y) represents the detail layer high dynamic range image, L H High dynamic range image representing the brightness layer and detail layer, l min Represents the minimum brightness value of the high dynamic range image of the brightness layer and the detail layer. α=0.5, β=1.5, γ=0.8, δ=0.2 are all fusion parameters.

[0120] It should be noted that in the fusion process, when the pixel in the HDR image is less than β times the minimum brightness value, the final brightness of the image does not fuse the detail layer information to reduce the impact of noise, otherwise it is weighted addition. This condition in the fusion process shows that for pixels with lower brightness in the HDR image, if its brightness is less than β times the minimum brightness value, the final brightness layer does not fuse the information of the detail layer, but directly uses the information of the brightness layer to reduce the impact of noise. If the brightness exceeds β times the minimum brightness value, then weighted addition is performed. The purpose of this mechanism is to deal with the noise problem that may exist in low brightness conditions. In low brightness areas, due to the low signal-to-noise ratio, the information of the detail layer may be interfered by noise, and direct fusion may introduce unnecessary noise. Therefore, by using the minimum brightness value and a multiple β, it is possible to control the fusion of the detail layer in the low brightness area, thereby maintaining the image quality. This design takes into account the characteristics of the image in different brightness areas, and through conditional judgment and adjustment of weights, the process of high dynamic range image fusion is more intelligent and adaptable to the needs of different scenes.

[0121] S107: Convert the high dynamic range fused image from the YUV space to the RGB space.

[0122] It should be noted that the specific conversion method is:

[0123] R=Y+1.402V

[0124] G=Y-0.3441U-0.7141V

[0125] B=Y+1.772U

[0126] The meanings of the variables are the same as those of the variables in the conversion method from RGB space to YUV space.

[0127] Reference Manual Attached Figure 5 , showing a target high dynamic range image obtained by dynamic range enhancement based on deep learning provided by the present invention.

[0128] Depend on Figure 5 It can be seen that the obtained target high dynamic range image shows tiny details clearly, avoiding the loss of image details of highly reflective surfaces caused by traditional overexposure or underexposure, and improving the detection effect of tiny defects. It eliminates image overexposure, enriches the light and dark details of the image, shortens the running time of the detection algorithm, and achieves more accurate and faster identification of tiny defects.

[0129] S108: De-noising the high dynamic range fused image converted to the RGB space to obtain a target high dynamic range image.

[0130] In a possible implementation manner, S108 specifically includes:

[0131] S1081: Denoise the high dynamic range fused image converted to RGB space using the Gaussian kernel function:

[0132]

[0133] Wherein, G(x,y) represents the Gaussian weight, (x,y) represents the pixel coordinates of the high dynamic range fusion image converted to the RGB space, and σ=0.5 represents the standard deviation of the Gaussian kernel function.

[0134] It should be noted that by applying the Gaussian kernel function to denoise the high dynamic range fused image converted to the RGB space, and using the convolution operation to smooth the image, it is helpful to reduce the noise components in the image and improve the image quality. By adjusting the standard deviation of the Gaussian kernel, the degree of denoising can be controlled to make the image clearer and smoother. Such denoising helps to produce more realistic, detailed, and easier to observe and analyze high dynamic range images, providing better input for visual and analytical tasks.

[0135] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0136] In the present invention, a detail layer image is separated from a brightness layer image of a low dynamic range image through a guided filtering algorithm, and a neural network model with a cascade structure is introduced to generate a low dynamic range image set of the collected low dynamic range image at different exposure levels. Then, the generated image set is subjected to multi-exposure fusion based on Gaussian decomposition and Laplace decomposition to generate a corresponding high dynamic range image. While shortening the algorithm running time, the image details are retained, and the problem of small defects being difficult to identify due to multiple exposures during the image acquisition process is avoided, thereby mining more detailed information of the image, obtaining an image with rich light and dark details and high contrast, solving the problem of local brightness oversaturation on the surface of highly reflective metals, improving the accuracy of defect detection, supporting online detection, improving detection efficiency, reducing labor costs, and promoting the development of automated detection of highly reflective metal products. Specific embodiment 2

[0138] In one embodiment, the present invention provides a high dynamic range image reconstruction system based on deep learning, comprising a processor and a memory for storing processor executable instructions; the processor is configured to call instructions stored in the memory to execute the high dynamic range image reconstruction method based on deep learning in Example 1.

[0139] A high dynamic range image reconstruction system based on deep learning provided by the present invention can realize the steps and effects of the high dynamic range image reconstruction method based on deep learning in the above-mentioned embodiment 1. To avoid repetition, the present invention will not go into details.

[0140] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0141] In the present invention, a detail layer image is separated from a brightness layer image of a low dynamic range image through a guided filtering algorithm, and a neural network model with a cascade structure is introduced to generate a low dynamic range image set of the collected low dynamic range image at different exposure levels. Then, the generated image set is subjected to multi-exposure fusion based on Gaussian decomposition and Laplace decomposition to generate a corresponding high dynamic range image. While shortening the algorithm running time, the image details are retained, and the problem of small defects being difficult to identify due to multiple exposures during the image acquisition process is avoided, thereby mining more detailed information of the image, obtaining an image with rich light and dark details and high contrast, solving the problem of local brightness oversaturation on the surface of highly reflective metals, improving the accuracy of defect detection, supporting online detection, improving detection efficiency, reducing labor costs, and promoting the development of automated detection of highly reflective metal products.

[0142] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A high dynamic range image reconstruction method based on deep learning, characterized in that: include: S101: acquiring a low dynamic range image of a target object in one exposure; S102: Convert the low dynamic range image from the RGB space to the YUV space to obtain a brightness layer image and a chrominance layer image of the low dynamic range image; S103: performing a guided filtering algorithm on the brightness layer image to obtain a base layer image of the low dynamic range image, and taking a difference between the brightness layer image and the base layer image as a detail layer image of the low dynamic range image; S104: using a cascade-structured neural network model to progressively generate a first low dynamic range image set and a second low dynamic range image set, respectively, wherein the first low dynamic range image set includes a plurality of brightness layer images at different exposure levels, and the second low dynamic range image set includes a plurality of detail layer images at different exposure levels; S105: performing multi-exposure fusion based on Gaussian decomposition and Laplace decomposition on the first low dynamic range image set and the second low dynamic range image set respectively to obtain a brightness layer high dynamic range image and a detail layer high dynamic range image; S106: Fusing the brightness layer high dynamic range image and the detail layer high dynamic range image to obtain a new brightness layer high dynamic range image, and combining the new brightness layer high dynamic range image with an image corresponding to the chroma layer to obtain a high dynamic range fused image; S107: Convert the high dynamic range fused image from the YUV space to the RGB space; S108: De-noising the high dynamic range fused image converted to the RGB space to obtain a target high dynamic range image.

2. The high dynamic range image reconstruction method based on deep learning according to claim 1, characterized in that: The S103 specifically includes: S1031: Input the brightness layer image into the guided filtering algorithm, and output the base layer image of the low dynamic range image: Among them, I i represents the guided graph corresponding to the guided filtering algorithm, q i represents the base layer image, i represents the pixel index, k represents the midpoint of the localized window, ω k The window corresponding to the midpoint of the localized window, (a k ,b k ) represents the coefficient of the guided filtering algorithm; S1032: Take the difference between the brightness layer image and the base layer image as the detail layer image T of the low dynamic range image i : T i =p i -q i Among them, p i represents the brightness layer image, q i Represents the base layer image.

3. The high dynamic range image reconstruction method based on deep learning according to claim 2, characterized in that: Solving the coefficients of the guided filtering algorithm by a ridge regression equation with a regularization term; The ridge regression equation E(a k ,b k ) specifically: Among them, γ is used to prevent a k The coefficient is too large for the regularization parameter, μ k Indicates that the guide graph is in window ω k The mean of Indicates that the guide graph is in window ω k The variance of |ω| represents the window ω k The number of pixels within Indicates that in window ω k The mean value of the brightness layer image within; The basic layer is replaced by taking the average of the coefficients of the guided filtering algorithm under multiple windows: in, It represents the mean of the coefficients of the guided filtering algorithm under multiple windows, i represents the pixel index, and k represents the midpoint of the localized window.

4. The high dynamic range image reconstruction method based on deep learning according to claim 1, characterized in that: The S105 specifically includes: S1051: Determine the number of pyramid decomposition levels n l : Wherein, min(x, y) represents the minimum value of the pixel row and column values ​​of the pyramid; S1052: respectively determine the contrast C of an image in the first low dynamic range image set or the second low dynamic range image set according to the contrast C of an image in the first low dynamic range image set or the second low dynamic range image set. n (x,y), saturation S n (x,y) and exposure E n (x,y) is used as the evaluation factor to establish the weight graph W n (x,y): W n (x,y)=C n (x,y)×S n (x,y)×E n (x,y) C n (x,y)=|f n (x,y)×H l | Where n represents the image number, H l represents the Laplacian operator matrix, (x, y) represents the pixel coordinates, f n (x,y) represents the pixel value of the coordinate, R n (x,y),G n (x,y),B n (x, y) represents the pixel value of the coordinate (x, y) of the nth image in the R channel, G channel, and B channel respectively. n (x, y) represents the intermediate variable, σ=0.2 represents the constant, and Respectively represent the exposure of the coordinates (x, y) of the nth image in the R channel, G channel, and B channel; S1053: Normalize the weight map to obtain a normalized weight map W n (x,y): Where N is the number of images in the image sequence, ε = 10 -12 A small non-zero constant to avoid the denominator being zero; S1054: performing Gaussian pyramid decomposition on the normalized weight map according to the number of pyramid decomposition layers to obtain a Gaussian pyramid; S1055: performing Laplace decomposition on the Gaussian pyramid to obtain a Laplace pyramid; S1056: performing a product operation on the Gaussian pyramid and the Laplacian pyramid according to the spatial positions corresponding to the pixels, and summing the product operation results to obtain a fused pyramid; S1057: Perform inverse Laplace transform on the fused pyramid to obtain the corresponding brightness layer high dynamic range image and the detail layer high dynamic range image.

5. The high dynamic range image reconstruction method based on deep learning according to claim 4, characterized in that: The S1054 specifically includes: S1054A: Setting the normalized weight map as the image to be processed; S1054B: performing convolution based on a Gaussian kernel on the image to be processed; S1054C: Downsampling the image to be processed obtained after the convolution based on size halving, using the downsampling result as the current layer, and adding the current layer to the Gaussian pyramid in sequence: Among them, x n Indicates the image size of the current layer that is an odd number, x n+1 represents the size corresponding to the image size that is an odd number in the current layer; S1054D: Use the current layer of the Gaussian pyramid as the image to be processed, and return to S1053B until the number of layers of the Gaussian pyramid reaches the number of layers of the pyramid decomposition, thereby obtaining the Gaussian pyramid.

6. The high dynamic range image reconstruction method based on deep learning according to claim 4, characterized in that: The S1055 specifically includes: S1055A: Upsampling each layer of the Gaussian pyramid to obtain multiple upsampled images; S1055B: Fill a 1-pixel border for the upsampled image using a nearest neighbor algorithm; S1055C: Double the size of the padded upsampled image; S1055D: Convolve the enlarged upsampled image with a Gaussian kernel; S1055E: Remove borders from upsampled images after convolution; S1055F: Calculate the difference between the up-sampled image after the border is removed and the corresponding original image of the up-sampled image to obtain each layer of the Laplacian pyramid, wherein the last layer of the Laplacian pyramid is the same as the last layer of the Gaussian pyramid.

7. The high dynamic range image reconstruction method based on deep learning according to claim 4, characterized in that: The S1057 specifically includes: S1057A: Up-sampling the last layer of the fusion pyramid, wherein the last layer of the fusion pyramid is a top layer of the fusion pyramid; S1057B: Add the upsampling result to the previous layer of the last layer as the target layer; S1057C: Determine whether the target layer is the first layer of the fusion pyramid. If so, use the image corresponding to the target layer as the brightness layer high dynamic range image or the detail layer high dynamic range image. Otherwise, use the target layer as the last layer and return to S1057A.

8. The high dynamic range image reconstruction method based on deep learning according to claim 1, characterized in that: The brightness layer high dynamic range image and the detail layer high dynamic range image are merged, specifically: Wherein, L(x, y) represents the new brightness layer high dynamic range image, I(x, y) represents the brightness layer high dynamic range image, and I d (x, y) represents the detail layer high dynamic range image, L H Represents the high dynamic range image of the brightness layer and the detail layer, l min It represents the minimum brightness value of the high dynamic range image of the brightness layer and the detail layer, and α=0.5, β=1.5, γ=0.8, and δ=0.2 are all fusion parameters.

9. The high dynamic range image reconstruction method based on deep learning according to claim 8, characterized in that: The S108 is specifically: S1081: De-noising the high dynamic range fused image converted to the RGB space using a Gaussian kernel function: Wherein, G(x, y) represents the Gaussian weight, (x, y) represents the pixel coordinates of the high dynamic range fused image converted to the RGB space, and σ=0.5 represents the standard deviation of the Gaussian kernel function.

10. A high dynamic range image reconstruction system based on deep learning, characterized in that: It comprises a processor and a memory for storing processor executable instructions; the processor is configured to call the instructions stored in the memory to execute the high dynamic range image reconstruction method based on deep learning as described in any one of claims 1 to 9.