A method and system for image acquisition of a dim-light high-temperature industrial endoscope

Abstract

The present invention discloses a method and system for capturing images of a dim-light high-temperature industrial endoscope. By acquiring digital images inside an industrial furnace and forming an original video stream, performing three-dimensional comb filtering on the original video stream, performing high-brightness video synthesis on the video stream after three-dimensional comb filtering to obtain a high-brightness video stream, and performing white balance processing on the image frames in the high-brightness video stream, high-quality images inside the furnace are obtained, solving the technical problems of insufficient imaging brightness, blurred details, and unclear contours of the images inside the industrial furnace under extremely low-light conditions. Moreover, the designed fidelity function, consistency function, and regularization function utilize the strong correlation between the optimized image and the original image, achieving the purpose of eliminating ghosting in the enhanced image while ensuring the enhancement of the image brightness, and effectively improving the image quality of the enhanced image.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of high-temperature industrial endoscopes, and particularly refers to a method and system for capturing images of a dim-light high-temperature industrial endoscope. Background Art

[0002] Industrial furnaces are important industrial production equipment and are widely used in fields such as building materials, metallurgy, and chemical engineering. Their operation status and efficiency have a great impact on products and the environment. To ensure the stable and efficient operation of the furnace, monitoring it is an essential step. Among them, imaging the interior of the furnace through a high-temperature industrial endoscope is an important means of furnace monitoring. However, due to the complex environment and poor lighting conditions inside the furnace, it is difficult for traditional imaging methods to capture satisfactory furnace images, and it is impossible to obtain effective images to guide operators, making it difficult to apply to the production monitoring of furnaces. Therefore, an industrial endoscope image acquisition method that can capture images inside the furnace under extremely weak light (illuminance below 0.0001 Lux) conditions is designed to provide high-quality furnace operation images for production operators to guide production, make timely adjustments according to the furnace conditions, ensure the stable operation of the furnace, and improve production efficiency.

[0003] Currently, industrial endoscopes can be mainly divided into non-light source types and light source types.

[0004] Non-light source industrial endoscopes directly use the lighting conditions inside the furnace for imaging without additional light source supplementation. Without other processing, this type of industrial endoscope cannot be applied to reliable imaging inside furnaces in extremely weak light environments. For the processing of the imaging method of non-light source endoscopes, there is processing from the imaging principle, which mainly obtains infrared images inside the furnace through infrared imaging to reflect the information inside the furnace; there is also post-processing after imaging, which mainly enhances the image through algorithms to improve the image quality in low illuminance environments.

[0005] Light source type endoscopes use the method of installing a light source at the front end to supplement the light source for the working environment to improve the imaging quality of the endoscope. However, due to the complex environment inside the furnace, simply using a set of LED lights at the front end for light supplementation will cause problems such as light dispersion and serious light loss, and cannot provide sufficient brightness for imaging inside the furnace, having little effect on improving the imaging quality. Moreover, in the high-temperature and highly corrosive environment inside the furnace, the lifespan of the LED will also be greatly affected. Therefore, this type of imaging device is not suitable for imaging inside the furnace.

[0006] The patent with the publication number CN109031646B is an industrial endoscope that simultaneously uses infrared light and visible light to obtain image information inside a furnace. This patent divides the light inside the furnace captured by the imaging lens into infrared light and visible light through a beam splitter, and sends them to the corresponding imaging chips through imaging tubes respectively to obtain digital images of infrared and visible light, and then transmits them to the host computer. By using this method, the influence of the harsh environment of high temperature and high dust inside the furnace can be overcome, the image information inside the furnace can be obtained, and the infrared image information inside the furnace can also be obtained through the infrared imaging part under low illumination conditions to guide the operators. Moreover, this method can also obtain the temperature information inside the furnace through the infrared image. These obtained image information can be used by the host computer to obtain and utilize the depth of field information to realize the online monitoring function of three-dimensional reconstruction. However, due to the lack of detailed information such as the contours and textures of the infrared images, the information provided by the infrared images obtained by this method is not sufficient in an extremely low light environment, and the reliability will decrease.

[0007] The patent with the publication number CN105913404A proposes a low-light imaging method based on frame accumulation. This method processes multiple frames of images obtained from the same scene to improve the signal-to-noise ratio and recognition rate of the scene image and increase the image quality under low illumination conditions. The specific method is as follows: First, preprocess the image, specifically including white balance processing, demosaicing, color correction, and converting to RGB format image; after the preprocessing, perform Surf feature point matching on the image to register the image; then perform weighted accumulation on the obtained multiple frames of images, and the weight values of each image are the same to ensure that the processed image will not be overexposed; finally, perform Gamma correction on the obtained image to obtain the corrected image, which is the image finally obtained by this method. The image obtained by this method will be improved in terms of signal-to-noise ratio and brightness, and the clarity can also be improved. However, if the illumination is extremely low (such as in a dim light environment, the illumination is lower than 0.0001 Lux), it is also difficult to obtain an image with sufficient brightness through this method, and the superimposition of multiple frames of images may cause ghosting in the enhanced image, reducing the quality of the enhanced image.

[0008] The patent with the publication number CN203838405U proposes an industrial endoscope probe with illumination compensation. This imaging device has a ring of LED lights around the imaging camera to increase the light in the observed environment, improve the intensity of the reflected light in the environment, increase the reflected light entering the camera, thereby enhancing the imaging brightness and improving the imaging quality. The volume of this device is not large, and a battery is used to power the LED light group. These two points also determine that the number of LED lights that can be added to this device is not large. Therefore, the amount of light compensation provided by the LED light group in the furnace is relatively limited. Moreover, due to the high-dust environment in the furnace, the light provided by the LED light group will be scattered by the dust, making it difficult for the light to compensate the part of interest, and the effect is very limited. And because a battery is used to power the LED light group, the battery life of the LED lights will not be very long, which is also not suitable for long-term video shooting. Also, due to the high temperature and complex environment in the furnace, it is not conducive to the operation of the LED lights and the battery, and may pose a danger. Therefore, this imaging method is also not suitable for imaging in a high-temperature and low-light furnace. Summary of the Invention

[0009] The imaging method and system for a low-light and high-temperature industrial endoscope provided by the present invention solve the technical problems of insufficient imaging brightness, blurred details, and unclear contours of the image inside the industrial furnace under extremely low-light conditions.

[0010] To solve the above technical problems, the imaging method for a low-light and high-temperature industrial endoscope proposed by the present invention includes:

[0011] Obtain a digital image inside the industrial furnace and form an original video stream;

[0012] Perform three-dimensional comb filtering on the original video stream;

[0013] Perform high-brightness video synthesis on the video stream after three-dimensional comb filtering to obtain a high-brightness video stream;

[0014] Perform white balance processing on the image frames in the high-brightness video stream to obtain a high-quality image inside the furnace.

[0015] Further, performing high-brightness video synthesis on the video stream after three-dimensional comb filtering to obtain a high-brightness video stream includes:

[0016] Collect video frames in the video stream after three-dimensional comb filtering. The video frames include the current video frame and the video frames adjacent to the current video frame;

[0017] Perform non-linear transformation on the video frames;

[0018] Overlay the video frames after non-linear transformation to obtain an enhanced image;

[0019] The maximum a posteriori model is used to optimize the enhanced image to obtain a synthesized video frame, thereby obtaining a highlighted video stream.

[0020] Furthermore, the calculation formula for the non-linear transformation of the video frame is:

[0021]

[0022] where P y and P x represent the gray value after transformation and the gray value before transformation respectively, and n is used to control the overall improvement degree of brightness, and n = 2.

[0023] Furthermore, using the maximum a posteriori model to optimize the enhanced image to obtain a synthesized video frame includes:

[0024] Calculating the fidelity function, where the calculation formula for the fidelity function is:

[0025]

[0026] where ψ(O t , Y j ) is the fidelity function, used to measure the similarity between the optimized image O t at time t and the enhanced image Y j at time j, is the variance of the joint distribution of O t and Y j , B and D represent the blurring matrix and the downsampling matrix respectively, S represents the covariance matrix, represents the motion compensation matrix from the optimized image O t at time t to the enhanced image Y j at time j;

[0027] Calculating the consistency function, where the calculation formula for the consistency function is:

[0028]

[0029] where, represents the output optimized image at time t-1, represents the consistency function, measuring the similarity between the optimized image O t at time t and the output optimized image at time t-1 , is the variance of the joint distribution of O t and , is the motion compensation matrix from the optimized image O t at time t to the output optimized image at time t-1 ;

[0030] Calculate the regularization function, where the calculation formula of the regularization function is:

[0031]

[0032] where L(O t ) is the regularization function, P is the size of the moving window, α is used to constrain the smoothness of the optimized image, and H l and V j represent the operators that move the optimized image O t by l and j pixels in the horizontal and vertical directions respectively at time t;

[0033] According to the fidelity function, the consistency function and the regularization function, obtain the posterior probability of the optimized image O t , and the calculation formula of the posterior probability is:

[0034]

[0035] where p(O t ) is the posterior probability of the optimized image O t , r is the number of video image frames of the original video used to output the optimized image before time t, and b is the number of video image frames of the original video used to output the optimized image after time t;

[0036] According to the posterior probability of the output optimized image, obtain the synthesized video frame, and the specific calculation formula is:

[0037]

[0038] Furthermore, perform white balance processing on the image frames in the highlight video stream to obtain high-quality in-furnace images, including:

[0039] Convert the image frames in the highlight video stream from the RGB space to the HSV space and the YCbCr space respectively to obtain the H channel, the S channel, the V channel, the Y channel, the Cb channel and the Cr channel;

[0040] Divide the image frames into a preset number of regions, and calculate the average absolute deviation of the H channel, the S channel, the Cb channel and the Cr channel of each region respectively;

[0041] According to the average absolute deviation of the H channel, the S channel, the Cb channel and the Cr channel, obtain the regions to be processed that need to be white balance processed;

[0042] Calculate the mean and deviation of the H channel, the S channel, the Cb channel and the Cr channel of the regions to be processed;

[0043] According to the mean and deviation of the H channel, the S channel, the Cb channel and the Cr channel of the regions to be processed, obtain the candidate white points;

[0044] Select a reference white point from the candidate white points, and fuse the reference white point with the luminance information in the HSV color space and the YCbCr color space.

[0045] Further, according to the means and variances of the H channel, S channel, Cb channel, and Cr channel in the area to be processed, the calculation formula for obtaining the candidate white points is:

[0046]

[0047] where H(i,j), S(i,j), Cb(i,j), and Cr(i,j) are the i-th pixel and j-th pixel of the H channel, S channel, Cb channel, and Cr channel respectively, AvrH all , AvrS all , AvrCb all , and AvrCr all are the means of the H channel, S channel, Cb channel, and Cr channel respectively, and DevH all , DevS all , DevCb all , and DevCr all are the variances of the H channel, S channel, Cb channel, and Cr channel respectively.

[0048] Further, the calculation formula for selecting a reference white point from the candidate white points and fusing the reference white point with the luminance information in the HSV color space and the YCbCr color space is:

[0049]

[0050] where img represents the fused image after fusing the reference white point with the luminance information in the HSV color space and the YCbCr color space, imgR, imgG, and imgB represent the channel images corresponding to the R channel, G channel, and B channel respectively, θ represents the multi-color domain fusion degree, V max , and Y max are the maximum value of lightness and the maximum value of luminance respectively, and R avgw , G avgw , B avgw are the average pixel grayscales of the R, G, and B channels respectively.

[0051] Further, after performing white balance processing on the image frames in the highlight video stream to obtain high-quality in-furnace images, it further includes:

[0052] Send the high-quality in-furnace images to the host computer for display.

[0053] The ghost light high-temperature industrial endoscope imaging system provided by the present invention includes:

[0054] A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for capturing images of a dim-light high-temperature industrial endoscope provided by the present invention are implemented.

[0055] Compared with the prior art, the advantages of the present invention are as follows:

[0056] The method and system for capturing images of a dim-light high-temperature industrial endoscope provided by the present invention obtain digital images inside an industrial furnace and form an original video stream, perform three-dimensional comb filtering on the original video stream, perform high-brightness video synthesis on the video stream after three-dimensional comb filtering to obtain a high-brightness video stream, and perform white balance processing on the image frames in the high-brightness video stream, thereby obtaining high-quality images inside the furnace. This solves the technical problems of insufficient imaging brightness, blurred details, and unclear contours of images inside an industrial furnace under extremely low-light conditions. Moreover, the designed fidelity function, consistency function, and regularization function utilize the strong correlation between the optimized image and the original image, achieving the purpose of eliminating ghosting in the enhanced image while ensuring the enhancement of the image brightness, and effectively improving the image quality of the enhanced image. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Schematic diagram of the three-dimensional comb filter in Embodiment 2 of the present invention;

[0058] Figure 2 Schematic diagram of the high-brightness video synthesis in Embodiment 2 of the present invention;

[0059] Figure 3 Flowchart of the white balance processing in Embodiment 2 of the present invention;

[0060] Figure 4 Flowchart of the method for capturing images of a dim-light high-temperature industrial endoscope in Embodiment 3 of the present invention;

[0061] Figure 5 Connection diagram of on-site equipment in Embodiment 3 of the present invention;

[0062] Figure 6 Block diagram of the structure of the system for capturing images of a dim-light high-temperature industrial endoscope in Embodiment of the present invention.

[0063] Reference Signs:

[0064] 1. Industrial furnace; 2. Dim-light level industrial endoscope; 3. Video stream processing module; 4. Host computer; 10. Memory; 20. Processor. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0065] To facilitate the understanding of the present invention, the following will describe the present invention more comprehensively and meticulously in conjunction with the accompanying drawings of the specification and preferred embodiments. However, the protection scope of the present invention is not limited to the following specific embodiments.

[0066] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways defined and covered by the claims.

[0067] Embodiment 1

[0068] The method for obtaining an image by a faint-light high-temperature industrial endoscope provided in Embodiment 1 of the present invention includes:

[0069] Step S101: Obtain digital images inside the industrial furnace and form an original video stream;

[0070] Step S102: Perform three-dimensional comb filtering on the original video stream;

[0071] Step S103: Perform high-brightness video synthesis on the video stream after three-dimensional comb filtering to obtain a high-brightness video stream;

[0072] Step S104: Perform white balance processing on the image frames in the high-brightness video stream to obtain a high-quality image inside the furnace.

[0073] The method for obtaining an image by a faint-light high-temperature industrial endoscope provided in the embodiments of the present invention obtains digital images inside the industrial furnace and forms an original video stream, performs three-dimensional comb filtering on the original video stream, performs high-brightness video synthesis on the video stream after three-dimensional comb filtering to obtain a high-brightness video stream, and performs white balance processing on the image frames in the high-brightness video stream, thereby obtaining a high-quality image inside the furnace. This solves the technical problems of insufficient imaging brightness, blurred details, and unclear contours of the images inside the industrial furnace under extremely weak light conditions. Moreover, the designed fidelity function, consistency function, and regularization function utilize the strong correlation between the optimized image and the original image, achieving the purpose of eliminating the ghosting in the enhanced image while ensuring the enhancement of the image brightness, and effectively improving the image quality of the enhanced image.

[0074] Specifically, in this embodiment, the image quality of the video stream is first improved through three-dimensional comb filtering, and then a high-brightness video stream synthesis method based on multi-frame synthesis and maximum a posteriori probability model is designed to perform high-brightness processing on the video stream. Finally, white balance processing is performed on the high-brightness video stream to approach the true color temperature inside the furnace, obtaining a high-quality image inside the furnace, providing a feasible and reliable imaging method for the acquisition of image video stream information inside industrial kilns under extremely weak light environments (illuminance lower than 0.0001 Lux).

[0075] Embodiment 2

[0076] The embodiments of the present invention provide a method for obtaining an image inside a furnace in a faint-light environment, which solves the technical problems of insufficient imaging brightness, blurred details, and unclear contours of the images inside the industrial furnace under extremely weak light conditions.

[0077] To solve these technical problems, the method for imaging in a dim light environment proposed in the embodiments of the present invention includes:

[0078] Obtain a digital image inside the furnace through a dim light level optical imaging endoscope and form a video stream.

[0079] In the original video stream signal, the luminance L and chrominance C are highly mixed, and there are problems such as bright color intermixing, color overlap, and severe clutter interference in the video image;

[0080] Therefore, perform comb filtering on the obtained video stream, use comb filtering to perform arithmetic processing on the video stream, solve the above problems, and at the same time retain the image details to obtain a clearer video image. Since the surface of the material in the furnace changes relatively slowly and the consecutive two frames in the video stream do not change much, three-dimensional comb filtering can be used to obtain better results. The schematic diagram of the three-dimensional comb filter used in this embodiment is as Figure 1 shown, and the specific method is as follows:

[0081] (1) Store the first two frames I t of the video frame I t-1 and I t-2 into the corresponding memories respectively.

[0082] (2) According to the color system adopted by the video, there is a 180° and 360° phase difference between the color components C of the image frame I t and I t-1 and I t-2 . Therefore, as long as they are operated, the luminance and chrominance information can be obtained.

[0083] After the comb filtering process, the image needs to be synthesized with a high-brightness video. The schematic diagram of realizing the high-brightness video synthesis in this embodiment is as Figure 2 shown. The specific steps are as follows:

[0084] (1) The original video image frame I t has a large difference in brightness. To avoid overexposure during the superposition process, perform a non-linear transformation on the image frame of the video to obtain I t '. Increase the gray value of the darker pixels in the original image and suppress the gray value of the brighter part. The following transformation can be used:

[0085]

[0086] P y and P x respectively represent the transformed gray value and the gray value before transformation. n can be used to control the overall increase degree of the brightness and is selected according to the gray distribution of the image. Here, n = 2 is taken.

[0087] (2) Superpose the transformed image. Select I′t-1 , I′ t and I′ t+1 are superimposed to obtain an enhanced image Y t . Since three frames are selected for image brightness enhancement, there may be a ghosting phenomenon in the enhanced image Y t . Therefore, the enhanced image needs to be optimized.

[0088] (3) The superimposed image Y t is optimized using the maximum a posteriori model. The posterior probability is jointly determined by the fidelity function ψ(·,·), the consistency function φ(·,·), and the regularization function L(·).

[0089] (4) The fidelity function ψ(·,·) measures the similarity between the optimized image and the original image and is calculated using the Mahalanobis distance. The calculation formula is

[0090]

[0091] where ψ(O t , Y j ) is the fidelity function used to measure the similarity between the optimized image O t at time t and the enhanced image Y j at time j. is the variance of the joint distribution of O t and Y j . B and D represent the blurring matrix and the downsampling matrix respectively, and S represents the covariance matrix. represents the motion compensation matrix from the optimized image O t at time t to the enhanced image Y j at time j.

[0092] (5) The consistency function φ(·,·) is used to control the temporal consistency of the synthesized frames. Its calculation formula is

[0093]

[0094] where represents the output optimized image at time t - 1, represents the consistency function that measures the similarity between the optimized image O t at time t and the output optimized image at time t - 1. is the variance of the joint distribution of O t and . is the motion compensation matrix from the optimized image O t at time t to the output optimized image at time t - 1.

[0095] (6) The regularization function L(·) is used to suppress the image noise information, and its calculation formula is

[0096]

[0097] where L(O t ) is the regularization function, P is the size of the moving window, α is used to constrain the smoothness of the optimized image, and H l and V j represent the operators that move the optimized image O t by l and j pixels in the horizontal and vertical directions respectively at time t. (7) The posterior probability of the synthesized video frame is:

[0098]

[0099] where p(O t ) is the posterior probability of the optimized image, which provides a direction for image optimization, r is the number of video image frames of the original video before time t used for outputting the optimized image, and b is the number of video image frames of the original video after time t used for outputting the optimized image. The fidelity function, consistency function, and regularization function designed in the embodiments of the present invention utilize the strong correlation between the optimized image and the original image, achieving the purpose of eliminating the ghosting in the enhanced image while ensuring the brightness of the enhanced image, and effectively improving the image quality of the enhanced image.

[0100] (8) The output synthesized image frame is:

[0101]

[0102] Combined with the above functions, the output synthesized image frame can be expressed as:

[0103]

[0104] (9) According to formula (7), the synthesized brightness enhanced image can be obtained by using the least squares method After processing the original video stream, a high-brightness video stream can be obtained.

[0105] There is a color difference in the video image after high-brightness processing. In order to improve the image quality, the multi-color dynamic domain algorithm is used to perform white balance processing on the image to obtain an image closer to the true color temperature in the furnace. The flowchart of realizing white balance processing in this embodiment is as Figure 3 shown, and the specific steps of the white balance algorithm are as follows:

[0106] (1) Convert the image from the RGB space to the HSV space and the YCbCr space respectively, and record the results as H, S, V, Y, Cb, Cr.

[0107] (2) Divide the image into 8 regions to enhance the robustness of the algorithm, and calculate the mean absolute deviations DevH, DevS, DevCb, and DevCr of H, S, Cb, and Cr for each region. The calculation formula is as follows:

[0108]

[0109] where N is the number of pixels in each region.

[0110] (3) Calculate |DevH + DevS| and |DevCb + DevCr| for each region. When one of the indicators in a region is small, it indicates that the color distribution in this region is uniform, which is not conducive to white balance processing, and this region is selected to be ignored.

[0111] (4) Without considering the regions that are not conducive to white balance processing in the previous step, calculate the mean values AvrH all , AvrS all , AvrCb all , AvrCr all and the variances DevH all , DevS all , DevCb all , DevCr all .

[0112] (5) Determine the candidate white point. The candidate white point meets the following requirements.

[0113]

[0114] (6) Select the reference white point. The method for selecting the reference white point is as follows: Arrange the brightness values of the candidate white point pixels from high to low, and respectively select the white points with the top 10% brightness values in the HSV space and the YCbCr space as the reference white points.

[0115] (7) Fuse the reference white point with the brightness information in the HSV space and the YCbCr space to complete the white balance calculation. The calculation formula is as follows.

[0116]

[0117] where V max is the maximum value of all image brightness in the image, Y max is the maximum value of all image luminance in the image, and θ is the multi-color domain fusion degree, and θ = 0.4 can be selected.

[0118] After video white balance processing, it is stored in the memory DDR SDRAM, and at the same time, it is decoded and sent for display through the output processing unit.

[0119] Through the above method, it is possible to obtain the image inside the furnace through an endoscope in an extremely low-light environment. Then, three-dimensional comb filtering is used to improve the image quality of the video stream. Next, the video stream is highlighted by multi-frame superposition to optimize the video stream. Then, white balance processing is performed on the highlighted video stream to approach the true color temperature inside the furnace, obtaining a high-quality image of the inside of the furnace. Finally, the video stream obtained inside the furnace is stored and sent to the host computer for display.

[0120] Embodiment 3

[0121] Refer to Figure 4 , the method for capturing images in a dim light environment proposed in this embodiment includes:

[0122] Step S101: Install the device in the furnace and obtain digital images through an endoscope.

[0123] Specifically, the connection diagram of the on-site equipment adopted in the embodiment of the present invention is as shown in Figure 5 shown, Figure 5 in which, 1 represents an industrial furnace, 2 represents a dim-light industrial endoscope, 3 represents a video stream processing module, and 4 represents a host computer.

[0124] Step S102: Store three consecutive frames in the obtained video stream for three-dimensional comb filtering.

[0125] Step S103: Perform operations on the stored images to obtain a clear video signal without crosstalk, dot noise, hanging points, and with a wider bandwidth for bright color signals.

[0126] Step S104: Perform non-linear image mapping on the image frames of the video to increase the gray value in darker areas and reduce the gray value in brighter areas, obtaining I t '. The transformation can be performed using formula (1) here.

[0127] Step S105: Superimpose the images after gray-scale transformation to obtain an enhanced image Y t .

[0128] Step S106: Optimize the enhanced image by the least squares method according to formula (7), and finally obtain a highlighted video stream.

[0129] Step S107: Perform white balance processing on the synthesized video. First, transform the image frames from the RGB space to the HSV space and the YCbCr space for representation, obtaining H, S, V, Y, Cb, Cr.

[0130] Step S108: Divide the image into 8 regions, and calculate the mean absolute deviations DevH, DevS, DevCb, and DevCr of H, S, Cb, and Cr for each region using formula (8). Calculate the |DevH + DevS| and |DevCb + DevCr| metrics for each region. When one of the metrics for a region is small, ignore that region and do not perform white balance processing on it. Calculate the mean AvrH of the entire image after ignoring the regions in step 6 all , AvrS all , AvrCb all , AvrCr all and the variances DevH all , DevS all , DevCb all , DevCr all .

[0131] Step S109: Calculate the candidate white points according to formula (9).

[0132] Step S110: Sort the candidate white points in descending order of pixel brightness values, and select the white points with the top 10% brightness values in the HSV space and the YCbCr space as the reference white points. And fuse the reference white points with the brightness information in the HSV space and the YCbCr space according to formula (10) according to the following formula to complete the white balance calculation.

[0133] Step S111: Store the video stream obtained after white balance processing into the DDR SDRAM.

[0134] Step S112: After the video stream obtained after white balance processing is processed by the output processing unit, send it to the host computer for display.

[0135] Refer to Figure 6 , the dim light high-temperature industrial endoscope imaging system proposed in the embodiment of the present invention includes:

[0136] A memory 10, a processor 20, and a computer program stored on the memory 10 and executable on the processor 20. Among them, when the processor 20 executes the computer program, it implements the steps of the dim light high-temperature industrial endoscope imaging method proposed in this embodiment.

[0137] For the specific working process and working principle of the dim light high-temperature industrial endoscope imaging system in this embodiment, refer to the working process and working principle of the dim light high-temperature industrial endoscope imaging method in this embodiment.

[0138] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An imaging method for a dim-light high-temperature industrial endoscope, characterized in that, The method includes: Obtaining digital images inside an industrial furnace and forming an original video stream; Performing three-dimensional comb filtering on the original video stream; Performing high-brightness video synthesis on the video stream after three-dimensional comb filtering to obtain a high-brightness video stream, where performing high-brightness video synthesis on the video stream after three-dimensional comb filtering to obtain a high-brightness video stream includes: Collecting video frames in the video stream after three-dimensional comb filtering, where the video frames include the current video frame and video frames adjacent to the current video frame; Performing non-linear transformation on the video frames; Superimposing the video frames after non-linear transformation to obtain an enhanced image; Optimizing the enhanced image using a maximum a posteriori model to obtain a synthesized video frame, thereby obtaining a high-brightness video stream, where optimizing the enhanced image using a maximum a posteriori model to obtain a synthesized video frame includes: Calculating a fidelity function, where the calculation formula of the fidelity function is: Among them, ψ(O t , Y j ) is the fidelity function, which is used to measure the similarity between the optimized image O t at time t and the enhanced image Y j at time j, is the variance of the joint distribution of O t and Y j . B and D respectively represent the blurring matrix and the downsampling matrix, S represents the covariance matrix, represents the motion compensation matrix from the optimized image O t at time t to the enhanced image Y j at time j; Calculating a consistency function, where the calculation formula of the consistency function is: Among them, represents the output optimized image at time t-1, represents the consistency function, which measures the optimized image O at time t t and the output optimized image O at time t-1 t-1 for similarity, is O t and O t-1 for the variance of the joint distribution, is the motion compensation matrix from the optimized image O at time t t to the output optimized image O at time t-1 t-1 ; Calculating a regularization function, where the calculation formula of the regularization function is: Among them, L(O t ) is the regularization function, P is the size of the moving window, α is used to constrain the smoothness of the optimized image, H l and V j represent the operators that move the optimized image O t by l and j pixels in the horizontal and vertical directions respectively; An optimized image O is obtained according to a fidelity function, a consistency function, and a regularization function t The posterior probability, and the calculation formula for the posterior probability is as follows: where p(O t ) is the posterior probability of the optimized image O t , r is the number of video frames of the original video used to output the optimized image before the t-th moment, and b is the number of video frames of the original video used to output the optimized image after the t-th moment; Obtaining a synthesized video frame according to the posterior probability of the output optimized image, and the specific calculation formula is: Performing white balance processing on the image frames in the high-brightness video stream to obtain a high-quality image inside the furnace.

2. The imaging method of the dim-light high-temperature industrial endoscope according to claim 1, wherein The calculation formula for performing non-linear transformation on the video frames is: where P y and P x represent the transformed gray value and the gray value before transformation respectively, and n is used to control the overall improvement degree of brightness, and n = 2.

3. The imaging method of the dim-light high-temperature industrial endoscope according to claim 2, wherein Performing white balance processing on the image frames in the high-brightness video stream to obtain a high-quality image inside the furnace includes: Converting the image frames in the high-brightness video stream from the RGB space to the HSV space and the YCbCr space respectively to obtain the H channel, S channel, V channel, Y channel, Cb channel, and Cr channel; Dividing the image frames into a preset number of regions and calculating the mean absolute deviation of the H channel, S channel, Cb channel, and Cr channel for each region respectively; Obtaining regions to be processed for white balance processing according to the mean absolute deviation of the H channel, S channel, Cb channel, and Cr channel; Calculating the mean and deviation of the H channel, S channel, Cb channel, and Cr channel of the regions to be processed; Obtaining candidate white points according to the mean and deviation of the H channel, S channel, Cb channel, and Cr channel of the regions to be processed; Selecting a reference white point from the candidate white points and fusing the reference white point with the luminance information in the HSV space and the YCbCr space.

4. The imaging method of the dim-light high-temperature industrial endoscope according to claim 3, characterized in that The calculation formula for obtaining candidate white points according to the mean and deviation of the H channel, S channel, Cb channel, and Cr channel of the regions to be processed is: Among them, H(i,j), S(i,j), Cb(i,j), and Cr(i,j) are the i-th pixel and j-th pixel of the H channel, S channel, Cb channel, and Cr channel respectively, and AvrH all , AvrS all , AvrCb all , and AvrCr all are the means of the H channel, S channel, Cb channel, and Cr channel respectively, and DevH all , DevS all , DevCb all , and DevCr all are the variances of the H channel, S channel, Cb channel, and Cr channel respectively.

5. The imaging method of the dim-light high-temperature industrial endoscope according to claim 4, wherein The calculation formula for selecting a reference white point from the candidate white points and fusing the reference white point with the luminance information in the HSV space and the YCbCr space is: Among them, img represents the fused image after fusing the reference white point with the luminance information in the HSV color space and the YCbCr color space. imgR, imgG, and imgB respectively represent the channel images corresponding to the R channel, G channel, and B channel. θ represents the multi-color domain fusion degree, V max and Y max are respectively the maximum value of lightness and the maximum value of luminance. R avgw 、G avgw 、B avgw are respectively the average pixel grayscale values of the R, G, and B channels.

6. The method for obtaining an image of a dim-light high-temperature industrial endoscope according to claim 5, characterized in that, After performing white balance processing on the image frames in the high-brightness video stream to obtain a high-quality image inside the furnace, it further includes: Sending the high-quality image inside the furnace to a host computer for display.

7. A low-light high-temperature industrial endoscope imaging system, the system includes: A memory (10), a processor (20), and a computer program stored on the memory (10) and executable on the processor (20), characterized in that when the processor (20) executes the computer program, it implements the steps of the method according to any one of claims 1 to 6 above.

Images