Unsupervised fruit brightness correction method driven by channel color correction and curve mapping

Abstract

The application discloses a kind of channel color correction and curve mapping drive's unattended fruit brightness correction method.The application can quickly generate the required brightness enhancement curve parameters by designing specific loss function and network structure, and carry out quick brightness correction through enhancement curve, better adapt to variable curvature and retain surface color information.The application overcomes the problem that large-scale data set is required for training deep learning brightness correction method, and also does not need to establish low-light condition and normal light matching data set, reduces the difficulty of data collection and processing.The application establishes a kind of spherical fruit image curve estimation network model structure, obtains different scale brightness information through jump layer connection, speeds up model operation speed through sparse convolution kernel, and simultaneously avoids model overfitting.

Description

Technical Field

[0001] This invention relates to a fruit brightness correction method in the field of post-harvest fruit processing technology, and more particularly to an unsupervised fruit brightness correction method driven by channel color correction and curve mapping. Background Technology

[0002] In the process of grading and sorting fruits using machine vision methods, spherical fruits such as apples, pears, and citrus fruits often exhibit a problem where the central area is brighter than the edge area during imaging (Li Jiangbo. Research on a rapid detection method for surface defects in navel oranges [D]. Zhejiang University, 2012.). Inappropriate lighting distribution can cause regional underexposure or overexposure in fruit imaging, resulting in the inaccurate representation of surface features in the image. Therefore, it is necessary to correct the brightness of the spherical fruit surface.

[0003] Traditional image processing methods such as histogram equalization, the Retinex method, and the Mask algorithm can improve image quality to some extent, but they have some limitations when processing images of spherical fruits. For example, histogram equalization may reduce gray levels and lose details; the Retinex method has high computational complexity and may produce light spots that mask defects; the Mask algorithm is quite sensitive to color interference.

[0004] Gomez-Sanchis et al. (2008) used a method based on the circumference of spherical fruits to create an elevation model. They assumed the three-dimensional structure of the spherical fruit was a perfect sphere, used the Lambertian reflection model to analyze the uneven distribution of brightness, and used the annular region of the static fruit to complete the brightness correction (Gomez-Sanchis J, Molto E, Camps-Valls G, et al. Automatic correction of the effects of the light source on spherical objects. An application to the analysis of hyperspectral images of citrusfruits[J]. Journal of Food Engineering, 2008, 85(2):191-200.). Zhang et al. (2015) simplified the calculation by using the angle between the point's normal vector and the light source (Zhang B, Huang W, Gong L, et al. Computervision detection of defective apples using automatic lightness correction and weighted RVM classifier[J]. Journal of Food Engineering, 2015, 146:143-151.). Such methods require modeling the relative positions of the fruit and the light source, which has limitations in dynamic real-time processing. Li et al. (2013) used digital signal processing and Butterworth filters for brightness correction, which worked well for monochrome fruits but was not suitable for fruits with complex surface color information (Li J, Rao X, Wang F, et al. Automatic detection of common surface defects on oranges using combined lighting transform and image ratio methods[J]. Postharvest Biology and Technology, 2013, 82: 59-69.).Chinese patent CN106124511A and Shang et al. (2023) respectively used the least squares regression method and the quadratic curve fitting method to quickly obtain the light field changes of segmented fruits in a specific direction, so as to correct the edge light intensity of the fruits (Shang M, Xue L, Zhang Y, et al. Full-surface defect detection of navel orange based on hyperspectral online sorting technology[J]. Journal of Food Science,2023,88(6):2488-2495.). However, before using these methods, accurate fruit segmentation results are required. In the calculation process, the segmentation edges are prone to pixel overflow due to division operations.

[0005] In summary, spherical fruits are susceptible to uneven lighting in machine vision imaging systems. Researchers have proposed a series of brightness correction methods based on shape features, spherical models, normal vector angles, and surface light distribution. However, these methods still struggle to adapt to variations in curvature and surface color interference. Summary of the Invention

[0006] To address the needs and problems in the background technology, this invention proposes an unsupervised fruit brightness correction method driven by channel color correction and curve mapping. This invention designs an unsupervised loss function and network model based on the surface color and curvature features of spherical fruits, establishing an unsupervised learning model driven by RGB three-channel color correction and curve mapping to achieve brightness correction of spherical fruit images. This aims to overcome the limitations of traditional methods in processing spherical fruit images, improve the performance and efficiency of brightness correction, and simultaneously reduce the requirements for data volume and hardware equipment.

[0007] The technical solution adopted in this invention is as follows:

[0008] Step 1: Establish a fruit brightness enhancement image dataset. The fruit brightness enhancement image dataset contains multiple images of spherical fruits collected under the lighting conditions to be corrected.

[0009] Step 2: Construct a curve estimation network model M for spherical fruit images. Use the fruit brightness enhancement image dataset to perform unsupervised training on the curve estimation network model M for spherical fruit images to obtain the trained curve estimation network model M for spherical fruit images.

[0010] Step 3: Input the spherical fruit image X to be corrected into the trained spherical fruit image curve estimation network model M to obtain the RGB three-channel curve parameter map Y. After using the RGB three-channel curve parameter map Y to perform point-by-point curve mapping estimation on the spherical fruit image X to be corrected, the final brightness correction result Z is obtained.

[0011] In step 2, the spherical fruit image curve estimation network model M includes 7 depthwise separable convolutional blocks, 3 max pooling layers, and 3 bilinear interpolation upsampling layers. The input of the spherical fruit image curve estimation network model M is used as the input of the first depthwise separable convolutional block. The first depthwise separable convolutional block is sequentially connected to the first max pooling layer, the second depthwise separable convolutional block, the second max pooling layer, the third depthwise separable convolutional block, and the third max pooling layer before being connected to the fourth depthwise separable convolutional block. The fourth depthwise separable convolutional block is connected to the first bilinear interpolation upsampling layer, and the output of the third depthwise separable convolutional block is connected to the first bilinear interpolation upsampling layer. The outputs of the first depthwise separable convolutional block are concatenated and fused before being input into the fifth depthwise separable convolutional block. The fifth depthwise separable convolutional block is connected to the second bilinear interpolation upsampling layer. The outputs of the second depthwise separable convolutional block and the second bilinear interpolation upsampling layer are concatenated and fused before being input into the sixth depthwise separable convolutional block. The sixth depthwise separable convolutional block is connected to the third bilinear interpolation upsampling layer. The outputs of the first depthwise separable convolutional block and the third bilinear interpolation upsampling layer are concatenated and fused before being input into the seventh depthwise separable convolutional block. The output of the seventh depthwise separable convolutional block is used as the output of the spherical fruit image curve estimation network model M.

[0012] The seven depthwise separable convolutional blocks have the same structure, each consisting of a depthwise separable convolutional layer, a batch normalization layer, and an activation layer connected in sequence.

[0013] In step 2, when the spherical fruit image curve estimation network model M undergoes unsupervised training, its unsupervised loss function consists of three parts: natural color contrast loss, spatial consistency loss, and feature information loss, as shown in the following formula:

[0014] L sum =L spa +α×L feature +β×L stable_col

[0015] Among them, L sum The total loss function value is represented by α and β, which are the first and second weighting coefficients, respectively.

[0016] Natural color contrast loss L stable_col The specific calculation formula is as follows:

[0017]

[0018] in, These are the mean values ​​of the red, green, and blue channels in the input image, respectively. These are the mean values ​​of the red, green, and blue channels in the baseline color image, respectively.

[0019] Spatial consistency loss L spa The specific calculation formula is as follows:

[0020]

[0021] Where K represents the number of 4×4 local regions, and Ω(i) represents the four adjacent regions centered on region i: top, bottom, left, and right; Y i The result of calculating region i after converting the enhanced image to grayscale and performing 4×4 average pooling; Y j The result of calculating region j after converting the enhanced image to grayscale and performing 4×4 average pooling; I i This is the calculation result for region i after converting the input image to grayscale and performing 4×4 average pooling; I j This is the calculation result for region j after converting the input image to grayscale and performing 4×4 average pooling; || represents taking the absolute value;

[0022] Feature information loss L feature The specific calculation formula is as follows:

[0023]

[0024] Among them, std ROI1 Let std be the standard deviation of the region of interest in the outer ring. ROI2 This represents the standard deviation of the region of interest within the inner ring.

[0025] In step 2, the training hyperparameters of the spherical fruit image curve estimation network model M include a batch size of 8, an initial learning rate of 0.0001, a weight decay coefficient of 0.0001, and 100 training iterations.

[0026] In step 3, the RGB three-channel curve parameter map Y is used to perform point-by-point curve mapping estimation on the spherical fruit image X to be corrected, specifically as follows:

[0027] LE n (x)=LE n-1 (x)+A(x)LE n-1 (x)(1-LE n-1 (x))

[0028] LE0(x)=I(x)

[0029] Where x is the image pixel coordinate, n is the number of iterations, A(x) is the pixel parameter value in the learnable parameter map; I(x) is the input image X of the spherical fruit to be corrected, LE n (x) is the enhanced image of the input image X (a spherical fruit class to be corrected) after n iterations, LE n-1 (x) is the enhanced image of the input image X of a spherical fruit to be corrected after n-1 iterations.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] 1. This invention establishes an unsupervised fruit brightness correction method driven by channel color correction and curve mapping. By designing a specific loss function and network structure, the required brightness enhancement curve parameters can be generated quickly, and the brightness can be corrected quickly through the enhancement curve. This method can better adapt to varying curvatures and retain surface color information.

[0032] 2. This invention overcomes the problem that training deep learning brightness correction methods requires large-scale datasets, while eliminating the need to establish paired datasets for low-light conditions and normal illumination, thus reducing the difficulty of data acquisition and processing.

[0033] 3. This invention establishes a network model structure for estimating the curve of images of spherical fruits. It obtains brightness information at different scales through skip connections, accelerates the model's computation speed through sparse convolutional kernels, and avoids model overfitting. Attached Figure Description

[0034] Figure 1 Framework diagram of an unsupervised fruit brightness correction method driven by channel color correction and curve mapping;

[0035] Figure 2 A diagram of the network structure for curve estimation of images of spherical fruits;

[0036] Figure 3 This is a schematic diagram of the ROI region where feature information is lost.

[0037] Figure 4 This is a schematic diagram of spatial consistency loss calculation; where (a) is the input image and (b) is the final brightness correction result Z (i.e., the enhanced image);

[0038] Figure 5 The images show a comparison of brightness correction methods; where (a) is the input image, (b) is the final brightness correction result Z (i.e., the enhanced image) obtained using the method of this invention, (c) is the enhanced image obtained using the fast adaptive brightness correction method, (d) is the enhanced image obtained using the EnlightenGAN method, (e) is the enhanced image obtained using Retinex, and (f) is the enhanced image obtained using the Histgram method. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] This invention proposes a brightness correction method for spherical fruits based on unsupervised learning and curve mapping, which includes the following process during the deep learning model training phase:

[0041] S1: Construct a spherical fruit imaging system, keeping the positions of the light source, camera, and imaging area relatively fixed. Place the fruit in the imaging area and acquire images.

[0042] S2: Divide the collected fruit images into a training set and a validation set in a 3:1 ratio to construct a low-light enhanced image dataset of fruits.

[0043] S3: Construct a curve estimation network model M for spherical fruit images, and use a fruit brightness enhancement image dataset to perform unsupervised training on the curve estimation network model M for spherical fruit images to obtain a well-trained curve estimation network model M for spherical fruit images.

[0044] like Figure 2 As shown, the curve estimation network model M for spherical fruit images includes 7 depthwise separable convolutional blocks, 3 max pooling layers, and 3 bilinear interpolation upsampling layers. The input of the spherical fruit image curve estimation network model M is used as the input of the first depthwise separable convolutional block. The first depthwise separable convolutional block is sequentially passed through the first max pooling layer, the second depthwise separable convolutional block, the second max pooling layer, the third depthwise separable convolutional block, and the third max pooling layer before being connected to the fourth depthwise separable convolutional block. The fourth depthwise separable convolutional block is connected to the first bilinear interpolation upsampling layer, and the output of the third depthwise separable convolutional block is connected to the first bilinear interpolation upsampling layer. The outputs of the sampling layers are concatenated and fused before being input into the fifth depthwise separable convolutional block. The fifth depthwise separable convolutional block is connected to the second bilinear interpolation upsampling layer. The outputs of the second depthwise separable convolutional block and the second bilinear interpolation upsampling layer are concatenated and fused before being input into the sixth depthwise separable convolutional block. The sixth depthwise separable convolutional block is connected to the third bilinear interpolation upsampling layer. The outputs of the first depthwise separable convolutional block and the third bilinear interpolation upsampling layer are concatenated and fused before being input into the seventh depthwise separable convolutional block. The output of the seventh depthwise separable convolutional block serves as the output of the spherical fruit image curve estimation network model M.

[0045] The seven depthwise separable convolutional blocks have identical structures, each consisting of a depthwise separable convolutional layer, a batch normalization layer, and an activation layer connected sequentially. Each depthwise separable convolutional layer comprises a depthwise convolution with a 3×3 kernel and a stride of 1, and a pointwise convolution with a 1×1 kernel and a stride of 1. The activation function for the activation layers of the first to sixth depthwise separable convolutional blocks is the ReLU function, while the activation function for the activation layer of the seventh depthwise separable convolutional block is the tanh function.

[0046] When the input RGB color image has dimensions of W×H×3, the specific dimensions are as follows:

[0047] 1) The input to the first depthwise separable convolutional layer is a W×H×3 image, and the output is a W×H×8 feature map, using the ReLU activation function.

[0048] 2) The first max pooling layer performs a 2×2 max pooling operation on the output of the previous layer, and the output is... The feature map.

[0049] 3) The input to the second depthwise separable convolutional layer is the output of the previous layer, and the output is... The feature map is activated using the ReLU activation function.

[0050] 4) The second max-pooling layer performs a 2×2 max-pooling operation on the output of the previous layer, and the output is... The feature map.

[0051] 5) The input to the third depthwise separable convolutional layer is the output of the previous layer, and the output is... The feature map is activated using the ReLU activation function.

[0052] 6) The third max-pooling layer performs a 2×2 max-pooling operation on the output of the previous layer, and the output is... The feature map.

[0053] 7) The input to the fourth depthwise separable convolutional layer is the output of the previous layer, and the output is... The feature map is activated using the ReLU activation function.

[0054] 8) The first bilinear interpolation upsampling layer performs a 2×2 upsampling operation on the output of the previous layer, and the output is... The feature map.

[0055] 9) The input to the fifth depthwise separable convolutional layer is a cross-layer connection between the output of the first bilinear interpolation upsampling layer and the output of the third depthwise separable convolutional layer. The output is... Use the ReLU activation function.

[0056] 10) The second bilinear interpolation upsampling layer performs a 2×2 upsampling operation on the output of the previous layer, and the output is... The feature map.

[0057] 11) The input to the sixth depthwise separable convolutional layer is a cross-layer connection between the output of the second bilinear interpolation upsampling layer and the output of the second depthwise separable convolutional layer; the output is... Use the ReLU activation function.

[0058] 12) The third bilinear interpolation upsampling layer performs a 2×2 upsampling operation on the output of the previous layer, and the output is a W×H×16 feature map.

[0059] 13) The input to the 7th depthwise separable convolutional layer is a cross-layer connection between the output of the 3rd bilinear interpolation upsampling layer and the output of the 1st depthwise separable convolutional layer. The output is W×H×3, using the tanh activation function.

[0060] When the network model M for estimating the curve of images of spherical fruits is trained in an unsupervised manner, its unsupervised loss function consists of three parts: natural color contrast loss, spatial consistency loss, and feature information loss, as shown in the following formula:

[0061] L sum =L spa +α×L feature +β×L stable_col

[0062] Among them, L sum The total loss function value is represented by α and β, which are the first and second weighting coefficients, respectively.

[0063] Natural color contrast loss L stable_col The color benchmark for this loss function is the color of images generated by a model trained on the ZeroDCE++ framework using the SICE multi-exposure dataset. The specific calculation formula is as follows:

[0064]

[0065] in, These are the mean values ​​of the red, green, and blue channels in the input image, respectively. These are the mean values ​​of the red, green, and blue channels in the baseline color image, respectively. Meanwhile, to avoid the deep learning model being unable to perform gradient calculations due to a loss function of 0, a coefficient of 0.0001 is introduced into the loss function.

[0066] Spatial consistency loss L spa Spatial consistency of an image is ensured by preserving the differences between adjacent regions, such as... Figure 4 (a) and Figure 4As shown in (b). The specific calculation formula is as follows:

[0067]

[0068] Where K represents the number of 4×4 local regions, and Ω(i) represents the four adjacent regions centered on region i: top, bottom, left, and right; Y i The result of calculating region i after converting the enhanced image to grayscale and performing 4×4 average pooling; Y j The result of calculating region j after converting the enhanced image to grayscale and performing 4×4 average pooling; I i This is the calculation result for region i after converting the input image to grayscale and performing 4×4 average pooling; I j This is the calculation result for region j after converting the input image to grayscale and performing 4×4 average pooling; || represents taking the absolute value; first, the enhanced image and the input image are converted to grayscale, then the grayscale image is subjected to 4×4 average pooling, simplifying K local regions to K pixels, resulting in the RGB three-channel curve parameter map Y and the input image I, as illustrated in the diagram. Figure 2 As shown.

[0069] Feature information loss L feature First, the enhanced image is converted to HSV color channels, and the V component is extracted. Then, the maximum inscribed circle of the apple object is defined. Based on the maximum inscribed circle, a region of interest (ROI) is generated in the V component, denoted as the outer ring ROI1. The radius of the maximum inscribed circle is then multiplied by 0.75 to generate the inner ring ROI2. The standard deviations of the outer ring ROI1 and the inner ring ROI2 in the V component are calculated respectively. Figure 3 As shown. The specific calculation formula is as follows:

[0070]

[0071] Among them, std ROI1 Let std be the standard deviation of the region of interest in the outer ring. ROI2 This represents the standard deviation of the region of interest within the inner ring.

[0072] The following hyperparameters and unsupervised loss function were used to train an unsupervised network model for estimating the curves of images of spherical fruits.

[0073] Table 1 shows the initialization parameters of the image curve estimation network model for spherical fruits.

[0074]

[0075] S4: Input the spherical fruit image X to be corrected into the trained spherical fruit image curve estimation network model M to obtain the RGB three-channel curve parameter map Y. After using the RGB three-channel curve parameter map Y to perform point-by-point curve mapping estimation on the spherical fruit image X to be corrected, the final brightness correction result Z is obtained, which is the final enhanced image.

[0076] The point-by-point curve mapping estimation of the RGB three-channel curve parameter map Y to be corrected spherical fruit image X is performed as follows:

[0077] LE n (x)=LE n-1 (x)+A(x)LE n-1 (x)(1-LE n-1 (x))

[0078] LE0(x)=I(x)

[0079] Where x is the image pixel coordinate, and each input pixel is normalized to the range [0,1]. n is the number of iterations, A(x) is the pixel parameter value in the learnable parameter map, satisfying A(x)∈[-1,1]; I(x) is the input image X of the spherical fruit to be corrected, LE n (x) is the enhanced image of the input image X (a spherical fruit class to be corrected) after n iterations, LE n-1 (x) is the enhanced image of the input spherical fruit image X after n-1 iterations. In practice, considering that the brightness changes on the fruit surface are more gradual than the brightness changes in nature, n is defined as 4. For color RGB images, each channel needs to be enhanced using this function.

[0080] The present invention will be described below with reference to specific test experiments and data.

[0081] The computers used in the experiment are shown in Table 2:

[0082] Table 2 shows the parameters of computer equipment.

[0083]

[0084] First, according to the present invention, 400 apple images were collected, including five poses: front view of the fruit stalk, front view of the calyx, side view, oblique view of the fruit stalk, and oblique view of the calyx.

[0085] In the low-light enhanced fruit image dataset, 400 images were divided into 300 training images and 100 validation images in a 3:1 ratio.

[0086] Using the training set of this dataset, along with the hyperparameters and unsupervised loss function shown in Table 1, we trained a network model for estimating the curves of images of spherical fruits. Finally, we evaluated the model's performance using the validation set of images.

[0087] The method proposed in this invention is compared with other illumination correction methods in Table 3. The invention provides a referenceless evaluation of image enhancement quality. Specifically, this invention uses UICM (Panetta K, Gao C, Agaian S. Human-Visual-System-Inspired underwater image quality measures[J].IEEE Journal of Oceanic Engineering, 2016, 41: 541-551.) in UIQM to evaluate color features, uses the image information entropy function (Roy P, Boddeti VN. Mitigating information leakage in image representations: A maximum entropy approach[J]. Computer Vision and Pattern Recognition, 2019, pp: 2586-2594.) to evaluate the overall structural clarity of the fruit, and uses bidirectional light distribution statistics (Rong D, Rao X, Ying Y. Computer vision detection of surface defect on oranges by means of asliding comparison window local segmentation algorithm[J]. Computers and Electronics in Agriculture, 2017, 137: 59-68.) to evaluate the overall illumination uniformity of the fruit surface.

[0088] The UICM method evaluates color features: This method evaluates the difference between enhanced images and human perception based on the white balance principle. First, it selects color parameters a and b from the Lab color space, which is based on human color perception, as an orthogonal basis, where a represents magenta to green and b represents yellow to blue. The UICM method for calculating the color richness measurement index of an image is shown in the following formula.

[0089] RG = RG

[0090] YB=(R+G) / 2-B

[0091] K = M × N

[0092]

[0093]

[0094] Intensity(I)=SORT(Flatten(I))

[0095]

[0096]

[0097]

[0098] Where R, G, and B are the red, green, and blue components of the image to be evaluated, respectively; RG and YB are the a and b components in the Lab color space, respectively; M, N, and K represent the number of rows, columns, and total pixels of the image to be evaluated, respectively; TαL and TαR represent the number of top and bottom pixel regions, respectively; Intensity(I) represents the image signal intensity; Flatten(I) expands the M×N two-dimensional matrix I into a one-dimensional array X of length K; SORT() sorts the values ​​of the one-dimensional array from smallest to largest; and in the calculation of UICM, μ I μ represents the mean of asymmetric pixel trimming in an image. I The closer the value is to 0, the smaller the mean color deviation and the more balanced the color distribution across different channels. This represents the second-order statistical variance of image intensity. The larger the value, the greater the dynamic range of the image colors.

[0099] Image information entropy function evaluates image sharpness features: it displays the overall sharpness of an apple image by using the amount of information contained in the clustered features of gray-level distribution in the image, as shown in the following formula:

[0100]

[0101] Here, Entropy represents the image information entropy. In this invention, the image to be evaluated is first converted into a grayscale image, then the corresponding grayscale histogram is calculated, and finally the probability p of a certain grayscale value appearing in the image is obtained from the grayscale histogram. i .

[0102] The bidirectional light distribution statistical evaluation of the overall brightness uniformity of fruit surface is as follows: First, the image is converted to grayscale. The fruit is defined as the largest inscribed circle of the object, and its radius is multiplied by 0.98 to avoid boundary point interference, generating a region of interest (ROI). The pixel average value of each row within the ROI is calculated to generate an array ROW, and the pixel average value of each column is calculated to generate an array COL. The coefficient of variation of the ROW and COL arrays is calculated separately. Finally, the overall uniformity of the apple surface, AppleAverage, is calculated, as shown in the following formula:

[0103]

[0104] AppleAverage=1-COV(ROW)-COV(COL)

[0105] Where COV() represents the calculation of array variance; n represents the length of array X. This represents the average value of array X.

[0106] Overall Evaluation: Because the magnitudes of the various evaluation indicators differ after calculation, it is necessary to normalize each parameter to facilitate comparison and analysis. The study will compare the performance of the original image and the enhanced image in various aspects. The specific calculations are shown in the formula:

[0107]

[0108]

[0109]

[0110] AIQM = AICM + AISM + AIAM

[0111] Wherein, AICM, AISM, and AIAM represent the color feature evaluation index, structural sharpness evaluation index, and overall brightness uniformity evaluation index of a uniformly illuminated image, respectively. In AIQM, "enhanced" and "origin" correspond to the calculated values ​​of the enhanced and original images, respectively. Evaluations were performed using 100 validation images, and the results were averaged.

[0112] Based on the production application scenario and deployment requirements of this project, the model needs to be designed to be lightweight. To evaluate the efficiency of the model, the inference time per frame is used as a metric.

[0113] Table 3 shows the rapid extraction effect of single fruit in the complete imaging region of each model.

[0114]

[0115] The horizontal comparison results are shown in Table 3. The spherical fruit brightness correction method based on unsupervised learning and curve mapping proposed in this study can significantly improve color features, sharpness features, and overall brightness uniformity of the fruit surface. Simultaneously, this method also shows a significant improvement in efficiency, with a single image processing time of 1.57 ms, which is 3.8 times faster than the fast adaptive brightness correction algorithm, 34.6 times faster than the EnlightenGAN algorithm, and 24.8 times faster than the Retinex algorithm. The comparison results are as follows: Figure 5 (a)- Figure 5As shown in (f).

[0116] Taking a PC equipped with an Nvidia RTX 3060 GPU as an example, it can complete the brightness correction task of 637 fruit images per second, which fully demonstrates the value of this invention in real-time sorting and detection of fruits in practical applications.

[0117] Finally, it should be noted that the above embodiments and descriptions are only used to illustrate the technical solutions of the present invention and not to limit it. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the disclosure of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the protection scope of the claims of the present invention.

Claims

1. An unsupervised fruit brightness correction method driven by channel color correction and curve mapping, characterized in that, Includes the following steps: Step 1: Establish a fruit brightness enhancement image dataset. The fruit brightness enhancement image dataset contains multiple images of spherical fruits collected under the lighting conditions to be corrected. Step 2: Construct a curve estimation network model M for spherical fruit images. Use the fruit brightness enhancement image dataset to perform unsupervised training on the curve estimation network model M for spherical fruit images to obtain the trained curve estimation network model M for spherical fruit images. In step 2, when the spherical fruit image curve estimation network model M undergoes unsupervised training, its unsupervised loss function consists of three parts: natural color contrast loss, spatial consistency loss, and feature information loss, as shown in the following formula: in, This represents the overall loss function value. and These are the first weighting coefficient and the second weighting coefficient, respectively. Natural color contrast loss L stable_col The specific calculation formula is as follows: in, These are the mean values ​​of the red, green, and blue channels in the input image, respectively. These are the mean values ​​of the red, green, and blue channels in the baseline color image, respectively. Spatial consistency loss L spa The specific calculation formula is as follows: in, represent The number of local regions It consists of four adjacent regions centered on region i: above, below, left, and right. To convert the enhanced image to grayscale and perform... The calculation results for region i after average pooling; To convert the enhanced image to grayscale and perform... The calculation results for region j after average pooling; To convert the input image to grayscale and perform... The calculation results for region i after average pooling; To convert the input image to grayscale and perform... The calculation results for region j after average pooling; Indicates taking the absolute value; Feature information loss L feature The specific calculation formula is as follows: in, Let be the standard deviation of the region of interest in the outer ring. The standard deviation of the region of interest within the inner ring; Step 3: Input the spherical fruit image X to be corrected into the trained spherical fruit image curve estimation network model M to obtain the RGB three-channel curve parameter map Y. After using the RGB three-channel curve parameter map Y to perform point-by-point curve mapping estimation on the spherical fruit image X to be corrected, the final brightness correction result Z is obtained.

2. The unsupervised fruit brightness correction method driven by channel color correction and curve mapping according to claim 1, characterized in that, In step 2, the spherical fruit image curve estimation network model M includes 7 depthwise separable convolutional blocks, 3 max pooling layers, and 3 bilinear interpolation upsampling layers. The input of the spherical fruit image curve estimation network model M is used as the input of the first depthwise separable convolutional block. The first depthwise separable convolutional block is sequentially connected to the first max pooling layer, the second depthwise separable convolutional block, the second max pooling layer, the third depthwise separable convolutional block, and the third max pooling layer before being connected to the fourth depthwise separable convolutional block. The fourth depthwise separable convolutional block is connected to the first bilinear interpolation upsampling layer. The output of the third depthwise separable convolutional block is spliced ​​and fused with the output of the first bilinear interpolation upsampling layer before being input into the fifth depthwise separable convolutional block. The fifth depthwise separable convolutional block is connected to the second bilinear interpolation upsampling layer. The output of the second depthwise separable convolutional block is concatenated and fused with the output of the second bilinear interpolation upsampling layer before being input into the sixth depthwise separable convolutional block. The sixth depthwise separable convolutional block is connected to the third bilinear interpolation upsampling layer. The output of the first depthwise separable convolutional block and the output of the third bilinear interpolation upsampling layer are concatenated and fused before being input into the seventh depthwise separable convolutional block. The output of the seventh depthwise separable convolutional block is used as the output of the spherical fruit image curve estimation network model M.

3. The unsupervised fruit brightness correction method driven by channel color correction and curve mapping according to claim 2, characterized in that, The seven depthwise separable convolutional blocks have the same structure, each consisting of a depthwise separable convolutional layer, a batch normalization layer, and an activation layer connected in sequence.

4. The unsupervised fruit brightness correction method driven by channel color correction and curve mapping according to claim 1, characterized in that, In step 2, the training hyperparameters of the spherical fruit image curve estimation network model M include a batch size of 8, an initial learning rate of 0.0001, a weight decay coefficient of 0.0001, and 100 training iterations.

5. The unsupervised fruit brightness correction method driven by channel color correction and curve mapping according to claim 1, characterized in that, In step 3, the RGB three-channel curve parameter map Y is used to perform point-by-point curve mapping estimation on the spherical fruit image X to be corrected, specifically as follows: Where x is the image pixel coordinate, and n is the number of iterations. These are the pixel parameter values ​​in the learnable parametric map; The input is an image X of a spherical fruit to be corrected. It is the enhanced image of the input image X of spherical fruit to be corrected after n iterations. It is the enhanced image of the input image X of spherical fruit to be corrected after n-1 iterations.

Images