A Method for Predicting the Optical Properties of Fruits Based on a Double-Layer Network with a Single Multi-Frequency Image

Abstract

The present invention relates to the technical field of fruit optical property detection, specifically: a method for predicting fruit optical properties based on a double-layer network using a single multi-frequency image, comprising the following steps: Step 1, the host computer generates multiple structured light fringe patterns, including a completely black image, a completely white image, a single-frequency fringe pattern, and a double-frequency orthogonal fringe pattern, and then projects the fringe patterns onto the fruit surface through a projector; Step 2, a CCD camera is used to collect the reflected light intensity distribution on the fruit surface under fringe illumination, and the collected images are saved in BMP format; Step 3, the completely black and completely white images are projected onto a standard diffuse reflection plate respectively, and the dark field and white board light intensity distributions of the standard reference light field are collected by the CCD camera, and then the collected fringe images are corrected for the black and white boards; The present invention realizes the prediction of fruit optical properties at different spatial frequencies through a single multi-frequency image. It solves the deficiencies in real-time performance, anti-interference ability, and information integrity in traditional detection methods.

Description

Technical Field

[0001] The present invention relates to the technical field of fruit optical property detection, and in particular to a method for predicting fruit optical properties based on a double-layer network of a single multi-frequency image. Background Technique

[0002] Fruit quality detection is of great significance for ensuring food safety, improving the competitiveness of the fruit market, and meeting the health needs of consumers. In fruit quality detection, spectral technology and optical imaging technology each play an irreplaceable role. Spectral technology can reflect its internal chemical composition, maturity, and potential defects by measuring the reflection or transmission spectra of fruits at different wavelengths, while optical imaging provides intuitive surface appearance, color, and structure information. The combination of the two makes non-destructive fruit quality assessment possible; however, the above methods still have certain deficiencies in terms of real-time performance, anti-interference ability, and response to sample non-uniformity. Summary of the Invention

[0003] In view of the technical problems raised in the background technique, the present invention provides a method for predicting fruit optical properties based on a double-layer network of a single multi-frequency image.

[0004] The technical solution adopted by the present invention is: a method for predicting fruit optical properties based on a double-layer network of a single multi-frequency image, including the following steps:

[0005] Step 1: The host computer generates multiple structured light fringe patterns, including a completely black picture, a completely white picture, a single-frequency fringe pattern, and a double-frequency orthogonal fringe pattern, and then projects the fringe patterns onto the fruit surface through a projector;

[0006] Step 2: Use a CCD camera to collect the reflected light intensity distribution on the fruit surface under fringe illumination, and save the collected image in BMP format;

[0007] Step 3: Project the completely black and completely white images onto a standard diffuse reflection plate respectively, and collect the dark field and white board light intensity distributions of the standard reference light field by a CCD camera, and then perform black and white board calibration on the collected fringe images;

[0008] Step 4: Calculate the pixel ratio of the corrected multi-frequency fringe image of the fruit and the corresponding corrected multi-frequency fringe image of the standard diffuse reflection plate and assign the obtained ratio image to the first channel of the input image, and set the remaining two channels to zero, thereby forming the input image for subsequent network processing;

[0009] Step 5: For the single-frequency fringe image, first, the corrected horizontal fringe pattern and vertical fringe pattern Perform ratio operations with the standard diffuse reflection plate image respectively. Assign the result of the horizontal stripe ratio map to the first channel, the result of the vertical stripe ratio map to the second channel, and set the third channel to zero;

[0010] Step 6: Obtain the optical characteristic values corresponding to the single-frequency stripe image. Use the light intensity distribution data after calibration with the black and white plates to perform three-phase demodulation to obtain the amplitude envelopes of the reflected photon densities of the AC and DC components and ;

[0011] Obtain the true diffuse reflectivities of the DC component and the AC component by comparing with the standard white plate and values;

[0012] Input the obtained values into the LUT, and further derive the optical characteristic parameters of the sample under SFDI based on the LUT method, namely: absorption coefficient and reduced scattering coefficient;

[0013] Step 7: Place the value corresponding to the spatial frequency in the first channel of the output image; place the value corresponding to in the second channel, and set the third channel to zero. As the output of the second-layer model, that is, the channel composition is The output of the absorption coefficient model is also constructed in the same way, and the output image channel composition is ;

[0014] Step 8: Construct a two-layer network architecture.

[0015] In one embodiment, in Step 8, the specific method for constructing the two-layer network architecture is as follows:

[0016] The first layer of the network: Adopt a U-net structure combined with the FSA attention mechanism, and embed the FSA attention mechanism in each upsampling and downsampling module;

[0017] The second layer of the network: An optimized network based on the generative adversarial network. Its generator adopts a ResNet-U-net structure, and uses feature addition to replace the traditional U-net skip connection;

[0018] Set the network model parameters: Set the input network image size to 600×600, select parameters for batch size, epoch, and learning rate, and set the network to be trained using a GPU. The specific parameters are as follows:

[0019] The batch size parameter is 4, the epoch parameter is 200, and the learning rate is 0.0002;

[0020] Train the double - layer optical characteristic network model according to the set network model parameters. During the training process, save the model with the highest accuracy in each round to the file best_model.pt;

[0021] The first - layer network outputs the multi - frequency separation result;

[0022] The second - layer network outputs respectively according to the training objective or the prediction result;

[0023] After the model training is completed, load best_model.pt through the test program for prediction. The output results include the predicted image and the normalized mean absolute error NMAE. The calculation formula of NMAE is as follows: ;

[0024] where and are the predicted value and the reference value respectively, and T is the total number of pixels.

[0025] In one embodiment, in step 1, the dual - frequency orthogonal fringe pattern is formed by the vertical superposition of the sine fringes with spatial frequencies and in the spatial domain. Among them, is the spatial frequency corresponding to the horizontal fringe pattern, is the spatial frequency corresponding to the vertical fringe pattern. The two sets of fringes are orthogonally arranged, and each set of frequencies uses the same initial phase α. At the same time, for the three single - frequency fringe patterns generated by the single spatial frequency and the single spatial frequency respectively, α is 0, and .

[0026] In one embodiment, in step 3, the collected fringe image is corrected for black - and - white plates through the following formula: ;

[0027] where is the corrected image intensity distribution, is the image intensity distribution before correction, is the dark - field image intensity distribution, is the image intensity distribution of the reference white plate.

[0028] In one embodiment, in step 4, the channel format of the input image is .

[0029] In one embodiment, in step 5, the single-frequency fringe image is used as the output of the first-layer network, and the channel composition is .

[0030] In one embodiment, in step 6, and are calculated as follows: ; ;

[0031] where, represents the spatial coordinates in the image plane and is used to identify the position of the pixel in the image, represents the spatial frequency of the projected fringe, , and are the relative image intensity distributions of the initial phase α after calibration with a black and white board under the selected spatial frequency illumination mode, and α is 0, and ; is the dark noise of the system, and the dark noise of the system is regarded as 0;

[0032] The calculation formula for the diffuse reflectance of the image is as follows: ;

[0033] where, is the diffuse reflectance of the image, is the amplitude envelope of the reflected photon density, is the amplitude envelope of the reflected photon density of the standard white board, is the diffuse reflectance of the standard white board;

[0034] Substitute the results of and into the formula respectively, and obtain and numerical values.

[0035] The beneficial effects of the present invention are as follows: Compared with the prior art, in the present invention, the optical properties of fruits at different spatial frequencies are predicted through a single multi-frequency image. By means of an improved U-Net network combined with the FSA attention mechanism, the adaptive separation of multi-frequency fringe signals is realized, effectively solving the problem of spectral leakage in traditional methods. Further, the GAN network is used to perform high-fidelity reconstruction and adversarial optimization on the separated frequency components, ensuring the quantitative accuracy of the optical property parameters. Applying the predicted optical property parameters to the analysis of the internal structure of fruits can more comprehensively and uniformly reflect the optical property changes of the internal microstructural defects of fruits, providing strong support for high-speed online detection and quality evaluation in the actual industry, and solving the deficiencies in real-time performance, anti-interference ability, and information integrity in traditional detection methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the implementation process of the present invention.

[0037] Figure 2 It is a schematic diagram of making a data set in the present invention.

[0038] Figure 3 It is a comparison chart of the predicted result of the optical property and the true value in the present invention.

[0039] Figure 4 It is the first-layer network architecture diagram of the present invention.

[0040] Figure 5 It is the architecture diagram of the second-layer GAN network generator in the present invention. Detailed implementation manners

[0041] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "front", "upper", "lower", "left", "right", "vertical", "horizontal", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0042] In order to solve the problems existing in the background technology, the present application proposes the following technical solution: A method for predicting the optical properties of fruits based on a single multi-frequency image with a double-layer network, specifically including the following steps:

[0043] Step 1: The host computer generates 11 structured light fringe patterns, including a completely black picture, a completely white picture, three groups of double-frequency orthogonal fringe patterns, and six single-frequency fringe patterns;

[0044] In Step 1, the double-frequency orthogonal fringe pattern is formed by the vertical superposition of the spatial frequencies and sinusoidal fringes in the spatial domain, where is the spatial frequency corresponding to the horizontal fringe pattern, is the spatial frequency corresponding to the vertical fringe pattern, and the two groups of fringes are arranged orthogonally. Each group of frequencies uses the same initial phase α. At the same time, for the single spatial frequency and the single spatial frequency respectively generated three single-frequency fringe patterns, and α is 0, and respectively. Finally, the fringe pattern is projected onto the fruit surface through a projector.

[0045] Step 2: Use a CCD camera to synchronously collect the reflected light intensity distribution on the fruit surface under fringe illumination with the projection system. The collected image is saved in the BMP format and named according to the regulations;

[0046] Step 3: Project the all - black and all - white images onto the standard diffuser plate respectively, and collect the dark field of the standard reference light field with a CCD camera and the white reference plate light intensity distribution. Use the following formula to correct the black - and - white plate for the collected fringe image: ;

[0047] where, is the corrected image intensity distribution, is the image intensity distribution before correction, is the dark - field image intensity distribution, is the image intensity distribution of the reference white plate;

[0048] Step 4: Calculate the pixel ratio of the corrected multi - frequency fringe image of the fruit and the corresponding corrected multi - frequency fringe image of the standard diffuser plate and assign the obtained ratio image to the first channel of the input image, and set the other two channels to zero, thus forming the input image for subsequent network processing; that is, the channel format of the input image is as Figure 2 shown, which is ;

[0049] Step 5: For the single - frequency fringe image, first perform ratio operations on the corrected horizontal fringe image and the vertical fringe image with the standard diffuser plate image respectively; the result of the horizontal fringe ratio image is assigned to the first channel, the result of the vertical fringe ratio image is assigned to the second channel, and the third channel is set to zero; this image is used as the output of the first - layer network, and the channel composition is as Figure 2 shown, which is .

[0050] Step 6: Obtain the optical characteristic values corresponding to the single - frequency fringe image, and use the light intensity distribution data after black - and - white plate correction to perform three - phase demodulation to obtain the amplitude envelopes of the reflected photon densities of the AC and DC components and , and the three - phase demodulation formula is calculated by the following formula: ; ;

[0051] where, represents the spatial coordinates in the image plane, used to identify the position of the pixel in the image, represents the spatial frequency of the projected fringe, , and are the initial phases α (taking 0, and The relative image intensity distribution under is the dark noise of the system, and the dark noise of the system is regarded as 0;

[0052] The true diffuse reflectance of the DC component and the AC component is obtained by comparing with the standard whiteboard and are both calculated and implemented through the following formula: ;

[0053] where is the diffuse reflectance of the image, is the amplitude envelope of the reflected photon density, is the amplitude envelope of the reflected photon density of the standard whiteboard, is the diffuse reflectance of the standard whiteboard;

[0054] Substitute and into respectively, and get and .

[0055] Input the obtained values into the LUT, and further derive the optical characteristic parameters of the sample under SFDI based on the LUT method, namely: absorption coefficient and reduced scattering coefficient .

[0056] Step 7: For the reduced scattering coefficient model: Place the value corresponding to the spatial frequency in the first channel of the output image; Place the value corresponding to in the second channel, and set the third channel to 0 as the output of the second layer model, that is, the channel composition is as shown in Figure 2 , which is . Similarly, the output of the absorption coefficient model is constructed in the same way, and the output image channel composition is .

[0057] Step 8: Construct a two-layer network architecture:

[0058] The first layer network: Adopt a U-net structure combined with the FSA attention mechanism. Embed the FSA attention mechanism in each upsampling and downsampling module. Its architecture diagram is as shown in Figure 4 .

[0059] The second layer network: An optimized network based on the generative adversarial network (GAN), its generator is as shown in Figure 5 , adopt a ResNet-U-net structure, and replace the traditional U-net skip connection with feature addition.

[0060] Step 9. Set the network model parameters: Set the input network image size to 600×600, and parameters such as batch size, epoch, and learning rate selection; Set the network to be trained using GPU, and the specific parameters are as follows:

[0061] The batch size parameter is 4, the epoch parameter is 200, and the learning rate is 0.0002.

[0062] Step 10. Train the double-layer optical property network model according to the parameters set in Step 9. During the training process, save the model with the highest accuracy in each round to the file best_model.pt. The first layer of the network outputs the multi-frequency separation result; the second layer of the network outputs or the prediction results of. After the model training is completed, load best_model.pt through the test program for prediction, and the output results include the prediction image and the normalized mean absolute error (NMAE), and its calculation formula is as follows: ;

[0063] where and are the predicted value and the reference value respectively, and T is the total number of pixels.

[0064] Step 11. Separate Channel 1 and Channel 2 of the prediction image to generate the optical property map corresponding to the spatial frequency as Figure 3 shown, and it can be seen that the relative error between the generated result and the true value is small, and the optical properties of the fruit can be well predicted.

[0065] In summary, the present invention realizes the prediction of the optical properties of fruits at different spatial frequencies through a single multi-frequency image. With the improved U-Net network combined with the FSA attention mechanism, the adaptive separation of multi-frequency fringe signals is realized, effectively solving the problem of spectral leakage in traditional methods. Further, the GAN network is used to perform high-fidelity reconstruction and adversarial optimization on the separated frequency components to ensure the quantitative accuracy of the optical property parameters ( and ). Applying the predicted optical property parameters to the analysis of the internal structure of fruits can more comprehensively and accurately reflect the optical property changes of the internal microstructural defects of fruits, providing strong support for high-speed online detection and quality evaluation in the actual industry.

[0066] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting the optical properties of fruits based on a single multi - frequency image, characterized in that: Step 1: The host computer generates multiple structured light fringe patterns, including a completely black image, a completely white image, a single - frequency fringe pattern, and a double - frequency orthogonal fringe pattern, and then projects the fringe patterns onto the fruit surface; Step 2: Use a camera to collect the reflected light intensity distribution on the fruit surface under fringe illumination; Step 3: Project full-black and full-white images onto a standard diffuser plate respectively, and acquire the dark field of the standard reference light field by a camera, and the white board light intensity distribution, and perform black-and-white board calibration; Step 4: The corrected multi-frequency fringe image of the fruit and the corresponding corrected multi-frequency fringe image of the standard diffuser plate are calculated according to the pixel ratio, and the obtained ratio image is assigned to the first channel of the input image, and the remaining two channels are set to zero, thereby constructing the input image for subsequent network processing; Step 5. For the single-frequency fringe image, perform ratio operations on the corrected horizontal fringe image and the vertical fringe image and the standard diffuse reflection plate image respectively. Assign the result of the horizontal fringe ratio image to the first channel, the result of the vertical fringe ratio image to the second channel, and set the third channel to zero; Step 6: Obtain the optical characteristic values corresponding to the single-frequency fringe image, perform three-phase demodulation on the light intensity distribution data after black-and-white board correction, and derive the optical characteristic parameters of the sample under SFDI through the LUT, namely: absorption coefficient and reduced scattering coefficient ; Step 7: Place the value corresponding to the spatial frequency in the first channel of the output image; place the value corresponding to in the second channel, set the third channel to zero, and use this as the output of the second-layer model, i.e., the channel composition is . The output of the absorption coefficient model is also constructed in the same way, and the channel composition of the output image is . Step 8: Construct a double - layer network architecture.

2. The method for predicting the optical properties of fruits based on a single multi - frequency image according to claim 1, characterized in that: In step 8, the specific method for constructing the double - layer network architecture is as follows: The first - layer network: Adopt a U - net structure combined with the FSA attention mechanism, and embed the FSA attention mechanism in each up - sampling and down - sampling module; The second - layer network: An optimized network based on a generative adversarial network, whose generator adopts a ResNet - U - net structure, and uses feature addition to replace the traditional U - net skip connection; Set network model parameters: Set the input network image size to 600×600, select parameters for batch size, epoch, and learning rate, and set the network to be trained using a GPU. The specific parameters are as follows: The batch size parameter is 4, the epoch parameter is 200, and the learning rate is 0.0002; Train the double - layer optical property network model according to the set network model parameters. During the training process, save the model with the highest accuracy in each round to the file best_model.pt; The first - layer network outputs the multi - frequency separation result; The second-layer network outputs the prediction results of or respectively according to the training objective; After the model training is completed, the best_model.pt is loaded through the test program for prediction. The output results include the predicted image and the normalized mean absolute error NMAE. The calculation formula of NMAE is as follows: ; Among them, and are the predicted value and the reference value respectively, and T is the total number of pixels.

3. The method for predicting the optical properties of fruits based on a single multi - frequency image according to claim 1, characterized in that: In Step 1, the dual-frequency orthogonal fringe pattern is formed by the vertical superposition of sine fringes with spatial frequencies and in the spatial domain, where is the spatial frequency corresponding to the transverse fringe pattern, is the spatial frequency corresponding to the longitudinal fringe pattern. The two sets of fringes are orthogonally arranged, and each set of frequencies uses the same initial phase α. At the same time, for a single spatial frequency and a single spatial frequency , three single-frequency fringe patterns are generated respectively, and α is 0, and respectively.

4. The method for predicting the optical properties of fruits based on a single multi - frequency image according to claim 1, characterized in that: In step 3, the collected fringe image is corrected for black and white plates using the following formula: ; Among them, is the corrected image intensity distribution, is the image intensity distribution before correction, is the dark field image intensity distribution, is the image intensity distribution of the reference white board.

5. A method for predicting the optical properties of fruits using a double-layer network based on a single multi-frequency image according to claim 1, characterized in that, In step 4, the channel format of the input image is .

6. The method for predicting the optical properties of fruits based on a single multi - frequency image according to claim 1, characterized in that: In step 5, the single-frequency fringe image is used as the output of the first-layer network, and the channel composition is .

7. The method for predicting the optical properties of fruits based on a single multi - frequency image according to claim 1, characterized in that: In step 6, and are calculated as follows: ; ； Among them, represents the spatial coordinates in the image plane and is used to identify the position of the pixel in the image. represents the spatial frequency of the projected fringe. , and are respectively the relative image intensity distributions of the initial phase α after calibration with a black-and-white board under the selected spatial frequency illumination mode. α is 0, and ; is the dark noise of the system, and the dark noise of the system is regarded as 0. The calculation formula for the diffuse reflectance of the image is as follows: ; wherein, is the diffuse reflectance of the image, is the amplitude envelope of the reflected photon density, is the amplitude envelope of the reflected photon density of the standard white board, is the diffuse reflectance of the standard white board; Substitute and into the formula respectively, and obtain and numerical values.

Images