Method and device for detecting quality of fruit, electronic equipment and storage medium

Abstract

The application provides a fruit quality detection method and device, electronic equipment and storage medium, and is applied to a fruit image acquisition module. The fruit image acquisition module comprises a micro-nano structure modulation layer and an image sensor layer. The micro-nano structure modulation layer comprises a plurality of micro-nano structure units. The method comprises the following steps: obtaining a to-be-processed image of a to-be-detected fruit based on the fruit image acquisition module; the to-be-processed image is a gray value image formed by modulating light information reflected by the to-be-detected fruit based on the micro-nano structure modulation layer, and processing the modulated light information based on the image sensor layer; determining characteristic values corresponding to the micro-nano structure units based on the coordinate positions and the gray values of each pixel point in the to-be-processed image; and inputting the characteristic values into a fruit quality detection model to obtain a fruit quality detection result. The fruit quality can be detected with low cost, non-destructively and high precision.

Description

Technical Field

[0001] This invention relates to the field of fruit quality testing technology, and in particular to a fruit quality testing method, apparatus, electronic device, and storage medium. Background Technology

[0002] As residents' consumption levels rise, consumers are demanding higher and higher quality fruits. In terms of appearance, consumers pay attention to the size, shape, color, and presence of diseases or blemishes. At the same time, they are also paying more attention to the internal quality of fruits, including their sugar content, acidity, and firmness.

[0003] According to relevant technologies, internal quality testing and grading of fruits requires juicing and destructive testing methods to obtain sugar or acidity values, resulting in long testing times and high costs. Visible / near-infrared spectroscopy can achieve non-destructive testing of fruits, but its use is limited by the high cost of spectroscopic detection equipment and the severe overlap and strong collinearity in the near-infrared spectral region, making one-to-one feature assignment difficult. This means that the concentration of substances inside the fruit does not have fixed absorption peak positions, leading to modeling difficulties and significant influence from the surrounding environment and sample variations.

[0004] Therefore, finding a low-cost, non-destructive, and highly accurate method for fruit quality testing has become a current research hotspot. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for fruit quality testing, enabling low-cost, non-destructive, and high-precision testing of fruit quality.

[0006] This invention provides a fruit quality detection method applied to a fruit image acquisition module. The fruit image acquisition module includes at least a micro / nano structure modulation layer and an image sensor layer. The micro / nano structure modulation layer partially covers the image sensor layer and includes multiple micro / nano structure units. The fruit quality detection method includes: acquiring a processed image of the fruit to be detected based on the fruit image acquisition module, wherein the processed image is a grayscale image formed by modulating the light information reflected from the fruit to be detected based on the micro / nano structure modulation layer to obtain modulated light information, and processing the modulated light information based on the image sensor layer; determining the coordinate positions of the micro / nano structure units on the image sensor layer; determining feature values ​​corresponding to the micro / nano structure units based on the coordinate positions and the grayscale values ​​of each pixel in the processed image; acquiring a pre-determined fruit quality detection model; and inputting the feature values ​​into the fruit quality detection model to obtain the fruit quality detection result of the fruit to be detected output by the fruit quality detection model.

[0007] According to a fruit quality detection method provided by the present invention, the step of determining the feature value corresponding to the micro / nano structure unit based on the coordinate position and the gray value of each pixel in the image to be processed specifically includes: determining a unit region image corresponding to each of the micro / nano structure units in the image to be processed based on the coordinate position; determining a target unit region image in a preset region in the unit region image; and determining the feature value corresponding to the micro / nano structure unit based on the average gray value of each pixel in the target unit region image.

[0008] According to a fruit quality detection method provided by the present invention, before inputting the feature values ​​into the fruit quality detection model, the fruit quality detection method further includes: filtering the feature values ​​to obtain target feature values, wherein the target feature values ​​are used to characterize feature values ​​whose influence on the fruit quality of the fruit to be detected exceeds an influence threshold; the step of inputting the feature values ​​into the fruit quality detection model to obtain the fruit quality detection result of the fruit to be detected output by the fruit quality detection model specifically includes: inputting the target feature values ​​into the fruit quality detection model to obtain the fruit quality detection result of the fruit to be detected output by the fruit quality detection model.

[0009] According to a fruit quality detection method provided by the present invention, the step of filtering the feature values ​​to obtain target feature values ​​specifically includes: acquiring multiple fruit samples to be tested; sampling the fruit samples to be tested according to a first preset sampling ratio to obtain a sample training set, wherein the sample training set includes a preset number of fruit samples to be tested; randomly sampling the feature values ​​corresponding to the preset number of fruit samples to be tested according to a second preset sampling ratio to obtain a training set, wherein the training set includes training feature values, which are feature values ​​sampled from the feature values ​​corresponding to the preset number of fruit samples to be tested; determining the sampling rounds; iteratively sampling the training feature values ​​in the training set based on the sampling rounds to obtain a subset of variables sampled in each sampling round, wherein the subset of variables includes sampled feature values ​​sampled from the training feature values; filtering the subset of variables sampled in each sampling round to obtain a target subset of variables; and obtaining the target feature value based on the target subset of variables.

[0010] According to a fruit quality detection method provided by the present invention, the step of iteratively sampling the training feature values ​​in the training set according to a second preset sampling ratio based on the sampling rounds to obtain a subset of variables sampled in each sampling round specifically includes: determining an initial subset of variables for each sampling, wherein the initial subset of variables for the later sampling is the subset of variables obtained in the previous sampling, and the initial subset of variables for the first sampling is a set composed of the training feature values; constructing a first partial least squares regression model corresponding to the initial subset of variables based on the initial subset of variables; determining the absolute value of the regression coefficient corresponding to each feature value in the initial subset of variables based on the first partial least squares regression model; selecting retained feature values ​​from the feature values ​​based on the absolute value of the regression coefficient; and obtaining a subset of variables sampled in each sampling round based on the retained feature values.

[0011] According to a fruit quality detection method provided by the present invention, the step of selecting retained feature values ​​from the feature values ​​based on the absolute values ​​of the regression coefficients specifically includes: sorting the absolute values ​​of the regression coefficients in descending order to obtain a sorted set; selecting a preset number of absolute values ​​of regression coefficients that are ranked first in the sorted set; and selecting retained feature values ​​from the feature values ​​based on the preset number of absolute values ​​of regression coefficients that are ranked first.

[0012] According to a fruit quality detection method provided by the present invention, the step of selecting a target variable subset based on the variable subsets sampled in each sampling round specifically includes: constructing a second partial least squares regression model corresponding to the variable subsets sampled in each sampling round; determining the root mean square error (RMSE) of the cross-mean square error corresponding to the variable subset based on the second partial least squares regression model; determining the minimum RMSE among the RMSEs, and taking the variable subset corresponding to the minimum RMSE as the target variable subset.

[0013] This invention also provides a fruit quality detection device applied to a fruit image acquisition module. The fruit image acquisition module includes at least a micro / nano structure modulation layer and an image sensor layer. The micro / nano structure modulation layer partially covers the image sensor layer. The micro / nano structure modulation layer includes multiple micro / nano structure units. The fruit quality detection device includes: a first module for acquiring a processed image of the fruit to be detected acquired by the fruit image acquisition module, wherein the processed image is a grayscale image formed by modulating the light information reflected from the fruit to be detected by the micro / nano structure modulation layer to obtain modulated light information, and processing the modulated light information by the image sensor layer; a second module for determining the coordinate positions of the micro / nano structure units in the image sensor layer; a third module for determining feature values ​​corresponding to the micro / nano structure units based on the coordinate positions and the grayscale values ​​of each pixel in the processed image; a fourth module for acquiring a pre-determined fruit quality detection model; and a fifth module for inputting the feature values ​​into the fruit quality detection model to obtain the fruit quality detection result of the fruit to be detected output by the fruit quality detection model.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the fruit quality detection method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the fruit quality detection method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the fruit quality detection method as described above.

[0017] The fruit quality detection method, apparatus, electronic device, and storage medium provided by this invention acquire a processed image of the fruit to be detected based on a fruit image acquisition module. The processed image is a grayscale image formed by modulating the light information reflected from the fruit by a micro / nano structure modulation layer, and then processing the modulated light information by an image sensor layer. In this invention, the modulation of the emitted light from the fruit by the micro / nano structure modulation layer allows the grayscale image formed by processing the modulated light information by the image sensor layer to carry information about the internal composition of the fruit, laying the foundation for obtaining high-precision fruit quality detection results. Based on the coordinate position and the grayscale value of each pixel in the processed image, feature values ​​corresponding to the micro / nano structure units are determined. These feature values ​​are input into a fruit quality detection model, and the model outputs the fruit quality detection result of the fruit. This enables low-cost, non-destructive, and high-precision fruit quality detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the fruit quality testing method provided by the present invention;

[0020] Figure 2 This is a schematic diagram of the structure of the fruit image acquisition module provided by the present invention;

[0021] Figure 3 This is a flowchart illustrating the process of determining the feature values ​​corresponding to micro / nano structural units based on coordinate positions and grayscale values ​​of each pixel in the image to be processed, as provided by the present invention.

[0022] Figure 4 This is a schematic diagram illustrating an application scenario for pixel selection below the micro / nano structure unit provided by the present invention;

[0023] Figure 5 This is a flowchart illustrating the process of filtering feature values ​​to obtain target feature values ​​provided by the present invention.

[0024] Figure 6 This invention provides a flowchart illustrating the process of iteratively sampling training feature values ​​in the training set based on sampling rounds to obtain a subset of variables sampled in each sampling round.

[0025] Figure 7This is a flowchart illustrating the process of obtaining a target variable subset based on the variable subsets sampled in each sampling round, as provided by the present invention.

[0026] Figure 8 This is a schematic diagram of the fruit quality testing device provided by the present invention;

[0027] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0028] Figure label:

[0029] 1: Fruit image sampling module; 2: Micro-nano structure modulation layer;

[0030] 3: Image sensor layer. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0032] The fruit quality detection method provided by this invention can be applied to fruit image acquisition modules.

[0033] Figure 2 This is a schematic diagram of the structure of the fruit image acquisition module provided by the present invention.

[0034] Combination Figure 2 It is understood that the fruit image acquisition module 1 may include at least a micro / nano structure modulation layer 2 and an image sensor layer 3. The micro / nano structure modulation layer 2 partially covers the image sensor layer 3, and the micro / nano structure modulation layer 2 may include multiple micro / nano structure units. In one embodiment, the micro / nano structure modulation layer 2 can cover a portion of the image sensor layer 3 by direct deposition or bonding.

[0035] It should be noted that micro- and nano-structure units can have different shapes and can modulate the spectrum of incident light in different ways.

[0036] To further introduce the fruit quality testing method provided by this invention, the following will be combined with... Figure 1 Please provide an explanation.

[0037] Figure 1 This is one of the flowcharts of the fruit quality testing method provided by the present invention.

[0038] In an exemplary embodiment of the present invention, combined with Figure 1As can be seen, the fruit quality testing method may include steps 110 to 150, and each step will be described below.

[0039] In step 110, an image of the fruit to be detected, acquired by the fruit image acquisition module, is obtained. This image is a grayscale image formed by modulating the light reflected from the fruit using a micro / nano structure modulation layer to obtain modulated light information, and then processing the modulated light information using an image sensor layer.

[0040] In one embodiment, a processing image of the fruit to be tested can be acquired. During application, an active light source can be applied to the fruit to be tested, and the light information reflected by the fruit can be obtained through this applied active light source. Further, the light information reflected by the fruit to be tested can be modulated based on a micro / nano structure modulation layer to obtain modulated light information, and the modulated light information can be processed based on an image sensor layer to form a processing image composed of the grayscale values ​​of each pixel. In this invention, by modulating the emitted light from the fruit to be tested through a micro / nano structure modulation layer, the grayscale image formed by processing the modulated light information based on the image sensor layer carries information about the internal material composition of the fruit to be tested, laying the foundation for obtaining high-precision fruit quality detection results.

[0041] In step 120, the coordinate positions of the micro / nano structure units in the image sensor layer are determined.

[0042] In step 130, feature values ​​corresponding to micro / nano structural units are determined based on coordinate positions and grayscale values ​​of each pixel in the image to be processed.

[0043] It should be noted that, since the micro-nano structure modulation layer only covers a part of the image sensor layer, and the image information after homogenization and without modulation does not have effective information, it is necessary to select the image sensor data, that is, to further select the gray values ​​of each pixel in the image to be processed corresponding to the region of the image sensor layer below the micro-nano structure modulation layer.

[0044] In one embodiment, the coordinate positions of each micro / nano structure unit in the image sensor layer can be determined. Furthermore, based on the coordinate positions and the gray values ​​of each pixel in the image to be processed, the feature values ​​corresponding to the micro / nano structure units can be determined.

[0045] Figure 3 This is a flowchart illustrating the process of determining the feature values ​​corresponding to micro / nano structural units based on their coordinate positions and the grayscale values ​​of each pixel in the image to be processed, as provided by the present invention.

[0046] In an exemplary embodiment of the present invention, combined with Figure 3As can be seen, determining the feature value corresponding to the micro / nano structure unit based on the coordinate position and the gray value of each pixel in the image to be processed can include steps 310 to 330, and each step will be described below.

[0047] In step 310, based on coordinate positions, the unit region images corresponding to each micro / nano structure unit are determined in the image to be processed.

[0048] In step 320, the target cell region image is determined in a preset region of the cell region image.

[0049] In step 330, the feature value corresponding to the micro / nano structure unit is determined based on the average gray value of each pixel in the target unit region image.

[0050] In one embodiment, the internal quality detection data of the fruit to be tested comes from the image data output by the fruit image acquisition module (corresponding to the image to be processed). Here, H and W represent the height and width of the image data, respectively. In the application, a monochrome camera is used, so the number of channels is 1.

[0051] Combination Figure 4 As can be seen, since the micro / nano structure modulation layer does not completely cover the surface of the image sensor layer, only a portion of the values ​​in X are obtained by modulating the optical signal and then summing the weighted signals by the image sensor layer to convert them into photocurrent signals. However, the gray values ​​without frequency domain weighting reflect the incident light intensity after homogenization, which is equivalent to summing the entire wavelength optical signal. This results in a high noise level and lacks the functions of filtering, modulation, and feature extraction. Therefore, it is necessary to remove these noise values ​​during data preprocessing, i.e., to determine the eigenvalues ​​corresponding to the micro / nano structure units.

[0052] In one embodiment, the pixel value below each micro / nanostructure unit can be accurately located based on the coordinate positions of each micro / nanostructure unit on the image sensor layer, thus determining the unit region image corresponding to the micro / nanostructure unit. In one example, the coordinate position can be represented as (h, w), where h ∈ H and w ∈ W, and the gray value corresponding to the pixel at the corresponding position can be represented as x. (h,w) .

[0053] Since there isn't a one-to-one correspondence between a micro / nanostructure unit and the pixel values ​​below it—meaning that a single micro / nanostructure unit can cover multiple image pixels—micro / nanostructure units are often fabricated as squares with varying internal shapes, covering d×d unit pixels below them (corresponding to the unit region image corresponding to the micro / nanostructure unit). Furthermore, based on the coordinate position of the micro / nanostructure unit, the d pixels below each micro / nanostructure unit can be selected. ′ ×d ′Pixel values, where d ′ ≤d, meaning the target cell region image is determined within a preset region in the cell region image. The preset region can be determined based on actual conditions. In one example, to prevent interference between cells, d can be set to a preset region. ′ Set to less than or equal to d / 2, and start selecting from the top left corner of each cell.

[0054] Furthermore, d can be ′ ×d ′ The average value x′ of the gray values ​​of each pixel is used as the feature value of the micro / nano structure unit. In one example, the feature value corresponding to the micro / nano structure unit can be determined based on the average gray value of each pixel in the target unit region image.

[0055] In step 140, a predetermined fruit quality detection model is obtained.

[0056] In step 150, the feature values ​​are input into the fruit quality detection model to obtain the fruit quality detection results of the fruit to be detected.

[0057] In one embodiment, a fruit quality detection model can be predetermined. In one instance, the fruit quality detection model can be a partial least squares regression model, or other machine learning models, such as a support vector machine (SVM) model. In this embodiment, no specific limitation is made on the fruit quality detection model.

[0058] In another embodiment, the fruit quality detection model can also be determined in the following manner:

[0059] Obtain the model training set, which includes multiple grayscale image samples. The grayscale image samples are the corresponding grayscale images obtained by the fruit image acquisition module based on multiple fruit samples to be detected.

[0060] Based on grayscale image samples, determine the feature values ​​of the grayscale image samples;

[0061] Based on the feature values ​​of grayscale image samples, the initial fruit quality detection model is trained to obtain the trained fruit quality detection model, which is then used as the final fruit quality detection model.

[0062] It should be noted that the process of determining the feature values ​​of grayscale image samples is the same as or similar to the process of determining the feature values ​​corresponding to micro / nano structural units described above, and will not be repeated in this embodiment.

[0063] In application, the feature values ​​corresponding to the micro / nano structure units can be input into the fruit quality detection model, thereby obtaining the fruit quality detection result of the fruit to be detected output by the fruit quality detection model. In this embodiment, since the micro / nano structure modulation layer modulates the emitted light of the fruit to be detected, the grayscale image formed by processing the modulated light information based on the image sensor layer carries information about the internal composition of the fruit to be detected. Consequently, the feature values ​​obtained based on the grayscale image carry information about the internal composition of the fruit to be detected. In the process of inputting the feature values ​​into the fruit quality detection model to obtain the fruit quality detection result, low-cost, non-damaging, and high-precision fruit quality detection is achieved.

[0064] The fruit quality detection method provided by this invention acquires a processed image of the fruit to be detected based on a fruit image acquisition module. This processed image is a grayscale image formed by modulating the light reflected from the fruit using a micro / nano structure modulation layer, and then processing the modulated light information using an image sensor layer. In this invention, the modulation of the emitted light from the fruit using a micro / nano structure modulation layer allows the grayscale image formed by processing the modulated light information using the image sensor layer to carry information about the internal composition of the fruit, laying the foundation for obtaining high-precision fruit quality detection. Based on the coordinate positions and grayscale values ​​of each pixel in the processed image, feature values ​​corresponding to the micro / nano structure units are determined. These feature values ​​are input into a fruit quality detection model, which outputs the fruit quality detection result of the fruit. This method enables low-cost, non-destructive, and high-precision fruit quality detection.

[0065] To further explain the fruit quality testing method provided by the present invention, the following embodiments will be used for illustration.

[0066] In an exemplary embodiment of the present invention, continuing with Figure 1 The above embodiment is used as an example for illustration. Before inputting the feature values ​​into the fruit quality detection model (corresponding to step 150), the fruit quality detection method also includes the following steps:

[0067] The feature values ​​are filtered to obtain the target feature values, which are used to characterize the feature values ​​whose influence on the quality of the fruit to be tested exceeds the influence threshold.

[0068] Furthermore, the feature values ​​are input into the fruit quality detection model to obtain the fruit quality detection results of the fruit to be tested, which can be achieved in the following way:

[0069] The target feature value is input into the fruit quality detection model, and the fruit quality detection result of the fruit to be tested is obtained from the output of the fruit quality detection model.

[0070] In one embodiment, the combination continues. Figure 4 The above embodiment is used as an example for illustration. After processing the feature values ​​corresponding to each micro / nano structure unit, a new set of data X can be obtained. ′ , where X ′ The d below each micro / nano structure unit ′ ×d ′ The average gray value (corresponding to the characteristic value of the micro / nano structure unit) is composed of the average gray value, which represents the photocurrent value after broadband filtering modulation. Different shapes of structures have different modulation effects, which are reflected in different photocurrent values.

[0071] However, micro- and nano-structure units of different shapes have different frequency domain modulation effects. In the field of fruit detection, the high-frequency band carries more information about the concentration of internal substances, which is reflected in the intensity changes of the absorption peaks of related functional groups. Therefore, it is necessary to further select the gray value (corresponding feature value) information below the micro- and nano-structure units. This can not only increase the signal-to-noise ratio of effective information, but also reduce the interference of redundant information, thereby enhancing the robustness and generalization ability of the model.

[0072] In one embodiment, feature values ​​can be filtered to obtain target feature values, wherein the target feature values ​​are used to characterize feature values ​​whose influence on the quality of the fruit to be tested exceeds an influence threshold. The influence threshold can be adjusted according to actual circumstances and is not specifically limited in this embodiment. In other words, the target feature value can be considered as a feature value that carries a large amount of information about the quality indicators of the fruit to be tested.

[0073] Furthermore, the target feature values ​​can be input into the fruit quality detection model to obtain the fruit quality detection results of the fruit to be tested, as output by the fruit quality detection model. In this embodiment, determining the fruit quality detection results based on the target feature values ​​can increase the accuracy of the detection results.

[0074] Figure 5 This is a schematic diagram of the process of filtering feature values ​​to obtain target feature values ​​provided by the present invention.

[0075] The following will combine Figure 5 The process of obtaining the target feature values ​​is explained.

[0076] In an exemplary embodiment of the present invention, combined with Figure 5 As can be seen, filtering the feature values ​​to obtain the target feature values ​​may include steps 510 to 570, and each step will be described below.

[0077] In step 510, multiple fruit samples to be tested are obtained.

[0078] In step 520, samples of the fruit to be tested are sampled according to a first preset sampling ratio to obtain a sample training set, wherein the sample training set includes a preset number of fruit samples to be tested. In step 530, feature values ​​corresponding to the preset number of fruit samples to be tested are randomly sampled according to a second preset sampling ratio to obtain a training set, wherein the training set includes training feature values, which are feature values ​​sampled from the feature values ​​corresponding to the preset number of fruit samples to be tested.

[0079] In one embodiment, the first preset sampling ratio can be determined according to actual conditions. In one example, the first preset sampling ratio can be 80%. This embodiment does not specifically limit the preset sampling ratio. During application, the samples to be tested can be sampled according to the first preset sampling ratio to obtain a sample training set. The sample training set includes a preset number of fruit samples to be tested. The preset number can be determined according to the first preset sampling ratio.

[0080] Furthermore, according to the second preset sampling ratio, the feature values ​​corresponding to the preset number of fruit samples to be tested are randomly sampled to obtain a training set. It is understood that the training set includes multiple training feature values, where the training feature values ​​are those sampled from the feature values ​​corresponding to the preset number of fruit samples to be tested. In this embodiment, the purpose of not using all feature values ​​is to ensure that the selected variables (corresponding to a subset of target variables) still have high adaptability when the samples change. It should be noted that the method for determining the feature values ​​corresponding to the preset number of fruit samples to be tested is the same as or similar to the method for determining the feature values ​​corresponding to the micro / nano structure units described above, and will not be repeated in this embodiment. The second preset sampling ratio can also be adjusted according to actual conditions; in this embodiment, it is not specifically limited.

[0081] In one example, if the feature values ​​corresponding to a predetermined number of fruit samples to be tested constitute a data matrix X′, where the data matrix X′ includes n samples (i.e., n fruit samples to be tested), and each sample contains m variables (corresponding to feature values), where each variable represents the photocurrent value modulated by a micro / nano structure, the data dimension of X′ can be represented as n×m, and the corresponding training set is obtained by sampling n′ from X′. i ×m′ i , where n′ i ∈n, m′ i ∈m.

[0082] In step 540, the sampling round is determined.

[0083] In step 550, based on the sampling rounds, the training feature values ​​in the training set are iteratively sampled to obtain a subset of variables sampled in each sampling round, wherein the subset of variables includes the sampled feature values ​​sampled from the training feature values.

[0084] In one embodiment, the sampling rounds can be determined. Further, based on the sampling rounds, the training feature values ​​in the training set are iteratively sampled to obtain a subset of variables sampled in each sampling round.

[0085] In step 560, a subset of target variables is obtained by filtering based on the subset of variables sampled in each sampling round.

[0086] Figure 7 This is a flowchart illustrating the process of obtaining a target variable subset based on the variable subsets sampled in each sampling round, as provided by the present invention.

[0087] In an exemplary embodiment of the present invention, combined with Figure 7 As can be seen, the selection of the target variable subset based on the variable subsets sampled in each sampling round can include steps 710 to 730, and each step will be described below.

[0088] In step 710, a second partial least squares regression model corresponding to the subset of variables is constructed based on the subset of variables sampled in each sampling round.

[0089] In step 720, based on the second partial least squares regression model, the cross root mean square error corresponding to the variable subset is determined.

[0090] In step 730, the minimum cross-mean square error is determined from the cross-mean square errors, and the subset of variables corresponding to the minimum cross-mean square error is taken as the target variable subset.

[0091] In one embodiment, the number of sampling rounds can be S. The number of sampling rounds can be adjusted according to actual conditions, and is not specifically limited in this embodiment.

[0092] In the application process, S samplings will generate S subsets of variables. Further, a second partial least squares regression model corresponding to each subset of variables can be constructed; then, based on the second partial least squares regression model, the root mean square error (RMSE) corresponding to each subset of variables can be determined. In the application process, the minimum RMSE can be determined from the RMSEs, and the subset of variables corresponding to the minimum RMSE is taken as the target subset of variables. In this embodiment, the selection of structural variables (corresponding to the target subset of variables) is completed; that is, the subset of variables that minimizes the model's RMSE indicates that this subset of variables (corresponding to the target subset of variables) has a stronger explanatory power for the dependent variable and is better able to predict the trend of the dependent variable's changes. This ensures that the fruit quality detection results obtained based on the target feature values ​​are more accurate. This embodiment can find structural variables (corresponding to the target subset of variables) that carry rich information, thereby establishing a more robust and accurate fruit detection model.

[0093] In step 570, target feature values ​​are obtained based on a subset of target variables.

[0094] In one embodiment, since the subset of target variables is composed of filtered feature values, the feature values ​​constituting the subset of target variables can be referred to as target feature values. It is understood that the target feature values ​​carry key information about the internal composition of the fruit to be tested, thereby ensuring that the fruit quality testing results obtained based on the target feature values ​​are more accurate.

[0095] Figure 6 This is a flowchart illustrating the process provided by the present invention of iteratively sampling training feature values ​​in the training set based on sampling rounds to obtain a subset of variables sampled in each sampling round.

[0096] In an exemplary embodiment of the present invention, combined with Figure 6 As can be seen, based on the sampling rounds, iterative sampling of the training feature values ​​in the training set to obtain the subset of variables sampled in each sampling round can include steps 610 to 650, and each step will be described below.

[0097] In step 610, the initial variable subset for each sampling is determined, wherein the initial variable subset for the subsequent sampling is the variable subset obtained from the previous sampling, and the initial variable subset for the first sampling is a set composed of training feature values.

[0098] It should be noted that the optimal subset of variables can be found through multiple samplings, such as S samplings, where the input data before each sampling is n′. i ×m′ i , where i represents the i-th iteration or sampling, and i∈{1,2,3,...,S}.

[0099] In one embodiment, an initial subset of variables can be determined for each sampling, wherein, for the first sampling, the initial subset of variables is a set consisting of training feature values. For the first sampling, the initial subset of variables V... old The set of variables is composed of the eigenvalues ​​corresponding to all micro / nano structural units, and its number is m. The initial subset of variables for the next sampling is the subset of variables obtained from the previous sampling.

[0100] In step 620, a first partial least squares regression model corresponding to the initial variable subset is constructed based on the initial variable subset.

[0101] In step 630, based on the first partial least squares regression model, the absolute values ​​of the regression coefficients corresponding to each feature value in the initial variable subset are determined.

[0102] In one embodiment, during each sampling process, a first partial least squares regression model corresponding to the initial subset of variables can be constructed based on the initial subset of variables. Furthermore, based on the first partial least squares regression model, the absolute values ​​of the regression coefficients corresponding to each feature value in the initial subset of variables during each sampling process are determined.

[0103] In application, the initial variable subset can be filtered based on the magnitude of the absolute value of the regression coefficients to obtain the variable subset (also known as the newly generated variable subset V) in each sampling process. new Since variables with larger absolute values ​​of regression coefficients in the first partial least squares regression model have a stronger explanatory power for the dependent variable, they are retained, while variables with smaller absolute values ​​of regression coefficients are removed, thus obtaining a subset of variables in each sampling process.

[0104] It should be noted that the first partial least squares regression model and the second partial least squares regression model can be the same model.

[0105] In step 640, based on the absolute value of the regression coefficient, retained feature values ​​are selected from the feature values.

[0106] In an exemplary embodiment of the present invention, the selection of retained feature values ​​from the feature values ​​based on the absolute value of the regression coefficients can be achieved in the following manner:

[0107] The absolute values ​​of the regression coefficients are sorted in descending order to obtain a sorted set;

[0108] Select the absolute values ​​of the regression coefficients of a predetermined number of items from the sorted set;

[0109] Based on the absolute values ​​of the regression coefficients of a predetermined number that are ranked first, the retained feature values ​​are selected from the feature values.

[0110] In one embodiment, the preset quantity can be adjusted according to actual conditions; for example, the preset quantity can be r. i =ke -a Where, k = (m / 2) 1 / S-1 ,

[0111] In step 650, a subset of variables sampled in each sampling round is obtained based on the retained feature values.

[0112] In one embodiment, the set of retained feature values ​​can be used as the subset of variables obtained in each sampling process (also known as the newly generated subset of variables V). new ), where V new =r*V old During application, V can be updated after each round of sampling. old For V new That is, the subset of variables V remaining after the last sampling. new The initial subset of variables V for the next sampling old In this embodiment, the selected subset of variables effectively retains key variables while removing irrelevant or redundant structural information variables, and therefore can be directly used for fruit quality testing.

[0113] It should be noted that this invention uses a micro / nano modulation structure (corresponding to the micro / nano modulation layer) and an image sensor device (corresponding to the image sensor layer) to detect the concentration of internal components in fruits. The incident light is modulated by the micro / nano modulation structure, and the photocurrent value beneath the structure is used for modeling and analysis. This approach is a non-destructive testing method. Using an image sensor and a micro / nano modulation structure enables batch testing of fruits. The selection of data and the choice of structure can improve the model's prediction accuracy, robustness, and generalization ability. Furthermore, the chip can be fabricated in a single CMOS process, reducing hardware costs and facilitating large-scale applications in various scenarios.

[0114] As described above, the fruit quality detection method provided by this invention acquires a processed image of the fruit to be detected based on a fruit image acquisition module. This processed image is a grayscale image formed by modulating the light reflected from the fruit using a micro / nano structure modulation layer, and then processing the modulated light information using an image sensor layer. In this invention, the modulation of the emitted light from the fruit using a micro / nano structure modulation layer allows the grayscale image formed by processing the modulated light information using an image sensor layer to carry information about the internal composition of the fruit, laying the foundation for obtaining high-precision fruit quality detection. Furthermore, based on the coordinate positions and grayscale values ​​of each pixel in the processed image, feature values ​​corresponding to the micro / nano structure units are determined. These feature values ​​are input into a fruit quality detection model, which outputs the fruit quality detection result of the fruit. This method achieves low-cost, non-destructive, and high-precision fruit quality detection.

[0115] Based on the same concept, the present invention also provides a fruit quality testing device.

[0116] The fruit quality testing device provided by the present invention is described below. The fruit quality testing device described below and the fruit quality testing method described above can be referred to in correspondence.

[0117] Figure 8 This is a schematic diagram of the fruit quality testing device provided by the present invention.

[0118] In an exemplary embodiment of the present invention, the fruit quality detection device can be applied to a fruit image acquisition module, wherein the fruit image acquisition module may include at least a micro-nano structure modulation layer and an image sensor layer, the micro-nano structure modulation layer partially covers the image sensor layer, and the micro-nano structure modulation layer may include multiple micro-nano structure units.

[0119] Combination Figure 8 As can be seen, the fruit quality testing device may include the first module 810 to the fifth module 850, and each module will be described in detail below.

[0120] The first module 810 can be configured to acquire a processing image of the fruit to be detected based on the fruit image acquisition module. The processing image is a grayscale image formed by modulating the light information reflected by the fruit to be detected based on the micro-nano structure modulation layer to obtain modulated light information and processing the modulated light information based on the image sensor layer.

[0121] The second module 820 can be configured to determine the coordinate positions of micro / nano structure units in the image sensor layer;

[0122] The third module 830 can be configured to determine the feature values ​​corresponding to the micro / nano structure units based on the coordinate positions and the gray values ​​of each pixel in the image to be processed.

[0123] The fourth module 840 can be configured to acquire a predetermined fruit quality detection model;

[0124] The fifth module 850 can be configured to input feature values ​​into the fruit quality detection model and obtain the fruit quality detection results of the fruit to be detected from the output of the fruit quality detection model.

[0125] In an exemplary embodiment of the present invention, the third module 830 can determine the feature value corresponding to the micro / nano structure unit based on the coordinate position and the gray value of each pixel in the image to be processed in the following manner:

[0126] Based on coordinate positions, the unit region images corresponding to each micro / nano structure unit are determined in the image to be processed;

[0127] The target cell region image is determined within a preset region of the cell region image;

[0128] Based on the average gray value of each pixel in the target unit region image, the feature value corresponding to the micro / nano structure unit is determined.

[0129] In an exemplary embodiment of the present invention, the fifth module 850 may further be configured to:

[0130] The feature values ​​are filtered to obtain the target feature values, which are used to characterize the feature values ​​whose influence on the quality of the fruit to be tested exceeds the influence threshold.

[0131] The fifth module 850 can input feature values ​​into the fruit quality detection model in the following way to obtain the fruit quality detection results of the fruit to be tested, which are output by the fruit quality detection model:

[0132] The target feature value is input into the fruit quality detection model, and the fruit quality detection result of the fruit to be tested is obtained from the output of the fruit quality detection model.

[0133] In an exemplary embodiment of the present invention, the fifth module 850 can filter the feature values ​​to obtain the target feature values ​​in the following manner:

[0134] Obtain multiple fruit samples to be tested;

[0135] According to the first preset sampling ratio, the fruit samples to be tested are sampled to obtain a sample training set, wherein the sample training set includes a preset number of fruit samples to be tested.

[0136] According to the second preset sampling ratio, the feature values ​​corresponding to the preset number of fruit samples to be tested are randomly sampled to obtain a training set. The training set includes training feature values, which are feature values ​​sampled from the feature values ​​corresponding to the preset number of fruit samples to be tested.

[0137] Determine the sampling rounds;

[0138] Based on the sampling rounds, the training feature values ​​in the training set are iteratively sampled to obtain a subset of variables sampled in each sampling round. The subset of variables includes the sampled feature values ​​sampled from the training feature values.

[0139] Based on the subset of variables sampled in each sampling round, a subset of target variables is obtained by filtering.

[0140] The target feature values ​​are obtained based on a subset of the target variables.

[0141] In an exemplary embodiment of the present invention, the fifth module 850 can perform iterative sampling of training feature values ​​in the training set based on sampling rounds to obtain a subset of variables sampled in each sampling round:

[0142] Determine the initial variable subset for each sampling, where the initial variable subset for the subsequent sampling is the variable subset obtained from the previous sampling, and the initial variable subset for the first sampling is a set composed of training feature values;

[0143] Based on the initial variable subset, construct the first partial least squares regression model corresponding to the initial variable subset;

[0144] Based on the first partial least squares regression model, determine the absolute values ​​of the regression coefficients corresponding to each feature value in the initial variable subset;

[0145] Based on the absolute value of the regression coefficient, the retained feature values ​​are obtained by filtering from the feature values;

[0146] Based on the retained feature values, a subset of variables sampled in each sampling round is obtained.

[0147] In an exemplary embodiment of the present invention, the fifth module 850 may achieve the following: Based on the absolute value of the regression coefficient, select and retain eigenvalues ​​from the eigenvalues.

[0148] The absolute values ​​of the regression coefficients are sorted in descending order to obtain a sorted set;

[0149] Select the absolute values ​​of the regression coefficients of a predetermined number of items from the sorted set;

[0150] Based on the absolute values ​​of the regression coefficients of a predetermined number that are ranked first, the retained feature values ​​are selected from the feature values.

[0151] In an exemplary embodiment of the present invention, the fifth module 850 may obtain a target variable subset based on the variable subsets sampled in each sampling round in the following manner:

[0152] Based on the subset of variables obtained from each sampling round, a second partial least squares regression model corresponding to the subset of variables is constructed.

[0153] Based on the second partial least squares regression model, determine the cross root mean square error corresponding to the variable subset;

[0154] The minimum cross-mean square error is determined from the cross-mean square error, and the subset of variables corresponding to the minimum cross-mean square error is taken as the subset of target variables.

[0155] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 can call logic instructions in the memory 930 to execute a fruit quality detection method applied to a fruit image acquisition module. The fruit image acquisition module includes at least a micro / nano structure modulation layer and an image sensor layer. The micro / nano structure modulation layer partially covers the image sensor layer. The micro / nano structure modulation layer includes multiple micro / nano structure units. The fruit quality detection method includes: acquiring a processed image of the fruit to be detected based on the fruit image acquisition module, wherein the processed image is a grayscale image formed by modulating the light information reflected from the fruit to be detected based on the micro / nano structure modulation layer to obtain modulated light information, and processing the modulated light information based on the image sensor layer; determining the coordinate positions of the micro / nano structure units in the image sensor layer; determining feature values ​​corresponding to the micro / nano structure units based on the coordinate positions and the grayscale values ​​of each pixel in the processed image; acquiring a pre-determined fruit quality detection model; inputting the feature values ​​into the fruit quality detection model to obtain the fruit quality detection result of the fruit to be detected output by the fruit quality detection model.

[0156] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the fruit quality detection methods provided by the above methods, applied to a fruit image acquisition module. The fruit image acquisition module includes at least a micro / nano structure modulation layer and an image sensor layer. The micro / nano structure modulation layer partially covers the image sensor layer. The micro / nano structure modulation layer includes multiple micro / nano structure units. The fruit quality detection method includes: acquiring the fruit to be detected based on the fruit image acquisition module. The image to be processed is a grayscale image formed by modulating the light information reflected by the fruit to be detected based on the micro-nano structure modulation layer to obtain modulated light information, and processing the modulated light information based on the image sensor layer; determining the coordinate position of the micro-nano structure unit on the image sensor layer; determining the feature value corresponding to the micro-nano structure unit based on the coordinate position and the grayscale value of each pixel in the image to be processed; obtaining a pre-determined fruit quality detection model; inputting the feature value into the fruit quality detection model to obtain the fruit quality detection result of the fruit to be detected output by the fruit quality detection model.

[0158] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the fruit quality detection method provided by the methods described above, applied to a fruit image acquisition module. The fruit image acquisition module includes at least a micro / nano structure modulation layer and an image sensor layer. The micro / nano structure modulation layer partially covers the image sensor layer. The micro / nano structure modulation layer includes multiple micro / nano structure units. The fruit quality detection method includes: acquiring a processing image of the fruit to be detected based on the fruit image acquisition module, wherein the processing image... To obtain modulated light information by modulating the light reflected from the fruit to be tested based on the micro / nano structure modulation layer, and to form a grayscale image by processing the modulated light information based on the image sensor layer; to determine the coordinate position of the micro / nano structure unit in the image sensor layer; to determine the feature value corresponding to the micro / nano structure unit based on the coordinate position and the grayscale value of each pixel in the image to be processed; to obtain a pre-determined fruit quality detection model; and to input the feature value into the fruit quality detection model to obtain the fruit quality detection result of the fruit to be tested output by the fruit quality detection model.

[0159] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0161] It is further understood that although the operations are described in a specific order in the accompanying drawings in the embodiments of the present invention, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting fruit quality, characterized in that, An image acquisition module for fruit is provided, wherein the image acquisition module includes at least a micro / nano structure modulation layer and an image sensor layer, the micro / nano structure modulation layer partially covers the image sensor layer, the micro / nano structure modulation layer includes multiple micro / nano structure units, and the fruit quality detection method includes: The image to be processed is obtained based on the fruit image acquisition module. The image to be processed is a grayscale image formed by modulating the light information reflected by the fruit to be detected based on the micro-nano structure modulation layer to obtain modulated light information, and processing the modulated light information based on the image sensor layer. Determine the coordinate position of the micro / nano structure unit in the image sensor layer; Based on the coordinate position and the grayscale value of each pixel in the image to be processed, a feature value corresponding to the micro / nano structure unit is determined. Specifically, determining the feature value corresponding to the micro / nano structure unit based on the coordinate position and the grayscale value of each pixel in the image to be processed includes: Based on the coordinate positions, the unit region images corresponding to each of the micro / nano structure units are determined in the image to be processed; The target unit region image is determined within a preset region of the unit region image; Based on the average gray value of each pixel in the target unit region image, the feature value corresponding to the micro / nano structure unit is determined. Obtain a pre-determined fruit quality detection model; The feature values ​​are input into the fruit quality detection model, and the fruit quality detection model outputs the fruit quality detection results of the fruit to be detected.

2. The fruit quality testing method according to claim 1, characterized in that, Before inputting the feature values ​​into the fruit quality detection model, the fruit quality detection method further includes: The feature values ​​are filtered to obtain target feature values, wherein the target feature values ​​are used to characterize feature values ​​whose influence on the quality of the fruit to be detected exceeds the influence threshold. The step of inputting the feature values ​​into the fruit quality detection model to obtain the fruit quality detection result of the fruit to be detected output by the fruit quality detection model specifically includes: The target feature value is input into the fruit quality detection model, and the fruit quality detection model outputs the fruit quality detection result of the fruit to be detected.

3. The fruit quality testing method according to claim 2, characterized in that, The process of filtering the feature values ​​to obtain the target feature values ​​specifically includes: Obtain multiple fruit samples to be tested; According to the first preset sampling ratio, the fruit samples to be tested are sampled to obtain a sample training set, wherein the sample training set includes a preset number of fruit samples to be tested; According to the second preset sampling ratio, the feature values ​​corresponding to the preset number of fruit samples to be tested are randomly sampled to obtain a training set, wherein the training set includes training feature values, which are feature values ​​sampled from the feature values ​​corresponding to the preset number of fruit samples to be tested. Determine the sampling rounds; Based on the sampling rounds, the training feature values ​​in the training set are iteratively sampled to obtain a subset of variables sampled in each sampling round, wherein the subset of variables includes the sampled feature values ​​sampled from the training feature values; Based on the subset of variables obtained from each sampling round, a subset of target variables is selected. The target feature values ​​are obtained based on the subset of target variables.

4. The fruit quality testing method according to claim 3, characterized in that, The step of iteratively sampling the training feature values ​​in the training set based on the sampling rounds to obtain a subset of variables sampled in each sampling round specifically includes: Determine the initial variable subset for each sampling, wherein the initial variable subset for the subsequent sampling is the variable subset obtained from the previous sampling, and the initial variable subset for the first sampling is the set composed of the training feature values; Based on the initial variable subset, a first partial least squares regression model corresponding to the initial variable subset is constructed; Based on the first partial least squares regression model, determine the absolute values ​​of the regression coefficients corresponding to each feature value in the initial variable subset; Based on the absolute values ​​of the regression coefficients, retained feature values ​​are obtained by filtering from the feature values. Based on the preserved feature values, a subset of variables sampled in each sampling round is obtained.

5. The fruit quality testing method according to claim 4, characterized in that, The step of selecting retained feature values ​​from the feature values ​​based on the absolute value of the regression coefficient specifically includes: The absolute values ​​of the regression coefficients are sorted in descending order to obtain a sorted set; Select the absolute values ​​of a predetermined number of regression coefficients from the sorted set; Based on the absolute values ​​of the predetermined number of regression coefficients ranked first, retained feature values ​​are selected from the feature values.

6. The fruit quality testing method according to claim 3, characterized in that, The process of selecting the target variable subset based on the variable subsets obtained from each sampling round specifically includes: Based on the subset of variables obtained from each sampling round, a second partial least squares regression model corresponding to the subset of variables is constructed. Based on the second partial least squares regression model, determine the cross root mean square error corresponding to the subset of variables; The minimum cross-mean square error is determined from the cross-mean square errors, and the subset of variables corresponding to the minimum cross-mean square error is taken as the target variable subset.

7. A fruit quality testing device, characterized in that, An apparatus for use in a fruit image acquisition module, wherein the fruit image acquisition module includes at least a micro / nano structure modulation layer and an image sensor layer, the micro / nano structure modulation layer partially covering the image sensor layer, the micro / nano structure modulation layer including multiple micro / nano structure units, the apparatus being used to implement the fruit quality detection method according to any one of claims 1 to 6, the fruit quality detection apparatus comprising: The first module is used to acquire a processing image of the fruit to be detected based on the fruit image acquisition module. The processing image is a grayscale image formed by modulating the light information reflected by the fruit to be detected based on the micro-nano structure modulation layer to obtain modulated light information, and processing the modulated light information based on the image sensor layer. The second module is used to determine the coordinate position of the micro / nano structure unit in the image sensor layer; The third module is used to determine the feature value corresponding to the micro / nano structure unit based on the coordinate position and the gray value of each pixel in the image to be processed. The fourth module is used to obtain a pre-determined fruit quality detection model; The fifth module is used to input the feature values ​​into the fruit quality detection model, and obtain the fruit quality detection result of the fruit to be detected output by the fruit quality detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the fruit quality detection method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the fruit quality detection method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the fruit quality detection method as described in any one of claims 1 to 6.

Images