Apple nondestructive testing method based on image processing

Abstract

The invention relates to the technical field of image processing, in particular to an apple nondestructive testing method based on image processing. The method comprises the following steps: respectively carrying out quantitative analysis on environmental factors, appearance images and internal spectrograms in an apple nondestructive testing process to obtain an environmental interference value, an image confidence value and a spectrum confidence value, and carrying out comprehensive analysis on the environmental interference value, the image confidence value and the spectrum confidence value to comprehensively evaluate the reliability of an apple detection result to obtain a confidence index; the apple nondestructive testing result is further confirmed according to the confidence index, the mechanical arm can be automatically controlled to sort apples into different collecting boxes, automatic operation is achieved, the sorting efficiency is improved, and errors and labor intensity of manual operation are reduced; and when the confidence index of the detection result is lower than a set threshold value, the system sorts the apples into a collection box for re-detection, so that no missing of any potential defect is ensured, a re-detection mechanism is formed, and the comprehensiveness and reliability of detection are further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an apple nondestructive testing method based on image processing. Background Art

[0002] Apple non-destructive testing is a technology that evaluates apple quality, defects, maturity, internal and external pests and diseases in a non-contact, non-destructive way. Non-destructive testing can conduct a comprehensive quality assessment on apples without damaging the appearance or internal quality of the apples. It is widely used in apple sorting, grading, storage, transportation and other links to ensure product quality.

[0003] Currently, non-destructive testing of apples mainly relies on visual image processing to identify whether the apples are in a non-destructive state. This type of identification and testing is easily affected by environmental factors (light, noise, dust, etc.) and the visual image itself is not accurate enough, resulting in insufficient monitoring accuracy and robustness, thereby reducing detection accuracy, especially when the assembly line is running at high speed. Summary of the invention

[0004] The purpose of the present invention is to provide an apple non-destructive inspection method based on image processing to solve the problems mentioned in the above background technology.

[0005] The purpose of the present invention can be achieved by the following technical solution: A nondestructive testing method for apples based on image processing, comprising the following steps:

[0006] R1: There are usually several inspection lines for non-destructive testing of apples. The appearance image, internal spectrum and environmental information of the apples are collected by communicating with each inspection line and each sensor mounted on the inspection line. The specific environmental information includes light intensity, background noise, vibration amplitude and dust content.

[0007] R2: Quantify the interference of environmental information on nondestructive testing of apples and obtain the environmental interference value;

[0008] R3: Quantitatively analyze the confidence level of non-destructive testing of the appearance of apples based on the appearance image to obtain the image confidence value;

[0009] R4: Quantitatively analyze the confidence level of internal nondestructive testing of apples based on the internal spectrum to obtain the spectral confidence value;

[0010] R5: The environmental interference value Y corresponding to the apple H , image confidence value Y T and the spectrum confidence value Y G Formulated calculation and analysis are performed to obtain the confidence index Y of apple detection HTG , the specific calculation formula is:

[0011]

[0012] Among them, η1, η2, and η3 are the set weight constants respectively;

[0013] R6: Use the trained appearance model to identify the appearance image to extract appearance features, and judge whether it has external defects based on the appearance features; if so, output appearance defects; if not, output appearance normal; use the trained internal model to identify the internal spectrum to extract internal features, and judge whether it has internal defects based on the internal features; if so, output internal defects; if not, output internal normal;

[0014] The test results are further confirmed based on the confidence index, and the apples are handled accordingly.

[0015] Preferably, the test results are further confirmed based on the confidence index, and the apples are processed accordingly; the specific confirmation and processing are:

[0016] When both the appearance and the interior are normal, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, the apple is output as normal, and the robot is controlled to sort the apple into a normal collection box; otherwise, the robot is controlled to sort the apple into a re-inspection collection box and re-inspect the apple.

[0017] If both the appearance defect and the internal condition are normal, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, the appearance defect of the apple is output, and the robot is controlled to sort the apple into the collection box for appearance defects; otherwise, the robot is controlled to sort the apple into the collection box for re-inspection, and the apple is re-inspected.

[0018] If both the appearance is normal and the internal defects exist, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, the internal defects of the apple are output, and the robot is controlled to sort the apples into the collection box for internal defects; otherwise, the robot is controlled to sort the apples into the collection box for re-inspection, and the apples are re-inspected;

[0019] If both appearance defects and internal defects exist at the same time, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, the appearance and internal defects of the apple are output, and the robot is controlled to sort the apples into collection boxes for appearance and internal defects; otherwise, the robot is controlled to sort the apples into a collection box for re-inspection and re-inspect the apples.

[0020] Preferably, the interference of environmental information on the non-destructive testing of apples is quantified according to the environmental information, and the specific quantification process is as follows:

[0021] Retrieve environmental information, including light intensity, background noise, vibration amplitude and dust content, and record them as G, Z, D and H respectively; set each detection line to correspond to a standard light intensity, thereby obtaining the standard light intensity of each detection line, and record it as BG;

[0022] Normalize the light intensity G, background noise Z, vibration amplitude D, dust content H and standard light intensity BG and take their values, and calculate and analyze the values ​​in a formula to obtain the environmental interference value Y H ; The specific calculation formula is:

[0023]

[0024] Among them, α1, α2, α3, and α4 are set weight constants respectively, and their values ​​are set by those skilled in the art according to actual needs.

[0025] Preferably, the specific process of quantitatively analyzing the confidence level of the non-destructive inspection of the appearance of apples based on the appearance image is as follows:

[0026] 4-1: Retrieve the appearance image and divide it into several blocks. Use the Laplace operator to identify the response value of each pixel in the block, and analyze the clarity of each block to obtain the clarity value of each block. Then, each block is divided into a clear block and a blurred block. The clarity values ​​corresponding to the cleaned block and the blurred block in the image are averaged to obtain the clarity mean values ​​corresponding to the cleaned block and the blurred block, and are recorded as and

[0027] 4-2: Perform grayscale processing on the appearance image, identify the brightness value of each block in the appearance image, and analyze the brightness distribution of the appearance image to obtain the brightness distribution value;

[0028] 4-3: Clear mean of the cleaned block The clear mean value corresponding to the blurred block The brightness distribution value Lγ is normalized and its value is taken. The value is calculated and analyzed by formula to obtain the image confidence value Y of the appearance image. T , the specific calculation formula is:

[0029]

[0030] Among them, γ3 and γ4 are the set weight constants respectively.

[0031] Preferably, the specific process of analyzing the clarity of each block is as follows:

[0032] Compare and analyze the response value of each pixel in the block with the set response interval. If the response value is greater than the upper limit of the set response interval, a high-definition point is accumulated; if the response value is within the set response interval, a medium-definition point is accumulated; if the response value is less than the lower limit of the set response interval, a low-definition point is accumulated; the cumulative number of high-definition points, medium-definition points and low-definition points in the block is counted respectively, and they are recorded as Q1, Q2 and Q3 respectively; the response values ​​corresponding to each high-definition point, medium-definition point and low-definition point in each block are summed up to obtain high-definition value, medium-definition value and low-definition value, and they are recorded as Q4, Q5 and Q6 respectively;

[0033] The cumulative number of high-definition points Q1, the cumulative number of medium-definition points Q2, the cumulative number of low-definition points Q3, the high-definition value Q4, the medium-definition value Q5 and the low-definition value Q6 are calculated and analyzed to obtain the clarity value Qβ of each block. The specific calculation formula is:

[0034]

[0035] Among them, β1, β2, and β3 are the set weight constants, and β1>β2>β3>1;

[0036] The clarity value of each block is compared and analyzed with the set clarity threshold. When the clarity value is greater than or equal to the set clarity threshold, the block is recorded as a clear block; when the clarity value is less than the set clarity threshold, the block is recorded as a fuzzy block.

[0037] Preferably, the specific process of analyzing the brightness distribution of the appearance image is:

[0038] Identify the brightness value of each block in the appearance image and record it as Li, where i = 1, 2, 3...I, I is a positive integer, I represents the total number of blocks in the appearance image, and i represents the number of any block; set a standard brightness interval [Lmin, Lmax], compare the brightness value of each block with the set standard brightness interval to obtain the brightness distance value of each block, and record it as ALi; the specific comparison method is:

[0039] When the brightness value is greater than the upper limit of the set standard interval, the brightness value and the upper limit of the standard interval are calculated to obtain the brightness distance value, ALi = Li-Lamx;

[0040] When the brightness value is at the upper limit of the set standard interval, the brightness distance value is assigned to zero;

[0041] When the brightness value is less than the lower limit of the set standard interval, the brightness value and the lower limit of the standard interval are calculated to obtain the brightness distance value, ALi = Lmin-Li;

[0042] The brightness value Li and the brightness distance value ALi of each block are normalized and their values ​​are taken, and the values ​​are calculated and analyzed by formula to obtain the brightness distribution value Lγ; the specific calculation formula is:

[0043]

[0044] Among them, γ1 and γ2 are the set weight constants, and e is a natural constant.

[0045] Preferably, the specific process of quantitatively analyzing the confidence level of the internal non-destructive testing of apples based on the internal spectrum is as follows:

[0046] 7-1: Retrieve the internal spectrum, identify the characteristic peaks in the internal spectrum and the non-absorption area corresponding to the characteristic peaks, and analyze the significance of the characteristic peaks to obtain the significance value;

[0047] 7-2: Assume that there is a standard spectrum, compare and analyze the internal spectrum with the standard spectrum to analyze and determine the degree of separation of characteristic peaks, and obtain the separation value accordingly;

[0048] 7-3: Normalize the significant value Fc and the separation value MN and take their numerical values, and perform formulaic calculation and analysis on the numerical values ​​to obtain the spectrum confidence value Y of the internal spectrum graph G , the specific calculation formula is:

[0049] Y G =μ1×e FC +μ2×e MN

[0050] Among them, μ1 and μ2 are the set weight constants respectively.

[0051] Preferably, the specific process of analyzing the significance of characteristic peaks is:

[0052] Retrieve the internal spectrum, identify the characteristic peak in the internal spectrum and the non-absorption area corresponding to the characteristic peak; use calculus to calculate the peak area of ​​the characteristic peak and record it as S peak. The specific calculation formula is: Where λ1 and λ2 are the start and end wavelengths of the characteristic peak, respectively, and S(λ) is the spectral signal expression;

[0053] Select several points in the non-absorption region of the characteristic peak and calculate the standard values ​​of the spectral intensity corresponding to the several points; divide the peak area by the signal-to-noise ratio of the characteristic peak, thereby obtaining the signal-to-noise ratio of each characteristic peak in the internal spectrum;

[0054] Compare and analyze the signal-to-noise ratio of the characteristic peak with the set signal-to-noise interval. When the signal-to-noise ratio is greater than the upper limit of the set signal-to-noise interval, a highly obvious peak is accumulated; when the signal-to-noise ratio is within the set signal-to-noise interval, a moderately obvious peak is accumulated; when the signal-to-noise ratio is less than the lower limit of the set signal-to-noise interval, a lowly obvious peak is accumulated; count the cumulative number of highly obvious peaks, moderately obvious peaks and lowly obvious peaks in the internal spectrum graph, and record them as F1, F2 and F3 respectively; calculate the mean of the signal-to-noise ratios corresponding to the highly obvious peaks, moderately obvious peaks and lowly obvious peaks to obtain highly obvious values, moderately obvious values ​​and slightly obvious values, and record them as F4, F5 and F6 respectively;

[0055] The cumulative number of highly obvious peaks F1, the cumulative number of moderately obvious peaks F2, the cumulative number of lowly obvious peaks F3, the highly obvious value F4, the moderately obvious value F5 and the slightly obvious value F6 are calculated and analyzed to obtain the significant value Fc of the characteristic peak. The specific calculation formula is:

[0056]

[0057] Wherein c1, c2, and c3 are respectively set weight constants, and c1>c2>c3>1.

[0058] Preferably, the specific process of comparing and analyzing the internal spectrum with the standard spectrum is as follows:

[0059] The internal spectrum and the standard spectrum are numbered synchronously one by one, so that the characteristic peaks of the internal spectrum and the standard spectrum can be recorded as j, j = 1, 2, 3 ... J, J is a positive integer, J represents the total number of characteristic peaks, and j represents any one of the characteristic peaks;

[0060] Calculate the wavelength interval between two adjacent characteristic peaks in the internal spectrum and the standard spectrum, and record it as and

[0061] The wavelength interval between two adjacent characteristic peaks in the internal spectrum The wavelength interval between two adjacent characteristic peaks in the standard spectrum The separation value MN of the characteristic peak is obtained by formula calculation and analysis. The specific calculation formula is:

[0062]

[0063] Beneficial effects of the present invention:

[0064] 1. Through the quantitative processing of environmental information such as light intensity, background noise, vibration amplitude and dust content, the degree of environmental interference on the test results can be accurately evaluated, providing data support for evaluating the confidence level of apple non-destructive testing results; at the same time, the staff can timely discover the environmental interference of the test line and make timely adjustments based on this;

[0065] 2. Through image block processing and Laplace operator calculation, the clarity of each block can be accurately quantified, and then the brightness distribution of the image can be evaluated based on the processing and calculation of the brightness value; the clarity and brightness distribution are comprehensively analyzed to further improve the accuracy of image quality assessment and provide data support for evaluating the confidence level of apple non-destructive testing results;

[0066] 3. Calculate the signal-to-noise ratio and wavelength interval of the characteristic peaks in the internal spectrum to quantify the significance and separation of the peaks in the internal spectrum, and conduct a comprehensive analysis to accurately evaluate the confidence level of the internal spectrum, providing data support for evaluating the confidence level of the non-destructive testing results of apples;

[0067] 4. By calculating the confidence index by integrating the environmental interference value, image confidence value and spectral confidence value, the reliability of apple inspection results can be more comprehensively evaluated, providing a basis for achieving the accuracy of non-destructive inspection of apples;

[0068] 5. The non-destructive testing results of apples are further confirmed based on the confidence index, and the robot can be automatically controlled to sort the apples into different collection boxes, realizing automated operation, which not only improves the sorting efficiency, but also reduces the errors and labor intensity of manual operation; when the confidence index of the test result is lower than the set threshold, the system will sort the apples into the collection box for re-testing to ensure that no potential defects are missed, forming a re-testing mechanism, and further improving the comprehensiveness and reliability of the test;

[0069] In summary, the present invention reduces manual intervention through intelligent processes, realizes automatic sorting and re-testing, improves the closed-loop nature of the testing process, reduces missed and misjudgment rates, increases the robustness of testing, and improves overall production efficiency and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The present invention will be further described below in conjunction with the accompanying drawings.

[0071] Figure 1 It is a schematic diagram of system module connection of the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] See also Figure 1 As shown, the present invention is a non-destructive testing method for apples based on image processing, comprising the following steps:

[0074] R1: There are usually several inspection lines for non-destructive testing of apples, and each inspection line is equipped with non-destructive testing devices (specific non-destructive testing devices include high-definition cameras and infrared spectrometers NIR), which inspect apples to select intact apples; communicate with each inspection line and each sensor mounted on the inspection line to collect the appearance image, internal spectrum and environmental information of the apple; specific environmental information includes light intensity, background noise, vibration amplitude and dust content; it should be noted that weak light intensity will blur the details, too strong light intensity will produce strong reflections, and uneven light will produce shadows or highlights, covering up defects (such as cracks, scratches, etc.), resulting in inaccurate identification; dust in the environment will interfere with the visual sensor, resulting in image errors or noise, thereby affecting the accuracy of defect detection; in high-speed assembly line inspection, equipment vibration will affect the stability of the high-definition camera, resulting in blurred or distorted images, especially in the case of dynamic inspection; there is a one-to-one correspondence between environmental information and the appearance image and internal spectrum of the apple, that is, the environmental information and the appearance image and internal spectrum of the apple are collected synchronously;

[0075] R2: Quantify the interference of environmental information on non-destructive testing of apples, specifically:

[0076] Retrieve environmental information, including light intensity, background noise, vibration amplitude and dust content, and record them as G, Z, D and H respectively; set each detection line to correspond to a standard light intensity, thereby obtaining the standard light intensity of each detection line, and record it as BG; it is specifically determined by the performance of the high-definition camera of the detection line. High-definition cameras with different performances have different sensitivities to light, so their corresponding optimal light intensities are different. If the actual light intensity of the environment differs greatly from the standard light intensity, it will lead to a decrease in image quality and color distortion, etc.

[0077] Normalize the light intensity G, background noise Z, vibration amplitude D, dust content H and standard light intensity BG and take their values, and calculate and analyze the values ​​in a formula to obtain the environmental interference value Y H ; The specific calculation formula is:

[0078]

[0079] Among them, α1, α2, α3, and α4 are respectively set weight constants, and their values ​​are set by technicians in this field according to actual needs. It can be seen from the formula that when the difference between the actual light intensity of the environment and the standard light intensity of the detection line is greater, the environmental interference value is greater; when the background noise is greater, the vibration amplitude is greater, and the dust content is greater, the environmental interference value is greater;

[0080] By quantifying environmental information such as light intensity, background noise, vibration amplitude and dust content, we can accurately assess the degree of environmental interference on the test results, providing data support for assessing the confidence level of apple's non-destructive testing results; at the same time, staff can promptly detect environmental interference on the test line and make timely adjustments.

[0081] R3: Quantitatively analyze the confidence level of the non-destructive inspection of the appearance of apples based on the appearance image to obtain the image confidence value, specifically:

[0082] The appearance image is retrieved and divided into several blocks. The Laplacian operator is used to identify and calculate the response value of each pixel in the block (i.e., the second-order derivative of the pixel). It should be noted that if the response value of each pixel in the block is larger, it means that the detail texture of the block is stronger and the clarity is higher; otherwise, it means that the block is relatively blurred and lacks clear details.

[0083] Compare and analyze the response value of each pixel in the block with the set response interval. If the response value is greater than the upper limit of the set response interval, a high-definition point is accumulated; if the response value is within the set response interval, a medium-definition point is accumulated; if the response value is less than the lower limit of the set response interval, a low-definition point is accumulated; the cumulative number of high-definition points, medium-definition points and low-definition points in the block is counted respectively, and they are recorded as Q1, Q2 and Q3 respectively; the response values ​​corresponding to each high-definition point, medium-definition point and low-definition point in each block are summed up to obtain the high-definition value, medium-definition value and low-definition value, and they are recorded as Q4, Q5 and Q6 respectively; the cumulative number of high-definition points Q1, the cumulative number of medium-definition points Q2, the cumulative number of low-definition points Q3, the high-definition value Q4, the medium-definition value Q5 and the low-definition value Q6 are calculated and analyzed by formula to obtain the clarity value Qβ of each block. The specific calculation formula is:

[0084]

[0085] Where β1, β2, and β3 are respectively set weight constants, and β1>β2>β3>1; the clarity value of each block is compared and analyzed with the set clarity threshold. When the clarity value is greater than or equal to the set clarity threshold, it means that the clarity of the block is high enough and the specific details of the block can be identified, and the block is recorded as a clear block; when the clarity value is less than the set clarity threshold, it means that the block is relatively blurred, and the block is recorded as a blurred block; the clarity values ​​corresponding to the cleaned block and the blurred block in the image are respectively calculated by averaging to obtain the clarity means corresponding to the cleaned block and the blurred block, and they are recorded as and

[0086] The appearance image is gray-scaled, and the brightness value of each block in the appearance image is identified (usually expressed as a gray value, ranging from 0 to 255), and is recorded as Li, where i = 1, 2, 3 ... I, I is a positive integer, I represents the total number of blocks in the appearance image, and i represents the number of any block in it; it should be noted that if the brightness value of each block in the appearance image is too large, it will cause overexposure; if the brightness value of each block in the appearance image is too small, it will cause shadows, affecting the quality of the appearance image;

[0087] It is assumed that there is a standard brightness interval [Lmin, Lmax]. For example, those skilled in the art usually set the standard brightness interval to [50, 200]. The brightness value of each block is compared with the set standard brightness interval to obtain the brightness distance value of each block, which is recorded as ALi. The specific comparison method is:

[0088] When the brightness value is greater than the upper limit of the set standard interval, it means that the block is overexposed, and the brightness value and the upper limit of the standard interval are calculated to obtain the brightness distance value, ALi = Li-Lamx;

[0089] When the brightness value is at the upper limit of the set standard range, it means that the brightness of the block is normal, and the brightness distance value is assigned to zero;

[0090] When the brightness value is less than the lower limit of the set standard interval, it means that there is a shadow in the block, and the brightness distance value is obtained by calculating the difference between the brightness value and the lower limit of the standard interval, ALi = Lmin-Li;

[0091] The brightness value Li and the brightness distance value ALi of each block are normalized and their values ​​are taken, and the values ​​are calculated and analyzed by formula to obtain the brightness distribution value Lγ; the specific calculation formula is:

[0092]

[0093] Among them, γ1 and γ2 are the set weight constants, and e is a natural constant. is the average brightness of each block. From the formula, we can see that the more uneven the brightness distribution in the appearance image is, the worse the quality of the appearance image is. The lower the confidence level of the nondestructive testing result based on the appearance image is, the larger the brightness distribution value is.

[0094] The clear mean of the clear blocks The clear mean value corresponding to the blurred block The brightness distribution value Lγ is normalized and its value is taken. The value is calculated and analyzed by formula to obtain the image confidence value Y of the appearance image. T , the specific calculation formula is:

[0095]

[0096] Among them, γ3 and γ4 are respectively the set weight constants. It can be seen from the formula that when the clear mean value corresponding to the blurred block is smaller than the clear mean value of the clear block, it means that the blurriness of the blurred block in the appearance image is greater, and the image confidence value is smaller; when the brightness distribution value is larger, the image confidence value is smaller;

[0097] Through image block processing and Laplace operator calculation, the clarity of each block can be accurately quantified, and then based on the processing and calculation of the brightness value, the brightness distribution of the image can be evaluated; the clarity and brightness distribution are comprehensively analyzed to further improve the accuracy of image quality assessment and provide data support for evaluating the confidence level of apple non-destructive testing results.

[0098] R4: Quantitatively analyze the confidence level of the internal nondestructive testing of apples based on the internal spectrum to obtain the spectral confidence value Y G , specifically:

[0099] Retrieve the internal spectrum, identify the characteristic peaks in the internal spectrum and the non-absorption area corresponding to the characteristic peaks (i.e., the background area of ​​the characteristic peaks). It should be noted that near-infrared light has absorption characteristics for the chemical components such as moisture, sugar, acidity, cellulose, etc. in the apple in the wavelength range of about 780nm to 2500nm. Different characteristic peaks usually correspond to different chemical components or physical properties (such as moisture, sugar, acidity, etc.). The peak area of ​​the characteristic peak is calculated by calculus and recorded as S peak. The specific calculation formula is: Where λ1 and λ2 are the starting and ending wavelengths of the characteristic peak, respectively, and S(λ) is the spectral signal expression; select a number of points in the non-absorption region of the characteristic peak, calculate the standard value of the spectral intensity corresponding to the points and record it as Nbackground; the specific number of points in the non-absorption region is set by the personnel in this field according to actual needs; divide the peak area Speak by Nbackground to get the signal-to-noise ratio of the characteristic peak, and thus get the signal-to-noise ratio of each characteristic peak in the internal spectrum; it should be noted that a low signal-to-noise ratio means that the characteristic peak is difficult to distinguish from the noise, which reduces the accuracy of the analysis results, makes it difficult to identify potential defects or decayed areas, and reduces the credibility of the image;

[0100] The signal-to-noise ratio of the characteristic peak is compared and analyzed with the set signal-to-noise interval. When the signal-to-noise ratio is greater than the upper limit of the set signal-to-noise interval, a highly obvious peak is accumulated; when the signal-to-noise ratio is in the set signal-to-noise interval, a moderately obvious peak is accumulated; when the signal-to-noise ratio is less than the lower limit of the set signal-to-noise interval, a lowly obvious peak is accumulated; the cumulative number of highly obvious peaks, moderately obvious peaks and lowly obvious peaks in the internal spectrum is counted, and they are recorded as F1, F2 and F3 respectively; the signal-to-noise ratios corresponding to the highly obvious peaks, moderately obvious peaks and lowly obvious peaks are respectively averaged to obtain highly obvious values, moderately obvious values ​​and slightly obvious values, and they are recorded as F4, F5 and F6 respectively; the cumulative number of highly obvious peaks F1, the cumulative number of moderately obvious peaks F2, the cumulative number of lowly obvious peaks F3, the highly obvious value F4, the moderately obvious value F5 and the slightly obvious value F6 are formulaically calculated and analyzed to obtain the significance value Fc of the characteristic peak. The specific calculation formula is:

[0101]

[0102] Where c1, c2, and c3 are the set weight constants, and c1>c2>c3>1. It can be seen from the formula that when the signal-to-noise ratio of the characteristic peak in the internal spectrum is larger, it means that the characteristic peak is less affected by the background noise in the non-absorption area, the peak is more obvious, and the higher the confidence level of the internal spectrum, the larger the significance value;

[0103] Assuming there is a standard spectrum, it should be noted that the wavelength position of the characteristic peak corresponds to the specific chemical components inside the apple (such as water, sugar, acidity, etc.), because different chemical components show unique absorption characteristics when absorbing infrared light of a specific wavelength;

[0104] The internal spectrum and the standard spectrum are numbered synchronously one by one, so that the characteristic peaks of the internal spectrum and the standard spectrum can be recorded as j, j = 1, 2, 3 ... J, J is a positive integer, J represents the total number of characteristic peaks, and j represents any one of the characteristic peaks; it should be noted that the characteristic peaks with the same number in the internal spectrum and the standard spectrum represent the same chemical substance; the wavelength interval between two adjacent characteristic peaks in the internal spectrum and the standard spectrum is calculated and recorded as and It should be noted that when the wavelength interval between characteristic peaks is small, two or more characteristic peaks may partially overlap, which will make the quantitative analysis of chemical components inaccurate;

[0105] The wavelength interval between two adjacent characteristic peaks in the internal spectrum The wavelength interval between two adjacent characteristic peaks in the standard spectrum The separation value MN of the characteristic peak is obtained by formula calculation and analysis. The specific calculation formula is:

[0106]

[0107] It can be seen from the formula that when the wavelength interval between two adjacent characteristic peaks in the calculated internal spectrum and the standard spectrum is farther away, it means that the position of the characteristic peak in the internal spectrum is more abnormal, which means that the characteristic peak is interfered by the adjacent characteristic peak or by other interferences (such as baseline drift, etc.), so that the position of the characteristic peak is different from the position of the characteristic peak in the standard spectrum, and the separation value is smaller;

[0108] The significance value Fc and the separation value MN are normalized and their values ​​are taken. The values ​​are calculated and analyzed by formula to obtain the spectrum confidence value Y of the internal spectrum. G , the specific calculation formula is:

[0109] Y G =μ1×e FC +μ2×e MN

[0110] Among them, μ1 and μ2 are the set weight constants respectively;

[0111] The signal-to-noise ratio and wavelength interval of the characteristic peaks in the internal spectrum are calculated to quantify the significance and separation of the peaks in the internal spectrum, and a comprehensive analysis is performed to accurately evaluate the confidence level of the internal spectrum, providing data support for evaluating the confidence level of apple non-destructive testing results.

[0112] R5: The environmental interference value Y corresponding to the apple H , image confidence value Y T and the spectrum confidence value Y GFormulated calculation and analysis are performed to obtain the confidence index Y of apple detection HTG , the specific calculation formula is:

[0113]

[0114] Among them, η1, η2, and η3 are the set weight constants. It can be seen from the formula that when the environmental interference value is larger, the confidence level of the apple detection result is lower, and the confidence index is smaller; when the image confidence value and the spectrum confidence value are larger, the confidence index is larger;

[0115] By calculating the confidence index by comprehensively considering the environmental interference value, image confidence value and spectral confidence value, the reliability of apple inspection results can be evaluated more comprehensively, providing a basis for achieving the accuracy of non-destructive inspection of apples.

[0116] R6: Use the trained appearance model to identify the appearance image to extract appearance features (color, GLCM contrast, circular contour, etc.), and judge whether there is an external defect based on the appearance features; if so, output the appearance defect; if not, output the appearance is normal; the specific performance is:

[0117] Set the mean threshold of the color channel. For example, if the mean of the red channel is less than 100, it is considered an appearance defect.

[0118] Set the GLCM contrast threshold, for example, a contrast greater than 0.5 is considered an appearance defect;

[0119] Set a circularity threshold, for example, a circularity less than 0.9 is considered an appearance defect;

[0120] Based on the threshold results of the above features, the defective area is marked and visualized;

[0121] The trained internal model will be used to identify the internal spectrum to extract internal features, and based on the internal features, it will be determined whether there are internal defects; if so, the internal defects will be output; if not, the internal normality will be output; the specific performance is as follows:

[0122] Extract the absorbance values ​​at specific wavelengths, such as 1450nm (water content), 1700nm (sugar content), etc. Use the partial least squares regression (PLS) model to correlate the spectral data with the chemical composition data measured in the laboratory, and use the model to predict the water content and sugar content of apples.

[0123] Set a moisture threshold. If the moisture is less than the set moisture threshold, it means the apple is not moist enough and an internal defect is output.

[0124] Set the sugar content threshold. If the sugar content is lower than the set sugar content threshold, it means that the apples do not contain enough sugar, and an internal defect is output.

[0125] When both the appearance and the interior are normal, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, it means that both the appearance and the interior of the apple are normal without obvious defects. Then the apple is output as normal and the robot is controlled to sort the apple into a normal collection box. Otherwise, the robot is controlled to sort the apple into a re-inspection collection box and re-inspect the apple.

[0126] If there are both appearance defects and normal internal parts, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, it means that the apple has appearance defects but the internal parts are normal. Then the appearance defects of the apple are output, and the robot is controlled to sort the apples into the collection box for appearance defects; otherwise, the robot is controlled to sort the apples into the collection box for re-inspection and re-inspect the apples.

[0127] If both the appearance is normal and the internal defects exist, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, it indicates that the apple has internal defects, but the appearance is normal. Then the internal defects of the apple are output, and the robot is controlled to sort the apples into the collection box for internal defects; otherwise, the robot is controlled to sort the apples into the collection box for re-inspection, and the apples are re-inspected.

[0128] If both appearance defects and internal defects exist, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, it indicates that both the appearance and the interior of the apple are defective. The appearance and internal defects of the apple are output, and the robot is controlled to sort the apples into the collection box for appearance and internal defects; otherwise, the robot is controlled to sort the apples into the collection box for re-inspection, and the apples are re-inspected.

[0129] The test results can automatically control the robot to sort the apples into different collection boxes, realizing automated operation, which not only improves the sorting efficiency, but also reduces the errors and labor intensity of manual operation. When the confidence index of the test results is lower than the set threshold, the system will sort the apples into the collection box for re-testing to ensure that no potential defects are missed, forming a re-testing mechanism, and further improving the comprehensiveness and reliability of the test.

[0130] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A nondestructive testing method for apples based on image processing, characterized in that: The following steps are involved: R1: Assume that there are several inspection lines for non-destructive inspection of apples, and communicate with each inspection line and each sensor mounted on the inspection line to collect the appearance image, internal spectrum and environmental information of the apple; the environmental information includes light intensity, background noise, vibration amplitude and dust content; R2: Quantify the interference of environmental information on nondestructive testing of apples and obtain the environmental interference value; R3: Quantitatively analyze the confidence level of non-destructive testing of the appearance of apples based on the appearance image to obtain the image confidence value; R4: Quantitatively analyze the confidence level of nondestructive testing inside apples based on the internal spectrum to obtain the spectral confidence value; R5: Normalize the environmental interference value, image confidence value and spectrum confidence value corresponding to the apple and take their numerical values, and perform formula calculation and analysis on the numerical values ​​to obtain the confidence index of apple detection; R6: Use the trained appearance model to identify the appearance image to extract appearance features, and determine whether there are external defects based on the appearance features; if so, output the appearance defects; If not, the output appearance is normal; The trained internal model will be used to identify the internal spectrum to extract internal features, and based on the internal features, it will be determined whether there are internal defects; if so, the internal defects will be output; if not, the internal normality will be output; The test results are further confirmed based on the confidence index, and the apples are handled accordingly.

2. The method for nondestructive testing of apples based on image processing according to claim 1, characterized in that: The test results are further confirmed based on the confidence index, and the apples are processed accordingly; the confirmation and processing are as follows: When both the appearance and the interior are normal, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, the apple is output as normal, and the robot is controlled to sort the apple into a normal collection box; otherwise, the robot is controlled to sort the apple into a re-inspection collection box and re-inspect the apple. If both the appearance defect and the internal condition are normal, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, the appearance defect of the apple is output, and the robot is controlled to sort the apple into the collection box for appearance defects; otherwise, the robot is controlled to sort the apple into the collection box for re-inspection, and the apple is re-inspected. If both the appearance is normal and the internal defects exist, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, the internal defects of the apple are output, and the robot is controlled to sort the apples into the collection box for internal defects; otherwise, the robot is controlled to sort the apples into the collection box for re-inspection, and the apples are re-inspected; If both appearance defects and internal defects exist at the same time, the confidence index corresponding to the apple is retrieved. When the confidence index is greater than or equal to the set confidence threshold, the appearance and internal defects of the apple are output, and the robot is controlled to sort the apples into collection boxes for appearance and internal defects; otherwise, the robot is controlled to sort the apples into a collection box for re-inspection and re-inspect the apples.

3. The method for nondestructive testing of apples based on image processing according to claim 1, characterized in that: The interference of environmental information on non-destructive testing of apples is quantified in the following process: Retrieve environmental information, including light intensity, background noise, vibration amplitude and dust content; set each detection line to correspond to a standard light intensity, thereby obtaining the standard light intensity of each detection line; The light intensity, background noise, vibration amplitude, dust content and standard light intensity are normalized and their values ​​are taken, and the values ​​are calculated and analyzed by formula to obtain the environmental interference value.

4. The method for nondestructive testing of apples based on image processing according to claim 1, characterized in that: The process of quantitative analysis of the confidence level of non-destructive testing of apple appearance based on appearance images is as follows: 4-1: Retrieve the appearance image and divide it into several blocks. Use the Laplace operator to identify the response value of each pixel in the block, and analyze the clarity of each block to obtain the clarity value of each block. Then, each block is divided into a clear block and a blurred block. The clarity values ​​corresponding to the cleaned blocks and the blurred blocks in the image are averaged to obtain the clarity average values ​​corresponding to the cleaned blocks and the blurred blocks. 4-2: Perform grayscale processing on the appearance image, identify the brightness value of each block in the appearance image, and analyze the brightness distribution of the appearance image to obtain the brightness distribution value; 4-3: The clear mean value of the cleaned block, the clear mean value corresponding to the blurred block, and the brightness distribution value are normalized and their values ​​are taken. The image confidence value of the appearance image is obtained by formulating and analyzing the values.

5. The method for nondestructive testing of apples based on image processing according to claim 4, characterized in that: The process of analyzing the clarity of each block is as follows: Compare and analyze the response value of each pixel in the block with the set response interval. If the response value is greater than the upper limit of the set response interval, a high-definition point is accumulated; if the response value is within the set response interval, a medium-definition point is accumulated; If the response value is less than the lower limit of the set response interval, a low definition point is accumulated; the accumulated number of high definition points, medium definition points and low definition points in the block is counted respectively, and the response values ​​corresponding to each high definition point, medium definition point and low definition point in each block are summed up to obtain the high definition value, medium definition value and low definition value; The cumulative number of high-definition points, the cumulative number of medium-definition points, the cumulative number of low-definition points, the high-definition value, the medium-definition value and the low-definition value are calculated and analyzed by formula to obtain the clarity value of each block; The clarity value of each block is compared and analyzed with the set clarity threshold. When the clarity value is greater than or equal to the set clarity threshold, the block is recorded as a clear block; when the clarity value is less than the set clarity threshold, the block is recorded as a fuzzy block.

6. The method for nondestructive testing of apples based on image processing according to claim 5, characterized in that: The specific process of analyzing the brightness distribution of the appearance image is as follows: Identify the brightness value of each block in the appearance image and record it as Li, where i = 1, 2, 3...I, I is a positive integer, I represents the total number of blocks in the appearance image, and i represents the number of any block; set a standard brightness interval [Lmin, Lmax], compare the brightness value of each block with the set standard brightness interval to obtain the brightness distance value of each block, and record it as ALi; the specific comparison method is: When the brightness value is greater than the upper limit of the set standard interval, the brightness value and the upper limit of the standard interval are calculated to obtain the brightness distance value, ALi = Li-Lamx; When the brightness value is at the upper limit of the set standard interval, the brightness distance value is assigned to zero; When the brightness value is less than the lower limit of the set standard interval, the brightness value and the lower limit of the standard interval are calculated to obtain the brightness distance value, ALi = Lmin-Li; The brightness value and brightness distance value of each block are normalized and their numerical values ​​are taken, and the numerical values ​​are calculated and analyzed by formula to obtain the brightness distribution value.

7. The method for nondestructive testing of apples based on image processing according to claim 1, characterized in that: The process of quantitative analysis of the confidence level of internal non-destructive testing of apples based on the internal spectrum is as follows: 7-1: Retrieve the internal spectrum, identify the characteristic peaks in the internal spectrum and the non-absorption area corresponding to the characteristic peaks, and analyze the significance of the characteristic peaks to obtain the significance value; 7-2: Assume that there is a standard spectrum, compare and analyze the internal spectrum with the standard spectrum to analyze and determine the degree of separation of characteristic peaks, and obtain the separation value accordingly; 7-3: Normalize the significant value and separation value and take their numerical value, and perform formulaic calculation and analysis on the numerical value to obtain the spectrum confidence value of the internal spectrum.

8. The method for nondestructive testing of apples based on image processing according to claim 7, characterized in that: The process of analyzing the significance of characteristic peaks is as follows: Retrieve the internal spectrum, identify the characteristic peak in the internal spectrum and the non-absorption area corresponding to the characteristic peak; use calculus to calculate the peak area of ​​the characteristic peak and record it as S peak. The specific calculation formula is: Where λ1 and λ2 are the start and end wavelengths of the characteristic peak, respectively, and S(λ) is the spectral signal expression; Select several points in the non-absorption region of the characteristic peak and calculate the standard values ​​of the spectral intensity corresponding to the several points; divide the peak area by the signal-to-noise ratio of the characteristic peak, thereby obtaining the signal-to-noise ratio of each characteristic peak in the internal spectrum; Compare and analyze the signal-to-noise ratio of the characteristic peak with the set signal-to-noise interval. When the signal-to-noise ratio is greater than the upper limit of the set signal-to-noise interval, a highly obvious peak is accumulated; when the signal-to-noise ratio is within the set signal-to-noise interval, a moderately obvious peak is accumulated; when the signal-to-noise ratio is less than the lower limit of the set signal-to-noise interval, a lowly obvious peak is accumulated; count the cumulative number of highly obvious peaks, moderately obvious peaks and lowly obvious peaks in the internal spectrum, and calculate the mean of the signal-to-noise ratios corresponding to the highly obvious peaks, moderately obvious peaks and lowly obvious peaks to obtain highly obvious values, moderately obvious values ​​and slightly obvious values ​​respectively; The cumulative number of highly obvious peaks, the cumulative number of moderately obvious peaks, the cumulative number of lowly obvious peaks, the highly obvious value, the moderately obvious value and the slightly obvious value are calculated and analyzed by formulating and analyzing to obtain the significance value of the characteristic peak.

9. The method for nondestructive testing of apples based on image processing according to claim 8, characterized in that: The process of comparing and analyzing the internal spectrum with the standard spectrum is as follows: The internal spectrum and the standard spectrum are numbered one by one synchronously, so that the characteristic peaks of the internal spectrum and the standard spectrum can be recorded as j, j = 1, 2, 3 ... J, J is a positive integer, J represents the total number of characteristic peaks, and j represents any one of the characteristic peaks; Calculate the wavelength interval between two adjacent characteristic peaks in the internal spectrum and the standard spectrum, and record it as and The wavelength interval between two adjacent characteristic peaks in the internal spectrum The wavelength interval between two adjacent characteristic peaks in the standard spectrum The separation value MN of the characteristic peak is obtained by formula calculation and analysis. The specific calculation formula is:

Images