Method for detecting mechanical sub-surface damage of thin-skinned fruits
By combining structured light illumination reflection imaging technology and a three-phase demodulation algorithm with a morphological reconstruction algorithm, and by selecting appropriate wavelengths and spatial frequencies, the problem of difficult detection of subsurface mechanical damage in fruits has been solved, achieving high-precision, low-cost, and rapid detection results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI AGRICULTURAL UNIVERSITY
- Filing Date
- 2023-06-21
- Publication Date
- 2026-04-10
AI Technical Summary
Existing technologies struggle to accurately detect mechanical damage to the subsurface of fruits, especially damage near the edges, and the detection process is complex and inefficient.
Structured light illumination reflection imaging technology, combined with three-phase demodulation algorithm and morphological reconstruction algorithm, is used to detect subsurface mechanical damage of fruits by selecting a suitable structured light stripe pattern with appropriate wavelength and spatial frequency, and by using illumination transformation and image ratio methods.
It enables accurate detection of subsurface mechanical damage to fruits, improves detection precision and efficiency, reduces costs, and is suitable for non-contact detection.
Smart Images
Figure CN116840236B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of fruit mechanical damage detection technology, and in particular to a method for detecting subsurface mechanical damage in thin-skinned fruits. Background Technology
[0002] Subsurface mechanical damage has a significant impact on the quality and marketability of fresh fruit. In the early stages after mechanical damage occurs, the damaged area is not obvious on the fruit's surface, making it difficult for traditional machine vision techniques to detect. To address this issue, structured light illumination reflection imaging technology has been proposed, which detects subsurface mechanical damage by changing the wavelength and spatial frequency of the structured light stripe pattern. For example, the near-spherical shape of fruits such as peaches, pears, and apples leads to uneven surface brightness distribution, resulting in darker edges and brighter centers. This phenomenon makes it difficult to detect subsurface mechanical damage near the edges. To address this problem, current methods propose constructing a 3D surface shape for each fruit to automatically correct for uneven surface brightness distribution caused by the shape. However, this requires constructing a 3D surface shape for each fruit, making the detection process very complex and inefficient. A method combining illumination transformation and image ratio has been used to automatically detect mechanical damage near the edges of fruits. This method is very effective for fruits of a single color but less effective for fruits with multiple colors. Therefore, accurately detecting subsurface mechanical damage in fruits is of great significance for improving the economic efficiency of the fruit sorting industry. Summary of the Invention
[0003] To address the technical problem of accurately detecting subsurface mechanical damage in fruits, this invention provides a method for detecting subsurface mechanical damage in thin-skinned fruits.
[0004] This invention is achieved using the following technical solution: a method for detecting subsurface mechanical damage in thin-skinned fruits, comprising the following steps:
[0005] S1 was used to build a light illumination reflection imaging system and acquire grayscale reflection images;
[0006] S2 uses a three-phase demodulation algorithm to extract the DC component image and AC component image from the grayscale reflection image. The AC component image can show the subsurface mechanical damage area of the fruit, so the subsurface mechanical damage area in the image can be displayed.
[0007] S3 selects a global threshold to binarize the DC component image, generating a mask image that only contains the apple region; and multiplies the DC component image and AC component image with the mask image respectively to remove the image background, reducing the interference of background noise on the identification of subsurface mechanical damage areas.
[0008] S4 selects structured light stripe patterns with different wavelengths and spatial frequencies for projection, and then calculates the contrast index of the subsurface damage region in the demodulated AC component image one by one to determine the wavelength and spatial frequency suitable for detecting subsurface mechanical damage in apples. Under the appropriate wavelength and spatial frequency, the features of the subsurface mechanical damage region are enhanced.
[0009] S5 divides the AC component image by the corresponding DC component image to obtain a ratio image. The ratio image is then reconstructed using a morphological reconstruction algorithm to remove local extreme value regions, resulting in a preprocessed image. The intensity values of the damaged and healthy regions are more evenly distributed in the preprocessed image, thus reducing the difficulty of segmenting the subsurface mechanical damage region.
[0010] S6 sets an appropriate grayscale constant, and obtains the labeled image by subtracting the grayscale constant from the preprocessed image; under the constraint of the preprocessed image, the labeled image is continuously morphologically dilated until a dilated image in a stable state is obtained.
[0011] S7 calculates the difference between the preprocessed image and the dilated image in a stable state to obtain a difference image; the difference image is then binarized using the Otsu thresholding method to obtain a binary image. Only the subsurface damage region is preserved in the difference image, allowing the image to be correctly segmented.
[0012] S8 uses morphological closing operations to eliminate holes and slits, obtaining a complete image of the mechanical damage area on the subsurface of an apple, and extracting the edge image and result image of the damage area.
[0013] As a further improvement to the above scheme, in step S1, the light illumination reflection imaging system includes a camera, a projector, and a computer;
[0014] Grayscale reflectance image I captured by the camera n (x,y),
[0015] Where I n (x,y)=I DC (x,y)+I AC (x,y)cos(2πf x x+δ n In the formula, n = 1, 2, 3; δ1 = 0, δ2 = 2π / 3, δ3 = 4π / 3; (x, y) represents the pixel coordinates of the structured light stripe pattern, f x It is the spatial frequency along the x-axis, I DC (x,y) represents the DC component image, I AC (x,y) represents the AC component image.
[0016] As a further improvement to the above scheme, in step S2, the DC component image I DC (x,y) is:
[0017] AC component image I AC (x,y) is:
[0018] As a further improvement to the above scheme, in step S3, the formula for calculating the mask image M(x,y) is as follows:
[0019]
[0020] As a further improvement to the above scheme, in step S4, the AC component image I AC The formula for calculating the contrast index CI of the subsurface mechanical damage region in (x,y) is as follows:
[0021]
[0022] Where n1, n2, and n represent the number of pixels in the subsurface damaged region, the healthy region, and the entire sample region, respectively; m1, m2, and m represent the average pixel intensity values in the subsurface damaged region, the healthy region, and the entire sample region, respectively; AC component image I AC The intensity value of the i-th pixel in (x,y) is represented as p i .
[0023] As a further improvement to the above scheme, in step S5, the formula for calculating the ratio image γ(x,y) is as follows:
[0024]
[0025] The ratio image γ(x,y) is used to obtain the preprocessed image I through a morphological reconstruction algorithm. P The expression for (x,y) is as follows:
[0026] I P (x,y)=imreconstruct[γ(x,y)]
[0027] In the formula, imreconstruct represents the morphological reconstruction function (IEEE transactions on image processing, 1993, 2(2): 176-201), which uses a morphological reconstruction algorithm to reconstruct the ratio image γ(x,y) to remove local extreme value regions, improve the surface intensity distribution, and reduce the interference of extreme values on damage detection.
[0028] As a further improvement to the above scheme, in step S6, the dilated image P in the stable state... M The formula for calculating (x, y) is as follows:
[0029]
[0030]
[0031] In the equation above, ∧ represents the operation of taking the minimum value pixel by pixel. This indicates that the structuring element E is used to label the image I. M Inflate (x,y); Represents the result of the i-th iteration; when The iteration stops when the time is reached, and the result of the last iteration is taken as the dilated image P in a stable state. M (x,y).
[0032] As a further improvement to the above scheme, in step S7, the difference image D h The formula for calculating (x, y) is as follows:
[0033] D h (x,y)=I P (x,y)-P M (x,y).
[0034] As a further improvement to the above scheme, in step S8, the threshold T is determined by the Otsu thresholding algorithm based on the difference image D. h The non-zero pixels in (x,y) are used to calculate and determine the binary image B(x,y); the calculation formula for the binary image B(x,y) is as follows:
[0035]
[0036] The morphological closing operation on the binary image B(x,y) to obtain the complete image Q(x,y) of the subsurface mechanical damage region of the fruit is calculated as follows:
[0037] Q(x,y) = imclose[B(x,y)]
[0038] In the formula, imclose represents the morphological closing operation of the image.
[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0040] 1. This invention uses structured light illumination reflection imaging technology to solve the problem of difficult detection of subsurface mechanical damage areas of fruits because the surface is almost invisible.
[0041] 2. This invention achieves better contrast in the subsurface mechanical damage area of fruit by selecting a structured light stripe pattern with suitable wavelength and spatial frequency.
[0042] 3. This invention solves the problem that subsurface mechanical damage near the edge area is difficult to detect due to the near-spherical shape of fruits; it has the advantages of being non-contact, low cost, high precision, and fast speed. Attached Figure Description
[0043] Figure 1 The corresponding grayscale reflectance image content acquired by this invention;
[0044] Figure 2 The flowchart illustrates a method for detecting subsurface mechanical damage in thin-skinned fruits provided by this invention. Detailed Implementation
[0045] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.
[0046] Example 1:
[0047] Please combine Figures 1-2 This embodiment of a method for detecting subsurface mechanical damage in thin-skinned fruits includes the following steps:
[0048] S1 constructs a light illumination reflection imaging system and acquires grayscale reflection images I. n (x,y);
[0049] S2 uses a three-phase demodulation algorithm to extract grayscale reflection image I n The DC component image I is obtained from (x,y). DC (x,y) and the AC component image I AC (x,y), where the AC component image I AC (x,y) can show the subsurface mechanical damage area of the fruit, so the subsurface mechanical damage area can be displayed in the image;
[0050] S3 selects the global threshold T g DC component image I DC Binarize (x,y) to generate a mask image M(x,y) containing only the fruit region; and then binarize the DC component image I... DC (x,y) and the AC component image I AC (x,y) are multiplied by the mask image M(x,y) respectively to remove the image background, thereby reducing the interference of background noise on the identification of subsurface mechanical damage areas;
[0051] S4 selects structured light stripe patterns with different wavelengths and spatial frequencies for projection, and then calculates the demodulated AC component image I one by one. AC The contrast index CI of the subsurface damage region in (x,y) is used to determine the wavelength and spatial frequency suitable for detecting subsurface mechanical damage in fruits. Under the appropriate wavelength and spatial frequency, the features of the subsurface mechanical damage region are enhanced.
[0052] S5 will convert the AC component image I AC (x,y) divided by the corresponding DC component image I DC The ratio image γ(x,y) is obtained from (x,y). A morphological reconstruction algorithm is used to reconstruct the ratio image γ(x,y) to remove local extrema, resulting in a preprocessed image I. P (x,y), the intensity values of the damaged and healthy regions in the preprocessed image are more evenly distributed, thus reducing the difficulty of segmenting the subsurface mechanical damage region;
[0053] S6 sets a suitable grayscale constant h, and preprocesses the image I... P The labeled image I is obtained by subtracting the gray level constant h from (x,y). M (x,y). In the preprocessed image I P Under the constraint of (x,y), the labeled image I M Perform continuous morphological dilation on (x,y) until a stable dilated image P is obtained. M (x,y);
[0054] S7 will preprocess image I P (x,y) and the dilation image P of the steady state M The difference between (x, y) is calculated to obtain the difference image D. h (x,y); The difference image D is obtained by using the Otsu thresholding method. h Binarize (x,y) to obtain a binary image B(x,y). Only the subsurface damage region is preserved in the difference image, so that the image is correctly segmented.
[0055] S8 eliminates holes and slits in the binary image through hole filling and morphological closing operations to obtain a complete image of the subsurface mechanical damage region of the fruit, Q(x,y); and extracts the edge image E(x,y) and the result image R(x,y) of the subsurface mechanical damage region image.
[0056] In step S1, the light illumination reflection imaging system includes a camera and a projector, as well as a computer connected to the camera and the projector. The projector is used to project sinusoidal fringe patterns with different phase shifts. The camera is used to collect the pattern projected from the projector onto the fruit after reflection. The computer is used to perform calculations on the pattern collected by the camera. Polarizers are installed at the front end of both the camera and the projector, and filters are also installed at the front end of both cameras.
[0057] Grayscale reflectance image I captured by the camera n (x,y),
[0058] Where I n (x,y)=I DC (x,y)+I AC(x,y)cos(2πf x x+δ n In the formula, n = 1, 2, 3; δ1 = 0, δ2 = 2π / 3, δ3 = 4π / 3; (x, y) represents the pixel coordinates of the structured light stripe pattern, f x It is the spatial frequency along the x-axis, I DC (x,y) represents the DC component image, I AC (x,y) represents the AC component image.
[0059] In step S2, the DC component image I DC (x,y) is:
[0060] AC component image I AC (x,y) is:
[0061] In step S3, the formula for calculating the mask image M(x,y) is as follows:
[0062]
[0063] In step S4, the AC component image I AC The formula for calculating the contrast index CI of the subsurface mechanical damage region in (x,y) is as follows:
[0064]
[0065] Where n1, n2, and n represent the number of pixels in the subsurface damaged region, the healthy region, and the entire sample region, respectively; m1, m2, and These represent the average pixel intensity values of the subsurface damaged region, the healthy region, and the entire sample region, respectively; AC component image I AC The intensity value of the i-th pixel in (x,y) is represented as p i .
[0066] In step S5, the formula for calculating the ratio image γ(x,y) is as follows:
[0067]
[0068] The ratio image γ(x,y) is used to obtain the preprocessed image I through a morphological reconstruction algorithm. P The expression for (x,y) is as follows:
[0069] I P (x,y)=imreconstruct[γ(x,y)]
[0070] In the formula, imreconstruct represents the morphological reconstruction function (IEEE transactions on image processing, 1993, 2(2): 176-201), which uses a morphological reconstruction algorithm to reconstruct the ratio image γ(x,y) to remove local extreme value regions.
[0071] In step S6, the dilated image P in the stable state M The formula for calculating (x, y) is as follows:
[0072]
[0073]
[0074] In the equation above, ∧ represents the operation of taking the minimum value pixel by pixel. This indicates that the structuring element E is used to label the image I. M Inflate (x,y); Represents the result of the i-th iteration; when The iteration stops when the time is reached, and the result of the last iteration is taken as the dilated image P in a stable state. M (x,y).
[0075] In step S7, the difference image D h The formula for calculating (x, y) is as follows:
[0076] D h (x,y)=I P (x,y)-P M (x,y).
[0077] In step S8, the threshold T is determined by the Otsu thresholding algorithm based on the difference image D. h The non-zero pixels in (x,y) are used to calculate and determine the binary image B(x,y); the calculation formula for the binary image B(x,y) is as follows:
[0078]
[0079] The morphological closing operation on the binary image B(x,y) to obtain the complete image Q(x,y) of the subsurface mechanical damage region of the fruit is calculated as follows:
[0080] Q(x,y) = imclose[B(x,y)]
[0081] In the formula, imclose represents the morphological closing operation of the image.
[0082] This design employs structured light illumination reflection imaging technology, which solves the problem of difficult detection of subsurface mechanical damage areas on fruits due to their near-invisibility on the surface. By selecting a structured light stripe pattern with suitable wavelength and spatial frequency, the subsurface mechanical damage areas of fruits exhibit better contrast. At the same time, it solves the problem that subsurface mechanical damage near the edges is difficult to detect effectively due to the near-spherical shape of fruits. It has the advantages of being non-contact, low-cost, highly accurate, and fast.
[0083] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.
Claims
1. A method of detecting sub-surface mechanical damage in thin-skinned fruit, characterised in that, The method comprises the following steps: S1: setting up a light illumination reflection imaging system and collecting a gray-scale reflection image; S2: using a three-phase demodulation algorithm to solve a direct current component image and an alternating current component image from the gray-scale reflection image; S3: selecting a global threshold, binarizing the direct current component image to generate a mask image containing only the apple area, and multiplying the direct current component image and the alternating current component image by the mask image to remove the image background; S4: selecting different wavelength and spatial frequency structured light fringe patterns for projection, and then calculating the contrast index of the subsurface damage area in the demodulated alternating current component image one by one to determine the wavelength and spatial frequency suitable for apple subsurface mechanical damage detection; S5: dividing the alternating current component image by the corresponding direct current component image to obtain a ratio image, and using a morphological reconstruction algorithm to reconstruct the ratio image to remove local extreme value areas to obtain a pretreatment image; S6: setting a suitable gray constant, obtaining a mark image by subtracting the gray constant from the pretreatment image, and continuously performing morphological dilation on the mark image under the constraint of the pretreatment image until a stable state of the dilated image is obtained; S7: subtracting the pretreatment image from the stable state of the dilated image to obtain a difference image, and using an Otsu threshold method to binarize the difference image to obtain a binary image; S8: eliminating holes and slits by morphological closing operation to obtain a complete apple subsurface mechanical damage area image, and extracting an edge image and a result image of the damage area.
2. A method of detecting sub-surface mechanical damage in thin-skinned fruit as claimed in claim 1, characterised in that, In the step S1, the light illumination reflection imaging system comprises a camera, a projector and a computer. Gray scale reflectance images captured by the camera , wherein , in the formula ; , , ; represents a pixel coordinate of a structured light fringe pattern, is a spatial frequency in the direction of the axis, represents a direct current component image, represents an alternating current component image.
3. A method of detecting sub-surface mechanical damage in thin-skinned fruit as claimed in claim 1, characterised in that, In the step S2, the direct current component image is: ; Alternating current component image To: .
4. A method of detecting sub-surface mechanical damage in thin-skinned fruit as claimed in claim 1, characterised in that, In the step S3, the mask image The calculation formula is as follows: ， wherein is a global threshold value.
5. A method of detecting sub-surface mechanical damage in thin-skinned fruit as claimed in claim 1, characterised in that, In the step S4, the AC component image Contrast index of the middle subsurface mechanical damage region The calculation formula is as follows: ; wherein , and represent the number of pixels of the subsurface damage area, the healthy area and the whole sample area, respectively; , and represent the average pixel intensity values of the subsurface damage area, the healthy area and the whole sample area, respectively; AC component image The i-th pixel intensity value in the AC component image is denoted as .
6. A method of detecting sub-surface mechanical damage in thin-skinned fruit as claimed in claim 1, characterised in that, The ratio image in the step S5 The calculation formula is as follows: ; Ratio image The pre-processed image is obtained by a morphological reconstruction algorithm The expression of the ratio image is as follows: ; In the formula denotes a morphological reconstruction function, the ratio image is reconstructed using a morphological reconstruction algorithm to remove local extremum regions.
7. A method of detecting sub-surface mechanical damage in thin-skinned fruit as claimed in claim 1, wherein, In the step S6, the steady-state expanded image The calculation formula is as follows: ; ; In the above equation, represents a pixel-wise minimum operation, denotes dilation of the marked image with the structuring element E; represents the result of the i-th iteration; the iteration stops when and the result of the last iteration is taken as the steady-state dilated image .
8. A method of detecting sub-surface mechanical damage in thin-skinned fruit as claimed in claim 1, characterised in that, In the step S7, the difference image The calculation formula is as follows: 。 9. A method of detecting sub-surface mechanical damage in thin-skinned fruit as claimed in claim 1, characterised in that, In the step S8, the threshold T is determined by the Otsu threshold algorithm according to the difference image with non-zero pixels; the binary image The calculation formula is as follows: ; To a binary image A morphological closing operation is performed to obtain a complete image of the sub-surface mechanical damage area of the fruit The calculation is as follows: ; In the formulae represents a morphological closing operation of the image.