Method for automatically analyzing deterioration degree of vegetables and fruits

Abstract

The invention belongs to the field of artificial intelligence and the technical field of new-generation information through a machine vision and artificial intelligence technology. Quality change conditions of vegetables and fruits (hereinafter referred to as vegetables and fruits) are automatically analyzed, whether the vegetables and the fruits go bad or not is automatically judged, and deterioration grades are given. The method can be used for quality detection and analysis in occasions such as purchase detection, storage and processing of agricultural products such as vegetables and fruits. The method can be applied to related industries of agricultural product production, processing, storage, logistics and the like. Vegetables and fruits can deteriorate after a certain period of time due to the natural attributes of the vegetables and fruits due to the reasons such as a storage mode, a logistics process, a storage environment and plant diseases and insect pests, and the phenomena such as color change, water loss, shrinkage, liquefaction, mildewing and rotting are reflected on the appearances of the vegetables and fruits. The quantitative deterioration degree of the vegetables and the fruits is calculated according to automatic analysis of the shapes of the vegetables and the fruits, so that corresponding treatment measures can be taken in time, and the method can also be used for routine quality inspection and analysis of the vegetables and the fruits. By analyzing the appearance characteristics of the vegetables and the fruits, whether the vegetables and the fruits go bad or not and the deterioration degree are judged under the condition that the vegetables and the fruits are not damaged. According to the method, the deterioration condition of the vegetables and fruits can be analyzed according to the images of the vegetables and fruits, and the method can play an important role in regular quality inspection of agricultural products and construction of intelligent agriculture and food safety systems.

Description

Technical Field

[0001] This invention uses machine vision and artificial intelligence technology to quantitatively analyze the degree of spoilage of fruits and vegetables, and belongs to the field of artificial intelligence and the next generation of information technology. Background Technology

[0002] Fruits and vegetables gradually spoil over time after harvesting. Generally, their outlines are approximately axially symmetrical. When fruits and vegetables spoil, they typically experience varying degrees of water loss or mold growth, leading to increased leaf wrinkling, changes in the smoothness and uniformity of the surface, and even leaf liquefaction and rotting. Spoilage causes leaves to wilt and lose their uprightness, resulting in a decrease in the continuity and symmetry of the fruit's edges. All of these factors contribute to a decrease in the symmetry of the fruit's outline.

[0003] This invention uses machine vision and artificial intelligence technologies to automatically detect the symmetry of the outer contour of fruits and vegetables, and to analyze and judge their degree of spoilage in a timely manner.

[0004] Spoilage of fruits and vegetables directly affects their quality and nutritional value, impacting market prices. Severe spoilage can even lead to food safety issues. This invention allows for quantitative analysis of the degree of spoilage in fruits and vegetables, enabling targeted treatment measures or early warning systems to mitigate the spoilage process and reduce the value loss caused by spoilage. Ultimately, this helps improve the quality of agricultural products and meet the growing material needs of consumers. Summary of the Invention

[0005] This invention proposes a method for determining whether fruits and vegetables have spoiled based on their outer contour features. This method is also effective in identifying feature changes in other targets with relatively stable outer contours.

[0006] The objective of this invention is achieved through the following technical methods:

[0007] Fresh fruits and vegetables typically exhibit symmetry in their side views, such as apples, watermelons, tomatoes, radishes, cabbage, bok choy, carrots, and garlic. As time passes, fruits and vegetables gradually spoil, and improper storage conditions accelerate this spoilage. Depending on the degree of spoilage, they may exhibit varying degrees of wrinkling, dents, mold, discoloration, liquefaction, and rotting, resulting in varying degrees of asymmetry in their outlines.

[0008] This invention determines whether fruits and vegetables have spoiled and outputs the spoilage level by following these steps:

[0009] 1. Take a picture of the fruit or vegetable from the side to obtain image P; the side view is defined as: the root of the vegetable facing down, the stem of the fruit facing down, and for slender fruits and vegetables, such as carrots, watermelons, and pumpkins, take the picture when they are laid flat.

[0010] 2. Extract the fruit and vegetable region in P as the region to be analyzed, i.e., the region of interest (ROI).

[0011] 3. Convert P to grayscale to obtain a grayscale image, denoted as GY;

[0012] 4. Calculate the gradient of GY using the Sobel operator, but not limited to the Sobel operator, to obtain the gradient map GR;

[0013] 5. Repair the broken points of the outer contour line of GR by expansion and erosion to obtain the closed outer contour map Contr; filter out the background noise of GR;

[0014] 6. The origin O of the rectangular coordinate system is the intersection of the diagonals of the smallest bounding rectangle of Contr;

[0015] 7. Calculate the lengths of the longer and shorter sides of the circumscribed rectangle, and use the line passing through point O and parallel to the longer side as the Y-axis. If all four sides of the circumscribed rectangle are of equal length, use the line parallel to any one of the sides as the Y-axis.

[0016] 8. Define the line perpendicular to Y passing through the origin O as the X-axis and establish an XOY rectangular coordinate system;

[0017] 9. Starting from point O, scan horizontally along the positive and negative Y-axis, incrementing by pixels, to obtain the scan line PY. The intersections of PY with the outer contour Contr on the left and right sides of the Y-axis are denoted as PYL and PYR, respectively. Calculate the pixel-level distance D from PYL and PYR to the Y-axis, respectively. PYL-Y and D PYR-Y ;

[0018] 10. After scanning from point O to the upper boundary of Contr, stop scanning in the positive Y-axis direction; after scanning to the lower boundary of Contr, stop scanning in the opposite Y-axis direction.

[0019] 11. Calculate the number of asymmetric pixels on both sides of the Y-axis defined by (1) according to formula (1), denoted as . ;

[0020] (1)

[0021] in These represent the minimum and maximum ordinate values ​​corresponding to the lower and upper boundaries of Contr during the scan, respectively. j is an integer.

[0022] 12. Calculate the corresponding formula according to formula (2). Single scan asymmetry

[0023] , (2)

[0024] 13. Calculate the area of ​​the closed region enclosed by the contour lines and coordinate axes in the first to fourth quadrants, and denote them as PS1, PS2, PS3 and PS4 respectively, using pixels as the unit of calculation. The purpose is to reduce the influence of noise on asymmetry.

[0025] 14. Calculate the proportion of the area in each quadrant to the total area, and use this as the weight of the asymmetry in each quadrant. 1, 2, 3, 4;

[0026] (3)

[0027] 15. Calculate according to formulas (4)-(7) Weighted sum

[0028] (4)

[0029] (5)

[0030] (6)

[0031] (7)

[0032] 16. x is the value of the abscissa in the corresponding quadrant. The average asymmetry is calculated according to formula (8) as the weighted asymmetry.

[0033] (8) Take the line passing through point O and parallel to the shorter side of the circumscribed rectangle as the Y-axis. If the circumscribed rectangle is a square, take the line passing through point O and perpendicular to the original Y-axis as the Y-axis, and recalculate the weighted asymmetry using the above method. The smaller of the two weighted asymmetry degrees calculated above is taken as the final weighted asymmetry degree. If the two are equal, either weighted asymmetry degree is selected as the final weighted asymmetry degree.

[0034] 17. According to the requirements for quality control of fruits and vegetables, set a monotonically increasing threshold sequence according to formula (9), with a total of 10 levels, but not limited to 10 levels.

[0035] (9)

[0036] It is the threshold for judging the degree of spoilage of fruits and vegetables, that is, the tolerable limit.

[0037] =0.025, but not limited to 0.025.

[0038] 18. Select threshold level i based on quality control requirements, when When vegetables and fruits are judged to be spoiled, the degree of spoilage is graded as i. The larger the i is, the higher the degree of spoilage of the vegetables and fruits.

[0039] The technical problem solved by this invention is to propose a method for quantitatively identifying the degree of spoilage of fruits and vegetables, which can determine the spoilage level as needed, and facilitate the quality grading and monitoring of fruits and vegetables.

[0040] This invention is applied to the quality testing of fruits and vegetables, and the method used is robust and noise-resistant.

[0041] Whether fruits and vegetables have spoiled is an important quality indicator. This invention uses the weighted symmetry of the outline of fruits and vegetables to identify spoilage, and has good versatility.

[0042] Since most fruits and vegetables have a certain degree of symmetry, this method can be used to analyze the degree of spoilage of any food with a certain degree of symmetry in its outer contour.

[0043] The beneficial effects that this invention can bring:

[0044] 1. This invention automatically identifies the spoilage status of fruits and vegetables based on images or video data, and judges their quality based on changes in their outlines. It is highly stable, does not damage the fruits and vegetables, and is a non-destructive testing method.

[0045] 2. This invention has a wide range of applications and can be used in fields such as quantitative analysis of the degree of spoilage of fruits and vegetables, quality grading of fruits and vegetables, and quality testing of fruits and vegetables. Attached Figure Description

[0046] Figure 1 A flowchart of an algorithm for determining the degree of spoilage of fruits and vegetables;

[0047] Figure 2 A flowchart for determining the degree of spoilage of fruits and vegetables. Detailed Implementation

[0048] The invention will now be further described with reference to the accompanying drawings.

[0049] like Figure 1 As shown, the method for identifying the degree of spoilage of fruits and vegetables according to the present invention includes a module for "extracting the fruit and vegetable region in the image as ROI", a module for "extracting the outer contour of the fruit and vegetable in the ROI and removing the background noise of the fruit and vegetable", and a module for "calculating the weighted asymmetry of the outer contour". The module "Output the degradation level based on the weighted asymmetry".

[0050] The "Extracting Vegetable and Fruit Regions from Images as ROIs" module implements steps 1-2 in the invention description of this specification. The "Extract the outer contour of fruits and vegetables in the ROI and remove background noise from fruits and vegetables" module implements steps 3-5 in the invention content of this specification. The "Calculate the weighted asymmetry of the outer contour" module implements steps 6-16 in the invention description of this specification. The "Output Deterioration Level Based on Weighted Asymmetry" module implements steps 17-18 in the invention description of this specification.

[0051] After extracting the outline of the fruit and vegetable area, the purpose of repair is to make the outer outline of the fruit and vegetable form a closed area so that symmetry can be calculated.

[0052] Figure 2 A flowchart of a method for identifying the degree of spoilage in fruits and vegetables is provided, in which:

[0053] Step "Input the captured image of the fruit and vegetable": This step implements the content of step 1 in the invention description of this specification. Step "Extract the fruit and vegetable region to be detected in the image as ROI": Extract the rectangular region containing the fruit and vegetable to be identified in the image as the region to be analyzed, thus realizing the content of step 2 in the invention description of this specification.

[0054] The purpose of extracting this rectangular ROI region from the fruit and vegetable image is to further extract the target image within that region;

[0055] Step “Converting the ROI image to grayscale and then performing gradient calculation”: Implements the content of step 3 in the invention description of this specification.

[0056] Step "Extracting the outer contour of fruits and vegetables and filtering out background noise": This step implements step 4 of the invention description in this specification. For the extracted smallest bounding rectangle region of the fruits and vegetables, the background image of the fruits and vegetables is removed based on the difference between the texture features of the fruits and vegetables and the background. Due to factors such as the focal length of the shooting lens, the background objects, and their integrity, the textures in these areas are significantly different from the characteristics of the fruits and vegetables within the smallest bounding rectangle. This step yields the foreground area of ​​the fruits and vegetables.

[0057] Step "Automatically repair the broken points of the contour line to make it a closed outer contour": This implements step 5 in the invention description of this specification. After extracting the normalized outer contour line of the fruit and vegetable region, the broken points are repaired to make it a closed outer contour.

[0058] Gradient calculations are used to extract the outer contour of the fruit and vegetable region. For non-closed line segments on the contour, a dilation method in digital image processing is used to repair them into continuous and closed lines.

[0059] Step “Calculate the weighted asymmetry of the closed contour”: Implement the content of steps 6-16 in the invention description of this specification.

[0060] An XOY Cartesian coordinate system is established with the geometric center of the smallest circumscribed rectangle of the outer contour as the origin. The weighted symmetry of the outer contour of fruits and vegetables is calculated. The method used for the weighted symmetry calculation is statistical and avoids being overly sensitive to noise.

[0061] The steps "Set the tolerance level corresponding to the spoilage of fruits and vegetables", "Below the level threshold", and "Determine that the fruits and vegetables are spoiled and output the spoilage level" implement the contents of steps 17-18 in the invention content of this specification.

[0062] Based on the degree of spoilage that users can tolerate, a tolerance level is determined, and fruits and vegetables that exceed this tolerance level are considered spoiled.

Claims

1. A method for automatically analyzing the degree of spoilage of fruits and vegetables, comprising the following features: (11) The image P of the vegetables and fruits to be detected is obtained by the detection camera; (12) Calculate the bounding rectangle Rec with the smallest area of ​​fruits and vegetables in P, extract the image within the rectangular area, and obtain PREc; (13) Convert PREc to grayscale image Pgray; perform Sobel operator filtering on Pgray, but not limited to Sobel operator filtering, to obtain the filtered image Pfil; extract the outer contour line Con of fruits and vegetables in Pfil and remove background noise to obtain PCon; repair the broken parts of the outer contour line Con on PCon by dilation and erosion methods to obtain the enhanced outer contour line image PEnh of the fruit and vegetable region. (14) Calculate the position of the intersection of the diagonals of the smallest circumscribed rectangle of PEnh. Take this point as the origin O, and establish an XOY rectangular coordinate system with the lines passing through O and parallel to the long side and the short side of the circumscribed rectangle as the vertical axis respectively. Calculate the weighted asymmetry of PEnh in the two established XOY coordinate systems respectively. (15) Calculate and output the spoilage level of fruits and vegetables based on the value of the weighted asymmetry.

2. The method according to claim 1, characterized in that, The method for step (11) is as follows: The area photographed is a frontal image of the fruits and vegetables; the area of ​​the fruits and vegetables photographed is fully exposed, that is, the image is not obscured.

3. The authentication method according to claim 1, characterized in that, The method for step (12) is as follows: When calculating the smallest bounding rectangle Rec for fruits and vegetables, the area of ​​the bounding rectangle is measured in pixels within the rectangle, and is calculated as the product of the rectangle's width and height.

4. The authentication method according to claim 1, characterized in that, The method for step (13) is as follows: Gradient calculations are performed on the transformed grayscale image Pgray to extract the outer contour boundary of the fruits and vegetables. Digital image processing methods are then used to filter out noise and repair the contour lines of this boundary, resulting in a continuous and closed outer contour boundary image PEnh of the fruits and vegetables.

5. The authentication method according to claim 1, characterized in that, The method for step (14) is as follows: Establish a Cartesian coordinate system with the origin O at the intersection of the diagonals of the smallest bounding rectangle of PEnh. Decompose PEnh into four quadrants in the Cartesian coordinate system. Calculate the lengths of the long and short sides of the bounding rectangle, and use the line parallel to the long side as the Y-axis. If the four sides of the bounding rectangle are of equal length, arbitrarily choose one side as the long side and any adjacent side as the short side, using the line parallel to any side as the Y-axis. Starting from point O, alternately scan horizontally along the positive and negative directions of the Y-axis, with pixels as the step unit, parallel to the X-axis, to obtain the scan line PY. The intersections of PY with the outer contour PEnh on the left and right sides of the Y-axis are denoted as PYL and PYR, respectively. Calculate the pixel-level distance D from PYL and PYR to the Y-axis, respectively. PYL-Y and D PYR-Y ; After scanning from point O along the positive Y-axis to the upper boundary of PEnh, stop scanning in the positive Y-axis direction; After scanning from point O along the negative Y-axis to the lower boundary of PEnh, stop scanning in the negative Y-axis direction; Calculate the number of asymmetrical pixels on both sides of the Y-axis on the scan line PY according to formula (1), denoted as ; (1) in These represent the lower and upper boundaries of PEnh during the scan, respectively, corresponding to the rows where the scan line is located when the ordinate reaches its minimum and maximum values. j is an integer representing the row number; when the scan line PY is above the X-axis, the row number is a positive integer; when it is below the X-axis, the row number is a negative integer. When it coincides with the X-axis, the row number is 0. Calculate the corresponding formula according to formula (2) Asymmetry of a single scan , (2) Calculate the area of ​​the closed region enclosed by the outline of fruits and vegetables and the coordinate axes in the first to fourth quadrants, and denote them as PS1, PS2, PS3 and PS4 respectively, with pixels as the unit of calculation. Calculate the proportion of each area in the total area according to formula (3), and use it as the weight of the asymmetry. 1, 2, 3, 4; (3) Calculate according to formulas (4)-(7) Weighted sum (4) (5) (6) (7) x is the value of the x-coordinate in the corresponding quadrant. The weighted asymmetry is calculated according to formula (8). (8) Using a straight line parallel to the shorter side of the circumscribed rectangle passing through point O as the Y-axis, calculate the weighted asymmetry according to the method described in point 5 above. The smaller of the above weighted asymmetry degrees is taken as the final weighted asymmetry degree at the end of this step. If the two are equal, then any weighted asymmetry degree calculated in this step is selected as the final weighted asymmetry degree at the end of this step.

6. The authentication method according to claim 1, characterized in that, The method for step (15) is as follows: According to the requirements for quality control of fruits and vegetables, a monotonically increasing threshold sequence is set according to formula (9), with a total of 10 levels, but not limited to 10 levels. (9) It is the threshold for judging the degree of spoilage of fruits and vegetables, that is, the tolerance limit, where i is an integer ranging from 1 to 10. =0.025, but not limited to 0.

025. Select threshold level i based on quality control requirements, when... When the time is right, the fruits and vegetables are judged to have spoiled, and the spoilage level is i. The larger i is, the higher the degree of spoilage of the fruits and vegetables.

Images