Mandarin orange quality inspection method, device, equipment and storage medium based on image processing

Abstract

The present application discloses a quality inspection method, device, equipment and storage medium for mandarin oranges based on image processing, wherein the method comprises: identifying a preset posture image of a mandarin orange in a preset posture from a plurality of mandarin orange images collected; performing roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image; outputting the quality inspection result of the mandarin orange according to the detection results of the roughness detection, prone deformity detection and lateral deformity detection; the device and equipment are used to implement the method, and the storage medium stores a program for executing the method. The present application is beneficial in that it provides a quality inspection method, device, equipment and storage medium for mandarin oranges based on image processing that can effectively detect deformities and rough skin defects of mandarin oranges at the same time.

Description

Technical Field

[0001] The present application relates to a fruit detection method, and more specifically, to a quality inspection method, device, equipment and storage medium for mandarin oranges based on image processing. Background Art

[0002] Deformed mandarin oranges affect the quality of commercial mandarin oranges, so they need to be detected and removed.

[0003] The current fruit shape description methods mainly include description methods based on geometric shape features, shape description methods based on frequency domain and shape description methods based on moments.

[0004] Fruit shape detection in static state can be described based on images under specific viewing angles. The above-mentioned shape description indicators can be selected according to the specific situation. Haruka Maeda (2018) extracted the elliptical Fourier descriptor coefficients of persimmon in horizontal and vertical views for principal component analysis, and then performed shape classification through clustering method (Maeda H, Akagi T, Tao R. Quantitative characterization of fruit shape and its differentiationpattern in diverse persimmon (Diospyros kaki) cultivars[J]. ScientiaHorticulturae, 2018, 228:41-48.).

[0005] Dynamic fruit shape detection focuses on constructing shape descriptors for fruits during high-speed grading, making them suitable for describing fruit shapes in situations with varying postures. Ying Yibin et al. (2016) (Ying Yibin, Wang Fujie, Rao Xiuqin. A method and device for fruit shape detection [P]. CN 103514452 B2016.09.28.) disclosed a fruit shape detection method that uses a threshold value set by the dispersion of the symmetry index of a well-formed fruit to determine whether the fruit's shape is well-formed.

[0006] However, the regular shape of the Wogan fruit is often "curling," exhibiting distinct shape characteristics from different viewing angles, making it difficult to effectively and reliably describe it using a single detection metric. Furthermore, current shape detection metrics focus primarily on describing the overall shape of the fruit, lacking a method for categorizing specific deformities.

[0007] In addition, classifying the thickness of the Wogan skin will help improve the quality of commercial Wogan.

[0008] In the relevant art, surface roughness measurement methods are mainly divided into two types: contact and non-contact. Contact measurement methods use a special stylus to move along the surface of the workpiece at a constant speed. The stylus moves up and down due to microscopic surface irregularities. The variable of the stylus movement is converted mechanically, optically, and electrically, and then amplified and calculated. The numerical value of the evaluation parameter of the measured surface roughness is displayed on an indicator. Contact measurement has high accuracy and a wide range of applications, but this method has disadvantages such as low measurement speed, easy scratching of the measured surface, and the possibility of filtering out some high-frequency surface information due to the probe diameter, which greatly limits its scope of application. Non-contact measurement mainly uses optical principles for measurement, and mainly includes interferometry, scatterometry, and light sectioning. Interferometry has high environmental requirements, a complex implementation process, and a long optical system adjustment time, making it difficult to achieve widespread application. Scatterometry can easily achieve automated high-speed measurement, but the two methods can only be used for comparative measurements, and it is difficult to use quantitative parameters to evaluate the workpiece's surface topography. Light sectioning is only suitable for applications with low measurement accuracy. The resolution and detection accuracy are not high, and the numerical calculation is time-consuming. In addition to the mature optical measurement methods mentioned above, researchers have also explored other methods for surface roughness measurement, including atomic force microscopy (ATM) and optical sensors. While ATM can accurately measure the surface topography of the object being measured, it is expensive. Optical sensors, utilizing the more mature optical triangulation method, are easy to implement, but their accuracy is limited.

[0009] In summary, when conducting quality inspection on Wogan oranges on the production line, the existing technology cannot simultaneously and efficiently detect the quality problems of deformity and rough skin of Wogan oranges. Summary of the Invention

[0010] The content of this application is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this application is not intended to identify key features or essential features of the technical solution for which protection is sought, nor is it intended to limit the scope of the technical solution for which protection is sought.

[0011] Some embodiments of the present application propose a method, device, equipment and storage medium for quality inspection of mandarin oranges based on image processing to solve the technical problems mentioned in the above background technology section.

[0012] As a first aspect of the present application, some embodiments of the present application provide a quality inspection method for mandarin oranges based on image processing, comprising: identifying a preset posture image of a mandarin orange in a preset posture from multiple collected images of the mandarin orange; performing roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image; outputting the quality inspection result of the mandarin orange based on the detection results of the roughness detection, prone deformity detection and lateral deformity detection; wherein, performing roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image comprises: when the preset posture of the preset posture image is a supine posture, performing roughness detection and prone deformity detection on the preset posture image; when the preset posture of the preset posture image is a prone posture, performing lateral deformity detection on the preset posture image; wherein, the roughness detection includes In summary: processing the preset posture image to obtain a sessile image with the stalk removed; performing feature extraction based on the sessile image to obtain feature values; inputting the feature values ​​into a decision tree model so that the decision tree model outputs a classification value representing the surface roughness classification of the mandarin orange; wherein, prone deformity detection includes: performing image segmentation processing on the preset posture image to obtain a contour image having the contour line of the mandarin orange in the preset posture image; extracting feature values ​​based on the mandarin orange contour image and inputting them into a deformity detection model for the preset posture so that the deformity detection model outputs the detection result of the deformity detection; wherein, side-lying deformity detection includes: performing image segmentation processing on the preset posture image to obtain a contour image having the contour line of the mandarin orange in the preset posture image; outputting the detection result of the deformity detection based on the result of the straight line fitting of the contour line of the mandarin orange in the contour image.

[0013] Furthermore, the decision tree model is constructed as an XGBoost decision tree model. Training the XGBoost decision tree model includes:

[0014] The original images of Wogan were collected and manually classified according to the roughness classification rules of Wogan;

[0015] Performing image segmentation on the original image to obtain a segmented image;

[0016] processing the segmented image to remove the fruit stalk in the segmented image to obtain a sessile image;

[0017] Performing feature extraction based on the sessile image to obtain a feature value of the sessile image;

[0018] The feature values ​​of the sessile image are used as input data, and the classification values ​​of the original image corresponding to the sessile image are used as output data to train the XGBoost decision tree model.

[0019] Furthermore, the roughness classification rules of Wogan divide the surface roughness of Wogan into rough skin and fine skin.

[0020] Furthermore, a preset posture image of the mandarin orange in a preset posture is identified from multiple collected images of the mandarin orange, including: inputting the mandarin orange image into a posture recognition model so that the posture recognition model outputs a posture recognition result; the posture recognition model is constructed as a MobileNetv3_small network model.

[0021] Furthermore, the preset posture image is subjected to image segmentation processing to obtain a contour picture having the contour line of the mandarin orange in the preset posture image, including: performing color space selection on the preset posture image; performing edge detection on the image after the color space selection of the preset posture image to obtain a fruit contour image; performing contour extraction on the fruit contour image to obtain a fruit contour mask image; and performing secondary edge detection on the fruit mask image to obtain a contour picture.

[0022] Furthermore, the deformity detection model is constructed as a LightGBM network model.

[0023] Furthermore, the deformity detection result is output based on the result of the straight-line fitting of the contour line of the Wogan in the Wogan contour picture, including: performing straight-line fitting on the contour line of the Wogan in the Wogan contour picture to obtain several fitted straight lines; calculating the slope of each fitted straight line in a coordinate system; calculating the absolute value of the difference between each slope, and if the minimum absolute value is greater than or equal to a preset value, outputting the result of the side-lying detection as a deformity.

[0024] As a second aspect of the present application, some embodiments of the present application provide a quality inspection device for mandarin oranges based on image processing, including: an identification module for identifying a preset posture image of a mandarin orange in a preset posture from multiple collected images of the mandarin orange; a detection module for performing roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image; an output module for outputting the quality inspection result of the mandarin orange based on the detection results of the roughness detection, prone deformity detection and lateral deformity detection; wherein, performing roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image includes: when the preset posture of the preset posture image is a supine posture, performing roughness detection and prone deformity detection on the preset posture image; when the preset posture of the preset posture image is a prone posture, performing lateral deformity detection on the preset posture image. Measurement; wherein, roughness detection includes: processing the preset posture image to obtain a sessile image with the stalk removed; performing feature extraction based on the sessile image to obtain feature values; inputting the feature values ​​into a decision tree model so that the decision tree model outputs a classification value representing the surface roughness classification of the mandarin orange; wherein, prone deformity detection includes: performing image segmentation processing on the preset posture image to obtain a contour picture having the contour line of the mandarin orange in the preset posture image; extracting feature values ​​based on the mandarin orange contour picture and inputting them into a deformity detection model for the preset posture so that the deformity detection model outputs the detection result of the deformity detection; wherein, side-lying deformity detection includes: performing image segmentation processing on the preset posture image to obtain a contour picture having the contour line of the mandarin orange in the preset posture image; outputting the detection result of the deformity detection based on the result of the straight line fitting of the contour line of the mandarin orange in the contour picture.

[0025] As the third aspect of the present application, some embodiments of the present application provide an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation method of the above-mentioned first aspect.

[0026] As a fourth aspect of the present application, some embodiments of the present application provide a computer storage medium on which a computer program is stored, wherein when the program is executed by a processor, the method described in any implementation of the above-mentioned first aspect is implemented.

[0027] The beneficial effect of the present application is that it provides a quality inspection method, device, equipment and storage medium for mandarin oranges based on image processing that can effectively detect deformities and rough skin defects of mandarin oranges at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings constituting a part of this application are used to provide a further understanding of this application and make other features, purposes and advantages of this application more apparent. The drawings and descriptions of the exemplary embodiments of this application are used to explain this application and do not constitute an improper limitation on this application.

[0029] In addition, throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the elements and components are not necessarily drawn to scale.

[0030] In the attached figure:

[0031] Figure 1 This is a schematic diagram of the main steps of a method for quality inspection of mandarin oranges based on image processing according to an embodiment of the present application;

[0032] Figure 2 This is a schematic diagram of the main process of a method for detecting deformity of mandarin oranges according to an embodiment of the present application;

[0033] Figure 3 This is a schematic diagram of an example process of a method for detecting deformity of mandarin oranges according to an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of an outline image of a fruit in a supine position according to an embodiment of the present application;

[0035] Figure 5 is a schematic diagram of an outline image of a fruit in a side-lying position according to an embodiment of the present application;

[0036] Figure 6 This is a schematic diagram of a process for processing a preset posture image to obtain a contour image according to an embodiment of the present application;

[0037] Figure 7 is a schematic diagram of the maximum inscribed circle of a contour line in a contour image according to an embodiment of the present application;

[0038] Figure 8 is a schematic diagram of the minimum circumscribed rectangle of a contour line in a contour image according to an embodiment of the present application;

[0039] Figure 9 is a schematic diagram of the slope of a contour line in a contour image according to an embodiment of the present application;

[0040] Figure 10 is a schematic diagram of a direct straight line fitting result of a contour line in a contour image according to an embodiment of the present application;

[0041] Figure 11 This is a schematic diagram of a straight line fitting result after algorithmic constraints are applied to contour lines in a contour image according to an embodiment of the present application;

[0042] Figure 12 This is a schematic diagram of the main process of roughness detection of mandarin oranges according to an embodiment of the present application;

[0043] Figure 13 is an original image of mandarin oranges collected in a classification method according to an embodiment of the present application;

[0044] Figure 14 is a segmented image obtained in a classification method according to an embodiment of the present application;

[0045] Figure 15 is a sessile image obtained in a classification method according to an embodiment of the present application;

[0046] Figure 16 is a grayscale compressed image obtained in a classification method according to an embodiment of the present application;

[0047] Figure 17 This is a structural diagram of a mandarin orange quality inspection device based on image processing according to an embodiment of the present application;

[0048] Figure 18 It is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0050] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0051] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0052] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0053] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0054] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0055] Reference Figure 1 As shown, the image processing-based quality inspection method of mandarin oranges of this application includes:

[0056] S100: Identifying a preset posture image of the mandarin orange in a preset posture from a plurality of collected mandarin orange images.

[0057] S200: performing roughness detection, prone deformity detection, and lateral deformity detection according to the type of the preset posture of the preset posture image.

[0058] S300: Outputting the quality inspection result of the mandarin orange according to the detection results of the roughness detection, the prone deformity detection and the lateral deformity detection.

[0059] Specifically, step S200 includes:

[0060] S201: When the preset posture of the preset posture image is a supine posture, performing roughness detection and prone deformity detection on the preset posture image.

[0061] S202: When the preset posture of the preset posture image is a prone posture, performing lateral deformity detection on the preset posture image.

[0062] The supine posture of the fruit referred to here refers to the posture in which the fruit stalk, calyx or other markers on the surface of the fruit are facing the image acquisition device (for example, a camera). Since the image acquisition equipment of most fruit production lines is installed above the production line, this posture in the fruit picture is defined as a supine posture.

[0063] The side-lying posture of the fruit referred to here refers to the posture in which the fruit stalk, calyx or other markers on the surface of the fruit are facing the side relative to the image acquisition device (for example, a camera). Since the image acquisition equipment of most fruit production lines is set above the production line, this posture is equivalent to the fruit stalk, calyx or other markers on the surface of the fruit facing the horizontal direction, so this posture in the fruit picture is defined as a supine posture.

[0064] In fact, on the production line, the cameras are positioned in multiple directions, so images from multiple directions can be obtained when the oranges are driven by the fruit trays. However, the posture of the oranges on the fruit trays is uncertain, so the camera above may not capture the image of the prone posture. Therefore, cameras at multiple angles are set up and then posture recognition is performed from the images of multiple cameras, which reduces the requirements for the posture of the oranges on the fruit trays.

[0065] More specifically, the roughness detection includes: processing the preset posture image to obtain a sessile image with the stalk removed; performing feature extraction based on the sessile image to obtain a feature value; inputting the feature value into a decision tree model so that the decision tree model outputs a classification value representing the surface roughness classification of the mandarin orange;

[0066] More specifically, prone deformity detection includes: performing image segmentation processing on a preset posture image to obtain a contour image having the contour line of the mandarin orange in the preset posture image; extracting feature values ​​from the mandarin orange contour image and inputting them into a deformity detection model for the preset posture so that the deformity detection model outputs a deformity detection result;

[0067] More specifically, the side-lying deformity detection includes: performing image segmentation processing on the preset posture image to obtain a contour picture having the contour line of the orange in the preset posture image; and outputting the deformity detection result based on the result of the straight line fitting of the contour line of the orange in the orange contour picture.

[0068] As an alternative, the decision tree model is constructed as an XGBoost decision tree model. Training the XGBoost decision tree model includes: collecting original images of mandarin oranges and manually classifying them according to the mandarin orange roughness classification rules; performing image segmentation on the original images to obtain segmented images; processing the segmented images to remove the stalks in the segmented images to obtain sessile images; performing feature extraction on the sessile images to obtain feature values ​​of the sessile images; and training the XGBoost decision tree model using the feature values ​​of the sessile images as input data and the classification values ​​of the original images corresponding to the sessile images as output data. The mandarin orange roughness classification rules classify the surface roughness of mandarin oranges into coarse skin and fine skin.

[0069] As an optional solution, step S100 includes: inputting the mandarin orange image into a posture recognition model so that the posture recognition model outputs a posture recognition result; the posture recognition model is constructed as a MobileNetv3_small network model.

[0070] As an optional solution, image segmentation is performed on the preset pose image to obtain a contour image of the mandarin orange in the preset pose image, including: performing color space selection on the preset pose image; performing edge detection on the image after the color space selection to obtain a fruit contour image; performing contour extraction on the fruit contour image to obtain a fruit contour mask image; and performing secondary edge detection on the fruit mask image to obtain a contour image. The deformity detection model is constructed as a LightGBM network model.

[0071] As an optional solution, the deformity detection result is output based on the result of straight line fitting of the contour line of the Wogan in the Wogan contour picture, including: performing straight line fitting on the contour line of the Wogan in the Wogan contour picture to obtain several fitted straight lines; calculating the slope of each fitted straight line in a coordinate system; calculating the absolute value of the difference between each slope, and if the minimum absolute value is greater than or equal to a preset value, outputting the result of side-lying detection as deformity.

[0072] The following combination Figures 2 to 11 As shown, the following is a more detailed introduction to the posture recognition and deformity detection in the quality inspection method of this application through an example.

[0073] The integrated development environment used in this embodiment is PyCharm2021.6, the programming language is Python3.6, and the GPU is NVIDIA GeForce RTX 3060. The specific implementation environment includes but is not limited to the above environment.

[0074] like Figure 2 and Figure 3 As shown, this embodiment specifically includes the following parts:

[0075] (1) Detection model establishment

[0076] 1) Original image collection: 4 images of Wogan in different postures were collected :

[0077] 2) Establishment of the Wogan posture detection model:

[0078] 2.1) We manually selected oranges in different poses from previously collected images to construct a pose detection dataset. Images with the stalk / calyx pointing vertically were labeled "supine," images with the stalk / calyx pointing horizontally were labeled "sideways," and all others were labeled "recumbent."

[0079] 2.2) Divide the dataset into training set, validation set, and test set according to the ratio of 7:2:1;

[0080] 2.3) Use the MobileNetv3_small network for training. Images are uniformly scaled to the same size. Image enhancement strategies include random brightness, random contrast, random horizontal flip, random vertical flip, random saturation, and random hue. The IMAGNET pre-trained model is used to accelerate model convergence, and the learning rate is optimized using linear descent.

[0081] 2.4) Save the optimal model during training and directly load the parameters of the optimal model during prediction;

[0082] 3) Wogan image segmentation: For the images in step 3) with supine and side-lying postures, and Perform color space transformation, edge detection, contour extraction, thresholding, and secondary edge detection in sequence to obtain the contour lines of the mandarin oranges. ( Figure 4 )and ( Figure 5 ), used for subsequent shape detection;

[0083] Step 4) The process is as follows Figure 6 As shown, specifically:

[0084] 3.1) Color Space Selection: Calculate the V component of the original image converted to the HSV color space to better highlight the features of the region of interest. R, G, and B are the values ​​of the original image in the red, green, and blue channels, respectively. The formula for calculating the V component is as follows;

[0085] (1)

[0086] 3.2) Edge Detection: Use the Canny operator to detect the edge of the V component of the mandarin orange image to obtain the mandarin orange contour image containing noise information. ;

[0087] 3.3) Contour extraction: Use the findContours function to find the image All contours contained in , use the contourArea function to find the contour with the largest area This is the outline of the Wogan. Use the fillPoly function to fill the interior of the Wogan with white [255,255,255] to obtain the mask image of the Wogan. ;

[0088] 3.4) Secondary edge detection: mask image Perform Canny edge detection again to obtain the final edge segmentation with clear and correct segmentation of the orange outline. and ;

[0089] 4) Establishment of supine Wogan deformity detection model: The outline of the Wogan Extract Area Intersection-Union Ratio , eccentricity , market share , roundness , contour fluctuation c and aspect ratio The six features are used as input to train the LightGBM model;

[0090] Step 4) is as follows:

[0091] 4.1) The outline of the mandarin orange in the front view Perform area intersection and union comparison , eccentricity , market share , roundness , contour fluctuation , aspect ratio The features of the fruit are extracted and divided into frontal deformed fruit and normal fruit according to their appearance. The feature value is calculated as follows:

[0092] Area Intersection and Union Ratio :Through the outline of the Wogan in the front view Get the maximum inscribed circle ( Figure 7 ), the difference between the area enclosed by the original outline of the Wogan and the area of ​​the largest inscribed circle is obtained, and then divided by the area of ​​the Wogan outline to eliminate the influence of scale, is the area enclosed by the outline of Wogan, represents the area of ​​the inscribed circle of the mandarin orange, is the intersection-union ratio. Calculated by formula (2):

[0093] (2)

[0094] Eccentricity :Assume the number of points on the contour is , the distance from the contour point to the centroid is , the average distance is , the eccentricity is . It is calculated by the following formula:

[0095] (3)

[0096] Market share : The ratio of the target area to the area of ​​the minimum enclosing rectangle, are the height and width of the minimum bounding rectangle respectively. Calculated by the following formula:

[0097] (4)

[0098] Roundness :Assume the target object area is , the perimeter is , the circumference of a circle of equal area with the target is , roundness Calculated by the following formula:

[0099] (5)

[0100] Contour Fluctuation The fluctuation rate of the distance from each point to the centroid is defined as the standard deviation of the distance from each point to the centroid of the edge contour of the orange. The normalized value is squared to improve the sensitivity of the algorithm to extreme points close to 1 or 0. is the distance between a point on the contour and the centroid, is the normalized distance value, is the number of contour points, and the contour fluctuation is calculated using the following formula:

[0101] (6)

[0102] (7)

[0103] Aspect ratio : minimum enclosing rectangle aspect ratio, are the height and width of the minimum bounding rectangle respectively ( Figure 8 ), the aspect ratio is calculated using the following formula:

[0104] (8)

[0105] 4.2) Divide the training set and test set into a ratio of 7:3;

[0106] 4.3) Training of the Wogan front shape detection model based on LightGBM: The hyperparameters selected are as follows: the learning rate (learning_rate) is set to [0.1, 0.3, 0.6]. A smaller learning rate can achieve better stability of model performance. The maximum depth of the tree model (max_depth) is set to 3-5, which is a core parameter that needs to be adjusted and has a greater impact on the performance of the model. The number of leaf nodes (num_leaves) of a single tree is set to 31 by default and is generally set to (0, ) and the number of boosting iterations (n_estimators) is set to 100 by default and is generally set between 100 and 1000, depending on the specific situation. Save the optimal model parameters.

[0107] 5) Development of a method for detecting deformity of the side-lying mandarin orange: From the outline of the side-lying mandarin orange The fitting lines on both sides were extracted and constrained to establish a deformity detection method for lateral-lying mandarin oranges.

[0108] Step 5) is as follows:

[0109] 5.1) Slope of the fitted line of the Wogan side: In the image coordinate system, the right direction is the positive direction of the x-axis, and the downward direction is the positive direction of the y-axis. The slope of the fitted line of the Wogan side is defined as:

[0110] (9)

[0111] in, 、 They are the coordinates of two points on the straight line, as shown in the schematic diagram Figure 9 As shown;

[0112] 5.2) Straight line fitting method for the side of Wogan:

[0113] like Figure 10 As shown, the straight line obtained by directly using the straight line detection method is , will detect the unnecessary upper and lower ends and oblique straight lines such as , and multiple straight lines may be detected on one side, which requires conditional constraints: Taking the left end straight line as an example, the left end fitting straight line is the left endpoint of the minimum circumscribed rectangle of the Wogan outline The horizontal axis value is the center, and the minimum enclosing rectangle width can fluctuate left and right 1 / 5 of the right side, the same is true for the right side. After the constraint, the side straight line fitting effect is as follows Figure 11 As shown;

[0114] 5.3) For the fitted lines calculated on the left and right sides Stored in collections and In the above example, the absolute value of the difference between the elements of two sets is calculated. Save in collection middle, If the minimum value is greater than 0.15, it is a laterally deformed fruit; otherwise, it is a normal fruit.

[0115] (2) Comprehensive detection of deformed mandarin oranges

[0116] After the above model and algorithm are established, the following steps are used to detect deformities of a mandarin orange.

[0117] 6) Image acquisition: 4 images of a mandarin orange in different poses are collected. .

[0118] 7) Mandarin orange posture recognition. Using the mandarin orange posture detection model, we can identify the mandarin orange posture from the image. Identify supine and side-lying image.

[0119] 8) Mandarin orange image segmentation: postures of supine and supine and side-lying The images of the supine position are transformed into color space, edge detected, contour extracted, thresholded, and secondarily edge detected. and side-lying The outline of the mandarin orange on the image and ;

[0120] 9) Supine Wogan deformity recognition: Load the supine Wogan deformity detection model to perform supine Wogan deformity recognition.

[0121] 10) Deformity identification of side-lying mandarin oranges: Load the side-lying mandarin orange deformity detection method to perform side-lying mandarin orange deformity identification.

[0122] If both the supine and side-lying deformity recognitions are passed, the deformity detection is considered to have been passed. Of course, only one of the supine or side-lying deformity detections can be performed.

[0123] The above detection method was used to detect deformities of three groups of manually sorted mandarin oranges. The detection results are shown in Table 1.

[0124]

[0125] As can be seen from the table above, for example, in the first row, of the 84 mandarin oranges that were manually labeled as having frontal deformities, 82 were correctly predicted as such using the deformity detection method of this application, with only 2 incorrectly identified. Similarly, two groups of mandarin oranges with side deformities and normal profiles were also identified. This shows that the deformity mandarin orange detection model using the above method can accurately classify the test set samples with a high accuracy rate.

[0126] The above technical solution makes the subsequent shape detection algorithm more targeted by detecting the posture of the fruit image, thereby improving the algorithm's ability to detect deformed fruits. The proposed frontal fruit trait detection algorithm improves the reliability of frontal shape detection by learning the six extracted feature quantities and using decision tree classification. For side fruit shape detection, a detection algorithm based on the difference in the slope of the fitting line is proposed, which has faster speed and better robustness. The technical solution of this application can realize real-time detection of deformed fruits on actual production lines and can realize the detection of specific deformities.

[0127] The prone posture image obtained through the above posture recognition can be used as the original image for roughness detection.

[0128] Combine Figures 12 to 16, the roughness detection scheme of this application is introduced in detail with an example.

[0129] Reference Figure 12 As shown, the classification method of this application specifically includes the following parts

[0130] 1) Image acquisition: After the above posture recognition, the prone image of the mandarin orange is used as the original image of the mandarin orange. ,like Figure 13 As shown, a dataset of coarse-skinned mandarin orange and fine-skinned mandarin orange was established;

[0131] 2) Wogan image segmentation: The original Wogan image Perform color space conversion, Otsu threshold segmentation, contour extraction, and image bitwise AND operation to obtain the segmented Wogan image. ,like Figure 14 As shown;

[0132] The specific segmentation of the mandarin orange image in step 2) is as follows:

[0133] 2.1) Color space transformation: In order to highlight the features of the region of interest more significantly, the original image of the mandarin orange was extracted. The R channel of the RGB space is used to obtain the R channel image of the original image of the Wogan. ;

[0134] 2.2) Image binarization: Using the threshold function, the image of the R channel of the original image of the Wogan is binarized. Perform Otsu threshold segmentation, set the threshold to 127, and get the binary image of the mandarin orange ;

[0135] 2.3) Maximum contour extraction: Use the findContours function to find the binary image of the mandarin orange. Grab all the contour points cnts included in the image, use the grab_contours function to return the contours in cnts, and then use the max function to get the maximum contour c, which is the outline of the mandarin orange. Use the fillPoly function to fill the interior of the mandarin orange with white [255,255,255] to obtain the mask image mask of the mandarin orange.

[0136] 2.4) Image bitwise and calculation: Through the bitwise_and function, the mask image of the mandarin orange is combined with the original image of the mandarin orange. Perform bitwise AND operation to obtain the image of the mandarin orange after image segmentation. .

[0137] 3) Removing the stem of mandarin oranges: Take the mandarin orange image after image segmentation The R channel of the image is obtained to obtain the R channel image of the mandarin orange after image segmentation , for the R channel of the mandarin orange image after image segmentation Perform Otsu threshold segmentation to obtain the R channel binary image after image segmentation , using a specified ellipse 11 11 structural nuclear elements The R channel of the image after segmentation is binarized. Erosion is performed to obtain a mask for removing the fruit stem of the Wogan. , the R channel image of the mandarin orange after image segmentation Then compare it with the image of the mandarin orange after image segmentation Perform bitwise AND operation to obtain the image of the stalkless mandarin orange ,like Figure 15 As shown;

[0138] The step 3) is specifically as follows:

[0139] 3.1) Image binarization: Take the R channel of the mandarin orange image after image segmentation , perform Otsu threshold segmentation on it, set the threshold to 50, and get the R channel binary image of the image after image segmentation ;

[0140] 3.2) Image morphological operation: Use the getStructuringElement function to define an ellipse11 11 structural nuclear elements , using the specified kernel element through the erode function The R channel of the image after segmentation is binarized. Erosion is performed to obtain a mask for removing the fruit stem of the Wogan. ;

[0141] 3.3) Image bitwise AND operation: Using the bitwise_and function, the mask of the stalk of the mandarin orange is removed. The image of mandarin orange after image segmentation Perform bitwise AND operation to obtain the image of the mandarin orange with the stem removed .

[0142] 4) Calculation of coefficient of variation (CV): Use the boundingRect function to obtain the segmented image of the mandarin orange. The circumscribed moment of , and the grayscale value of the stripe of 25% of the circumscribed moment is obtained , and obtain the coefficient of variation CV.

[0143] The specific steps of step 4) are:

[0144] 4.1) Grayscale transformation: Grayscale transformation of the segmented orange image The grayscale image of the orange after image segmentation is obtained by grayscale conversion through the cvtColor function. ;

[0145] 4.2) Image binarization: The R channel of the image after image segmentation Perform Otsu threshold segmentation, set the threshold to 127, and get the binary image of the mandarin orange ;

[0146] 4.3) Maximum Contour Extraction: Use the findContours function to find the binarized Wogan image. All the contour points cnts included in , use the grab_contours function to return the contours contours in cnts, and then use the max function to obtain the maximum contour c, c is the outline of the mandarin orange;

[0147] 4.4) Grayscale value of the 25% band of the circumscribed moment Selection: Use the boundingRect function to extract the upper left corner coordinates of the circumscribed moment of the maximum contour c obtained , as well as width w and height h, with its vertical direction as x-axis and horizontal direction as y-axis, get the coordinates of the position of the strip of 25% of the circumscribed moment , get the gray value of 25% of the stripe of the circumscribed moment , calculate the gray value mean through the average function :

[0148]

[0149] The standard deviation SD of the grayscale value is calculated by the std function:

[0150]

[0151] Divide the standard deviation by the mean SD / MN to obtain the coefficient of variation CV. This is calculated using the following formula:

[0152]

[0153] Where, is the standard deviation, is the number of pixels in the strip that is 25% of the circumscribed matrix of the orange. It is the gray value of the stripe that is 25% of the circumscribed moment of the orange , is the grayscale value of the stripe that is 25% of the circumscribed moment The average value of .

[0154] 5) The average value of the 20%-80% grayscale value of the V component in HSV Calculation: Get the image of the mandarin orange with the stem removed The value of three channels , the values ​​of the three channels Divide by 255 respectively, and we get , take the largest value among the three as the grayscale value , for grayscale values Sort by large to small, for gray values ​​between 20% and 80% Find the average value, which is recorded as the average value of the 20%-80% grayscale value of the V component in HSV ;

[0155] The specific steps in step 5) are:

[0156] 5.1) Color Space Transformation: Obtaining an image of a mandarin orange with the stem removed The value of three channels , the values ​​of the three channels Divide by 255 respectively, and we get , take the largest value among the three as the grayscale value .

[0157] 5.2) Grayscale value sorting: The image of the mandarin orange with the stem removed is calculated. The two-dimensional grayscale value Convert it into a one-dimensional grayscale value list through the flatten function, and then sort the grayscale value list from large to small according to the sorted function;

[0158] 5.3) Get the grayscale mean feature: For the sorted grayscale list, take the top 20%-80% grayscale values , the average value is calculated by the mean function, in order to remove the brightest and darkest areas and reduce the interference of noise as the feature quantity.

[0159] 6) Obtaining the gray-level co-occurrence matrix features: Calculating the gray-level compression image of the mandarin orange with the stem removed The 8-level gray-level co-occurrence matrix of the Wogan thick and thin peel is extracted. , the standard deviation of the gray-level co-occurrence matrix , contrast of gray-level co-occurrence matrix , the dissimilarity of the gray-level co-occurrence matrix .

[0160] The specific steps in step 6) are:

[0161] 6.1) Grayscale conversion: First, the image of the mandarin orange with the stem removed is The grayscale image of the mandarin orange with the stem removed is obtained by grayscale conversion using the cvtColor function. .

[0162] 6.2) Acquisition Feature quantity: Create gray-level co-occurrence matrix , set the parameters of the gray level co-occurrence matrix, gray level mapping range [0,255], the number of gray levels of the image scaling is 8, the distance between the pixel of interest and its adjacent pixels is 1, angle is 0°, thus obtaining the gray-level co-occurrence matrix and its image, namely Figure 16 .

[0163] Among them, the gray level co-occurrence matrix , obtained by the following formula:

[0164]

[0165] The four selected features can be obtained through the gray level co-occurrence matrix:

[0166] 1. Mean of the gray-level co-occurrence matrix , obtained through the following public announcement:

[0167]

[0168] 2. Standard deviation of the gray-level co-occurrence matrix , obtained through the following public announcement:

[0169]

[0170] 3. Contrast of Gray Level Co-occurrence Matrix , obtained through the following public announcement:

[0171]

[0172] 4. Dissimilarity of Gray-Level Co-occurrence Matrix , obtained through the following public announcement:

[0173]

[0174] Where d is the relative distance expressed in pixels; ; are the pixel coordinates in the image, The number of gray levels to scale the image to.

[0175] 7) Classification of coarse-skinned and fine-skinned mandarins: The coefficient of variation (CV) of the 25% vertical section of the circumscribed moment of the G component in RGB and the average grayscale value of the 20%-80% of the V component in HSV are used. , the mean of the gray-level co-occurrence matrix , the standard deviation of the gray-level co-occurrence matrix , contrast of gray-level co-occurrence matrix and the dissimilarity of the gray-level co-occurrence matrix As the input of the XGBoost model, the corresponding coarse-skinned mandarin oranges and fine-skinned mandarin oranges are marked as 1 and 0 respectively, and used as the output for model training.

[0176] The step 7) is specifically as follows:

[0177] 7.1) Dataset feature extraction: original image of Wogan Calculate the average grayscale value of the V component 20%-80% in HSV , the coefficient of variation of the 25% vertical section of the G component circumscribed moment in RGB, the characteristic variable CV, the mean of the gray-level co-occurrence matrix , the standard deviation of the gray-level co-occurrence matrix , contrast of gray-level co-occurrence matrix , the dissimilarity of the gray-level co-occurrence matrix Feature extraction;

[0178] 7.2) Classification Labeling: We manually selected coarse-skinned and fine-skinned mandarins to construct a dataset for coarse-skin and fine-skin detection, assigning them the labels "coarse-skin" and "fine-skin" respectively.

[0179] 7.3) Dataset Partitioning: Divide the dataset into training and test sets using a ratio of 8:2.

[0180] 7.4) XGBoost Model Training: Train a model for classifying the thickness and fineness of mandarin orange peels based on an XGBoost decision tree model. The hyperparameters selected are as follows: gbtree as the booster, binary:logistic as the loss function, binary classification error rate (error) as the evaluation metric, maximum tree depth of 5, L2 regularization parameter of 10, sample sampling ratio of 0.75, column sampling ratio of 0.75, minimum weight sum of each leaf of 2, learning rate of 0.2, random seed (default 0), number of threads of 8, and parameter controlling the number of leaves of 0.15. Save the optimal model parameters using save_model.

[0181] 7.5) XGBoost Model Testing: During testing, the load_model function was used to call the coarse-peeled and fine-peeled mandarin orange classification model to classify coarse-peeled and fine-peeled mandarins. The classification accuracy was 0.9728, and the algorithm inference time was 50ms.

[0182] After the model training is completed, the prone original mandarin orange image is processed through the above steps to obtain feature values ​​and then input into the XGBoost model (i.e., decision tree model) to output the classification of coarse skin or fine skin.

[0183] Table 2 shows a specific example of the four feature quantities of the gray-level co-occurrence matrix for the coarse and fine peels of mandarin oranges. Table 2 shows the feature quantities obtained after performing the above processing on a group of mandarin oranges. After grouping the data according to the manually classified coarse and fine peels, a comparison shows that the four feature quantities have significant differences between the coarse and fine peels. Such feature values ​​are used as training data to facilitate model convergence.

[0184]

[0185] Table 3 shows the results of the classification method used in this application for a group of mandarin oranges, compared to the actual classification results. As shown in Table 2, after identifying a group of mandarin oranges, when compared to the manual classification results, 239 of the manually classified oranges were found to have fine skin, while the machine incorrectly identified one as coarse skin. Of the 92 manually classified oranges with coarse skin, the machine incorrectly identified eight as fine skin, while the rest were correctly identified. Furthermore, the identification was completed at a relatively high sorting speed.

[0186]

[0187] Based on the above results, it can be seen that the technical solution of the present application can be used to construct a mandarin orange image dataset for the images of mandarin oranges on an actual fruit classification production line, extract the surface features of mandarin oranges (the average value I_V of the 20%-80% grayscale value of the V component in HSV, the coefficient of variation CV of the 25% longitudinal section of the circumscribed moment of the G component in RGB, the mean Mean of the grayscale co-occurrence matrix, the standard deviation Std of the grayscale co-occurrence matrix, the contrast Contrast of the grayscale co-occurrence matrix, and the dissmilarity Dissmilarity of the grayscale co-occurrence matrix), and use the XGBoost model to divide the mandarin oranges into coarse-skinned mandarin oranges and fine-skinned mandarin oranges, with high detection accuracy and fast detection speed.

[0188] Finally, the quality inspection results are output based on the deformity detection and roughness detection results. If the deformity detection results show that both the prone and lateral postures are normal, and the roughness detection shows that the skin is fine, the quality inspection result is output as qualified. If one of the deformity detection and roughness detection fails, that is, deformity or rough skin appears, the quality inspection result is output as unqualified.

[0189] Reference Figure 17As shown, the image processing-based quality inspection device for mandarin oranges of the present application includes: a recognition module for identifying a preset posture image of the mandarin orange in a preset posture from multiple collected images of the mandarin orange; a detection module for performing roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image; an output module for outputting the quality inspection result of the mandarin orange according to the detection results of the roughness detection, prone deformity detection and lateral deformity detection; wherein, performing roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image includes: when the preset posture of the preset posture image is a supine posture, performing roughness detection and prone deformity detection on the preset posture image; when the preset posture of the preset posture image is a prone posture, performing lateral deformity detection on the preset posture image; wherein, the roughness detection The detection includes: processing the preset posture image to obtain a sessile image with the stalk removed; performing feature extraction based on the sessile image to obtain feature values; inputting the feature values ​​into a decision tree model so that the decision tree model outputs a classification value representing the surface roughness classification of the mandarin orange; wherein, the prone deformity detection includes: performing image segmentation processing on the preset posture image to obtain a contour picture having the contour line of the mandarin orange in the preset posture image; extracting feature values ​​based on the mandarin orange contour picture and inputting them into a deformity detection model for the preset posture so that the deformity detection model outputs the detection result of the deformity detection; wherein, the side-lying deformity detection includes: performing image segmentation processing on the preset posture image to obtain a contour picture having the contour line of the mandarin orange in the preset posture image; outputting the detection result of the deformity detection based on the result of the straight line fitting of the contour line of the mandarin orange in the mandarin orange contour picture.

[0190] like Figure 18 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0191] Typically, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 18 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 18 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0192] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0193] It should be noted that the computer-readable medium described above in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, (but not limited to) an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0194] In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0195] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP, and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an adhoc peer-to-peer network), as well as any currently known or later developed network.

[0196] The computer-readable medium may be included in the electronic device or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: (Continue with method steps).

[0197] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0198] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function.

[0199] It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures.

[0200] For example, two blocks shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow charts, and combinations of blocks in the block diagrams and / or flow charts, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or may be implemented using a combination of dedicated hardware and computer instructions.

[0201] The units described in some embodiments of the present disclosure may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as follows: a processor including a pickup information query unit, a pickup code generation unit, and a pickup unit. The names of these units do not, in some cases, limit the units themselves. For example, the pickup code generation unit may also be described as a "unit for generating a code corresponding to pickup information."

[0202] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0203] The above descriptions are merely some preferred embodiments of the present disclosure and illustrate the underlying technical principles. Those skilled in the art should understand that the scope of the invention encompassed by the embodiments of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A quality inspection method for mandarin oranges based on image processing, comprising: Recognize a preset posture image of the mandarin orange in a preset posture from multiple collected mandarin orange images; Performing roughness detection, prone deformity detection, and lateral deformity detection according to the type of the preset posture of the preset posture image; Outputting the quality inspection result of the mandarin orange according to the detection results of the roughness detection, the prone deformity detection and the lateral deformity detection; The performing of roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image includes: When the preset posture of the preset posture image is a supine posture, performing roughness detection and prone deformity detection on the preset posture image; When the preset posture of the preset posture image is a prone posture, performing lateral lying deformity detection on the preset posture image; Wherein, the roughness detection includes: Processing the preset posture image to obtain a sessile image with the fruit stalk removed; Performing feature extraction based on the sessile image to obtain feature values; Inputting the feature value into a decision tree model so that the decision tree model outputs a classification value representing the classification of the surface roughness of the mandarin orange; Wherein, the prone deformity detection includes: Performing image segmentation processing on the preset posture image to obtain a contour image having the contour line of the orange in the preset posture image; Extracting feature values ​​according to the mandarin orange contour image and inputting them into a deformity detection model for the preset posture so that the deformity detection model outputs a detection result of the deformity detection; Wherein, the lateral lying deformity detection includes: Performing image segmentation processing on the preset posture image to obtain a contour image having the contour line of the orange in the preset posture image; The deformity detection result is output according to the result of the straight line fitting of the contour line of the mandarin orange in the mandarin orange contour picture.

2. The method for quality inspection of mandarin oranges based on image processing according to claim 1, characterized in that: The decision tree model is constructed as an XGBoost decision tree model, and training the XGBoost decision tree model includes: The original images of Wogan were collected and manually classified according to the roughness classification rules of Wogan; Performing image segmentation on the original image to obtain a segmented image; Processing the segmented image to remove the stalk in the segmented image to obtain the stalk-free image; Performing feature extraction according to the sessile image to obtain a feature value of the sessile image; The XGBoost decision tree model is trained by taking the feature value of the sessile image as input data and taking the classification value of the original image corresponding to the sessile image as output data.

3. The method for quality inspection of mandarin oranges based on image processing according to claim 1, characterized in that: The roughness classification rule of Wogan oranges classifies the surface roughness of Wogan oranges into rough skin and fine skin.

4. The quality inspection method for mandarin oranges based on image processing according to claim 1, Features: in, The step of identifying a preset posture image of an orange in a preset posture from a plurality of collected orange images includes: Inputting the orange image into a gesture recognition model so that the gesture recognition model outputs a gesture recognition result; The posture recognition model is constructed as a MobileNetv3_small network model.

5. The quality inspection method for mandarin oranges based on image processing according to claim 1, Features: in, The performing image segmentation processing on the preset posture image to obtain a contour image having the contour line of the orange in the preset posture image includes: Performing color space selection on the preset posture image; Performing edge detection on the image after color space selection of the preset posture image to obtain a fruit contour image; Performing contour extraction on the fruit contour image to obtain a fruit contour mask image; The contour image is obtained by performing secondary edge detection on the fruit mask image.

6. The method for quality inspection of mandarin oranges based on image processing according to claim 1, characterized in that: in, The deformity detection model is constructed as a LightGBM network model.

7. The quality inspection method for mandarin oranges based on image processing according to claim 1, Features: in, The outputting of the deformity detection result according to the straight line fitting result of the contour line of the mandarin orange in the mandarin orange contour picture includes: Performing straight line fitting on the contour lines of the mandarin oranges in the mandarin orange contour picture to obtain a plurality of fitting straight lines; Calculate the slope of each fitted straight line in a coordinate system; The absolute values ​​of the differences between the slopes are calculated, and if the minimum absolute value is greater than or equal to a preset value, the result that the side lying detection is deformed is output.

8. A quality inspection device for mandarin oranges based on image processing, comprising: A recognition module, used to recognize a preset posture image of the mandarin orange in a preset posture from a plurality of collected mandarin orange images; A detection module, used for performing roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image; An output module, used for outputting the quality inspection result of the mandarin orange according to the detection results of the roughness detection, the prone deformity detection and the lateral deformity detection; The performing of roughness detection, prone deformity detection and lateral deformity detection according to the type of the preset posture of the preset posture image includes: When the preset posture of the preset posture image is a supine posture, performing roughness detection and prone deformity detection on the preset posture image; When the preset posture of the preset posture image is a prone posture, performing lateral lying deformity detection on the preset posture image; Wherein, the roughness detection includes: Processing the preset posture image to obtain a sessile image with the fruit stalk removed; Performing feature extraction based on the sessile image to obtain feature values; Inputting the feature value into a decision tree model so that the decision tree model outputs a classification value representing the classification of the surface roughness of the mandarin orange; Wherein, the prone deformity detection includes: Performing image segmentation processing on the preset posture image to obtain a contour image having the contour line of the orange in the preset posture image; Extracting feature values ​​according to the mandarin orange contour image and inputting them into a deformity detection model for the preset posture so that the deformity detection model outputs a detection result of the deformity detection; Wherein, the lateral deformity detection includes: Performing image segmentation processing on the preset posture image to obtain a contour image having the contour line of the orange in the preset posture image; The deformity detection result is output according to the result of the straight line fitting of the contour line of the mandarin orange in the mandarin orange contour picture.

9. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the processors implement the method according to any one of claims 1 to 7.

10. A computer storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images