Fruit defect detection method, device, equipment and medium

Abstract

This application relates to a method, apparatus, device, and medium for fruit defect detection. The method includes: obtaining multiple images of the fruit to be detected from different light source directions, segmenting the fruit images based on a global thresholding algorithm to determine a region of interest in the fruit images; extracting texture features in the region of interest based on a feature extraction and classification algorithm, inputting the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit, and determining defective fruit images in the multiple fruit images; segmenting defective region images in the defective fruit images based on a clustering algorithm, and filtering the defective region images using a median filtering algorithm; calculating the weight between the median filtered image area of ​​the defective region image and the area of ​​the defective fruit image, and sorting the corresponding fruit to a first conveyor belt when the weight exceeds a preset threshold. This application can accurately and quickly sort out fruit that meets industry standards, defective fruit, and other fruit.

Description

Technical Field

[0001] The present application relates to the field of fruit planting and production, and in particular to a fruit defect detection method, corresponding device, electronic equipment and computer-readable storage medium. Background Art

[0002] The fruit planting industry is one of the important industries in my country's agriculture. Apples are susceptible to various surface defects, such as insect spots, rot, bruises on the fruit surface caused by insects, fungi, diseases, or picking and transportation processes.

[0003] At present, fruit defect detection mainly relies on manual sorting or fruit non-destructive testing systems. Manual sorting is time-consuming and labor-intensive. If it relies on manual work, the labor intensity is high and the production efficiency is low. The fruit defect detection system in the existing technology takes too long to collect and process data, the detection efficiency is low, and the recognition accuracy is unstable for feature selection and extraction. At the same time, the detection equipment is complicated, and the purchase, installation and maintenance costs are high, which greatly affects the detection efficiency and economic benefits of fruit defect detection.

[0004] In summary, the existing technology for fruit detection has problems such as long data collection and processing time, low detection efficiency, unstable recognition accuracy for feature selection and extraction, complicated detection equipment, and high purchase, installation and maintenance costs. The applicant has made corresponding explorations to solve these problems. Summary of the Invention

[0005] The purpose of this application is to solve the above problems and provide a fruit defect detection method, corresponding device, electronic device and computer-readable storage medium.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A fruit defect detection method proposed to meet one of the purposes of this application includes the following steps:

[0008] In response to a fruit defect detection instruction, a plurality of fruit images of the fruit to be detected in different light source directions are acquired, and the fruit images are segmented based on a global thresholding algorithm to determine a region of interest in the fruit image;

[0009] Extracting texture features in the region of interest according to a feature extraction and classification algorithm, inputting the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit, and determining defective fruit images among the multiple fruit images;

[0010] Segmenting a defective region image from the defective fruit image based on a clustering algorithm, filtering the defective region image using a median filtering algorithm, and determining a median filtered image of the defective region image;

[0011] The weight between the median filter image area of ​​the defective region image and the area of ​​the defective fruit image is calculated. When it is detected that the weight exceeds a preset threshold, the fruit corresponding to the defective fruit image is sorted to a first conveyor belt. When it is detected that the weight is less than or equal to the preset threshold, the fruit corresponding to the defective fruit image is sorted to a second conveyor belt to complete the defect detection of the fruit.

[0012] Optionally, the step of segmenting the fruit image based on a global thresholding algorithm to determine a region of interest in the fruit image comprises the following steps:

[0013] Converting the fruit image into grayscale image data to obtain multiple color channel matrices in the fruit image;

[0014] Calculating global thresholds corresponding to the plurality of color channel matrices in the grayscale image data of the fruit image according to a preset global thresholding algorithm;

[0015] The grayscale values ​​corresponding to the multiple color channel matrices are subjected to image binarization processing according to the global threshold value, the binarized image data corresponding to the multiple color channel matrices are determined, and the region of interest in the fruit image is determined according to the binarized image data of the fruit image.

[0016] Optionally, the step of extracting texture features in the region of interest according to a feature extraction and classification algorithm and inputting the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit includes the following steps:

[0017] Using a Haar filter to obtain diagonal pixel values, vertical pixel values, horizontal pixel values, uniformity, correlation, and contrast in a region of interest of the fruit image;

[0018] The diagonal pixel values, vertical pixel values, horizontal pixel values, uniformity, correlation and contrast in the region of interest of the fruit image are input into the naive Bayes classification algorithm to preliminarily detect defective fruit images in the fruit image.

[0019] Optionally, the step of extracting texture features in the region of interest according to a feature extraction and classification algorithm includes the following steps:

[0020] Acquire multiple pixel values ​​of the region of interest of the fruit image, and determine pixel combinations between the pixel values ​​within a preset pixel value interval;

[0021] Counting the frequencies of pixels corresponding to the pixel values ​​in the pixel combination appearing in the region of interest of the fruit image to determine a frequency set of the pixels;

[0022] The frequencies in the frequency set are converted into probabilities, and the texture features in the region of interest are extracted according to the probability of the pixels appearing in the region of interest of the fruit image.

[0023] Optionally, the step of counting the frequencies of pixels corresponding to the pixel values ​​in the pixel combination appearing in the region of interest of the fruit image to determine a frequency set of the pixels comprises the following steps:

[0024] Determining a target distance and a plurality of statistical directions, and acquiring pixels in a region of interest of the fruit image that meet the target distance;

[0025] According to each statistical direction, respectively counting the number of matches between pixels corresponding to the pixel values ​​in each pixel combination and pixels that meet the target distance;

[0026] The number of matches is determined as a frequency, and the frequency set corresponding to the statistical direction is obtained.

[0027] Optionally, the step of segmenting the defective region image from the defective fruit image based on a clustering algorithm, filtering the defective region image using a median filtering algorithm, and determining a median filtered image of the defective region image comprises the following steps:

[0028] Acquire defective fruit images from the plurality of fruit images, convert grayscale image data of the defective fruit images into color image data, wherein the clustering algorithm includes a K-means clustering algorithm;

[0029] Segmenting the defective area image from the color image data of the defective fruit image based on a K-means clustering algorithm;

[0030] The defect area image is filtered based on the median filtering algorithm, the pixel values ​​of the current pixel point of the defect area image and the adjacent pixel points of the current pixel point are extracted, the pixel values ​​are sorted, and the pixel value at the middle position is used as the pixel value of the current pixel point to determine the median filtered image of the defect area image.

[0031] Optionally, the step of calculating a weight between the median filter image area of ​​the defective region image and the defective fruit image area, and sorting the fruit corresponding to the defective fruit image to a first conveyor belt when detecting that the weight exceeds a preset threshold, and sorting the fruit corresponding to the defective fruit image to a second conveyor belt when detecting that the weight is less than or equal to the preset threshold, to complete the fruit defect detection step includes the following steps:

[0032] performing image binarization processing on the defective fruit image and the defective region image in the defective fruit image to determine binary image data corresponding to the defective fruit image and the defective region image;

[0033] Determine the areas corresponding to the defective fruit image and the defective region image according to the binarized image data;

[0034] In response to the timing monitoring instruction, the weight between the area of ​​the defective region image and the area of ​​the defective fruit image is calculated, the comparison result between the weight and the preset threshold is determined, and the fruits corresponding to the defective fruit image are sorted according to the comparison result.

[0035] A fruit defect detection device provided for another purpose of the present application includes:

[0036] a region of interest determination module configured to respond to a fruit defect detection instruction, obtain a plurality of fruit images of the fruit to be detected in different light source directions, and segment the fruit images based on a global thresholding algorithm to determine a region of interest in the fruit images;

[0037] a preliminary detection module configured to extract texture features in the region of interest based on a feature extraction and classification algorithm, input the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit, and determine defective fruit images among the multiple fruit images;

[0038] a defective area processing module configured to segment a defective area image from the defective fruit image based on a clustering algorithm, filter the defective area image using a median filtering algorithm, and determine a median filtered image of the defective area image;

[0039] The defective fruit determination module is configured to calculate the weight between the median filter image area of ​​the defective region image and the area of ​​the defective fruit image. When it is detected that the weight exceeds a preset threshold, the fruit corresponding to the defective fruit image is sorted to the first conveyor belt. When it is detected that the weight is less than or equal to the preset threshold, the fruit corresponding to the defective fruit image is sorted to the second conveyor belt to complete the defect detection of the fruit.

[0040] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the fruit defect detection method described in the present application.

[0041] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the fruit defect detection method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0042] Compared with the existing technology, the present application addresses the problems in the existing technology of fruit detection such as long data collection and processing time, low detection efficiency, unstable recognition accuracy for feature selection and extraction, complicated detection equipment, and high purchase, installation and maintenance costs. The present application segments the fruit image based on a global thresholding algorithm to determine the region of interest in the fruit image, extracts texture features in the region of interest based on a feature extraction and classification algorithm, inputs the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit, determines defective fruit images in multiple fruit images, segments defective area images in the defective fruit images based on a clustering algorithm, filters the defective area images using a median filtering algorithm, calculates the weight between the median filtered image area of ​​the defective area image and the defective fruit image area, and when it is detected that the weight exceeds a preset threshold, the fruit segment corresponding to the defective fruit image is segmented. When the fruit is picked up to the first conveyor belt and the weight is detected to be less than or equal to the preset threshold, the fruit corresponding to the defective fruit image is sorted to the second conveyor belt. This not only significantly improves the sorting efficiency and accuracy of the fruit, but also accurately and quickly detects defects on the surface of the fruit, greatly saving the time of data processing in the fruit detection process. On the basis of classifying the fruits that meet the industry standards and the defective fruits, the defective fruits can be further classified according to their defect conditions, and the fruits that meet the industry standards, defective fruits and other categories of fruits can be accurately and quickly sorted out, greatly improving the detection accuracy of the fruit. The fruit defect detection method of the present application has fast recognition speed, high accuracy and good stability. At the same time, it solves the problems of low efficiency, high cost and poor stability of manual detection, greatly saves manpower and material resources, significantly improves the production efficiency of the fruit, and makes full use of resources, thereby improving the economic benefits of the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0044] Figure 1 This is an exemplary device used in the fruit defect detection method in the embodiments of the present application;

[0045] Figure 2 These are illustrations of various fruit defects in the embodiments of this application;

[0046] Figure 3 Schematic diagram of the process of the fruit defect detection method in the embodiment of the present application;

[0047] Figure 4 This is a schematic diagram of a process for determining a region of interest in the fruit image in an embodiment of the present application;

[0048] Figure 5 This is a schematic diagram of the process of performing preliminary defect detection on the fruit to be inspected in an embodiment of the present application;

[0049] Figure 6 Schematic diagram of a process for extracting texture features in a region of interest according to a feature extraction and classification algorithm in an embodiment of the present application;

[0050] Figure 7 Schematic diagram of the process of determining the median filter image of the defect area image in an embodiment of the present application;

[0051] Figure 8 This is a schematic diagram of a process for determining defective fruits according to weights in an embodiment of the present application;

[0052] Figure 9 This is a principle block diagram of the fruit defect detection device in an embodiment of the present application;

[0053] Figure 10 Schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0054] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0055] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0057] It will be understood by those skilled in the art that the terms "client," "terminal," and "terminal device" as used herein include both devices that are wireless signal receivers, i.e., devices that only have wireless signal receivers without transmission capabilities, and devices that have receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers and tablet computers, which have single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / Intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; and conventional laptop and / or palmtop computers or other devices, which have and / or include a radio frequency receiver. As used herein, the terms "client," "terminal," or "terminal device" may be portable, transportable, or installed in a vehicle (air, sea, and / or land), or may be adapted and / or configured to operate locally and / or in a distributed manner at any other location on Earth and / or in space. As used herein, the terms "client," "terminal," or "terminal device" may also refer to a communication terminal, an Internet terminal, or a music / video playback terminal, such as a PDA, an MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or may include a smart TV, a set-top box, or other device.

[0058] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. Computer programs are stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0059] It should be noted that the concept of "server" referred to in this application can also be extended to server clusters. Based on the network deployment principles understood by those skilled in the art, the servers described should be logically divided. In physical space, these servers can be independent of each other but callable through interfaces, or integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method of this application.

[0060] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for implementation.

[0061] Unless expressly specified otherwise, the neural network models referenced or may be referenced in this application may be deployed on a remote server and remotely called on the client, or may be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence may be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0062] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as they are suitable for being called by the technical solution of this application.

[0063] Those skilled in the art should be aware that although the various methods of this application are described based on the same concept and thus exhibit commonality, unless otherwise specified, these methods can be independently executed. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept. Therefore, concepts with the same expression, as well as concepts that are appropriately transformed for convenience despite different expression, should be understood as equivalent.

[0064] Unless expressly stated to be mutually exclusive, the various embodiments disclosed in this application may be cross-combined with the relevant technical features of the various embodiments to flexibly construct new embodiments, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or resolve certain deficiencies in the prior art. Those skilled in the art should be aware of such flexibility.

[0065] In an exemplary embodiment of the present invention, the fruit defect detection method can be based on Figure 1 The fruit sorting device shown is implemented, see Figure 1 and Figure 2, the hardware foundation required for the implementation of the relevant technical solutions of this application can be deployed according to the structure shown in the figure. The fruit sorting device includes a camera 1, a light source 2, a feeding conveyor belt 3, a fruit clamp 4, a manipulator 5 and a material distribution conveyor belt 6, etc. The fruit 7 is fixed by the fruit clamp 4, and the fruit clamp 4 is composed of a limiting column and a base. The fruit is placed in the limiting column and fixed, and is sent to the inside of the fruit defect detection system via the feeding conveyor belt 3. The camera 1 inside the fruit defect detection system captures the fruit to determine multiple fruit images of the fruit to be detected in different light source directions. The fruit includes apples, pears or peaches, etc. The camera can capture multiple angles of the fruit, such as the front, back, left and right, to obtain fruit images of the fruit to be detected at multiple angles in different light source directions, and obtain all-round surface images of the fruit to be detected. A plurality of light sources 3 are distributed around the fruit sorting device, and the plurality of light sources 3 can provide a bright lighting environment for the camera to capture the fruit to be detected. After the camera captures the fruit, the defect area of ​​the plurality of fruit surface images of each fruit to be detected can be detected by a visual algorithm. When the defect area of ​​the fruit to be detected is less than the preset threshold, the robot 4 grabs it and places it into the dividing conveyor belt ② and the dividing conveyor belt ③. When the defect area of ​​the fruit to be detected exceeds the preset threshold, it is grabbed and placed into the dividing conveyor belt ① and the dividing conveyor belt ④. The intact fruit that meets the industry standards remains on the feeding conveyor belt, thereby completing the detection and sorting of fruit defects. The preset threshold can be 1%, 1.5% or 2%, etc. Those skilled in the art can determine the preset threshold as needed according to actual conditions, and no limitation is made here.

[0066] Based on the above example scenarios, please refer to Figure 3 In one embodiment, the fruit defect detection method of the present application comprises the following steps:

[0067] Step S10: responding to a fruit defect detection instruction, acquiring a plurality of fruit images of the fruit to be detected in different light source directions, and segmenting the fruit images based on a global thresholding algorithm to determine a region of interest in the fruit images;

[0068] The terminal device within the fruit defect detection system can respond to a fruit defect detection instruction, obtain multiple fruit images of the fruit to be detected in different light source directions, preprocess the multiple fruit images, crop the multiple fruit images, unify the image size and resolution of the multiple fruit images, segment the fruit images based on a global thresholding algorithm to determine the region of interest in the fruit images, obtain color image data of the multiple fruit images, convert the color image data of the multiple fruit images into grayscale images, and obtain a color channel matrix of each pixel in the grayscale image of the fruit image. The global thresholding algorithm can be:

[0069]

[0070] Among them, T represents the global threshold, n represents the row of the color channel matrix, m represents the column of the color channel matrix, and Gimage represents the gray value of each pixel in the gray image. After determining the global threshold corresponding to the color channel matrix of each pixel in the fruit image, the gray value corresponding to the color channel matrix of each pixel is subjected to image binarization processing according to the global threshold, and the binarized image data corresponding to the color channel matrix of each pixel is determined. The region of interest in the fruit image is determined according to the binarized image data of the fruit image. For example: if Gimage ij < T, then Simage(i,j) = 0; if Gimage ij > T, then Simage(i,j) = 1. Among them, Simage represents the pixel gray value of the binary image. The pixel values corresponding to each pixel in the gray image of the fruit image are subjected to binarization processing to determine the binarized image data values corresponding to each pixel in the gray image of the fruit image, that is, the pixel gray value of the binary image, so as to extract and segment the region of interest in the fruit image.

[0071] Step S20: Extract the texture features in the region of interest according to the feature extraction and classification algorithm, input the texture features into the naive Bayes classification algorithm to perform preliminary defect detection on the fruit, and determine the defective fruit images in the multiple fruit images;

[0072] The defective fruit image represents an image of a fruit with defects among the multiple fruit images. After the fruit image is segmented based on a global thresholding algorithm to determine the region of interest in the fruit image, a feature extraction and classification algorithm in a terminal device within the fruit defect detection system can extract texture features in the region of interest. The feature extraction and classification algorithm can be a wavelet transform algorithm, etc. First, a Haar filter is used to obtain the discrete wavelet transform coefficients DWT (Discrete Wavelet Transform) of the diagonal (CDCoefficients for Diagonal), vertical (CV-Coefficients for Vertical) and horizontal pixel values ​​(CHCoefficients for Horizontal) of each pixel in the region of interest of the multiple fruit images. Wavelet Transformation) is used to extract texture features. A shear wavelet transform can be performed on each image sub-block in the region of interest based on cone shear wavelet to obtain a first-level decomposition coefficient after the shear wavelet transform. Then, the first-level decomposition coefficient x is denoised according to a preset threshold W to obtain a second-level decomposition coefficient. Each image sub-block is detected according to the shear wavelet edge detection method based on the second-level decomposition coefficient to obtain the texture features of each image sub-block, wherein the first-level decomposition coefficient and the second-level decomposition coefficient are both wavelet coefficients. The mean and standard deviation can be calculated based on the first-level decomposition coefficient and the second-level decomposition coefficient to further obtain the statistical features of the region of interest of the multiple fruit images. The statistical features include the mean, standard deviation, energy, uniformity, contrast and correlation between each pixel in the region of interest, etc. Each of the statistical features is expressed as follows:

[0073]

[0074] Among them, the mean μ represents the mean between each pixel point in the region of interest, the standard deviation σ represents the standard deviation between each pixel point in the region of interest, the uniformity H represents the uniformity between each pixel point in the region of interest, the contrast Ct represents the contrast between each pixel point in the region of interest, the correlation Co represents the correlation between each pixel point in the region of interest, the n represents the row of the color channel matrix of each pixel point in the region of interest, and m represents the column of the color channel matrix of the color channel matrix of each pixel point in the region of interest.

[0075] Obtain pixel grayscale values ​​of a binary image in a region of interest of the fruit image, determine all vertical, horizontal, and diagonal coefficients obtained by primary decomposition and secondary decomposition, apply a wavelet transform algorithm based on the vertical, horizontal, and diagonal coefficients to extract texture features and statistical features of the region of interest of the fruit image, extract texture features such as energy, uniformity, contrast, and correlation of diagonal [CD], vertical [CV], and horizontal [CH] from the region of interest of the fruit image, and input the texture features such as energy, uniformity, contrast, and correlation of diagonal [CD], vertical [CV], and horizontal [CH] into a naive Bayes classification method to distinguish defective fruit images (DimageRGB) from non-defective fruit images (SoundimageRGB) among the multiple fruit images.

[0076] Specifically, the naive Bayes classification algorithm is expressed as follows:

[0077]

[0078] Assuming B has already happened, we can find A i The probability of occurrence. B is evidence, A i Assume that the extracted features are independent, and the presence of any feature does not affect the others. In this case, the classification of the fruit samples is accomplished by assigning texture and statistically independent features as predictors, with both features having an equal impact on the experimental results. Therefore, the maximum probability value of the probability y = ("defective" or "intact") can be calculated using the Naive Bayes formula, and this is used as the criterion for classifying the defective and non-defective fruit images.

[0079] Step S30: segmenting the defective region image in the defective fruit image based on a clustering algorithm, filtering the defective region image using a median filtering algorithm, and determining a median filtered image of the defective region image;

[0080] The defective region image represents an image of the defective region portion of the defective fruit image. After determining the defective fruit image from the plurality of fruit images, the intact fruit and the defective fruit are classified, and the defective fruit image is further processed based on a clustering algorithm. The clustering algorithm includes a K-means clustering algorithm to segment the defective region image from the defective fruit image. The defective fruit image from the plurality of fruit images is obtained, the grayscale image data of the defective fruit image is converted into color image data, the defective region image is segmented from the color image data of the defective fruit image based on the K-means clustering algorithm, and the defective region image is filtered based on a median filtering algorithm. The pixel values ​​of the current pixel point of the defective region image and the pixel values ​​of the adjacent pixels of the current pixel point are extracted, the pixel values ​​are sorted, and the pixel value at the middle position is used as the pixel value of the current pixel point to determine a median filtered image of the defective region image, thereby enhancing the clarity of the defective region portion of the defective fruit image.

[0081] In a K-means clustering-based image segmentation algorithm, the acquired defective fruit image is converted from grayscale image data to color image data, and ImageRGB is converted to the L*A*B* color space. The L*A*B* color space easily distinguishes visual differences in color. This color space includes a luminosity layer (L*), a chromaticity layer (A*), colors along the blue-yellow axis (B*), and colors along the red-green axis. The differences between these colors can be determined using the Euclidean distance metric. The defective areas in the defective fruit image are segmented using the K-Means clustering algorithm. The K-means clustering algorithm uses color counts as cluster counts and cluster centers as color values. The K-Means algorithm first creates "K" clusters. For each pixel in the defective fruit image, the Euclidean distance metric is used to find the cluster with the minimum distance. If the pixel is already in another cluster, it is removed from the cluster and added to a new cluster. If a pixel is added to a cluster, the cluster center value is adjusted and the pixel's color value is removed from the old cluster. Repeat the above steps until there are no more pixel values ​​left for separation, locate the correctly segmented image, and use the median filtering algorithm to enhance the clarity of the defective area in the defective fruit image. The median filtering algorithm can obtain the pixel values ​​of the current pixel point and the surrounding adjacent pixels (a total of an odd number of pixels) of the defective area image, sort the current pixel point and the surrounding adjacent pixels, and then use the pixel value in the middle position as the pixel value of the current pixel point. The median filtering algorithm has a good filtering effect on isolated noise pixels, such as salt and pepper noise, impulse noise, etc., can maintain the edge characteristics of the image, avoid significant blurring of the image, and greatly improve the clarity of the defective area in the defective fruit image.

[0082] Step 40: Calculate the weight between the median filter image area of ​​the defective region image and the defective fruit image area. When it is detected that the weight exceeds a preset threshold, sort the fruit corresponding to the defective fruit image to the first conveyor belt. When it is detected that the weight is less than or equal to the preset threshold, sort the fruit corresponding to the defective fruit image to the second conveyor belt to complete the defect detection of the fruit.

[0083] After determining the median filter image corresponding to the defective region image in the defective fruit image, a weight between the median filter image area of ​​the defective region image and the area of ​​the defective fruit image can be calculated based on a defect area calculation algorithm. Based on the weight between the median filter image area of ​​the defective region image and the area of ​​the defective fruit image, it can be determined whether the fruit to be inspected meets industry standards. When it is detected that the weight exceeds a preset threshold, the fruit corresponding to the defective fruit image is determined to be unqualified fruit, thereby completing the defect detection of the fruit. The preset threshold can be 1%, 1.5%, or 2%, etc.

[0084] In some embodiments, the percentage of the defective area corresponding to the defective area portion in the defective fruit image is calculated based on a defective area calculation algorithm (DACA Defective Area Calculation Algorithm). A global thresholding algorithm (GTA) can be used to convert the defective fruit image and the defective area image in the defective fruit image into binary image data, and the area of ​​the defective fruit image and the defective area image is determined by calculating the number of pixel values ​​of the defective fruit image and the defective area image in the defective fruit image that are equal to 1. The area of ​​the defective fruit image and the defective area image determined above is calculated as a percentage, and the calculation formula is expressed as follows:

[0085]

[0086] D represents the defect percentage, A0 represents the image area of ​​the defective region, and A1 represents the area of ​​the segmented image feature. The first conveyor belt includes the dividing conveyor belt ① and the dividing conveyor belt ④, etc., and the second conveyor belt includes the dividing conveyor belt ② and the dividing conveyor belt ③, etc. If the defect percentage D of the fruit to be detected is greater than 1%, the defect area of ​​the fruit is too large, and the fruit to be detected is far below the industry standard, then it will be grabbed and placed in the dividing conveyor belt ① and the dividing conveyor belt ④. If the defect percentage D of the fruit to be detected is less than or equal to 1%, the robot 4 will grab it and place it in the dividing conveyor belt ② and the dividing conveyor belt ③, and the intact fruit that meets the industry standard will remain on the feeding conveyor, so as to complete the detection and sorting of fruit defects.

[0087] In some embodiments, the defective area of ​​the fruit is detected by a machine vision algorithm, and the defective fruit is further classified, which can effectively avoid the waste of fruit products, accurately and quickly detect and sort out different categories of fruit products, and greatly improve resource utilization. For example, fruits with a defective area exceeding 1% can be used as animal feed, etc., and fruits with a defective area less than or equal to 1% can be used as raw materials for the production of fruit wine, thereby making full use of resources and greatly improving economic benefits.

[0088] It can be seen from the above embodiments that, compared with the prior art, the present application addresses the problems in the prior art of fruit detection such as long data collection and processing time, low detection efficiency, unstable recognition accuracy for feature selection and extraction, complicated detection equipment, and high purchase, installation and maintenance costs. The present application segments the fruit image based on a global thresholding algorithm to determine the region of interest in the fruit image, extracts texture features in the region of interest based on a feature extraction and classification algorithm, inputs the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit, determines defective fruit images in multiple fruit images, segments defective area images in defective fruit images based on a clustering algorithm, filters the defective area images using a median filtering algorithm, calculates the weight between the median filtered image area of ​​the defective area image and the area of ​​the defective fruit image, and when it is detected that the weight exceeds a preset threshold, the defective fruit image is segmented. The fruits are sorted to the first conveyor belt, and when it is detected that the weight is less than or equal to the preset threshold, the fruits corresponding to the defective fruit images are sorted to the second conveyor belt. This not only significantly improves the sorting efficiency and sorting accuracy of the fruits, but also can accurately and quickly detect defects on the surface of the fruits, greatly saving the time of data processing in the fruit detection process. On the basis of classifying the fruits that meet the industry standards and the defective fruits, the defective fruits can be further classified according to their defect conditions, and the fruits that meet the industry standards, defective fruits and other categories of fruits can be accurately and quickly sorted out, greatly improving the detection accuracy of the fruits. The fruit defect detection method of the present application has fast recognition speed, high accuracy and good stability. At the same time, it solves the problems of low efficiency, high cost and poor stability of manual detection, greatly saves manpower and material resources, significantly improves the production efficiency of the fruit, and makes full use of resources, thereby improving the economic benefits of the enterprise.

[0089] Based on any embodiment of this application, please refer to Figure 4 The step of segmenting the fruit image based on a global thresholding algorithm to determine a region of interest in the fruit image comprises the following steps:

[0090] Step S101: converting the fruit image into grayscale image data to obtain multiple color channel matrices in the fruit image;

[0091] Step S103: Calculate the global thresholds corresponding to the multiple color channel matrices in the grayscale image data of the fruit image according to a preset global thresholding algorithm;

[0092] Step S105: Perform image binarization on the grayscale values corresponding to the multiple color channel matrices according to the global threshold, determine the binarized image data corresponding to the multiple color channel matrices, and determine the region of interest in the fruit image according to the binarized image data of the fruit image.

[0093] Specifically, the color image data of the multiple fruit images can be obtained, and the color image data of the multiple fruit images can be converted into a grayscale image, so as to obtain the color channel matrix of each pixel in the grayscale image of the fruit image. Calculate the global thresholds corresponding to the multiple color channel matrices in the grayscale image data of the fruit image according to a preset global thresholding algorithm. After determining the global thresholds corresponding to the color channel matrices of each pixel in the fruit image, perform image binarization on the grayscale values corresponding to the color channel matrices of each pixel according to the global threshold, determine the binarized image data corresponding to the color channel matrices of each pixel, and determine the region of interest in the fruit image according to the binarized image data of the fruit image.

[0094] In some embodiments, if Gimage ij < T, then Simage(i,j) = 0. If Gimage ij > T, then Simage(i,j) = 1. Wherein, the Gimage represents the grayscale value of each pixel in the grayscale image, the T represents the global threshold, and the Simage represents the pixel grayscale value of the binary image. Perform binarization on the pixel values corresponding to each pixel in the grayscale image of the fruit image, determine the binarized image data values corresponding to each pixel in the grayscale image of the fruit image, that is, the pixel grayscale value of the binary image, so as to extract and segment the region of interest in the fruit image.

[0095] Based on any embodiment of the present application, please refer to Figure 5 , the steps of extracting the texture features in the region of interest according to the feature extraction and classification algorithm and inputting the texture features into the naive Bayes classification algorithm for preliminary defect detection of the fruit include the following steps:

[0096] Step S201: Use a Haar filter to obtain the diagonal pixel values, vertical pixel values, horizontal pixel values, uniformity, correlation, and contrast in the region of interest of the fruit image;

[0097] Step S203: input the diagonal pixel values, vertical pixel values, horizontal pixel values, uniformity, correlation and contrast in the region of interest of the fruit image into the naive Bayes classification algorithm to preliminarily detect defective fruit images in the fruit image.

[0098] Specifically, the pixel grayscale values ​​of the binary image in the region of interest of the fruit image are obtained, all vertical, horizontal and diagonal coefficients obtained by the first-level decomposition and the second-level decomposition are determined, and a wavelet transform algorithm is applied according to the vertical, horizontal and diagonal coefficients to extract the texture features and statistical features of the region of interest of the fruit image, and the energy, uniformity, contrast, correlation and other texture features of the diagonal [CD], vertical [CV] and horizontal [CH] are extracted from the region of interest of the fruit image, and the energy, uniformity, contrast, correlation and other texture features of the diagonal [CD], vertical [CV] and horizontal [CH] are input into the naive Bayes classification method to distinguish the defective fruit image (DimageRGB) and the non-defective fruit image (SoundimageRGB) among the multiple fruit images.

[0099] Based on any embodiment of this application, please refer to Figure 6 The step of extracting the texture features in the region of interest according to the feature extraction and classification algorithm comprises the following steps:

[0100] Step S202: obtaining a plurality of pixel values ​​of a region of interest of the fruit image, and determining pixel combinations between respective pixel values ​​within a preset pixel value interval;

[0101] In one embodiment, the terminal device within the fruit defect detection system can set a pixel value interval based on the pixel values ​​in the multiple fruit images. For example, if the pixel values ​​of the fruit image's region of interest (ROI) are primarily concentrated within the range of 0 to N, the pixel value interval is [0, N]. When a large number of ROIs are acquired, the terminal device normalizes the pixel values ​​of the ROIs within the same data domain so that the pixel values ​​of the ROIs within the same data domain are within a unified pixel interval. For example, if the pixel values ​​of the fruit image's ROI are primarily concentrated within the range of 0 to 100, then pixel values ​​greater than 100 within the ROI of each fruit image are converted to within the range of 0 to 100, or pixel values ​​greater than 100 within the ROI of each fruit image are deleted. If the pixel value interval is [0, N], the number of all possible pixel combinations within the pixel value interval [0, N] is F(n) = n! = n × (n-1) × ... × 2 × 1. For example, assuming that N=12, the number of pixel combinations f(n)=12×11×...×2×1, such as the number of pixel combinations (0,0), (0,1), (1,0), ..., (12,12), etc. It should be noted that a pixel combination can be a combination of two or three pixel values, and those skilled in the art can determine the pixel combination as needed according to actual conditions, and this is not limited here.

[0102] Step S204, counting the frequency of pixels corresponding to the pixel values ​​in the pixel combination appearing in the region of interest of the fruit image, and determining a frequency set of the pixels;

[0103] Determine a target distance and multiple statistical directions, the number of directions of the statistical directions is multiple, obtain pixels that meet the target distance in the area of ​​interest of the fruit image, and count the number of pixels corresponding to the pixel values ​​in each pixel combination that match the pixels that meet the target distance according to each statistical direction, determine the number of matches as the frequency, and obtain multiple sets of frequency numbers corresponding to the number of directions. Among them, for the area of ​​interest of the fruit image, there can be four statistical directions, namely 0°, 45°, 90° and 135°. For the area of ​​interest of the fruit image, calculations can be performed separately according to the pixel values ​​of different channels. The target distance can generally be set to 1. Those skilled in the art can determine it as needed according to actual conditions, and this is not limited here.

[0104] Step S206: convert the frequencies in the frequency set into probabilities, and extract the texture features in the region of interest according to the probability of the pixel appearing in the region of interest of the fruit image.

[0105] The terminal device determines the number of channels of the region of interest of the fruit image, converts the frequencies in the multiple frequency sets into probabilities, calculates the probability means of the corresponding positions in the multiple frequency sets after conversion, obtains an averaged probability set whose number of sets is consistent with the number of channels based on the probabilities, and determines the averaged probability set as a texture feature.

[0106] It can be seen from the above embodiments that by converting the frequencies in the frequency set into probabilities and extracting the texture features in the region of interest based on the probability of the pixels appearing in the region of interest of the fruit image, the algorithm model can be focused on the region of interest of the fruit image, making the extracted features more accurate, ensuring the accuracy of the fruit defect detection results, avoiding the misdetection of intact fruits as defective fruits, and greatly improving the accuracy of fruit defect detection.

[0107] Based on any embodiment of this application, please refer to Figure 7 The steps of segmenting the defective region image from the defective fruit image based on a clustering algorithm, filtering the defective region image using a median filtering algorithm, and determining a median filtered image of the defective region image include the following steps:

[0108] Step S301: acquiring defective fruit images from the plurality of fruit images, converting grayscale image data of the defective fruit images into color image data, wherein the clustering algorithm includes a K-means clustering algorithm;

[0109] Step S303: segmenting the defective area image from the color image data of the defective fruit image based on a K-means clustering algorithm;

[0110] Step S305: Filter the defect area image based on a median filtering algorithm, extract the pixel values ​​of the current pixel point of the defect area image and the adjacent pixels of the current pixel point, sort the pixel values, and use the pixel value at the middle position as the pixel value of the current pixel point to determine the median filtered image of the defect area image.

[0111] Specifically, a defective fruit image from the multiple fruit images is obtained, the obtained defective fruit image is converted, the grayscale image data of the defective fruit image is converted into color image data, the defective area image is segmented from the color image data of the defective fruit image based on the K-means clustering algorithm, and a median filtering algorithm is used to enhance the clarity of the defective area portion in the defective fruit image. The median filtering algorithm can obtain the pixel values ​​of the current pixel point and the surrounding adjacent pixel points (a total of an odd number of pixel points) of the defective area image, sort the current pixel point and the surrounding adjacent pixel points, and then use the pixel value at the middle position as the pixel value of the current pixel point. The median filtering algorithm has a good filtering effect on isolated noise pixels, such as salt and pepper noise, pulse noise, etc., can maintain the edge characteristics of the image, avoid significant blurring of the image, and greatly improve the clarity of the defective area portion in the defective fruit image.

[0112] Based on any embodiment of this application, please refer to Figure 8 , calculating the weight between the median filter image area of ​​the defective region image and the area of ​​the defective fruit image, and when detecting that the weight exceeds a preset threshold, sorting the fruit corresponding to the defective fruit image to a first conveyor belt; when detecting that the weight is less than or equal to the preset threshold, sorting the fruit corresponding to the defective fruit image to a second conveyor belt, thereby completing the step of fruit defect detection, comprising the following steps:

[0113] Step 401: performing image binarization processing on the defective fruit image and the defective region image in the defective fruit image to determine binary image data corresponding to the defective fruit image and the defective region image;

[0114] Step 403: determining the areas corresponding to the defective fruit image and the defective region image based on the binarized image data;

[0115] Step 405, responding to the timing monitoring instruction, calculating the weight between the area of ​​the defective region image and the area of ​​the defective fruit image, determining the comparison result between the weight and the preset threshold, and sorting the fruit corresponding to the defective fruit image according to the comparison result.

[0116] Specifically, the percentage of the defective area corresponding to the defective area portion in the defective fruit image is calculated based on the defective area calculation algorithm (DACA Defective Area Calculation Algorithm). The global thresholding algorithm (GTA) can be used to convert the defective fruit image and the defective area image in the defective fruit image into binary image data, and the area of ​​the defective fruit image and the defective area image is determined by calculating the number of pixel values ​​of the defective fruit image and the defective area image in the defective fruit image that are equal to 1. The area of ​​the defective fruit image and the defective area image determined above is calculated as a percentage, and the calculation formula is expressed as follows:

[0117]

[0118] D represents the defect percentage, A0 represents the image area of ​​the defect region, and A1 represents the area of ​​the segmented image feature. If the defect percentage D of the fruit to be detected is greater than 1%, the defect area of ​​the fruit is too large, and the fruit to be detected is lower than the industry standard, then the fruit to be detected is classified as defective fruit. If the defect percentage D of the fruit to be detected is less than or equal to 1%, then the fruit to be detected is classified as intact fruit, thereby completing the detection of the fruit defects.

[0119] It can be seen from the above embodiments that the percentage of the defect area corresponding to the defective area part in the defective fruit image is calculated based on the defect area calculation algorithm (DACA Defective Area Calculation Algorithm). The algorithm is simple and efficient, and can accurately and quickly sort out fruits that meet industry standards, defective fruits and other categories of fruits, thereby greatly improving the detection accuracy of fruits. The fruit defect detection method of the present application has fast recognition speed, high accuracy and good stability. At the same time, it solves the problems of low efficiency, high cost and poor stability of manual detection, greatly saves manpower and material resources, significantly improves the production efficiency of fruits, and makes full use of resources, thereby improving the economic benefits of the enterprise.

[0120] See also Figure 9, a fruit defect detection device provided to meet one of the purposes of this application includes. Among them, the region of interest determination module 1100 is configured to respond to the fruit defect detection instruction, obtain multiple fruit images of the fruit to be detected in different light source directions, and segment the fruit images based on the global thresholding algorithm to determine the region of interest in the fruit image; the preliminary detection module 1200 is configured to extract texture features in the region of interest based on the feature extraction and classification algorithm, input the texture features into the naive Bayes classification algorithm to perform preliminary defect detection on the fruit, and determine the defective fruit image in the multiple fruit images; the defect area processing module 1300 is configured to segment the defect area image in the defective fruit image based on the clustering algorithm, filter the defect area image using the median filtering algorithm, and determine the median filtered image of the defect area image; the defective fruit determination module 1400 is configured to calculate the weight between the median filtered image area of ​​the defect area image and the area of ​​the defective fruit image, and when it is detected that the weight exceeds a preset threshold, the fruit corresponding to the defective fruit image is sorted to the first conveyor belt; when it is detected that the weight is less than or equal to the preset threshold, the fruit corresponding to the defective fruit image is sorted to the second conveyor belt to complete the defect detection of the fruit.

[0121] Based on any embodiment of the present application, the region of interest determination module 1100 includes:

[0122] a color channel acquisition unit configured to convert the fruit image into grayscale image data and acquire a plurality of color channel matrices in the fruit image;

[0123] A global threshold determination unit is configured to calculate a global threshold corresponding to the plurality of color channel matrices in the grayscale image data of the fruit image according to a preset global thresholding algorithm;

[0124] The region of interest determination unit is configured to perform image binarization processing on the grayscale values ​​corresponding to the multiple color channel matrices according to the global threshold, determine the binarized image data corresponding to the multiple color channel matrices, and determine the region of interest in the fruit image based on the binarized image data of the fruit image.

[0125] Based on any embodiment of the present application, the preliminary detection module 1200 includes:

[0126] an acquisition unit configured to acquire diagonal pixel values, vertical pixel values, horizontal pixel values, uniformity, correlation, and contrast in a region of interest of the fruit image using a Haar filter;

[0127] The preliminary detection unit is configured to input the diagonal pixel values, vertical pixel values, horizontal pixel values, uniformity, correlation and contrast in the area of ​​interest of the fruit image into the naive Bayes classification algorithm to preliminarily detect defective fruit images in the fruit image.

[0128] Based on any embodiment of the present application, the preliminary detection module 1200 includes:

[0129] a pixel combination determining unit configured to obtain a plurality of pixel values ​​of an area of ​​interest of the fruit image and determine a pixel combination between respective pixel values ​​within a preset pixel value interval;

[0130] a frequency set determining unit configured to count the frequencies of pixels corresponding to the pixel values ​​in the pixel combination appearing in the region of interest of the fruit image to determine the frequency set of the pixels;

[0131] The texture feature determination unit is configured to convert the frequencies in the frequency set into probabilities, and extract the texture features in the region of interest according to the probability of the pixels appearing in the region of interest of the fruit image.

[0132] Based on any embodiment of the present application, the frequency set determining unit includes:

[0133] a pixel acquisition subunit, configured to determine a target distance and a plurality of statistical directions, and acquire pixels in a region of interest of the fruit image that meet the target distance;

[0134] a statistical subunit configured to count, according to each statistical direction, the number of matches between pixels corresponding to the pixel values ​​in each pixel combination and pixels meeting the target distance;

[0135] The frequency set determining unit is configured to determine the number of matches as a frequency, and obtain the frequency set corresponding to the statistical direction.

[0136] Based on any embodiment of the present application, the defective area processing module 1300 includes:

[0137] an image conversion unit configured to obtain a defective fruit image from the plurality of fruit images, and convert grayscale image data of the defective fruit image into color image data, wherein the clustering algorithm comprises a K-means clustering algorithm;

[0138] a defect area segmentation unit configured to segment the defect area image from the color image data of the defective fruit image based on a K-means clustering algorithm;

[0139] The filtered image determination unit is configured to filter the defect area image based on a median filtering algorithm, extract the pixel values ​​of the current pixel point of the defect area image and the pixel values ​​of the adjacent pixels of the current pixel point, sort the pixel values, and use the pixel value at the middle position as the pixel value of the current pixel point to determine the median filtered image of the defect area image.

[0140] Based on any embodiment of the present application, the defective fruit determination module 1400 includes:

[0141] A binarization processing unit is configured to perform image binarization processing on the defective fruit image and the defective region image in the defective fruit image, and determine binary image data corresponding to the defective fruit image and the defective region image;

[0142] an area determining unit, configured to determine the areas corresponding to the defective fruit image and the defective region image according to the binarized image data;

[0143] The defective fruit determination unit is configured to respond to a timing monitoring instruction, calculate the weight between the area of ​​the defective region image and the area of ​​the defective fruit image, determine a comparison result between the weight and the preset threshold, and sort the fruit corresponding to the defective fruit image according to the comparison result.

[0144] Based on any embodiment of this application, please refer to Figure 10 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 10 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a fruit defect detection method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor may execute the fruit defect detection method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0145] In this embodiment, the processor is used to execute Figure 9 The memory stores the program code and various data required to execute the modules and submodules. The network interface is used to transmit data between user terminals and servers. The memory in this embodiment stores the program code and data required to execute all modules and submodules in the fruit defect detection device of this application. The server can call the server's program code and data to execute the functions of all submodules.

[0146] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the fruit defect detection method described in any embodiment of the present application.

[0147] The present application also provides a computer program product, comprising a computer program / instruction, which, when executed by one or more processors, implements the steps of the fruit defect detection method described in any embodiment of the present application.

[0148] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the method. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0149] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0150] To sum up, the present application can accurately and quickly sort out fruits that meet industry standards, defective fruits and other categories of fruits, greatly improving the detection accuracy of fruits. The fruit defect detection method of the present application has fast recognition speed, high accuracy and good stability. At the same time, it solves the problems of low efficiency, high cost and poor stability of manual inspection, greatly saves manpower and material resources, significantly improves the production efficiency of fruits, and makes full use of resources, thereby improving the economic benefits of the enterprise.

Claims

1. A method for detecting fruit defects, characterized in that: The steps include: In response to a fruit defect detection instruction, a plurality of fruit images of the fruit to be detected in different light source directions are acquired, and the fruit images are segmented based on a global thresholding algorithm to determine a region of interest in the fruit image; Extracting texture features in the region of interest according to a feature extraction and classification algorithm, inputting the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit, and determining defective fruit images among the multiple fruit images, wherein the step of extracting texture features in the region of interest according to the feature extraction and classification algorithm includes: Acquire multiple pixel values ​​of the region of interest of the fruit image, and determine pixel combinations between the pixel values ​​within a preset pixel value interval; Counting the frequency of pixels corresponding to the pixel values ​​in the pixel combinations appearing in the region of interest of the fruit image to determine a frequency set of the pixels, which includes: determining a target distance and multiple statistical directions, obtaining pixels in the region of interest of the fruit image that meet the target distance; counting the number of matches between pixels corresponding to the pixel values ​​in each pixel combination and pixels that meet the target distance according to each statistical direction; determining the number of matches as a frequency to obtain the frequency set corresponding to the statistical direction; Converting the frequencies in the frequency set into probabilities, and extracting texture features in the region of interest according to the probability of the pixels appearing in the region of interest of the fruit image; Segmenting a defective region image from the defective fruit image based on a clustering algorithm, filtering the defective region image using a median filtering algorithm, and determining a median filtered image of the defective region image; The weight between the median filter image area of ​​the defective region image and the area of ​​the defective fruit image is calculated. When it is detected that the weight exceeds a preset threshold, the fruit corresponding to the defective fruit image is sorted to a first conveyor belt. When it is detected that the weight is less than or equal to the preset threshold, the fruit corresponding to the defective fruit image is sorted to a second conveyor belt to complete the defect detection of the fruit.

2. The fruit defect detection method according to claim 1, characterized in that: The step of segmenting the fruit image based on a global thresholding algorithm to determine a region of interest in the fruit image comprises the following steps: Converting the fruit image into grayscale image data to obtain multiple color channel matrices in the fruit image; Calculating global thresholds corresponding to the plurality of color channel matrices in the grayscale image data of the fruit image according to a preset global thresholding algorithm; The grayscale values ​​corresponding to the multiple color channel matrices are subjected to image binarization processing according to the global threshold value, the binarized image data corresponding to the multiple color channel matrices are determined, and the region of interest in the fruit image is determined according to the binarized image data of the fruit image.

3. The fruit defect detection method according to claim 2, characterized in that: The step of extracting texture features in the region of interest according to a feature extraction and classification algorithm and inputting the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit comprises the following steps: Using a Haar filter to obtain diagonal pixel values, vertical pixel values, horizontal pixel values, uniformity, correlation, and contrast in a region of interest of the fruit image; The diagonal pixel values, vertical pixel values, horizontal pixel values, uniformity, correlation and contrast in the region of interest of the fruit image are input into the naive Bayes classification algorithm to preliminarily detect defective fruit images in the fruit image.

4. The fruit defect detection method according to claim 1, characterized in that: The step of segmenting a defective region image from the defective fruit image based on a clustering algorithm, filtering the defective region image using a median filtering algorithm, and determining a median filtered image of the defective region image comprises the following steps: Acquire a defective fruit image from the plurality of fruit images, convert grayscale image data of the defective fruit image into color image data, wherein the clustering algorithm includes a K-means clustering algorithm; Segmenting the defective area image from the color image data of the defective fruit image based on a K-means clustering algorithm; The defect area image is filtered based on the median filtering algorithm, the pixel values ​​of the current pixel point of the defect area image and the adjacent pixel points of the current pixel point are extracted, the pixel values ​​are sorted, and the pixel value at the middle position is used as the pixel value of the current pixel point to determine the median filtered image of the defect area image.

5. The fruit defect detection method according to claim 1, characterized in that: The step of calculating a weight between the median filter image area of ​​the defective region image and the defective fruit image area, and sorting the fruit corresponding to the defective fruit image to a first conveyor belt when detecting that the weight exceeds a preset threshold, and sorting the fruit corresponding to the defective fruit image to a second conveyor belt when detecting that the weight is less than or equal to the preset threshold, so as to complete the defect detection of the fruit, comprises the following steps: performing image binarization processing on the defective fruit image and the defective region image in the defective fruit image to determine binary image data corresponding to the defective fruit image and the defective region image; Determine the areas corresponding to the defective fruit image and the defective region image according to the binarized image data; In response to the timing monitoring instruction, the weight between the area of ​​the defective region image and the area of ​​the defective fruit image is calculated, the comparison result between the weight and the preset threshold is determined, and the fruits corresponding to the defective fruit image are sorted according to the comparison result.

6. A fruit defect detection device, characterized in that: include: a region of interest determination module configured to respond to a fruit defect detection instruction, obtain a plurality of fruit images of the fruit to be detected in different light source directions, and segment the fruit images based on a global thresholding algorithm to determine a region of interest in the fruit images; The preliminary detection module is configured to extract texture features in the region of interest based on a feature extraction and classification algorithm, input the texture features into a naive Bayes classification algorithm to perform preliminary defect detection on the fruit, and determine defective fruit images among the multiple fruit images, wherein the step of extracting texture features in the region of interest based on the feature extraction and classification algorithm includes: Acquire multiple pixel values ​​of the region of interest of the fruit image, and determine pixel combinations between the pixel values ​​within a preset pixel value interval; Counting the frequency of pixels corresponding to the pixel values ​​in the pixel combinations appearing in the region of interest of the fruit image to determine a frequency set of the pixels, which includes: determining a target distance and multiple statistical directions, obtaining pixels in the region of interest of the fruit image that meet the target distance; counting the number of matches between pixels corresponding to the pixel values ​​in each pixel combination and pixels that meet the target distance according to each statistical direction; determining the number of matches as a frequency to obtain the frequency set corresponding to the statistical direction; Converting the frequencies in the frequency set into probabilities, and extracting texture features in the region of interest according to the probability of the pixels appearing in the region of interest of the fruit image; a defective area processing module configured to segment a defective area image from the defective fruit image based on a clustering algorithm, filter the defective area image using a median filtering algorithm, and determine a median filtered image of the defective area image; The defective fruit determination module is configured to calculate the weight between the median filter image area of ​​the defective region image and the area of ​​the defective fruit image. When it is detected that the weight exceeds a preset threshold, the fruit corresponding to the defective fruit image is sorted to the first conveyor belt. When it is detected that the weight is less than or equal to the preset threshold, the fruit corresponding to the defective fruit image is sorted to the second conveyor belt to complete the defect detection of the fruit.

7. An electronic device comprising a central processing unit and a memory, characterized in that: The central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that It stores a computer program implemented according to the method described in any one of claims 1 to 5 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images