Red date defect identification method and system based on machine vision

Abstract

The invention relates to the technical field of machine vision, in particular to a red date defect identification method and system based on machine vision, and the method comprises the steps: obtaining original images of a plurality of red dates at a plurality of overturning angles, enabling the plurality of red dates to move along with a transmission belt and rotate along with a rotating mechanism on the transmission belt, and obtaining the original images of the plurality of red dates; the original images are collected through a camera located above the conveying belt, and the original images of the multiple overturning angles are used for covering all the surfaces of the red dates; and then carrying out mixed identification on the original images of the plurality of overturning angles in sequence to obtain a defect identification result of the images in the image group. The red dates are conveyed and automatically turned over through the conveying belt and the rotating mechanism, and the camera shoots original images of multiple turning angles so as to cover the whole surfaces of the red dates. And then performing hybrid recognition on the original image, and performing automatic recognition on red date defects by using a traditional vision algorithm and an AI algorithm. Compared with manual screening, the problems of low screening precision, low efficiency and the like are solved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically a method and system for identifying defects in jujubes based on machine vision. Background Technology

[0002] As a widely consumed food, the quality of jujubes directly impacts consumer health. Effective defect detection ensures that jujube products on the market meet safety standards and are free from harmful substances or pests. High-quality jujubes are more competitive in the market. Accurate defect detection helps identify high-quality products, promoting brand building and enhancing market trust. For producers and sellers, grading and screening allow for pricing jujubes according to quality grades, thereby improving economic efficiency.

[0003] However, current technology still relies on manual inspection of jujubes. Traditional methods of manually inspecting jujubes for defects have limited speed and cannot meet the rapid demands of large-scale production. In addition, human judgment is easily affected by factors such as personal experience and visual fatigue, resulting in unstable test results. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a machine vision-based method and system for identifying defects in jujubes, so as to solve the problems in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] The present invention provides a machine vision-based method for identifying defects in jujubes, comprising the following steps:

[0007] The original images of multiple jujubes at multiple flip angles are acquired, wherein the multiple jujubes are displaced by the conveyor belt and rotated by the rotating mechanism on the conveyor belt. The original images are captured by a camera located above the conveyor belt, and the original images at multiple flip angles are used to cover the entire surface of the jujubes.

[0008] Multiple original images at various flip angles are sequentially subjected to hybrid recognition to obtain defect recognition results for the images in the image group. The hybrid recognition includes visual processing algorithm recognition and AI algorithm recognition.

[0009] In one embodiment of this application, multiple original images at different flip angles are sequentially mixed for identification to obtain defect identification results for the images in the image group, including:

[0010] S1, take the original image from the first angle as the image to be processed;

[0011] S2, preprocess the image to be processed to obtain a preprocessed image, wherein the preprocessing includes background removal, channel separation and high-pass filtering;

[0012] S3, Perform a visual processing algorithm on the preprocessed image to obtain a first recognition result, wherein the first recognition result includes defect features of a determined type, potential defect features, or no defect features;

[0013] S4, when the first identification result includes defect features, the defect features are added to a pre-built defect pool; when the first identification result does not include defect features or potential defect features, the pre-processed image is input into a pre-built defect identification model to obtain a second identification result; when the first identification result includes potential defect features, the pre-processed image is input into a pre-built defect identification model to obtain a third identification result.

[0014] S5, when the second identification result includes defect features, add the defect features to the pre-built defect pool;

[0015] S6, when the third identification result includes a target detection result corresponding to the potential defect feature, the detection confidence threshold of the target detection result is reduced to obtain an updated detection confidence threshold; and the third identification result is filtered based on the updated detection confidence threshold to obtain the defect identification result of the image; when the third identification result does not include a target detection result corresponding to the potential defect feature, the corresponding position of the image to be processed is marked and sent to the target object;

[0016] S7, take the original image from the next angle as the image to be processed, and return to step S2 until the recognition of the original images from all angles is completed.

[0017] In one embodiment of this application, the image to be processed is preprocessed to obtain a preprocessed image, including:

[0018] The image to be processed is converted to HSV format, and the luminance channel image, resolution channel image, and color channel image are extracted.

[0019] The brightness channel image is merged with a pre-constructed brightness compensation map, and the pixel value of each pixel in the merged image is subtracted from the pre-constructed correction value to obtain the compensated brightness channel image. The brightness compensation map and the correction value are both constructed in the pre-debugging stage.

[0020] The resolution channel image is subjected to morphological processing to obtain a morphologically processed image, wherein the morphological processing includes dilation and erosion; the morphologically processed image is binarized to obtain a binarized image; the top-level contour features in the binarized image are extracted, and the contour features that meet the preset size screening conditions are taken as the jujube contour; the minimum bounding rectangle of the jujube contour is calculated, and the jujube contour is numbered based on the center of the minimum bounding rectangle to obtain the resolution channel preprocessed image;

[0021] The compensated luminance channel image, the preprocessed resolution channel image, and the color channel image are merged to obtain a merged image;

[0022] The merged image is segmented into a grid based on the positions of multiple rotating mechanisms to obtain a preprocessed image.

[0023] In one embodiment of this application, the debugging phase includes:

[0024] The camera is controlled to capture a pure white background, and a brightness map is obtained.

[0025] The luminance map is converted to HSV format to obtain the luminance channel image in the luminance map; and the pixel value of each pixel in the luminance channel image in the luminance map is subtracted from the maximum luminance value to obtain a new luminance channel image.

[0026] For each pixel in the new luminance channel image, calculate the average pixel value of that pixel with the pixel values ​​of all its neighboring pixels;

[0027] Find the maximum and minimum average values ​​from the average values ​​of the multiple pixel values, and take half of the difference between the maximum and minimum average values ​​as the correction value; subtract the minimum average value from the pixel value of each pixel in the brightness channel image of the brightness map to obtain the brightness compensation image.

[0028] In one embodiment of this application, a visual processing algorithm is performed on the preprocessed image to obtain defect features, including:

[0029] Extract the minimum bounding rectangle from the preprocessed images at multiple flip angles at the target location, and extract the area or height of the minimum bounding rectangle corresponding to multiple flip angles; calculate the area or height difference of the minimum bounding rectangle corresponding to any two flip angles, and determine that the jujube at the target location has a deformation defect when the area difference or height difference is greater than a set area threshold.

[0030] Extract the contour features of the preprocessed image and filter out the outer contour; calculate the convex hull and convex defect depth of the outer contour; when the convex defect depth exceeds a preset depth threshold, determine that the jujube corresponding to the preprocessed image has a broken or moldy defect.

[0031] Extract the contour features of the preprocessed image and filter out the largest closed contour. Count the number of contours and the area ratio of the contours within the closed contour. If the number of contours within the closed contour is greater than a preset number threshold, or if the area ratio within the closed contour is greater than a preset ratio threshold, it is determined that the jujubes corresponding to the preprocessed image have a shriveled defect.

[0032] In one embodiment of this application, a visual processing algorithm is performed on the preprocessed image to obtain potential defect features, including:

[0033] The contour features of the preprocessed image are extracted, and the largest closed contour is selected. Ellipse fitting is performed on the closed contour to obtain an approximate ellipse of the jujube. The major axis of the smallest bounding rectangle of the preprocessed image, the major axis of the approximate ellipse, and the minor axis of the approximate ellipse are extracted. The angle between the major axis of the smallest bounding rectangle and the major axis of the approximate ellipse is calculated, and the ratio of the major axis of the approximate ellipse to the minor axis of the approximate ellipse is calculated. When the angle is greater than a preset angle threshold, or when the aspect ratio exceeds a preset aspect ratio range, it is determined that the jujube corresponding to the preprocessed image has a potential morphological defect.

[0034] The preprocessed image is converted to the RGB color space to obtain an RGB image; the R channel image is extracted from the RGB image, the average pixel value of the R channel image is calculated, and the pixel value variance is calculated after normalizing the pixel value of the R channel image; when the average pixel value exceeds a preset pixel value range or the pixel value variance is greater than a preset variance threshold, it is determined that the jujube corresponding to the preprocessed image has a potential color defect.

[0035] In one embodiment of this application, reducing the detection confidence threshold of the target detection result to obtain an updated detection confidence threshold includes:

[0036] Extract the probability value P of all current detection results corresponding to the current potential defect from the pre-constructed database of probability relationships between detection results and potential defect features, wherein the database of probability relationships between detection results and potential defect features includes the correspondence between potential defect features and various detection results and the corresponding probabilities.

[0037] The detection confidence threshold C for the target detection result R based on the probability value P. R After adjustment, the updated detection confidence threshold C' is obtained. RAmong them, the updated detection confidence threshold C' R The mathematical expression is:

[0038] C' R =C R -P×C max

[0039] In the formula, C max This represents the maximum adjustment amount for the confidence threshold.

[0040] In one embodiment of this application, the method for constructing the detection result-potential defect feature corresponding probability relationship database includes:

[0041] Acquire multiple jujube sample images;

[0042] The jujube sample image is preprocessed to obtain a preprocessed image sample;

[0043] The preprocessed image samples are input into the defect recognition model to obtain the sample recognition results; and a visual processing algorithm is performed on the preprocessed image samples to obtain potential defect features.

[0044] Multiple preprocessed image samples are classified based on potential defect features to obtain multiple sample sets; and the probability of each detection result in each sample set is calculated to obtain a database of probability relationships between detection results and potential defect features.

[0045] In one embodiment of this application, the preprocessed image is input into a pre-built defect recognition model to obtain a second or third recognition result, including:

[0046] The preprocessed image is normalized and enhanced to obtain the input image;

[0047] The input image is input into a pre-built defect recognition model to obtain a second recognition result or a third recognition result. The second recognition result or the third recognition result includes multiple defect labeling types and confidence levels of multiple defect labeling types. The defect recognition model is obtained by training an artificial neural network.

[0048] This application also provides a machine vision-based jujube defect recognition system, including:

[0049] An image acquisition module is used to acquire original images of multiple jujubes at multiple flip angles. The multiple jujubes are displaced by the conveyor belt and rotated by the rotating mechanism on the conveyor belt. The original images are acquired by a camera located above the conveyor belt. The original images at multiple flip angles are used to cover the entire surface of the jujubes.

[0050] The defect recognition module is used to sequentially perform mixed recognition on the original images at multiple flip angles to obtain the defect recognition results of the images in the image group. The mixed recognition includes visual processing algorithm recognition and AI algorithm recognition.

[0051] The beneficial effects of this invention are as follows: This invention provides a machine vision-based method and system for identifying defects in jujubes. It acquires original images of multiple jujubes from multiple flipping angles. The jujubes are displaced by a conveyor belt and rotated by a rotating mechanism on the belt. The original images are captured by a camera located above the conveyor belt, and the images from multiple flipping angles cover the entire surface of the jujubes. Then, the original images from multiple flipping angles are sequentially mixed and recognized to obtain the defect identification results of the images in the image group. This application utilizes a conveyor belt and a rotating mechanism to transport and automatically flip the jujubes. A camera captures original images from multiple flipping angles to cover the entire surface of the jujubes. Then, mixed recognition is performed on the original images, using both traditional visual algorithms and AI algorithms to automatically identify defects in the jujubes. Compared to manual sorting, it solves the problems of low sorting accuracy and low efficiency, and can perform multi-type sorting according to needs, allowing for refined sorting and diversion of jujubes of different grades, thereby improving the overall profitability of the processing plant. Attached Figure Description

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0053] Figure 1 This is a sorting structure diagram illustrating a machine vision-based method for identifying defects in jujubes, as shown in one embodiment of this application.

[0054] Figure 2 The system block diagram of a machine vision-based jujube defect recognition method shown in one embodiment of the application is provided.

[0055] Figure 3 This is a flowchart illustrating a machine vision-based method for identifying defects in jujubes in one embodiment of this application;

[0056] Figure 4 This is a schematic diagram showing the positions of the original images at three flip angles in one embodiment of this application;

[0057] Figure 5 This is a flowchart illustrating the implementation of the hybrid algorithm in this application;

[0058] Figure 6 This is a flowchart illustrating the debugging and preprocessing process in one embodiment of this application;

[0059] Figure 7 This is a schematic diagram of a brightness compensation diagram in one embodiment of this application;

[0060] Figure 8 This is a luminance channel image before luminance compensation in one embodiment of this application;

[0061] Figure 9 This is a brightness channel image after brightness compensation in one embodiment of this application;

[0062] Figure 10 This is a schematic diagram of a merged image in one embodiment of this application;

[0063] Figure 11 This is a schematic diagram of a preprocessed image in one embodiment of this application;

[0064] Figure 12 This is a schematic diagram of a preprocessed image before brightness compensation in one embodiment of this application;

[0065] Figure 13 This is a schematic diagram of a preprocessed image after brightness compensation in one embodiment of this application;

[0066] Figure 14 This is a structural diagram of a machine vision-based jujube defect recognition system shown in one embodiment of this application. Detailed Implementation

[0067] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0068] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0069] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of the invention; however, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details.

[0070] Figure 1 This is a sorting structure diagram illustrating a machine vision-based method for identifying defects in jujubes, as shown in one embodiment of this application. Figure 1As shown, the sorting mechanism upon which this application relies includes a conveyor belt 110 located on a frame and a turning roller 120 mounted on the conveyor belt 110. The roller surface of the turning roller 120 is concave to adapt to the shape of the jujubes. The turning roller 120 is mounted on a rotating shaft, which is fixed to a bracket located on the conveyor belt 110. Gears 130 are provided at both ends of the rotating shaft. A fixed rack 140 is provided on the frame. The gears 130 and the fixed rack 140 mesh. When the gears 130 move with the bracket and the conveyor belt 110, the gears 130 and the rack 140 cooperate to make the gears 130 and the turning roller 120 rotate, thereby turning the jujubes.

[0071] The transmission belt 110 is driven by the drive motor 150 and the drive gear. The top of the frame is also equipped with a camera mounting bracket 160, which is equipped with a camera to capture images of the jujubes during the flipping process.

[0072] In addition, the supporting equipment includes upstream feeding equipment, conveying equipment, and downstream pneumatic sorting equipment. These devices are not related to the main solution of this application and are only used to describe their supporting functions, so they will not be described in detail here.

[0073] Figure 2 The system block diagram of a machine vision-based jujube defect recognition method shown in one embodiment of the application is as follows: Figure 2 As shown, the sorting system in this application includes a drive control unit (DCU) for controlling the conveyor belt drive motor, an algorithm execution unit (APU) for executing the algorithm, and a human-machine interface (HMI). The HMI is used to input / output some instructions or status information. The image captured by the camera is input to the algorithm execution unit (APU) to execute the hybrid recognition algorithm in this application, thereby completing the automatic sorting.

[0074] The drive control unit (DCU) and algorithm execution unit (APU) are located in two separate systems. To synchronize the two systems, a synchronization system is added. Each time a row of red dates passes through the visual area, a photoelectric signal is mechanically triggered. This photoelectric signal is simultaneously sent to both the APU and the DCU. The APU starts taking pictures when the photoelectric signal arrives, and the data transmission process can only be initiated when this photoelectric signal arrives.

[0075] Figure 3 This is a flowchart illustrating a machine vision-based defect identification method for jujubes in one embodiment of this application, as shown below. Figure 3 As shown in the figure, a machine vision-based method for identifying defects in jujubes in this embodiment may include the following steps:

[0076] S310, acquire original images of multiple jujubes at multiple flip angles, wherein the multiple jujubes are displaced by the conveyor belt and rotated by the rotating mechanism on the conveyor belt, and the original images are acquired by a camera located above the conveyor belt, and the original images at multiple flip angles are used to cover the entire surface of the jujubes.

[0077] In this application, original images are captured from three flip angles to achieve full coverage acquisition of image information on the surface of the jujube. Figure 4 This is a schematic diagram showing the positions of the original images at three flip angles in one embodiment of this application, as shown below. Figure 4 As shown, the height of the camera above the conveyor belt is H, the width of the visual area projected onto the conveyor belt by the camera is L, the diameter of the conveyor belt is D2, the diameter of the mechanism driving the rotation is D1, and the horizontal speed of the conveyor belt is v.

[0078] To ensure the camera rotates at least 240° within the visual field, the gear must rotate at least 240° and no more than 360° when passing through the visual field, and the rotation must not exceed L. This must satisfy: D² × π × 2 / 3 <L<D2×π。

[0079] The relationship between the motor speed and the horizontal speed of the conveyor belt is: v=(D1×π) / 60.

[0080] In addition, the distance between adjacent red dates is relatively large. If only the top surface light source is used, there will be a lot of interfering colors between adjacent red dates. Therefore, an auxiliary light source needs to be added at the bottom.

[0081] S320, perform hybrid recognition on the original images at multiple flip angles in sequence to obtain the defect recognition results of the images in the image group, wherein the hybrid recognition includes visual processing algorithm recognition and AI algorithm recognition.

[0082] This application employs a hybrid algorithm combining traditional visual algorithms and AI algorithms for automatic sorting of jujubes. However, due to the large size and high resolution of the acquired images, directly feeding them into the algorithm would consume a significant amount of time. Therefore, this system incorporates visual processing algorithms, such as OpenCV, on top of the AI ​​algorithms.

[0083] Figure 5 The following is a flowchart illustrating the implementation of the hybrid algorithm in this application: Figure 5 As shown, the steps of the hybrid algorithm are as follows:

[0084] S1, take the original image from the first angle as the image to be processed;

[0085] S2, preprocess the image to be processed to obtain a preprocessed image, wherein the preprocessing includes background removal, channel separation and high-pass filtering;

[0086] In this application, preprocessing is first used to remove redundant image information to facilitate subsequent processing. Figure 6 This is a flowchart illustrating the debugging and preprocessing process in one embodiment of this application, as shown below. Figure 6 As shown, in one embodiment of this application, the debugging phase includes:

[0087] S2-1, Control the camera to capture a pure white background and obtain a brightness map;

[0088] Under normal working conditions, pure white paper without any other colors is used to cover the field of view of an industrial camera instead of red dates. After taking a picture (referred to as a brightness map), it is kept for later use.

[0089] S2-2, Convert the luminance map to HSV format to obtain the luminance channel image in the luminance map; and subtract the pixel value of each pixel in the luminance channel image in the luminance map from the maximum luminance value to obtain a new luminance channel image;

[0090] S2-3, For each pixel in the new luminance channel image, calculate the average pixel value of that pixel and the pixel values ​​of all its neighboring pixels;

[0091] S2-4, find the maximum average value and the minimum average value from the average values ​​of the multiple pixel values, and take half of the difference between the maximum average value and the minimum average value as the correction value; and subtract the minimum average value from the pixel value of each pixel in the brightness channel image of the brightness map to obtain the brightness compensation image.

[0092] During the debugging process, the color space of the luminance map is converted to HSV format, and the V channel is separated. Each pixel of the V channel is subtracted from the maximum luminance 0xFF to obtain a new V channel. The average value of the nine pixels (top, bottom, left, right, top left, bottom left, top right, and bottom right) of each pixel is calculated to find the maximum and minimum values ​​of all averages. Half of the difference between the maximum and minimum values ​​is the uncorrected value. The minimum value is subtracted from each pixel of the V channel to obtain the compensation map. The compensation map and the correction value are kept for later use.

[0093] Figure 7 This is a schematic diagram of a brightness compensation image in one embodiment of this application. The obtained brightness compensation image is as follows: Figure 7 As shown.

[0094] The preprocessing process includes:

[0095] S2-5, Convert the image to be processed to HSV format and extract the luminance channel image, resolution channel image and color channel image;

[0096] S2-6, merge the brightness channel image with the pre-built brightness compensation map, and subtract the pre-built correction value from the pixel value of each pixel in the merged image to obtain the compensated brightness channel image, wherein the brightness compensation map and the correction value are both constructed in the pre-debugging stage;

[0097] The image resolution is 4096*3000, which is relatively high, resulting in uneven brightness across the entire image, with high brightness in the center and low brightness at the edges. A visual processing algorithm was used to convert the image to HSV format and separate the V channel (luminance) image. The V channel image and the compensation image were then merged, and the correction value was subtracted to obtain the luminance-compensated V channel image.

[0098] Figure 8 This is a luminance channel image before luminance compensation in one embodiment of this application. Figure 9 This is a brightness channel image after brightness compensation in one embodiment of this application. The brightness compensation effect is as follows: Figure 8 and Figure 9 As shown.

[0099] S2-7, Perform morphological processing on the resolution channel image to obtain a morphologically processed image, wherein the morphological processing includes dilation and erosion; perform binarization on the morphologically processed image to obtain a binarized image; extract the top-level contour features from the binarized image, and take the contour features that meet the preset size screening conditions as the jujube contour; calculate the minimum bounding rectangle of the jujube contour, and number the jujube contour based on the center of the minimum bounding rectangle to obtain the resolution channel preprocessed image;

[0100] Specifically, for the S-channel image, after dilation and erosion, binarization is performed. Top-level contour search is performed on the binary image, the area of ​​all contours is calculated, all contours with areas within a preset range are found, and the minimum bounding rectangle and center of these contours are calculated. Based on the landing point of the rectangle center, each rectangle is numbered, and all bounding rectangles and contour information are retained for later use.

[0101] S2-8, merge the compensated luminance channel image, the resolution channel preprocessed image, and the color channel image to obtain a merged image;

[0102] Furthermore, images formed by rectangles and outlines are rewritten as pure white in the new image after brightness compensation. Figure 10 This is a schematic diagram of a merged image in one embodiment of this application. The merged image is as follows: Figure 10 As shown.

[0103] S2-9, The merged image is divided into grids based on the positions of multiple rotating mechanisms to obtain a preprocessed image.

[0104] The location of the jujubes in the captured images is defined. For example, if there is a jujube within the red rectangle in the image, then there will also be a jujube in that location in the next captured image. Based on this characteristic, the image is initially divided into grids, resulting in smaller images.

[0105] Furthermore, since the positions of the jujubes differ in different original images, the position of the same jujube in different original images can be matched using P' = P + vt, where P' is the position of the jujube in the later image, P is the position of the jujube in the earlier image, v is the conveyor belt speed, and t is the time difference between the capture of the preceding and following images.

[0106] The preprocessed image after segmentation is a single jujube image. Figure 11 This is a schematic diagram of a preprocessed image in one embodiment of this application. The preprocessed image obtained after segmentation is as follows: Figure 11 shown.

[0107] By using visual processing algorithms such as dilation, erosion, edge detection, and bounding rectangle, the minimum bounding rectangle of the jujube is found. Secondary image segmentation is then performed based on the minimum bounding rectangle, which greatly reduces the amount of data fed to the AI ​​algorithm and saves inference time.

[0108] Due to the introduction of the synchronization signal, the position of the jujube in each image is relatively fixed. Therefore, the large image is cut into small images of individual jujubes using a visual processing algorithm, and the background interference around the jujubes in the small images is removed, which can remove nearly 1 / 2 of the interference pixels. Figure 12 This is a schematic diagram of a preprocessed image before brightness compensation in one embodiment of this application. Figure 13 This is a schematic diagram of a preprocessed image after brightness compensation according to one embodiment of this application. After brightness compensation, the problems of uneven image brightness, high brightness in the center, and low brightness at the edges can be significantly improved.

[0109] Furthermore, the length and width of the smallest bounding rectangle of a single jujube can be used to determine its grade. If a task is to select first-grade jujubes, then jujubes with a length or width exceeding the first-grade standard are classified as premium-grade, while those below this standard are classified as second-grade. Therefore, preprocessing images allows for convenient and quick visual screening of jujubes, and the screening results can be transmitted to downstream sorting equipment.

[0110] S3, Perform a visual processing algorithm on the preprocessed image to obtain a first recognition result, wherein the first recognition result includes defect features of a determined type, potential defect features, or no defect features;

[0111] After preprocessing, a visual processing algorithm is first applied to the preprocessed image. In this application, if the visual processing algorithm can quickly detect the exact defect features, there is no need to execute further AI algorithms, thus saving detection time. If the exact defect features cannot be detected, further processing is performed by the AI ​​algorithm.

[0112] The inability to detect exact defect features in this application means: (1) the defect features cannot be detected; (2) there are potential defect features, but the defect type is unknown.

[0113] The process of visual processing algorithms includes:

[0114] (1) The exact type of defect

[0115] (1-1) Extract the minimum bounding rectangle from the preprocessed images of multiple flip angles at the target location, and extract the area or height of the minimum bounding rectangle corresponding to multiple flip angles; calculate the area or height difference of the minimum bounding rectangle corresponding to any two flip angles, and determine that the jujube at the target location has a deformation defect when the area difference or height difference is greater than the set area threshold.

[0116] The minimum bounding rectangles for the three flip angles are rect1, rect2, and rect3, with areas of A(rect1), A(rect2), and A(rect3) and heights of H(rect1), H(rect2), and H(rect3).

[0117] If the difference between any two areas or heights between the three flipping angles is too large (exceeding a certain threshold), the jujube can be considered a deformed defective jujube.

[0118] That is, |A(rect1)-A(rect2)|>ΔA max Alternatively, |A(rect1)-A(rect3)|>ΔA max Or, |A(rect2)-A(rect2)|>ΔA max At that time, the jujubes were determined to be deformed and defective.

[0119] And, |H(rect1)-H(rect2)|>ΔH max Alternatively, |H(rect1)-H(rect3)|>ΔH max Or, |H(rect2)-H(rect2)|>ΔH max At that time, the jujube was determined to be a deformed, defective jujube; ΔA max As the area threshold, ΔH max This is the height threshold.

[0120] (1-2) Extract the contour features of the preprocessed image and filter out the outer contour; calculate the convex hull and convex defect depth D of the outer contour; when the convex defect depth exceeds a preset depth threshold D... max When the preprocessed image corresponds to a jujube, it is determined that the jujube has defects such as breakage or mold.

[0121] The jujube with a dent could be a damaged jujube or a jujube that has become moldy. If the moldy area happens to fall on the edge of the jujube in this photo, the surface color will be darkened, and it will be "misjudged" as black when the image is binarized, thus creating an extra dent.

[0122] (1-3) Extract the contour features of the preprocessed image and filter out the largest closed contour. Count the number of contours N and the contour area ratio K within the closed contour. If the number of contours N within the closed contour is greater than a preset threshold N, then... max Alternatively, the area ratio K within the closed contour is greater than a preset ratio threshold K. max At that time, it was determined that the jujubes corresponding to the preprocessed image had a shriveled defect.

[0123] (2) Potential defects

[0124] In some cases, image features cannot directly infer the existence of a specific type of defect, but their features differ from the normal range, indicating a variety of defect types. When there are more than three such features, they are defined as potential defect features in this application, requiring further inference by an AI model.

[0125] (2-1) Extract the contour features of the preprocessed image and filter out the largest closed contour. max For the closed contour outline max Perform ellipse fitting to obtain an approximate ellipse of the jujube; the approximate ellipse fitting can be performed by calling the fitEllipse function.

[0126] Extract the major axis LA of the minimum bounding rectangle of the preprocessed image. rect The major axis LA of the approximate ellipse elli and the minor axis SA of the approximate ellipse elli ;

[0127] Calculate the major axis LA of the minimum bounding rectangle. rect and the major axis LA of the approximate ellipse elli Calculate the major axis LA of the approximate ellipse by taking the included angle θ. elli and the minor axis SA of the approximate ellipse elliThe ratio of the major and minor axes is abs;

[0128] When the included angle θ is greater than a preset included angle threshold θ max Alternatively, the aspect ratio abs exceeds a preset aspect ratio range (i.e., abs > abs). max And abs <abs min When the preprocessed image corresponds to a jujube, it is determined that the jujube has a potential morphological defect; (abs) min abs max () represents the aspect ratio range.

[0129] Red dates are typically oval-shaped, with an aspect ratio between 1.5 and 3.0. If the aspect ratio is too small (close to 1), the date will be too flat and may become deformed due to crushing or abnormal internal structure. If the aspect ratio is too large (over 3), the date will be elongated and may become unbalanced due to cracks or external compression. Furthermore, if the date's outline is regular and symmetrical, the major axis of the fitted ellipse should be roughly aligned with the major axis of the smallest bounding rectangle, with an angle close to 0°. For example, in a standard oval-shaped date, the major axis of the ellipse completely coincides with the major axis of the smallest bounding rectangle.

[0130] When jujubes have irregular outlines due to cracks, deformation, or localized collapse, the major axis of the fitted ellipse will deviate from the major axis of the smallest bounding rectangle. The included angle of the major axis reflects the consistency of the outline direction; an excessively large angle indicates that the outline has deviated from its direction due to cracks or deformation. The aspect ratio reflects the regularity of the shape; exceeding a threshold indicates that the jujube's shape is unbalanced due to compression, collapse, or cracking. By comprehensively judging and combining both, and utilizing the degree of deviation of geometric features, efficient detection of jujube morphological defects can be achieved.

[0131] For example, the outline of a normal jujube should be close to a regular ellipse, and the major axis of the ellipse should be aligned with the major axis of the smallest bounding rectangle. If the included angle is too large or the aspect ratio is abnormal, it indicates that the outline deviates from a regular ellipse, suggesting the presence of shape irregularities (such as cracks or deformation).

[0132] The aspect ratio of a jujube directly reflects the regularity of its shape. If defects cause an imbalance in the elliptical shape, the aspect ratio will deviate from the normal range. For example, cracks can alter the outline distribution of a jujube, leading to abnormal lengths of either the major or minor axis. For instance, if a crack extends along the major axis, it may shorten the major axis while keeping the minor axis unchanged, resulting in a lower aspect ratio. Furthermore, uneven stress causing partial collapse of the jujube may significantly shorten the minor axis, leading to an abnormally high aspect ratio.

[0133] Therefore, this application uses the angle between the major axis of the approximate ellipse and the major axis of the smallest circumscribed rectangle, as well as the ratio of the major and minor axes of the approximate ellipse, to verify potential defects.

[0134] (2-2) Convert the preprocessed image to the RGB color space to obtain an RGB image;

[0135] Extract the R (red) channel image from the RGB image, calculate the average pixel value of the R channel image, and calculate the pixel value variance after normalizing the pixel values ​​of the R channel image.

[0136] Among them, the color characteristic of jujubes is mainly red (healthy jujubes are usually reddish-brown or dark red), and the R channel is more sensitive to the red component. By extracting the R channel, the distribution characteristics of red color in jujubes can be reflected more directly.

[0137] The formula for calculating the average is: in, R is the average pixel value, R(i,j) is the pixel value of the pixel at coordinate (i,j), and n is the number of pixels.

[0138] Normalization is calculated using the Max-Min normalization formula:

[0139]

[0140] In the formula, nor(R(i,j)) is the normalized pixel value of the pixel at coordinate (i,j), and R... min R is the minimum pixel value. max This is the maximum pixel value.

[0141] The formula for calculating variance is:

[0142]

[0143] In the formula, σ represents the variance, and μ is the normalized pixel mean.

[0144] In the average pixel Pixel values ​​outside the preset range (i.e.) and Alternatively, the pixel value variance σ is greater than a preset variance threshold σ. max At that time, it was determined that the jujubes corresponding to the preprocessed image had potential color defects.

[0145] In this embodiment, the average pixel value reflects the overall brightness level of the jujube's color. If the jujube deteriorates (e.g., mold, sugarification) or is affected by ambient light, the overall R-channel pixel value may be too high (too bright) or too low (too dark). For example, internal sugar decomposition or mold growth may cause the color to darken (lower R value), and jujubes treated with sulfur may appear an abnormally bright red (higher R value). Variance measures the uniformity of the jujube's color distribution. Excessive variance indicates uneven color distribution, which may include local discoloration, spots, or damaged areas. For example, the tissue around the wormholes in insect-eaten fruit may oxidize and discolor, causing a significant increase or decrease in the local R value. The internal tissue exposed at the cracks in cracked fruit may appear dark brown due to oxidation (lower R value).

[0146] This application utilizes mean and variance analysis of the R channel to effectively detect color defects in jujubes caused by biological spoilage, processing problems, abnormal storage, or varietal differences. Its principle is based on the statistical characteristics of color distribution, combined with threshold settings for practical applications, enabling rapid and automated evaluation of jujube color quality.

[0147] S4, when the first identification result includes defect features, the defect features are added to a pre-built defect pool; when the first identification result does not include defect features or potential defect features, the pre-processed image is input into a pre-built defect identification model to obtain a second identification result; when the first identification result includes potential defect features, the pre-processed image is input into a pre-built defect identification model to obtain a third identification result.

[0148] If the visual processing algorithm extracts defect features of a defined type, these features are added to the defect pool (defect_pool). If the visual processing algorithm does not extract any defect features or potential defect features, the algorithm proceeds to the subsequent AI recognition process. If the visual processing algorithm only extracts potential defect features, the algorithm also proceeds to the subsequent AI recognition process, but the detection threshold needs to be adjusted accordingly to increase detection speed. For example, if a potential morphological defect feature is detected, the detection threshold for the AI ​​detection result corresponding to the potential morphological defect feature is lowered, making it easier to detect defects related to the potential defect feature.

[0149] S5, when the second identification result includes defect features, add the defect features to the pre-built defect pool;

[0150] S6, when the third identification result includes a target detection result corresponding to the potential defect feature, the detection confidence threshold of the target detection result is reduced to obtain an updated detection confidence threshold; and the third identification result is filtered based on the updated detection confidence threshold to obtain the defect identification result of the image; when the third identification result does not include a target detection result corresponding to the potential defect feature, the corresponding position of the image to be processed is marked and sent to the target object;

[0151] Both the second and third recognition results are from AI-generated recognition. The recognition process includes:

[0152] The preprocessed image is normalized and enhanced to obtain the input image; the robustness of the model is improved by image normalization and color enhancement.

[0153] The input image is input into a pre-built defect recognition model to obtain a second recognition result or a third recognition result. The second recognition result or the third recognition result includes multiple defect labeling types and confidence levels of multiple defect labeling types. The defect recognition model is obtained by training an artificial neural network.

[0154] The training data is categorized by defect type, such as cracks, mold, shriveling, compression deformation, asymmetric defects, and normal. By introducing the Faster R-CNN deep object detection model, the appearance quality inspection of jujubes has shifted from a "human experience" to a "data-driven" model, significantly improving inspection efficiency, accuracy, and scalability. This provides strong technical support for modern agricultural product grading and sorting systems. The specific framework and training metrics include:

[0155] Framework platform: TensorFlow + PyTorch + Caffe + Detectron2

[0156] Backbone: ResNet-50with FPN (Feature PyramidNetwork)

[0157] Optimizer: SGD (momentum 0.9, weight decay 0.0001)

[0158] Training strategy:

[0159] Batch size: 16

[0160] Learning rate: Initially set to 0.005, using a StepLR decay strategy.

[0161] Epochs: 50–100 (depending on convergence)

[0162] Use the following metrics to evaluate performance:

[0163]

[0164] In actual testing, the model inference time on a high-performance GPU is approximately 80–100 ms per image, which meets the requirements of industrial applications.

[0165] The third identification result is based on the potential defect features mentioned above. Therefore, this application uses potential identification features to adjust the relevant detection threshold, thereby making it easier to detect results related to potential identification features. The specific process includes:

[0166] S6-1, Extract the probability value P of all detection results corresponding to the current potential defect from the pre-constructed detection result-potential defect feature corresponding probability relationship database, wherein the detection result-potential defect feature corresponding probability relationship database includes the correspondence between potential defect features and multiple detection results and the corresponding probability;

[0167] In this application, a database of probability relationships between detection results and potential defect features is constructed in advance using samples, thereby finding the correspondence between various detection results and potential defect features, as well as the probability of each correspondence.

[0168] For example, the current potential defect characteristics and morphological potential defect characteristics can be respectively associated with mold defects (probability 15%), extrusion deformation defects (probability 45%), breakage defects (probability 24%), and asymmetric defects (foreign countries) (probability 16%).

[0169] The process of constructing the database of probability relationships between detected results and potential defect features includes:

[0170] Acquire multiple jujube sample images;

[0171] The jujube sample image is preprocessed to obtain a preprocessed image sample; the preprocessing process is as described above and will not be repeated here.

[0172] The preprocessed image samples are input into the defect recognition model to obtain the sample recognition results; and a visual processing algorithm is performed on the preprocessed image samples to obtain potential defect features.

[0173] Multiple preprocessed image samples are classified based on potential defect features to obtain multiple sample sets; and the probability of each detection result in each sample set is calculated to obtain a database of probability relationships between detection results and potential defect features.

[0174] In this embodiment, probability is used to reflect the likelihood of multiple detection results corresponding to each potential defect. The detection threshold of each detection result is adjusted according to the probability, making it easier to detect the results corresponding to defects in the visual processing algorithm. The two mutually verify each other and improve the detection accuracy.

[0175] S6-2, The detection confidence threshold C for the target detection result R based on the probability value P. R After adjustment, the updated detection confidence threshold C' is obtained. R Among them, the updated detection confidence threshold C' R The mathematical expression is:

[0176] C' R =C R -P×C max

[0177] In the formula, C max This represents the maximum adjustment amount for the confidence threshold.

[0178] Finally, probability is introduced to calculate the adjustment amount. The higher the probability of the detection result, the greater the reduction in the corresponding confidence threshold.

[0179] In one embodiment of this application, the detection result and the corresponding confidence level are as follows:

[0180] Mold Defect: 35% Confidence Level; Detection Threshold: 60%

[0181] Extrusion deformation defects: 55% confidence level; detection threshold: 60%.

[0182] Damage / Defect: Confidence probability 7%, Detection threshold 60%

[0183] Asymmetric defects (foreign): 3% confidence level, detection threshold: 60%

[0184] Detection result: Not detected

[0185] The detection threshold in this application is 60%. Based on the probability of multiple detection results corresponding to potential defects, the detection threshold is adjusted, and the results are as follows:

[0186] Mold defects: Detection threshold at 35% confidence level: 60% - 30% * 15% = 55.5%

[0187] Extrusion deformation defects: Detection threshold at 55% confidence level: 60% - 30% * 45% = 46.5%

[0188] Damage / defect: 7% confidence level detection threshold: 60% - 30% * 24% = 52.8%

[0189] Asymmetric defects (foreign): Detection threshold at 3% confidence level: 60% - 30% * 16% = 55.2%

[0190] Detection result: Extrusion deformation defect

[0191] Therefore, as can be seen from the above examples, this application has the function of re-verification for defects with high similarity, thereby making the detection results more accurate.

[0192] S7, take the original image from the next angle as the image to be processed, and return to step S2 until the recognition of the original images from all angles is completed.

[0193] Finally, the above process is repeated to complete the recognition of the original images from three angles. Defects are then counted from the defect pool. If no defect features are found, the jujubes pass the inspection. If any defect features are present, they are marked and sent to downstream equipment or personnel for automatic or manual sorting.

[0194] This invention discloses a machine vision-based method for identifying defects in jujubes. The method acquires original images of multiple jujubes from multiple flipping angles. The jujubes are displaced by a conveyor belt and rotated by a rotating mechanism on the belt. The original images are captured by a camera located above the conveyor belt, and the images from multiple flipping angles cover the entire surface of the jujubes. Then, the original images from multiple flipping angles are sequentially mixed for identification to obtain the defect identification results of the images in the image group. This application utilizes a conveyor belt and a rotating mechanism to transport and automatically flip the jujubes. A camera captures original images from multiple flipping angles to cover the entire surface of the jujubes. Then, mixed identification is performed on the original images, using both traditional visual algorithms and AI algorithms to automatically identify defects in the jujubes. Compared to manual sorting, this method solves the problems of low sorting accuracy and low efficiency, and can perform multi-type sorting according to needs, allowing for refined sorting and diversion of jujubes of different grades, thereby improving the overall profitability of the processing plant.

[0195] like Figure 14 As shown, this application also provides a machine vision-based jujube defect recognition system, comprising:

[0196] An image acquisition module is used to acquire original images of multiple jujubes at multiple flip angles. The multiple jujubes are displaced by the conveyor belt and rotated by the rotating mechanism on the conveyor belt. The original images are acquired by a camera located above the conveyor belt. The original images at multiple flip angles are used to cover the entire surface of the jujubes.

[0197] The defect recognition module is used to sequentially perform mixed recognition on the original images at multiple flip angles to obtain the defect recognition results of the images in the image group. The mixed recognition includes visual processing algorithm recognition and AI algorithm recognition.

[0198] This invention discloses a machine vision-based jujube defect identification system. It acquires original images of multiple jujubes from multiple flipping angles, wherein the jujubes are displaced by a conveyor belt and rotated by a rotating mechanism on the conveyor belt. The original images are captured by a camera located above the conveyor belt, and the multiple flipping angle images are used to cover the entire surface of the jujubes. Then, the original images from multiple flipping angles are sequentially mixed for identification to obtain the defect identification results of the images in the image group. This application utilizes a conveyor belt and a rotating mechanism to transport and automatically flip the jujubes, and a camera captures original images from multiple flipping angles to cover the entire surface of the jujubes. Then, mixed identification is performed on the original images, using traditional vision algorithms and AI algorithms to automatically identify defects in the jujubes. Compared to manual sorting, it solves the problems of low sorting accuracy and low efficiency, and can perform multi-type sorting according to needs, allowing for refined sorting and diversion of jujubes of different grades, thereby improving the overall profitability of the processing plant.

[0199] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0200] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0201] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0202] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0203] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0204] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0205] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0206] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A machine vision-based method for identifying defects in jujubes, characterized in that, Including the following steps: The original images of multiple jujubes at multiple flip angles are acquired, wherein the multiple jujubes are displaced by the conveyor belt and rotated by the rotating mechanism on the conveyor belt. The original images are captured by a camera located above the conveyor belt, and the original images at multiple flip angles are used to cover the entire surface of the jujubes. Multiple original images at various flip angles are sequentially subjected to hybrid recognition to obtain defect recognition results for the images in the image group. The hybrid recognition includes visual processing algorithm recognition and AI algorithm recognition.

2. The method for identifying defects in jujubes based on machine vision according to claim 1, characterized in that, By sequentially performing mixed recognition on the original images at multiple flip angles, the defect recognition results of the images in the image group are obtained, including: S1, take the original image from the first angle as the image to be processed; S2, preprocess the image to be processed to obtain a preprocessed image, wherein the preprocessing includes background removal, channel separation and high-pass filtering; S3, Perform a visual processing algorithm on the preprocessed image to obtain a first recognition result, wherein the first recognition result includes defect features of a determined type, potential defect features, or no defect features; S4, when the first identification result includes defect features, the defect features are added to a pre-built defect pool; when the first identification result does not include defect features or potential defect features, the pre-processed image is input into a pre-built defect identification model to obtain a second identification result; when the first identification result includes potential defect features, the pre-processed image is input into a pre-built defect identification model to obtain a third identification result. S5, when the second identification result includes defect features, add the defect features to the pre-built defect pool; S6, when the third identification result includes a target detection result corresponding to the potential defect feature, the detection confidence threshold of the target detection result is reduced to obtain an updated detection confidence threshold; and the third identification result is filtered based on the updated detection confidence threshold to obtain the defect identification result of the image; when the third identification result does not include a target detection result corresponding to the potential defect feature, the corresponding position of the image to be processed is marked and sent to the target object; S7, take the original image from the next angle as the image to be processed, and return to step S2 until the recognition of the original images from all angles is completed.

3. The method for identifying defects in jujubes based on machine vision according to claim 2, characterized in that, The image to be processed is preprocessed to obtain a preprocessed image, including: The image to be processed is converted to HSV format, and the luminance channel image, resolution channel image, and color channel image are extracted. The brightness channel image is merged with a pre-constructed brightness compensation map, and the pixel value of each pixel in the merged image is subtracted from the pre-constructed correction value to obtain the compensated brightness channel image. The brightness compensation map and the correction value are both constructed in the pre-debugging stage. The resolution channel image is subjected to morphological processing to obtain a morphologically processed image, wherein the morphological processing includes dilation and erosion; the morphologically processed image is binarized to obtain a binarized image; the top-level contour features in the binarized image are extracted, and the contour features that meet the preset size screening conditions are taken as the jujube contour; the minimum bounding rectangle of the jujube contour is calculated, and the jujube contour is numbered based on the center of the minimum bounding rectangle to obtain the resolution channel preprocessed image; The compensated luminance channel image, the preprocessed resolution channel image, and the color channel image are merged to obtain a merged image; The merged image is segmented into a grid based on the positions of multiple rotating mechanisms to obtain a preprocessed image.

4. The method for identifying defects in jujubes based on machine vision according to claim 3, characterized in that, The debugging phase includes: The camera is controlled to capture a pure white background, and a brightness map is obtained. The luminance map is converted to HSV format to obtain the luminance channel image in the luminance map; and the pixel value of each pixel in the luminance channel image in the luminance map is subtracted from the maximum luminance value to obtain a new luminance channel image. For each pixel in the new luminance channel image, calculate the average pixel value of that pixel with the pixel values ​​of all its neighboring pixels; Find the maximum and minimum average values ​​from the average values ​​of the multiple pixel values, and take half of the difference between the maximum and minimum average values ​​as the correction value; subtract the minimum average value from the pixel value of each pixel in the brightness channel image of the brightness map to obtain the brightness compensation image.

5. The method for identifying defects in jujubes based on machine vision according to claim 3, characterized in that, A visual processing algorithm is performed on the preprocessed image to obtain defect features, including: Extract the minimum bounding rectangle from the preprocessed images at multiple flip angles at the target location, and extract the area or height of the minimum bounding rectangle corresponding to multiple flip angles; calculate the area or height difference of the minimum bounding rectangle corresponding to any two flip angles, and determine that the jujube at the target location has a deformation defect when the area difference or height difference is greater than a set area threshold. Extract the contour features of the preprocessed image and filter out the outer contour; calculate the convex hull and convex defect depth of the outer contour; when the convex defect depth exceeds a preset depth threshold, determine that the jujube corresponding to the preprocessed image has a broken or moldy defect. Extract the contour features of the preprocessed image and filter out the largest closed contour. Count the number of contours and the area ratio of the contours within the closed contour. If the number of contours within the closed contour is greater than a preset number threshold, or if the area ratio within the closed contour is greater than a preset ratio threshold, it is determined that the jujubes corresponding to the preprocessed image have a shriveled defect.

6. The method for identifying defects in jujubes based on machine vision according to claim 2, characterized in that, A visual processing algorithm is performed on the preprocessed image to obtain potential defect features, including: The contour features of the preprocessed image are extracted, and the largest closed contour is selected. Ellipse fitting is performed on the closed contour to obtain an approximate ellipse of the jujube. The major axis of the smallest bounding rectangle of the preprocessed image, the major axis of the approximate ellipse, and the minor axis of the approximate ellipse are extracted. The angle between the major axis of the smallest bounding rectangle and the major axis of the approximate ellipse is calculated, and the ratio of the major axis of the approximate ellipse to the minor axis of the approximate ellipse is calculated. When the angle is greater than a preset angle threshold, or when the aspect ratio exceeds a preset aspect ratio range, it is determined that the jujube corresponding to the preprocessed image has a potential morphological defect. The preprocessed image is converted to the RGB color space to obtain an RGB image; the R channel image is extracted from the RGB image, the average pixel value of the R channel image is calculated, and the pixel value variance is calculated after normalizing the pixel value of the R channel image; when the average pixel value exceeds a preset pixel value range or the pixel value variance is greater than a preset variance threshold, it is determined that the jujube corresponding to the preprocessed image has a potential color defect.

7. The method for identifying defects in jujubes based on machine vision according to claim 2, characterized in that, Lowering the detection confidence threshold of the target detection result to obtain the updated detection confidence threshold includes: Extract the probability value P of all current detection results corresponding to the current potential defect from the pre-constructed database of probability relationships between detection results and potential defect features, wherein the database of probability relationships between detection results and potential defect features includes the correspondence between potential defect features and various detection results and the corresponding probabilities. The detection confidence threshold C for the target detection result R based on the probability value P. R After adjustment, the updated detection confidence threshold C' is obtained. R Among them, the updated detection confidence threshold C' R The mathematical expression is: C' R =C R -P×C max In the formula, C max This is the maximum adjustment amount for the confidence threshold.

8. The method for identifying defects in jujubes based on machine vision according to claim 7, characterized in that, The method for constructing the probability relation database corresponding to the detection results and potential defect features includes: Acquire multiple jujube sample images; The jujube sample image is preprocessed to obtain a preprocessed image sample; The preprocessed image samples are input into the defect recognition model to obtain the sample recognition results; and a visual processing algorithm is performed on the preprocessed image samples to obtain potential defect features. Multiple preprocessed image samples are classified based on potential defect features to obtain multiple sample sets; and the probability of each detection result in each sample set is calculated to obtain a database of probability relationships between detection results and potential defect features.

9. A machine vision-based method for identifying defects in jujubes according to claim 2, characterized in that, The preprocessed image is input into a pre-built defect recognition model to obtain a second or third recognition result, including: The preprocessed image is normalized and enhanced to obtain the input image; The input image is input into a pre-built defect recognition model to obtain a second recognition result or a third recognition result. The second recognition result or the third recognition result includes multiple defect labeling types and confidence levels of multiple defect labeling types. The defect recognition model is obtained by training an artificial neural network.

10. A machine vision-based defect recognition system for jujubes, characterized in that, include: An image acquisition module is used to acquire original images of multiple jujubes at multiple flip angles. The multiple jujubes are displaced by the conveyor belt and rotated by the rotating mechanism on the conveyor belt. The original images are acquired by a camera located above the conveyor belt. The original images at multiple flip angles are used to cover the entire surface of the jujubes. The defect recognition module is used to sequentially perform mixed recognition on the original images at multiple flip angles to obtain the defect recognition results of the images in the image group. The mixed recognition includes visual processing algorithm recognition and AI algorithm recognition.

Images