Image recognition method of fruit and vegetable production and impurity removal all-in-one machine based on video monitoring

Abstract

The invention discloses an image recognition method of a fruit and vegetable production and impurity removal all-in-one machine based on video monitoring, and relates to the technical field of fruit and vegetable impurity removal recognition. Performing image preprocessing on the fruit and vegetable image to obtain a preliminary image; contour extraction is carried out on fruits and vegetables in the preliminary image, image feature extraction is carried out on fruit and vegetable images in the fruit and vegetable contour range, and fruit and vegetable image features are obtained; normal fruits and vegetables and abnormal fruits and vegetables are screened, normal features of the fruits and vegetables and abnormal features of the fruits and vegetables are extracted, and a fruit and vegetable abnormality recognition model is trained; after training is completed, abnormal fruits and vegetables in the fruit and vegetable production impurity removal production line are recognized through a fruit and vegetable abnormity recognition model; the method is used for solving the problems that an existing fruit and vegetable impurity removal recognition technology is insufficient in generalization ability and high in model training cost, and consequently the fruit and vegetable impurity removal recognition technology is limited in use.

Description

Technical Field

[0001] The present invention relates to the technical field of fruit and vegetable impurity removal and recognition, and specifically to an image recognition method for a fruit and vegetable production impurity removal all-in-one machine based on video monitoring. Background Technique

[0002] The fruit and vegetable impurity removal and recognition technology refers to an intelligent detection method based on computer vision, deep learning, and automation technology. Its purpose is to automatically identify foreign objects, defects, or non-target objects mixed in the fruit and vegetable production and processing process through image or video analysis, and drive the actuator to remove them, so as to achieve the automation of fruit and vegetable quality grading and impurity separation. Its core goal is to improve fruit and vegetable processing efficiency, reduce labor costs, and ensure food safety.

[0003] When the existing fruit and vegetable impurity removal and recognition technology identifies defective fruits and vegetables, it usually identifies a single type of fruit and vegetable, with insufficient generalization ability and inability to adapt to different varieties of fruits and vegetables. Moreover, the existing fruit and vegetable impurity removal and recognition technology requires a large number of defective fruits and vegetables for training the model. Training the model with defective fruits and vegetables increases the cost of data acquisition, and the defective fruits and vegetables cannot be converted into products, but only serve as training materials for the model and cannot generate income. For example, in the patent application with the publication number CN115908257A, a defective recognition model training method and a fruit and vegetable defect recognition method are disclosed. This solution trains the defective recognition model with defective fruits and vegetables, which will lead to a significant increase in training costs. The existing fruit and vegetable impurity removal and recognition technology also has problems such as insufficient generalization ability and high model training costs, resulting in limitations in the use of the fruit and vegetable impurity removal and recognition technology. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems in the prior art to some extent. By video monitoring the fruit and vegetable production and impurity removal line, fruit and vegetable images are extracted, and then the fruit and vegetable images are preprocessed to obtain preliminary images after processing. After that, the outlines of the fruits and vegetables in the preliminary images are extracted to obtain fruit and vegetable outlines, and then the fruit and vegetable images within the closed range enclosed by the fruit and vegetable outlines are extracted and named single-fruit images. Then, image features of the single-fruit images are extracted to obtain fruit and vegetable image features, and a fruit and vegetable anomaly recognition model is constructed. Normal fruits and vegetables and abnormal fruits and vegetables are screened, and their fruit and vegetable image features are extracted to obtain fruit and vegetable normal features and fruit and vegetable abnormal features. Based on the extracted fruit and vegetable normal features and fruit and vegetable abnormal features, the fruit and vegetable anomaly recognition model is trained. However, during the training process, only the fruit and vegetable normal features are used to train the fruit and vegetable anomaly recognition model, and the fruit and vegetable abnormal features are only used to confirm the accuracy rate of the fruit and vegetable anomaly recognition model. To save training costs, verification using abnormal fruits and vegetables may not be adopted either. After training, the fruit and vegetable anomaly recognition model is used to identify abnormal fruits and vegetables in the fruit and vegetable production and impurity removal line, so as to solve the problems that the existing fruit and vegetable impurity removal recognition technology still has insufficient generalization ability and relatively high model training cost, resulting in limitations in the use of the fruit and vegetable impurity removal recognition technology.

[0005] To achieve the above object, in the first aspect, the present application provides an image recognition method for a fruit and vegetable production and impurity removal all-in-one machine based on video monitoring, including the following steps:

[0006] Perform video monitoring on the fruit and vegetable production and impurity removal line to extract fruit and vegetable images;

[0007] Preprocess the fruit and vegetable images, and obtain preliminary images after processing;

[0008] Extract the outlines of the fruits and vegetables in the preliminary images. After extracting the fruit and vegetable outlines, perform image feature extraction on the fruit and vegetable images within the fruit and vegetable outline range to obtain fruit and vegetable image features;

[0009] Construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables, extract the fruit and vegetable image features of normal fruits and vegetables and abnormal fruits and vegetables to obtain fruit and vegetable normal features and fruit and vegetable abnormal features, and then train the fruit and vegetable anomaly recognition model based on the fruit and vegetable normal features and fruit and vegetable abnormal features;

[0010] After training, use the fruit and vegetable anomaly recognition model to identify abnormal fruits and vegetables in the fruit and vegetable production and impurity removal line.

[0011] Further, performing video monitoring on the fruit and vegetable production and impurity removal line to extract fruit and vegetable images includes the following sub-steps:

[0012] A high-resolution industrial camera is provided inside the fruit and vegetable production and impurity removal all-in-one machine for photographing the fruits and vegetables on the fruit and vegetable production and impurity removal line;

[0013] Name the image of the fruit and vegetable production impurity removal line obtained by shooting as the fruit and vegetable image.

[0014] Further, perform image preprocessing on the fruit and vegetable image. After the processing is completed, the preliminary image is obtained, including the following sub-steps:

[0015] Obtain the image of the fruit and vegetable production impurity removal line when it is not running, named the production line image, and there are no fruits and vegetables in the production line image;

[0016] Compare the production line image with the fruit and vegetable image, remove the pixel points with the same color value, and name the remaining fruit and vegetable image after removal as the background-free image;

[0017] Perform grayscale processing on the background-free image to obtain the preliminary image.

[0018] Further, extract the contours of the fruits and vegetables in the preliminary image. After extracting the fruit and vegetable contours, extract the image features of the fruit and vegetable image within the range of the fruit and vegetable contours, and obtain the fruit and vegetable image features, including the following sub-steps:

[0019] Extract the contours of the fruits and vegetables in the preliminary image to obtain the fruit and vegetable contours, and then extract the fruit and vegetable image within the enclosed range of the fruit and vegetable contours, named the single fruit image;

[0020] Extract the image features of the single fruit image to obtain the fruit and vegetable image features.

[0021] Further, extract the contours of the fruits and vegetables in the preliminary image to obtain the fruit and vegetable contours, and then extract the fruit and vegetable image within the enclosed range of the fruit and vegetable contours, named the single fruit image, including the following sub-steps:

[0022] Extract the contours of the preliminary image through the OpenCV contour extraction technology to obtain the fruit and vegetable contours;

[0023] Name the pixel points on the fruit and vegetable contours as contour points. For any fruit and vegetable contour, mark it as the target contour, and obtain the contour points of the target contour, named the target range points;

[0024] Extract the pixel points within the enclosed range of the target range points, excluding the target range points, to obtain the single fruit image.

[0025] Further, extract the image features of the single fruit image to obtain the fruit and vegetable image features, including the following sub-steps:

[0026] Edit serial numbers for the pixel points in the single fruit image, named pixel serial numbers, and represent the pixel serial numbers by the symbol P(n,m), where both n and m are positive integers and (n,m) is the serial number of P, and P(n,m) represents the pixel point at the nth row and the mth column;

[0027] Extract the grayscale value of each P(n, m), denoted as h(n, m), and then remove the h(n, m) with a value of 255 to obtain H(n, m).

[0028] Name the pixel points in the single-fruit image as single-fruit points. For any P(n, m), obtain the number of single-fruit points in its eight-neighborhood, denoted as the number of adjacent points. Group the P(n, m) with the same number of adjacent points into the same group, named the same-adjacent-quantity group.

[0029] For any same-adjacent-quantity group, label it as the group to be analyzed. For any single-fruit point in the group to be analyzed, label it as the point to be analyzed.

[0030] Label the P(n, m) adjacent to the point to be analyzed as the adjacent point, number the H(n, m) of the adjacent point, represented by the symbol T i where i is a positive integer and i is the serial number of T. Label the H(n, m) of the point to be analyzed as F, and calculate the absolute value of the difference between F and T i using the formula R i = |F - T i |, where R i is the absolute value of the difference between F and T i ;

[0031] Number R i in ascending order, represented by the symbol E i . Establish a plane rectangular coordinate system with i as the X-axis and E i as the Y-axis, named the pixel feature map, and input E i into the pixel feature map according to i;

[0032] Conduct a linear regression analysis on the pixel feature map to obtain the slope of the regression line, named the adjacent-point grayscale change amplitude. Analyze each single-fruit point in the same-adjacent-quantity group and analyze each same-adjacent-quantity group to obtain the adjacent-point grayscale change amplitudes of all single-fruit points;

[0033] Establish a plane rectangular coordinate system with the H(n, m) of P(n, m) as the horizontal axis and the corresponding adjacent-point grayscale change amplitude of P(n, m) as the vertical axis, named the single-fruit feature map. Input P(n, m) and its corresponding adjacent-point grayscale change amplitude into the single-fruit feature map, and each same-adjacent-quantity group has an independent single-fruit feature map;

[0034] Conduct a clustering analysis on the single-fruit feature map through a clustering algorithm to obtain different feature clustering sets. Mark the range of the feature clustering sets on the horizontal axis as the clustering range. Merge the feature clustering sets whose clustering ranges have intersections to obtain the merged clustering sets. If the feature clustering sets are not merged, mark themselves as the merged clustering sets;

[0035] Obtain the range of any merged clustering set on the vertical axis, mark it as the feature range, and find the maximum value of the feature range, which is marked as the fruit and vegetable image feature.

[0036] Furthermore, construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables, extract the fruit and vegetable image features of normal fruits and vegetables and abnormal fruits and vegetables to obtain fruit and vegetable normal features and fruit and vegetable abnormal features, and then train the fruit and vegetable anomaly recognition model based on the fruit and vegetable normal features and fruit and vegetable abnormal features, including the following sub-steps:

[0037] Construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables and extract their fruit and vegetable image features to obtain fruit and vegetable normal features and fruit and vegetable abnormal features;

[0038] Train the fruit and vegetable anomaly recognition model based on the obtained fruit and vegetable normal features and fruit and vegetable abnormal features.

[0039] Furthermore, construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables and extract their fruit and vegetable image features to obtain fruit and vegetable normal features and fruit and vegetable abnormal features, including the following sub-steps:

[0040] Construct a fruit and vegetable anomaly recognition model;

[0041] Screen normal fruits and vegetables and abnormal fruits and vegetables, and both the normal fruits and vegetables and the abnormal fruits and vegetables are screened for samples of the first sample quantity;

[0042] Extract the fruit and vegetable image features of normal fruits and vegetables and abnormal fruits and vegetables to obtain the first sample quantity of fruit and vegetable normal features and the first sample quantity of fruit and vegetable abnormal features.

[0043] Furthermore, train the fruit and vegetable anomaly recognition model based on the obtained fruit and vegetable normal features and fruit and vegetable abnormal features, including the following sub-steps:

[0044] Find the maximum value in the fruit and vegetable normal features, mark it as the fruit and vegetable normal maximum feature, find the minimum value in the fruit and vegetable abnormal features, and mark it as the fruit and vegetable abnormal minimum feature;

[0045] Compare the fruit and vegetable normal maximum feature with the fruit and vegetable abnormal minimum feature. If the fruit and vegetable normal maximum feature is less than the fruit and vegetable abnormal minimum feature, then output a perfect training signal; otherwise, output a correct rate test signal;

[0046] If a perfect training signal is output, mark the recognition accuracy of the fruit and vegetable anomaly recognition model as 100%, and at the same time mark the median of the fruit and vegetable normal maximum feature and the fruit and vegetable abnormal minimum feature as the fruit and vegetable feature normal threshold;

[0047] If an output correct rate test signal is generated, obtain the number of abnormal fruit and vegetable features that are less than or equal to the normal maximum features of the fruit and vegetables, mark it as A, mark the first sample quantity as B, calculate A / B to obtain the recognition accuracy of the fruit and vegetable abnormal recognition model, and at the same time mark the minimum abnormal fruit and vegetable feature as the normal threshold of the fruit and vegetable features.

[0048] Further, after the training is completed, the steps for identifying abnormal fruit and vegetables in the fruit and vegetable production impurity removal line through the fruit and vegetable abnormal recognition model include the following sub-steps:

[0049] Use the fruit and vegetable abnormal recognition model to identify the fruit and vegetables in the fruit and vegetable production impurity removal line. If the extracted fruit and vegetable image features are greater than the normal threshold of the fruit and vegetable features, mark the corresponding fruit and vegetables as abnormal fruit and vegetables;

[0050] If abnormal fruit and vegetables are identified, when the abnormal fruit and vegetables reach the end of the conveyor belt, use the abnormal handling container to pick up the abnormal fruit and vegetables.

[0051] Advantages of the present invention: The present invention conducts video monitoring on the fruit and vegetable production impurity removal line, extracts fruit and vegetable images, then performs image preprocessing on the fruit and vegetable images, obtains a preliminary image after the processing is completed, then extracts the contours of the fruit and vegetables in the preliminary image to obtain fruit and vegetable contours, and then extracts the fruit and vegetable images within the closed range surrounded by the fruit and vegetable contours, named single-fruit images. Then, image feature extraction is performed on the single-fruit images to obtain the amplitude of the gray-scale change of adjacent points of each fruit and vegetable. Then, through clustering analysis, fruit and vegetable image features are obtained. The advantage is that for the same type of fruit and vegetables, the color distribution and color changes on their surfaces are relatively small. When there are defects in the fruit and vegetables, there will be color regions on the surface that are different from the normal colors of the fruit and vegetables. Therefore, by analyzing the amplitude of the gray-scale change of adjacent points of each fruit and vegetable, it can be preliminarily determined whether there are defects in the fruit and vegetables. Then, through clustering analysis, the characteristics of the amplitude of the gray-scale change of adjacent points of the fruit and vegetables, that is, the fruit and vegetable image features, are determined, improving the accuracy and rationality of fruit and vegetable impurity removal recognition;

[0052] The present invention constructs a fruit and vegetable abnormal recognition model, screens normal fruit and vegetables and abnormal fruit and vegetables and extracts their fruit and vegetable image features to obtain normal fruit and vegetable features and abnormal fruit and vegetable features. Based on the extracted normal fruit and vegetable features and abnormal fruit and vegetable features, the fruit and vegetable abnormal recognition model is trained. After the training is completed, the fruit and vegetable abnormal recognition model is used to identify abnormal fruit and vegetables in the fruit and vegetable production impurity removal line. The advantage is that during the training process, only the normal fruit and vegetable features are used to train the fruit and vegetable abnormal recognition model, and the abnormal fruit and vegetable features are only used to confirm the correct rate of the fruit and vegetable abnormal recognition model. To save training costs, abnormal fruit and vegetables can also not be used for verification, improving the effectiveness of fruit and vegetable impurity removal recognition and reducing the cost of model training. Description of the Drawings

[0053] Figure 1 is the step flow chart of the method of the present invention;

[0054] Figure 2 Schematic diagram of the fruit and vegetable image of the present invention;

[0055] Figure 3 Schematic diagram of the single fruit image of the present invention;

[0056] Figure 4 Schematic diagram of the pixel feature map of the present invention;

[0057] Figure 5 Schematic diagram of the single fruit feature map of the present invention. Detailed implementation manners

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0059] Embodiment 1, please refer to Figure 1 As shown, the present application provides an image recognition method for a fruit and vegetable production impurity removal all-in-one machine based on video monitoring, including the following steps:

[0060] Step S1, perform video monitoring on the fruit and vegetable production impurity removal production line, and extract the fruit and vegetable image; Step S1 includes the following sub-steps:

[0061] Please refer to Figure 2 As shown, in step S101, a high-resolution industrial camera is provided inside the fruit and vegetable production impurity removal all-in-one machine for photographing the fruits and vegetables on the fruit and vegetable production impurity removal production line;

[0062] Step S102, name the image of the fruit and vegetable production impurity removal production line obtained by photographing as the fruit and vegetable image;

[0063] In specific implementation, the obtained fruit and vegetable image is as Figure 2 shown, Figure 2 in which is the front view of the fruit and vegetable production impurity removal production line. In addition, there are also left and right views for analyzing whether there are defects on the side of the fruits and vegetables. The analysis process is the same as that of the front view. Therefore, only the analysis process of the front view is listed in this embodiment.

[0064] Step S2, perform image preprocessing on the fruit and vegetable image, and obtain a preliminary image after the processing; Step S2 includes the following sub-steps:

[0065] Step S201, obtain the image of the fruit and vegetable production impurity removal production line when it is not running, name it the production line image, and there are no fruits and vegetables in the production line image;

[0066] Step S202: Compare the production line image with the fruit and vegetable image, eliminate the pixel points with the same color value, and name the remaining fruit and vegetable image after elimination as the background-free image.

[0067] Step S203: Perform grayscale processing on the background-free image to obtain a preliminary image.

[0068] In specific implementation, when shooting the fruit and vegetable image and the production line image, illuminate the fruit and vegetable production impurity removal production line. When illuminating, it is necessary to ensure that there are no shadows in the fruit and vegetable image and the production line image, that is, provide 360° lighting for the fruit and vegetable production impurity removal production line, and then compare to obtain the background-free image. Only fruits and vegetables exist in the background-free image, and non-fruit and vegetable parts are all transparent. Then, a preliminary image is obtained through grayscale processing. It should be noted that the fruits and vegetables have undergone a cleaning step before shooting the fruit and vegetable image. Therefore, the surface of the fruits and vegetables is only the color of the fruits and vegetables themselves, without other impurities blocking. The impurity removal recognition in this embodiment is to identify defective fruits and vegetables.

[0069] Step S3: Extract the contours of the fruits and vegetables in the preliminary image. After extracting the fruit and vegetable contours, extract the image features of the fruit and vegetable image within the range of the fruit and vegetable contours to obtain the fruit and vegetable image features. Step S3 includes the following sub-steps:

[0070] Step S301: Extract the contours of the fruits and vegetables in the preliminary image to obtain the fruit and vegetable contours, and then extract the fruit and vegetable image within the enclosed range surrounded by the fruit and vegetable contours, and name it the single-fruit image.

[0071] Step S301 includes the following sub-steps:

[0072] Step S301.1: Use the OpenCV contour extraction technology to extract the contours of the preliminary image to obtain the fruit and vegetable contours.

[0073] Step S301.2: Name the pixel points on the fruit and vegetable contours as contour points. For any fruit and vegetable contour, mark it as the target contour, and obtain the contour points of the target contour, and name them target range points.

[0074] Please refer to Figure 3 As shown, Step S301.3: Extract the pixel points within the enclosed range surrounded by the target range points, excluding the target range points, to obtain the single-fruit image.

[0075] In specific implementation, use the existing OpenCV contour extraction technology to extract the contours of the preliminary image. The single-fruit image obtained based on the fruit and vegetable contours is as Figure 3 shown;

[0076] Step S302: Extract the image features of the single-fruit image to obtain the fruit and vegetable image features.

[0077] Step S302 includes the following sub-steps:

[0078] Step S302.1: Edit serial numbers for the pixel points in the single-fruit image, named pixel serial numbers, and represented by the symbol P(n, m). Here, both n and m are positive integers, and (n, m) is the serial number of P. P(n, m) represents the pixel point at the nth row and the mth column.

[0079] Step S302.2: Extract the gray value of each P(n, m), marked as h(n, m), and then eliminate the h(n, m) with a value of 255 to obtain H(n, m).

[0080] Step S302.3: Name the pixel points in the single-fruit image as single-fruit points. For any P(n, m), obtain the number of single-fruit points in its eight-neighborhood, named the adjacent-point number; group the P(n, m) with the same adjacent-point number into the same group, named the same-adjacent-quantity group.

[0081] In specific implementation, the numbering of P(n, m) includes transparent pixel points. P(n, m) is obtained by editing the serial number, where 1 ≤ n ≤ 41 and 1 ≤ m ≤ 30. Then, eliminate the P(n, m) with a gray value of 255. The gray value of the remaining P(n, m) is H(n, m). Taking H(7, 8) as an example, the H(n, m) in its eight-neighborhood are H(7, 7), H(8, 7), H(6, 8), H(8, 8), H(6, 9), H(7, 9), and H(8, 9) respectively. There is no P(n, m) with h(n, m) being 255. Therefore, the adjacent-point number is 8. Group the P(n, m) with the same adjacent-point number into the same same-adjacent-quantity group. Through analysis, 6 same-adjacent-quantity groups are obtained, and the corresponding adjacent-point numbers are 3, 4, 5, 6, 7, and 8 respectively. In this embodiment, they are called the 3-same-adjacent group, 4-same-adjacent group, 5-same-adjacent group, 6-same-adjacent group, 7-same-adjacent group, and 8-same-adjacent group in sequence.

[0082] Step S302.4: For any same-adjacent-quantity group, mark it as the group to be analyzed. For any single-fruit point in the group to be analyzed, mark it as the point to be analyzed.

[0083] Step S302.5: Mark the P(n, m) adjacent to the point to be analyzed as the adjacent point, and number the H(n, m) of the adjacent point, represented by the symbol T i where i is a positive integer and i is the serial number of T. Mark the H(n, m) of the point to be analyzed as F. Calculate the absolute value of the difference between F and T i using the formula R i = |F - T i |. Here, R i is the absolute value of the difference between F and T i .

[0084] Please refer to Figure 4As shown, in step S302.6, number R i in ascending order, and represent it with the symbol E i . Taking i as the X-axis and E i as the Y-axis, establish a plane rectangular coordinate system, name it the pixel feature map, and input E i into the pixel feature map according to i;

[0085] In step S302.7, perform a linear regression analysis on the pixel feature map to obtain the slope of the regression line, name it the adjacent point gray-scale change amplitude, analyze each single fruit point in the same adjacent quantity group, and analyze each same adjacent quantity group to obtain the adjacent point gray-scale change amplitude of all single fruit points;

[0086] In a specific implementation, still taking H(7,8) as an example, it is obtained that H(7,8) is 198, and H(7,7), H(8,7), H(6,8), H(8,8), H(6,9), H(7,9), and H(8,9) are 176, 192, 194, 190, 198, 195, 198, and 198 in sequence, corresponding to T1 to T8 respectively, F is 198, and it is calculated that R1 to R8 are 22, 6, 4, 8, 0, 3, 0, and 0 in sequence. After sorting, E1 to E8 are 0, 0, 0, 3, 4, 6, 8, and 22 in sequence. Thus, the pixel feature map of P(n,m) is constructed as Figure 4 shown. Through linear regression analysis, the adjacent point gray-scale change amplitude of P(n,m) is 2.5357. In this embodiment, it is not necessary to use the formula of the regression line for calculation. Therefore, only the slope of the regression line is given in this embodiment, and the regression line is not described; analyze each single fruit point in the same adjacent quantity group, that is, analyze all single fruit points in the 3-adjacent group, 4-adjacent group, 5-adjacent group, 6-adjacent group, 7-adjacent group, and 8-adjacent group. The adjacent point gray-scale change amplitudes obtained from the analysis of each different same adjacent quantity group are independent of each other;

[0087] Please refer to Figure 5 shown. In step S302.8, taking the H(n,m) of P(n,m) as the horizontal axis and the adjacent point gray-scale change amplitude corresponding to P(n,m) as the vertical axis, establish a plane rectangular coordinate system, name it the single fruit feature map, input P(n,m) and its corresponding adjacent point gray-scale change amplitude into the single fruit feature map, and each same adjacent quantity group has an independent single fruit feature map;

[0088] In step S302.9, perform a clustering analysis on the single fruit feature map through a clustering algorithm to obtain different feature clustering sets. Mark the range of the feature clustering set on the horizontal axis as the clustering range, merge the feature clustering sets whose clustering ranges have intersections to obtain a merged clustering set. If the feature clustering set is not merged, mark itself as the merged clustering set;

[0089] Step S302.10, obtain the range of any merged clustering set on the vertical axis, mark it as the feature range, and find the maximum value of the feature range, which is marked as the fruit and vegetable image feature;

[0090] In specific implementation, taking the 8-connected neighborhood group as an example, a single fruit feature map is constructed as shown in Figure 5 shown, Figure 5 The coordinate points within a rectangular range in it are a feature clustering set. The numbers within the rectangle are the serial numbers of the feature clustering sets. There are a total of 8 feature clustering sets. Among them, the 1st feature clustering set and the 6th feature clustering set do not intersect with any feature clustering set on the horizontal axis. Therefore, the 1st feature clustering set and the 7th feature clustering set each form a merged clustering set. The 2nd feature clustering set and the 3rd feature clustering set have an intersecting range on the horizontal axis. At the same time, both the 4th feature clustering set and the 5th feature clustering set intersect with the 3rd feature clustering set on the horizontal axis. Therefore, the 2nd feature clustering set, the 3rd feature clustering set, the 4th feature clustering set, and the 5th feature clustering set are merged into a merged clustering set. Similarly, the 7th feature clustering set and the 8th feature clustering set are merged into a merged clustering set. Taking the merged clustering set obtained by merging the 2nd feature clustering set, the 3rd feature clustering set, the 4th feature clustering set, and the 5th feature clustering set as an example, the minimum and maximum values of the vertical axis among the coordinate points are 1.1579 and 3.0683 respectively. Then its feature range is [1.1579, 3.0683]. The size of the feature range is the difference between the maximum value and the minimum value. Obtaining [1.1579, 3.0683] is the maximum value of the feature range. Therefore, mark 3.0683 - 1.1579 = 1.9104 as the fruit and vegetable image feature of the 8-connected neighborhood group. Each different connected neighborhood quantity group has an independent fruit and vegetable image feature.

[0091] Step S4, construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables, extract the fruit and vegetable image features of normal fruits and vegetables and abnormal fruits and vegetables to obtain the normal fruit and vegetable features and the abnormal fruit and vegetable features, and then train the fruit and vegetable anomaly recognition model based on the normal fruit and vegetable features and the abnormal fruit and vegetable features; Step S4 includes the following sub-steps:

[0092] Step S401, construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables and extract their fruit and vegetable image features to obtain the normal fruit and vegetable features and the abnormal fruit and vegetable features;

[0093] Step S401 includes the following sub-steps:

[0094] Step S401.1, construct a fruit and vegetable anomaly recognition model;

[0095] Step S401.2, screen normal fruits and vegetables and abnormal fruits and vegetables. Both normal fruits and vegetables and abnormal fruits and vegetables screen samples of the first sample quantity;

[0096] Step S401.3: Extract the fruit and vegetable image features of normal fruits and vegetables and abnormal fruits and vegetables to obtain the fruit and vegetable normal features of the first sample quantity and the fruit and vegetable abnormal features of the first sample quantity;

[0097] In specific implementation, a fruit and vegetable abnormality recognition model is constructed. There is no special requirement for the first sample quantity, which can be set by the user. Whether to test the accuracy rate of the fruit and vegetable abnormality recognition model through abnormal fruits and vegetables is also determined by the user. In this embodiment, listing the test of the accuracy rate of the fruit and vegetable abnormality recognition model through abnormal fruits and vegetables is only to provide a test method for the accuracy rate of the fruit and vegetable abnormality recognition model. If the user chooses to save costs, it is also possible to choose not to conduct the test and only select normal fruits and vegetables to analyze the normal threshold of fruit and vegetable features. If only normal fruits and vegetables are selected for analysis, then the minimum fruit and vegetable abnormality feature in step S402 is the normal threshold of fruit and vegetable features. In this embodiment, the first sample quantity is set to 500, and abnormal fruits and vegetables are not selected for testing, that is, only 500 fruit and vegetable normal features are extracted. Each fruit and vegetable normal feature is further divided into fruit and vegetable image features of different co-neighbor quantity groups;

[0098] Step S402: Train the fruit and vegetable abnormality recognition model based on the extracted fruit and vegetable normal features and fruit and vegetable abnormal features;

[0099] Step S402 includes the following sub-steps:

[0100] Step S402.1: Find the maximum value in the fruit and vegetable normal features, mark it as the fruit and vegetable normal maximum feature, and find the minimum value in the fruit and vegetable abnormal features, mark it as the fruit and vegetable abnormal minimum feature;

[0101] Step S402.2: Compare the fruit and vegetable normal maximum feature with the fruit and vegetable abnormal minimum feature. If the fruit and vegetable normal maximum feature is less than the fruit and vegetable abnormal minimum feature, output a perfect training signal; otherwise, output a correct rate test signal;

[0102] Step S402.3: If a perfect training signal is output, mark the recognition accuracy of the fruit and vegetable abnormality recognition model as 100%, and at the same time mark the median of the fruit and vegetable normal maximum feature and the fruit and vegetable abnormal minimum feature as the normal threshold of fruit and vegetable features;

[0103] Step S402.4: If a correct rate test signal is output, obtain the number of fruit and vegetable abnormal features less than or equal to the fruit and vegetable normal maximum feature, mark it as A, mark the first sample quantity as B, calculate A / B to obtain the recognition accuracy of the fruit and vegetable abnormality recognition model, and at the same time mark the fruit and vegetable abnormal minimum feature as the normal threshold of fruit and vegetable features;

[0104] In specific implementation, there is a normal threshold for fruit and vegetable features in each co-neighbor quantity group. In this embodiment, only normal fruits and vegetables are selected for analysis. When analyzing through abnormal fruits and vegetables, the test process of the correct rate has been given in detail and will not be specifically described in this embodiment. When analyzing only through normal fruits and vegetables, for each co-neighbor quantity group, the maximum value of the fruit and vegetable image features in the same co-neighbor quantity group is selected as the normal threshold for fruit and vegetable features.

[0105] Step S5: After training, use the fruit and vegetable abnormality recognition model to identify abnormal fruits and vegetables in the fruit and vegetable production impurity removal line; Step S5 includes the following sub-steps:

[0106] Step S501: Use the fruit and vegetable abnormality recognition model to identify the fruits and vegetables in the fruit and vegetable production impurity removal line. If the extracted fruit and vegetable image features are greater than the normal threshold for fruit and vegetable features, mark the corresponding fruits and vegetables as abnormal fruits and vegetables;

[0107] Step S502: If abnormal fruits and vegetables are identified, when the abnormal fruits and vegetables reach the end of the conveyor belt, intercept the abnormal fruits and vegetables with an abnormal handling container;

[0108] In specific implementation, the normal threshold for fruit and vegetable features represents the maximum value of the color change range on the surface of normal fruits and vegetables. If the fruit and vegetable image features are larger than the normal threshold for fruit and vegetable features, it means that the color change on the surface of the fruits and vegetables exceeds that of normal fruits and vegetables, generally due to defects in the fruits and vegetables, resulting in the color change presented on their surfaces; there is a normal threshold for fruit and vegetable features in each co-neighbor quantity group, and different co-neighbor quantity groups also exist when extracting fruit and vegetable image features. Compare the same co-neighbor quantity groups. If the fruit and vegetable image features of any co-neighbor quantity group are greater than the corresponding normal threshold for fruit and vegetable features, mark the fruits and vegetables as abnormal fruits and vegetables; when dealing with abnormal fruits and vegetables, if abnormal fruits and vegetables are identified, mark them in the monitoring screen. When they move to the end of the conveyor belt, the fruits and vegetables on the rest of the conveyor belt will freely fall to the sorting opening of the next process, and a container will extend under the conveyor belt where the abnormal fruits and vegetables are located to load the abnormal fruits and vegetables. After the abnormal fruits and vegetables fall into the container, the container will retract.

[0109] Embodiment 2. The present application provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus. The memory stores computer-readable instructions, and the processor can call the instructions in the memory. When the computer-readable instructions are executed by the processor, the steps in the image recognition method of the fruit and vegetable production impurity removal integrated machine based on video monitoring are run to achieve the following functions: extracting fruit and vegetable images; performing image preprocessing on the fruit and vegetable images to obtain preliminary images; extracting the contours of the fruits and vegetables in the preliminary images, and extracting image features of the fruit and vegetable images within the fruit and vegetable contour ranges to obtain fruit and vegetable image features; screening normal fruits and vegetables from abnormal fruits and vegetables, extracting normal fruit and vegetable features and abnormal fruit and vegetable features, and training the fruit and vegetable abnormal recognition model; after the training is completed, identifying abnormal fruits and vegetables in the fruit and vegetable production impurity removal production line through the fruit and vegetable abnormal recognition model.

[0110] In addition, when the logical instructions in the above-mentioned memory are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs, Read-Only Memories), random access memories (RAMs, Random Access Memories), magnetic disks, or optical discs that can store program codes.

[0111] Embodiment 3. The present application also provides a computer-readable storage medium. The present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the image recognition method of the fruit and vegetable production impurity removal integrated machine based on video monitoring are run to achieve the following functions: extracting fruit and vegetable images; performing image preprocessing on the fruit and vegetable images to obtain preliminary images; extracting the contours of the fruits and vegetables in the preliminary images, and extracting image features of the fruit and vegetable images within the fruit and vegetable contour ranges to obtain fruit and vegetable image features; screening normal fruits and vegetables from abnormal fruits and vegetables, extracting normal fruit and vegetable features and abnormal fruit and vegetable features, and training the fruit and vegetable abnormal recognition model; after the training is completed, identifying abnormal fruits and vegetables in the fruit and vegetable production impurity removal production line through the fruit and vegetable abnormal recognition model.

[0112] Through the description of the above embodiments, the embodiments of the present invention can be provided as a method, a system or a computer program product. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0113] In the embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. The above-described embodiments are merely illustrative. For example, the division of modules or units is only a logical function division, and there can be other division methods in actual implementation. For another example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of systems, modules, and units can be electrical, mechanical, or other forms.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. Image recognition method of a fruit and vegetable production impurity removal all-in-one machine based on video surveillance, characterized in that, The steps include the following: Video monitor the fruit and vegetable production and impurity removal line, and extract fruit and vegetable images; Perform image preprocessing on the fruit and vegetable images, and obtain preliminary images after the processing; Extract the contours of the fruits and vegetables in the preliminary images. After extracting the fruit and vegetable contours, extract the image features of the fruit and vegetable images within the range of the fruit and vegetable contours to obtain fruit and vegetable image features; Construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables, extract the fruit and vegetable image features of normal fruits and vegetables and abnormal fruits and vegetables to obtain normal fruit and vegetable features and abnormal fruit and vegetable features, and then train the fruit and vegetable anomaly recognition model based on the normal fruit and vegetable features and abnormal fruit and vegetable features; After the training is completed, identify the abnormal fruits and vegetables in the fruit and vegetable production and impurity removal line through the fruit and vegetable anomaly recognition model.

2. The image recognition method of the fruit and vegetable production impurity removal integrated machine based on video surveillance according to claim 1, characterized in that, Video monitor the fruit and vegetable production and impurity removal line, and extracting fruit and vegetable images includes the following sub-steps: A high-resolution industrial camera is installed inside the fruit and vegetable production and impurity removal all-in-one machine, which is used to photograph the fruits and vegetables on the fruit and vegetable production and impurity removal line; Name the image of the fruit and vegetable production and impurity removal line obtained by photographing as the fruit and vegetable image.

3. The image recognition method of the fruit and vegetable production impurity removal integrated machine based on video monitoring according to claim 2, characterized in that, Perform image preprocessing on the fruit and vegetable images, and obtaining preliminary images after the processing includes the following sub-steps: Obtain the image of the fruit and vegetable production and impurity removal line when it is not running, and name it the production line image. There are no fruits and vegetables in the production line image; Compare the production line image with the fruit and vegetable image, remove the pixel points with the same color value, and name the remaining fruit and vegetable image after the removal as the background-free image; Perform grayscale processing on the background-free image to obtain the preliminary image.

4. The image recognition method of the fruit and vegetable production impurity removal all-in-one machine based on video monitoring according to claim 3, characterized in that, Extract the contours of the fruits and vegetables in the preliminary images. After extracting the fruit and vegetable contours, extract the image features of the fruit and vegetable images within the range of the fruit and vegetable contours to obtain fruit and vegetable image features includes the following sub-steps: Extract the contours of the fruits and vegetables in the preliminary images to obtain fruit and vegetable contours, and then extract the fruit and vegetable images within the enclosed range surrounded by the fruit and vegetable contours, and name it the single fruit image; Extract the image features of the single fruit image to obtain the fruit and vegetable image features.

5. The image recognition method of the fruit and vegetable production impurity removal integrated machine based on video monitoring according to claim 4, wherein, Extract the contours of the fruits and vegetables in the preliminary images to obtain fruit and vegetable contours, and then extract the fruit and vegetable images within the enclosed range surrounded by the fruit and vegetable contours, and name it the single fruit image includes the following sub-steps: Extract the contours of the preliminary images through the OpenCV contour extraction technology to obtain fruit and vegetable contours; Name the pixel points on the fruit and vegetable contours as contour points. For any fruit and vegetable contour, mark it as the target contour, and obtain the contour points of the target contour, and name them target range points; Extract the pixel points within the enclosed range surrounded by the target range points, excluding the target range points, to obtain the single fruit image.

6. The image recognition method of the fruit and vegetable production impurity removal all-in-one machine based on video monitoring according to claim 5, characterized in that, Extract the image features of the single fruit image to obtain the fruit and vegetable image features includes the following sub-steps: Edit serial numbers for the pixel points in the single fruit image, and name them pixel serial numbers, which are represented by the symbol P(n,m). Among them, both n and m are positive integers and (n,m) is the serial number of P. P(n,m) represents the pixel point at the nth row and the mth column; Extract the grayscale value of each P(n,m), mark it as h(n,m), and then remove the h(n,m) with the value of 255 to obtain H(n,m); Name the pixel points in the single fruit image as single fruit points. For any P(n,m), obtain the number of single fruit points in its eight-neighborhood, named the number of adjacent points; group the P(n,m) with the same number of adjacent points into the same group, named the group with the same adjacent quantity. For any group with the same adjacent quantity, label it as the group to be analyzed. For any single fruit point in the group to be analyzed, label it as the point to be analyzed. Mark the P(n,m) adjacent to the point to be analyzed as the adjacent point, number the H(n,m) of the adjacent point, and represent it by the symbol T i where i is a positive integer and i is the serial number of T. Mark the H(n,m) of the point to be analyzed as F, and calculate the absolute value of the difference between F and T through the formula R i = |F - T i |, where R i is the absolute value of the difference between F and T i ; i ​ Number R in ascending order i and represent it with the symbol E i Taking i as the X-axis and E i as the Y-axis, establish a plane rectangular coordinate system, named the pixel feature map, and input E i into the pixel feature map according to i; Conduct a linear regression analysis on the pixel feature map to obtain the slope of the regression line, named the amplitude of adjacent point gray-scale change. Analyze each single fruit point in the group with the same adjacent quantity and analyze each group with the same adjacent quantity to obtain the amplitude of adjacent point gray-scale change for all single fruit points. Establish a plane rectangular coordinate system with H(n,m) of P(n,m) as the horizontal axis and the amplitude of adjacent point gray-scale change corresponding to P(n,m) as the vertical axis, named the single fruit feature map. Enter P(n,m) and its corresponding amplitude of adjacent point gray-scale change into the single fruit feature map. Each group with the same adjacent quantity has an independent single fruit feature map. Conduct a clustering analysis on the single fruit feature map through a clustering algorithm to obtain different feature clustering sets. Mark the range of the feature clustering set on the horizontal axis as the clustering range. Merge the feature clustering sets whose clustering ranges have intersections to obtain the merged clustering set. If the feature clustering set is not merged, mark itself as the merged clustering set. Obtain the range of any merged clustering set on the vertical axis, mark it as the feature range, and find the maximum value of the feature range, marked as the fruit and vegetable image feature.

7. The image recognition method of the fruit and vegetable production impurity removal integrated machine based on video monitoring according to claim 6, characterized in that, Construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables, extract the fruit and vegetable image features of normal fruits and vegetables and abnormal fruits and vegetables to obtain the normal fruit and vegetable features and abnormal fruit and vegetable features. Then, train the fruit and vegetable anomaly recognition model based on the normal fruit and vegetable features and abnormal fruit and vegetable features, including the following sub-steps: Construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables and extract their fruit and vegetable image features to obtain the normal fruit and vegetable features and abnormal fruit and vegetable features. Train the fruit and vegetable anomaly recognition model based on the extracted normal fruit and vegetable features and abnormal fruit and vegetable features.

8. The image recognition method of the fruit and vegetable production impurity removal integrated machine based on video monitoring according to claim 7, characterized in that, Construct a fruit and vegetable anomaly recognition model, screen normal fruits and vegetables and abnormal fruits and vegetables and extract their fruit and vegetable image features to obtain the normal fruit and vegetable features and abnormal fruit and vegetable features, including the following sub-steps: Construct a fruit and vegetable anomaly recognition model. Screen normal fruits and vegetables and abnormal fruits and vegetables. Both the normal fruits and vegetables and the abnormal fruits and vegetables are screened for a first sample quantity of samples. Extract the fruit and vegetable image features of normal fruits and vegetables and abnormal fruits and vegetables to obtain a first sample quantity of normal fruit and vegetable features and a first sample quantity of abnormal fruit and vegetable features.

9. The image recognition method of the fruit and vegetable production impurity removal integrated machine based on video monitoring according to claim 8, characterized in that, Train the fruit and vegetable anomaly recognition model based on the extracted normal fruit and vegetable features and abnormal fruit and vegetable features, including the following sub-steps: Find the maximum value in the normal fruit and vegetable features, marked as the maximum normal fruit and vegetable feature. Find the minimum value in the abnormal fruit and vegetable features, marked as the minimum abnormal fruit and vegetable feature. Compare the maximum normal fruit and vegetable feature with the minimum abnormal fruit and vegetable feature. If the maximum normal fruit and vegetable feature is less than the minimum abnormal fruit and vegetable feature, output a perfect training signal; otherwise, output a correct rate test signal. If a perfect training signal is output, the recognition accuracy of the fruit and vegetable anomaly recognition model is marked as 100%, and the median of the maximum normal fruit and vegetable feature and the minimum abnormal fruit and vegetable feature is marked as the normal fruit and vegetable feature threshold; If a correct rate test signal is output, obtain the number of abnormal fruit and vegetable features that are less than or equal to the maximum normal fruit and vegetable feature, mark it as A, mark the first sample quantity as B, calculate A / B to obtain the recognition accuracy of the fruit and vegetable anomaly recognition model, and at the same time mark the minimum abnormal fruit and vegetable feature as the normal fruit and vegetable feature threshold.

10. The image recognition method of the fruit and vegetable production impurity removal integrated machine based on video monitoring according to claim 9, characterized in that, After training is completed, the steps for identifying abnormal fruits and vegetables in the fruit and vegetable production impurity removal line through the fruit and vegetable anomaly recognition model include the following sub-steps: Use the fruit and vegetable anomaly recognition model to identify the fruits and vegetables in the fruit and vegetable production impurity removal line. If the extracted fruit and vegetable image feature is greater than the normal fruit and vegetable feature threshold, mark the corresponding fruit and vegetable as abnormal fruit and vegetable; If an abnormal fruit and vegetable is recognized, when the abnormal fruit and vegetable reaches the end of the conveyor belt, use the abnormal processing container to pick up the abnormal fruit and vegetable.

Images