Image recognition method of fruit and vegetable production impurity integrated machine based on video monitoring

Abstract

The application discloses a fruit and vegetable production impurity removal integrated machine image recognition method based on video monitoring, relates to the fruit and vegetable impurity removal recognition technical field, and comprises the following steps: fruit and vegetable image extraction; fruit and vegetable image pre-processing is carried out, and a preliminary image is obtained; the fruit and vegetable in the preliminary image is profiled, the fruit and vegetable image in the fruit and vegetable profile range is image feature extracted, and fruit and vegetable image features are obtained; normal fruit and vegetable and abnormal fruit and vegetable are screened, fruit and vegetable normal features and fruit and vegetable abnormal features are extracted, and a fruit and vegetable abnormal recognition model is trained; after training is completed, the fruit and vegetable abnormal recognition model is used for identifying abnormal fruit and vegetable in a fruit and vegetable production impurity removal production line; the application is used for solving the problems that the existing fruit and vegetable impurity removal recognition technology still has insufficient generalization ability and high model training cost, and leads to the limitation of fruit and vegetable impurity removal recognition technology in use.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fruit and vegetable impurity recognition, in particular to an image recognition method of a fruit and vegetable production impurity removal integrated machine based on video monitoring. BACKGROUND

[0002] Fruit and vegetable impurity recognition technology refers to an intelligent detection method based on computer vision, deep learning and automation technology, aiming to automatically identify mixed foreign matter, defects or non-target objects in fruit and vegetable production and processing through image or video analysis, and drive the execution mechanism to remove them, so as to realize the automation of fruit and vegetable quality grading and impurity separation. The core goal is to improve fruit and vegetable processing efficiency, reduce labor costs, and ensure food safety.

[0003] The existing fruit and vegetable impurity recognition technology usually identifies single kind of fruit and vegetable when identifying fruit and vegetable with defects, has insufficient generalization ability, cannot adapt to different varieties of fruit and vegetable, and needs a large number of defective fruit and vegetable for training model. Training the model with defective fruit and vegetable increases the cost of data acquisition, and the defective fruit and vegetable cannot be converted into products, only serving as training materials for the model, and cannot generate revenue. For example, in the patent application with publication number CN115908257A, a defect recognition model training method and a fruit and vegetable defect recognition method are disclosed. This scheme trains the defect recognition model with defective fruit and vegetable, which will greatly increase the training cost. The existing fruit and vegetable impurity recognition technology also has the problems of insufficient generalization ability and high model training cost, which limits the use of fruit and vegetable impurity recognition technology. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the prior art by video monitoring of the fruit and vegetable production impurity removal production line, extracting fruit and vegetable images, then performing image preprocessing on the fruit and vegetable images, obtaining preliminary images after processing, performing contour extraction on the fruit and vegetable in the preliminary images, obtaining fruit and vegetable contours, then extracting fruit and vegetable images in the closed range surrounded by the fruit and vegetable contours, naming them as single fruit images, then performing image feature extraction on the single fruit images, obtaining fruit and vegetable image features, constructing a fruit and vegetable anomaly recognition model, screening normal fruit and vegetables and abnormal fruit and vegetables and extracting their fruit and vegetable image features, obtaining fruit and vegetable normal features and fruit and vegetable abnormal features, training the fruit and vegetable anomaly recognition model based on the extracted fruit and vegetable normal features and fruit and vegetable abnormal features, but only training the fruit and vegetable anomaly recognition model with fruit and vegetable normal features during the training process, and fruit and vegetable abnormal features are only used to confirm the accuracy of the fruit and vegetable anomaly recognition model, to save training cost, abnormal fruit and vegetables can also be used for verification, and after training, the fruit and vegetable anomaly recognition model is used to identify abnormal fruit and vegetables in the fruit and vegetable production impurity removal production line, to solve the problems of insufficient generalization ability and high model training cost of existing fruit and vegetable impurity removal recognition technology, resulting in limitations of fruit and vegetable impurity removal recognition technology in use.

[0005] To achieve the above-mentioned purpose, in a first aspect, the present application provides an image recognition method of a fruit and vegetable production impurity removal all-in-one machine based on video monitoring, comprising the following steps:

[0006] Video monitoring of the fruit and vegetable production impurity removal production line is performed to extract fruit and vegetable images;

[0007] Image preprocessing is performed on the fruit and vegetable images, and preliminary images are obtained after processing;

[0008] Contour extraction is performed on the fruit and vegetable in the preliminary images, and after the fruit and vegetable contours are extracted, image feature extraction is performed on the fruit and vegetable images in the fruit and vegetable contour range to obtain fruit and vegetable image features;

[0009] A fruit and vegetable anomaly recognition model is constructed, normal fruit and vegetables and abnormal fruit and vegetables are screened, fruit and vegetable image features of normal fruit and vegetables and abnormal fruit and vegetables are extracted, fruit and vegetable normal features and fruit and vegetable abnormal features are obtained, and then the fruit and vegetable anomaly recognition model is trained based on the fruit and vegetable normal features and the fruit and vegetable abnormal features;

[0010] After training, the fruit and vegetable anomaly recognition model is used to identify abnormal fruit and vegetables in the fruit and vegetable production impurity removal production line.

[0011] Further, video monitoring of the fruit and vegetable production impurity removal production line to extract fruit and vegetable images comprises the following sub-steps:

[0012] A high-resolution industrial camera is arranged inside the fruit and vegetable production impurity removal all-in-one machine to take pictures of the fruit and vegetables on the fruit and vegetable production impurity removal production line;

[0013] The images of the fruit and vegetable production line that were captured were named "Fruit and Vegetable Images".

[0014] Further, image preprocessing is performed on the fruit and vegetable images. The preliminary images obtained after processing include the following sub-steps:

[0015] Obtain an image of the fruit and vegetable production line when it is not running, and name it the production line image. The production line image does not contain any fruits or vegetables.

[0016] Compare the production line image with the fruit and vegetable image, remove pixels with the same color value, and name the remaining fruit and vegetable image after removal as "no background image".

[0017] The image without background is converted to grayscale to obtain a preliminary image.

[0018] Further, contour extraction is performed on the fruits and vegetables in the preliminary image. After the contours of the fruits and vegetables are extracted, image feature extraction is performed on the fruit and vegetable images within the contour range to obtain the fruit and vegetable image features, including the following sub-steps:

[0019] Contour extraction is performed on the fruits and vegetables in the preliminary image to obtain the fruit and vegetable contours. Then, the fruit and vegetable images within the closed area enclosed by the fruit and vegetable contours are extracted and named as single fruit images.

[0020] Image features are extracted from single fruit images to obtain fruit and vegetable image features.

[0021] Further, the contours of the fruits and vegetables in the preliminary image are extracted to obtain the fruit and vegetable contours. Then, the fruit and vegetable images within the closed area enclosed by the fruit and vegetable contours are extracted and named as single fruit images. This includes the following sub-steps:

[0022] The contours of fruits and vegetables are obtained by extracting contours from the preliminary image using OpenCV contour extraction technology.

[0023] Name the pixels on the fruit and vegetable outline as outline points. For any fruit and vegetable outline, mark it as the target outline. Obtain the outline points of the target outline and name them as target range points.

[0024] Extract the pixels within the closed area enclosed by the target range points, excluding the target range points, to obtain a single-fruit image.

[0025] Furthermore, image feature extraction is performed on single fruit images to obtain fruit and vegetable image features, including the following sub-steps:

[0026] Edit the serial number of each pixel in a single fruit image and name it as pixel number. The pixel number is represented by the symbol P(n,m), where n and m are both positive integers and (n,m) is the serial number of P. P(n,m) represents the pixel in the nth row and mth column.

[0027] Extract the grayscale value of each P(n,m) and label it as h(n,m). Then remove the h(n,m) with a value of 255 to obtain H(n,m).

[0028] The pixels in a single fruit image are named single fruit points. For any P(n,m), the number of single fruit points in its eight neighborhoods is obtained and named the neighbor number. P(n,m) with the same number of neighbor points are grouped into the same group and named the same neighbor group.

[0029] For any group of adjacent quantities, mark it as the group to be analyzed; for any single result point in the group to be analyzed, mark it as the point to be analyzed.

[0030] Mark the points P(n,m) adjacent to the point to be analyzed as neighboring points, and number the H(n,m) of the neighboring points using the symbol T. i Let F be the index of T, where i is a positive integer. The H(n,m) of the points to be analyzed is labeled as F, and the result is obtained using the formula R. i =|FT i Calculate F and T i The absolute value of the difference, where R i For F and T i The absolute value of the difference;

[0031] Arrange R in ascending order i Number them using the symbol E i This means that, with i as the X-axis, E i Establish a Cartesian coordinate system for the Y-axis, name it pixel feature map, and set E i Enter the pixel feature map according to i;

[0032] Linear regression analysis is performed on the pixel feature map to obtain the slope of the regression line, which is named the gray value change amplitude of neighboring points. Each single fruit point in the same neighbor group is analyzed, and each same neighbor group is analyzed to obtain the gray value change amplitude of neighboring points of all single fruit points.

[0033] A Cartesian coordinate system is established with H(n,m) of P(n,m) as the horizontal axis and the gray level change range of the neighboring points corresponding to P(n,m) as the vertical axis. This system is named the single fruit feature map. The gray level change range of P(n,m) and its corresponding neighboring points is recorded in the single fruit feature map. Each group of neighboring points has an independent single fruit feature map.

[0034] Clustering analysis is performed on the single fruit feature map using a clustering algorithm to obtain different feature cluster sets. The range of the feature cluster set on the horizontal axis is marked as the cluster range. Feature cluster sets with overlapping cluster ranges are merged to obtain a merged cluster set. If a feature cluster set is not merged, it is marked as a merged cluster set.

[0035] Obtain the range of any merged cluster set on the vertical axis and mark it as the feature range. Find the maximum value of the feature range and mark it as the fruit and vegetable image feature.

[0036] Furthermore, a fruit and vegetable anomaly recognition model is constructed, screening normal and abnormal fruits and vegetables, extracting image features of normal and abnormal fruits and vegetables to obtain normal and abnormal fruit and vegetable features, and then training the fruit and vegetable anomaly recognition model based on the normal and abnormal fruit and vegetable features, including the following sub-steps:

[0037] A fruit and vegetable anomaly identification model was constructed to screen normal and abnormal fruits and vegetables and extract their fruit and vegetable image features to obtain normal and abnormal fruit and vegetable features.

[0038] The abnormal fruit and vegetable identification model is trained based on the extracted normal and abnormal fruit and vegetable features.

[0039] Furthermore, a fruit and vegetable anomaly identification model is constructed, and normal and abnormal fruits and vegetables are screened and their image features are extracted to obtain normal and abnormal fruit and vegetable features, including the following sub-steps:

[0040] Construct a model for identifying anomalies in fruits and vegetables;

[0041] Normal fruits and vegetables are screened from abnormal fruits and vegetables, and both normal and abnormal fruits and vegetables are screened from a first sample size.

[0042] Extract the image features of normal and abnormal fruits and vegetables to obtain the normal features of a first sample number of fruits and vegetables and the abnormal features of a first sample number of fruits and vegetables.

[0043] Furthermore, training the fruit and vegetable anomaly recognition model based on the extracted normal and abnormal features of fruits and vegetables includes the following sub-steps:

[0044] Find the maximum value among the normal characteristics of fruits and vegetables and mark it as the maximum normal characteristic of fruits and vegetables; find the minimum value among the abnormal characteristics of fruits and vegetables and mark it as the minimum abnormal characteristic of fruits and vegetables.

[0045] The maximum normal feature of fruits and vegetables is compared with the minimum abnormal feature of fruits and vegetables. If the maximum normal feature of fruits and vegetables is less than the minimum abnormal feature of fruits and vegetables, the perfect training signal is output; otherwise, the accuracy test signal is output.

[0046] If a perfect training signal is output, the recognition accuracy of the fruit and vegetable anomaly recognition model is marked as 100%, and the median of the maximum normal feature of fruits and vegetables and the minimum abnormal feature of fruits and vegetables is marked as the normal threshold of fruit and vegetable features.

[0047] If the accuracy test signal is output, the number of abnormal fruit and vegetable features that are less than or equal to the maximum normal feature of fruit and vegetables is obtained and marked as A. The number of the first sample is marked as B. The recognition accuracy of the fruit and vegetable abnormality recognition model is calculated by A / B. At the same time, the minimum abnormal fruit and vegetable feature is marked as the normal threshold of fruit and vegetable features.

[0048] Furthermore, after training, the identification of abnormal fruits and vegetables in the fruit and vegetable production line through the fruit and vegetable anomaly recognition model includes the following sub-steps:

[0049] The abnormal fruit and vegetable identification model is used to identify the fruits and vegetables in the fruit and vegetable production line. If the extracted fruit and vegetable image features are greater than the normal threshold of fruit and vegetable features, the corresponding fruits and vegetables are marked as abnormal fruits and vegetables.

[0050] If abnormal fruits and vegetables are detected, they will be picked up by an abnormality handling container when they reach the end of the conveyor belt.

[0051] The beneficial effects of this invention are as follows: This invention uses video monitoring of the fruit and vegetable production line to extract fruit and vegetable images. These images are then preprocessed to obtain preliminary images. The contours of the fruits and vegetables in the preliminary images are then extracted to obtain fruit and vegetable contours. Images of the fruits and vegetables within the closed area enclosed by these contours are then extracted and named as single-fruit images. Image features are then extracted from the single-fruit images to obtain the grayscale variation of neighboring points for each fruit and vegetable. Cluster analysis is then used to obtain the fruit and vegetable image features. The advantage is that for fruits and vegetables of the same type, the color distribution and color variation on their surfaces are relatively similar. However, when fruits and vegetables have defects, color areas different from the normal colors will appear on the surface. Therefore, by analyzing the grayscale variation of neighboring points for each fruit and vegetable, it is possible to preliminarily determine whether the fruit and vegetable have defects. Then, cluster analysis is used to determine the characteristics of the grayscale variation of neighboring points, i.e., the fruit and vegetable image features, thus improving the accuracy and rationality of fruit and vegetable impurity removal identification.

[0052] This invention constructs a fruit and vegetable anomaly recognition model, filters normal and abnormal fruits and vegetables, and extracts their image features to obtain normal and abnormal fruit and vegetable features. Based on the extracted normal and abnormal fruit and vegetable features, the fruit and vegetable anomaly recognition model is trained. After training, the model is used to identify abnormal fruits and vegetables in the fruit and vegetable production line. The advantage is that during training, the fruit and vegetable anomaly recognition model is trained only using normal fruit and vegetable features. The abnormal fruit and vegetable features are only used to confirm the accuracy of the model. To save training costs, abnormal fruits and vegetables can also be omitted for verification, which improves the effectiveness of fruit and vegetable impurity removal recognition and reduces the cost of model training. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the steps of the method of the present invention;

[0054] Figure 2 This is a schematic diagram of fruit and vegetable images according to the present invention;

[0055] Figure 3 This is a schematic diagram of a single fruit image according to the present invention;

[0056] Figure 4 This is a schematic diagram of the pixel feature map of the present invention;

[0057] Figure 5 This is a schematic diagram of the single-fruit feature diagram of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Example 1, please refer to Figure 1 As shown, this application provides an image recognition method for an integrated fruit and vegetable production cleaning machine based on video surveillance, including the following steps:

[0060] Step S1 involves video monitoring of the fruit and vegetable production line for impurity removal and extracting images of the fruits and vegetables. Step S1 includes the following sub-steps:

[0061] Please see Figure 2 As shown, in step S101, a high-resolution industrial camera is installed inside the fruit and vegetable production impurity removal machine to take pictures of the fruits and vegetables on the fruit and vegetable production impurity removal line.

[0062] Step S102: Name the captured images of the fruit and vegetable production line for removing impurities as fruit and vegetable images;

[0063] In practice, the captured images of fruits and vegetables are as follows: Figure 2 As shown, Figure 2 The middle view is the front view of the fruit and vegetable production line for removing impurities. In addition, there are left and right views, which are used to analyze whether there are defects on the sides of the fruits and vegetables. The analysis process is the same as that of the front view. Therefore, this embodiment only lists the analysis process of the front view.

[0064] Step S2 involves preprocessing the fruit and vegetable images to obtain preliminary images. Step S2 includes the following sub-steps:

[0065] Step S201: Obtain an image of the fruit and vegetable production line when it is not running, and name it the production line image. The production line image does not contain any fruits or vegetables.

[0066] Step S202: Compare the production line image with the fruit and vegetable image, remove pixels with the same color value, and name the remaining fruit and vegetable image after removal as a background-free image.

[0067] Step S203: Perform grayscale processing on the backgroundless image to obtain a preliminary image;

[0068] In practice, when taking images of fruits and vegetables and the production line, the fruit and vegetable production impurity removal line is illuminated. The illumination must ensure that there are no shadows in the fruit and vegetable images and the production line images, that is, the fruit and vegetable production impurity removal line is provided with 360° lighting. Then, a background-free image is obtained by comparison. The background-free image only contains fruits and vegetables, and the non-fruit and vegetable parts are transparent. Then, a preliminary image is obtained through grayscale processing. It should be noted that the fruits and vegetables have already undergone a cleaning step before the fruit and vegetable images are taken. Therefore, the surface of the fruits and vegetables only shows the color of the fruits and vegetables themselves, without any other impurities obscuring them. The impurity removal identification in this embodiment is to identify fruits and vegetables with defects.

[0069] Step S3 involves extracting the contours of the fruits and vegetables in the initial image. After extracting the contours, image features are extracted from the fruit and vegetable images within the contour range to obtain the fruit and vegetable image features. Step S3 includes the following sub-steps:

[0070] Step S301: Extract the contours of the fruits and vegetables in the preliminary image to obtain the fruit and vegetable contours, and then extract the fruit and vegetable images within the closed area enclosed by the fruit and vegetable contours, and name them as single fruit images.

[0071] Step S301 includes the following sub-steps:

[0072] Step S301.1: Extract contours from the preliminary image using OpenCV contour extraction technology to obtain the contours of fruits and vegetables;

[0073] Step S301.2: Name the pixels on the fruit and vegetable outline as outline points. For any fruit and vegetable outline, mark it as the target outline, obtain the outline points of the target outline, and name them as target range points.

[0074] Please see Figure 3 As shown, in step S301.3, the pixels within the closed area enclosed by the target range points are extracted, excluding the target range points, to obtain a single-fruit image;

[0075] In practice, existing OpenCV contour extraction technology is used to extract contours from the initial image. The resulting single-fruit image based on the fruit and vegetable contour extraction is shown below. Figure 3 As shown;

[0076] Step S302: Extract image features from single fruit images to obtain fruit and vegetable image features;

[0077] Step S302 includes the following sub-steps:

[0078] Step S302.1: Edit the sequence number of the pixels in the single fruit image and name it as pixel number. The pixel number is represented by the symbol P(n,m), where n and m are both positive integers and (n,m) is the sequence number of P. P(n,m) represents the pixel in the nth row and mth column.

[0079] Step S302.2: Extract the grayscale value of each P(n,m) and label it as h(n,m). Then remove the h(n,m) with a value of 255 to obtain H(n,m).

[0080] Step S302.3: Name the pixels in the single fruit image as single fruit points. For any P(n,m), obtain the number of single fruit points in its eight neighborhoods and name them as the number of neighboring points. Group P(n,m) with the same number of neighboring points into the same group and name them as the same neighbor group.

[0081] In practice, the numbering of P(n,m) includes transparent pixels. P(n,m) is obtained by editing the sequence number, where 1≤n≤41 and 1≤m≤30. Then, P(n,m) with a grayscale value of 255 is removed. The grayscale value of the remaining P(n,m) is H(n,m). Taking H(7,8) as an example, its eight neighboring H(n,m) are H(7,7), H(8,7), H(6,8), and H(8,8). H(6,9), H(7,9) and H(8,9) do not have a P(n,m) with h(n,m) of 255, so the number of neighboring points is 8. P(n,m) with the same number of neighboring points are grouped into the same group of same neighboring points. Through analysis, 6 groups of same neighboring points are obtained, with the corresponding number of neighboring points being 3, 4, 5, 6, 7 and 8, respectively. In this embodiment, they are referred to as the 3-same-neighbor group, the 4-same-neighbor group, the 5-same-neighbor group, the 6-same-neighbor group, the 7-same-neighbor group and the 8-same-neighbor group, respectively.

[0082] Step S302.4: For any group of adjacent quantities, mark it as the group to be analyzed; for any single result point in the group to be analyzed, mark it as the point to be analyzed.

[0083] Step S302.5: Mark P(n,m) adjacent to the point to be analyzed as neighboring points, and number the H(n,m) of the neighboring points using the symbol T. i Let F be the index of T, where i is a positive integer. The H(n,m) of the points to be analyzed is labeled as F, and the result is obtained using the formula R. i =|FT i Calculate F and T i The absolute value of the difference, where R i For F and T i The absolute value of the difference;

[0084] Please see Figure 4As shown, in step S302.6, R is processed in ascending order. i Number them using the symbol E i This means that, with i as the X-axis, E i Establish a Cartesian coordinate system for the Y-axis, name it pixel feature map, and set E i Enter the pixel feature map according to i;

[0085] Step S302.7: Perform linear regression analysis on the pixel feature map, obtain the slope of the regression line, and name it the gray value change amplitude of neighboring points. Analyze each single fruit point in the same neighbor group, and analyze each same neighbor group to obtain the gray value change amplitude of neighboring points of all single fruit points.

[0086] In specific implementation, taking H(7,8) as an example, we obtain H(7,8) as 198, and H(7,7), H(8,7), H(6,8), H(8,8), H(6,9), H(7,9), and H(8,9) as 176, 192, 194, 190, 198, 195, 198, and 198 respectively, corresponding to T1 to T8. F is 198, and we calculate R1 to R8 as 22, 6, 4, 8, 0, 3, 0, and 0 respectively. We sort E1 to E8 as 0, 0, 0, 3, 4, 6, 8, and 22 respectively. Based on this, we construct the pixel feature map of P(n,m) as follows. Figure 4 As shown, the gray-scale change of neighboring points of P(n,m) is 2.5357 obtained through linear regression analysis. In this embodiment, it is not necessary to use the formula of the regression line for calculation. Therefore, this embodiment only gives the slope of the regression line without explaining the regression line. Each single result point in the same neighbor group is analyzed, that is, all single result points in the 3 same neighbor group, 4 same neighbor group, 5 same neighbor group, 6 same neighbor group, 7 same neighbor group and 8 same neighbor group are analyzed. The gray-scale change of neighboring points obtained in each different same neighbor group are independent of each other.

[0087] Please see Figure 5 As shown in step S302.8, a Cartesian coordinate system is established with H(n,m) of P(n,m) as the horizontal axis and the gray level change range of the neighboring points corresponding to P(n,m) as the vertical axis. This system is named the single fruit feature map. The gray level change range of P(n,m) and its corresponding neighboring points is recorded in the single fruit feature map. Each group of neighboring points has an independent single fruit feature map.

[0088] Step S302.9: Perform cluster analysis on the single fruit feature map using a clustering algorithm to obtain different feature cluster sets. Mark the range of the feature cluster set on the horizontal axis as the cluster range. Merge the feature cluster sets whose cluster ranges have intersections to obtain a merged cluster set. If the feature cluster set is not merged, mark it as the merged cluster set.

[0089] Step S302.10: Obtain the range of any merged cluster set on the vertical axis and mark it as the feature range; find the maximum value of the feature range and mark it as the fruit and vegetable image feature.

[0090] In specific implementation, taking the 8 neighboring groups as an example, the single-fruit feature map is constructed as follows: Figure 5 As shown, Figure 5 The coordinates within a rectangular area represent a feature cluster set. The numbers within the rectangle indicate the cluster numbers. There are eight feature cluster sets in total. Cluster sets 1 and 6 do not intersect with any other cluster set on the horizontal axis; therefore, cluster sets 1 and 7 each form a merged cluster set. Cluster sets 2 and 3 intersect on the horizontal axis, and cluster sets 4 and 5 both intersect with cluster set 3 on the horizontal axis. Therefore, cluster sets 2, 3, 4, and 5 are merged into a single merged cluster set. Similarly, cluster sets 7 and... The 8 feature clusters are merged into a single merged cluster set. Taking the merged cluster set obtained by merging feature clusters 2, 3, 4, and 5 as an example, the minimum and maximum values ​​of the vertical axis in the coordinate points are 1.1579 and 3.0683, respectively. Therefore, its feature range is [1.1579, 3.0683]. The size of the feature range is the difference between the maximum and minimum values. Obtaining [1.1579, 3.0683] is the maximum value of the feature range. Therefore, 3.0683 - 1.1579 = 1.9104 is marked as the fruit and vegetable image feature of the 8 neighboring groups. Each different neighboring group has independent fruit and vegetable image features.

[0091] Step S4 involves constructing a fruit and vegetable anomaly recognition model, filtering normal and abnormal fruits and vegetables, extracting image features from both types of fruits and vegetables to obtain normal and abnormal features, and then training the fruit and vegetable anomaly recognition model based on these features. Step S4 includes the following sub-steps:

[0092] Step S401: Construct a fruit and vegetable anomaly recognition model, filter normal and abnormal fruits and vegetables and extract their fruit and vegetable image features to obtain normal fruit and vegetable features and abnormal fruit and vegetable features.

[0093] Step S401 includes the following sub-steps:

[0094] Step S401.1: Construct a fruit and vegetable anomaly identification model;

[0095] Step S401.2: Screen normal fruits and vegetables and abnormal fruits and vegetables. For both normal and abnormal fruits and vegetables, select the first sample size.

[0096] Step S401.3: Extract the fruit and vegetable image features of normal and abnormal fruits and vegetables to obtain the first sample number of normal fruit and vegetable features and the first sample number of abnormal fruit and vegetable features.

[0097] In specific implementation, the number of samples in the first sample of the fruit and vegetable anomaly identification model is not specifically set and can be set by the user. Whether to test the accuracy of the fruit and vegetable anomaly identification model with abnormal fruits and vegetables is also decided by the user. The example of testing the accuracy of the fruit and vegetable anomaly identification model with abnormal fruits and vegetables in this embodiment is only to provide a method for testing the accuracy of the fruit and vegetable anomaly identification model. If the user chooses to save costs, they can also choose not to perform the test and only select normal fruits and vegetables to analyze the normal threshold of fruit and vegetable features. If only normal fruits and vegetables are selected for analysis, the minimum abnormal feature of fruit and vegetables in step S402 is the normal threshold of fruit and vegetable features. In this embodiment, the number of samples in the first sample is set to 500, and abnormal fruits and vegetables are not selected for testing, that is, only 500 normal fruit and vegetable features are extracted, and each normal fruit and vegetable feature is further divided into different neighboring groups of fruit and vegetable image features.

[0098] Step S402: Train the fruit and vegetable abnormality recognition model based on the extracted normal and abnormal features of fruits and vegetables;

[0099] Step S402 includes the following sub-steps:

[0100] Step S402.1: Find the maximum value among the normal characteristics of fruits and vegetables and mark it as the maximum normal characteristic of fruits and vegetables; find the minimum value among the abnormal characteristics of fruits and vegetables and mark it as the minimum abnormal characteristic of fruits and vegetables.

[0101] Step S402.2: Compare the maximum normal feature of fruits and vegetables with the minimum abnormal feature of fruits and vegetables. If the maximum normal feature of fruits and vegetables is less than the minimum abnormal feature of fruits and vegetables, output the perfect training signal; otherwise, output the accuracy test signal.

[0102] Step S402.3: If a perfect training signal is output, the recognition accuracy of the fruit and vegetable anomaly recognition model is marked as 100%, and the median of the maximum normal feature of fruit and vegetables and the minimum abnormal feature of fruit and vegetables is marked as the normal threshold of fruit and vegetable features.

[0103] Step S402.4: If the accuracy test signal is output, the number of abnormal fruit and vegetable features that are less than or equal to the maximum normal feature of fruit and vegetables is obtained and marked as A. The number of the first sample is marked as B. The recognition accuracy of the fruit and vegetable abnormality recognition model is calculated by A / B. At the same time, the minimum abnormal fruit and vegetable feature is marked as the normal threshold of fruit and vegetable features.

[0104] In practice, each neighboring data group has a normal threshold for fruit and vegetable features. In this embodiment, only normal fruits and vegetables are selected for analysis. When analyzing abnormal fruits and vegetables, the accuracy test process has been given in detail and will not be described in detail in this embodiment. When analyzing only normal fruits and vegetables, for each neighboring data group, the maximum value of the fruit and vegetable image features in the same neighboring data group is selected as the normal threshold for fruit and vegetable features.

[0105] Step S5: After training, the abnormal fruits and vegetables in the fruit and vegetable production line are identified using the fruit and vegetable anomaly recognition model. Step S5 includes the following sub-steps:

[0106] Step S501: The fruit and vegetable in the fruit and vegetable production line is identified by the fruit and vegetable abnormality identification model. If the extracted fruit and vegetable image features are greater than the normal threshold of fruit and vegetable features, the corresponding fruit and vegetable is marked as abnormal fruit and vegetable.

[0107] Step S502: If abnormal fruits and vegetables are detected, the abnormal fruits and vegetables are intercepted by the abnormal handling container when they reach the end of the conveyor belt.

[0108] In practice, the normal threshold for fruit and vegetable features represents the maximum value of color variation on the surface of normal fruits and vegetables. If the fruit and vegetable image features are larger than the normal threshold, it means that the color variation on the surface of the fruit and vegetable exceeds that of normal fruits and vegetables, which is generally caused by defects in the fruit and vegetable. Each neighboring quantity group has a normal threshold for fruit and vegetable features. Different neighboring quantity groups also exist when extracting fruit and vegetable image features. The same neighboring quantity groups are compared. If the fruit and vegetable image features of any neighboring quantity group are greater than the corresponding normal threshold, the fruit and vegetable is marked as abnormal. When processing abnormal fruits and vegetables, if an abnormal fruit and vegetable is identified, it is marked on the monitoring screen. When it moves to the end of the conveyor belt, the fruits and vegetables on the other conveyor belt fall freely to the sorting port of the next process. A container will extend from under the conveyor belt where the abnormal fruit and vegetable is located to pick up the abnormal fruit and vegetable. After the abnormal fruit and vegetable falls into the container, the container will be retracted.

[0109] Example 2: This application provides an electronic device, which may include a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The memory stores computer-readable instructions, and the processor can call these instructions. When the processor executes a computer-readable instruction, it performs steps such as those in the image recognition method for a fruit and vegetable production cleaning machine based on video surveillance, to achieve the following functions: extracting fruit and vegetable images; performing image preprocessing on the fruit and vegetable images to obtain preliminary images; extracting contours of the fruits and vegetables in the preliminary images; extracting image features from the fruit and vegetable images within the contour range to obtain fruit and vegetable image features; screening normal and abnormal fruits and vegetables; extracting normal and abnormal features of the fruits and vegetables; training a fruit and vegetable anomaly recognition model; and, after training, identifying abnormal fruits and vegetables in the fruit and vegetable production cleaning line using the fruit and vegetable anomaly recognition model.

[0110] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] Example 3: This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the steps of the image recognition method for the integrated fruit and vegetable production cleaning machine based on video surveillance, as described above, to achieve the following functions: extracting fruit and vegetable images; performing image preprocessing on the fruit and vegetable images to obtain preliminary images; extracting contours of the fruits and vegetables in the preliminary images; extracting image features of the fruit and vegetable images within the contour range to obtain fruit and vegetable image features; screening normal and abnormal fruits and vegetables; extracting normal and abnormal features of fruits and vegetables; training a fruit and vegetable abnormality recognition model; and identifying abnormal fruits and vegetables in the fruit and vegetable production cleaning line through the fruit and vegetable abnormality recognition model after training.

[0112] Based on the above description of the embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the technical solutions described above, or the parts that contribute to the prior art, can be embodied in the form of software products. These computer software products can be stored in computer-readable storage media, such as ROM / RAM, magnetic disks, optical disks, etc., and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments.

[0113] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces. The indirect coupling or communication connection between systems, modules, and units may be electrical, mechanical, or other forms.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image recognition method for an integrated fruit and vegetable production impurity removal machine based on video surveillance, characterized in that, Includes the following steps: Video monitoring was conducted on the fruit and vegetable production line for removing impurities to extract images of the fruits and vegetables. Image preprocessing is performed on fruit and vegetable images to obtain preliminary images; Contour extraction is performed on the fruits and vegetables in the preliminary image. After the fruit and vegetable contours are extracted, image features are extracted from the fruit and vegetable images within the contour range to obtain the fruit and vegetable image features. A fruit and vegetable anomaly identification model is constructed. Normal and abnormal fruits and vegetables are screened, and the image features of normal and abnormal fruits and vegetables are extracted to obtain normal and abnormal fruit and vegetable features. The fruit and vegetable anomaly identification model is then trained based on the normal and abnormal fruit and vegetable features. After training, the abnormal fruits and vegetables in the fruit and vegetable production line are identified using the fruit and vegetable anomaly recognition model. The process involves extracting the contours of fruits and vegetables from the initial image, and then extracting image features from the fruit and vegetable images within the contour range. This process includes the following sub-steps: Contour extraction is performed on the fruits and vegetables in the preliminary image to obtain the fruit and vegetable contours. Then, the fruit and vegetable images within the closed area enclosed by the fruit and vegetable contours are extracted and named as single fruit images. Image features are extracted from single fruit images to obtain fruit and vegetable image features; The process of extracting the contours of fruits and vegetables from the initial image, and then extracting the fruit and vegetable images within the closed area enclosed by the contours, naming them as single-fruit images, includes the following sub-steps: The contours of fruits and vegetables are obtained by extracting contours from the preliminary image using OpenCV contour extraction technology. Name the pixels on the fruit and vegetable outline as outline points. For any fruit and vegetable outline, mark it as the target outline. Obtain the outline points of the target outline and name them as target range points. Extract the pixels within the closed area enclosed by the target range points, excluding the target range points, to obtain a single fruit image; Extracting image features from single fruit images to obtain fruit and vegetable image features includes the following sub-steps: Edit the serial number of each pixel in a single fruit image and name it as pixel number. The pixel number is represented by the symbol P(n,m), where n and m are both positive integers and (n,m) is the serial number of P. P(n,m) represents the pixel in the nth row and mth column. Extract the grayscale value of each P(n,m) and label it as h(n,m). Then remove the h(n,m) with a value of 255 to obtain H(n,m). The pixels in a single fruit image are named single fruit points. For any P(n,m), the number of single fruit points in its eight neighborhoods is obtained and named the neighbor number. P(n,m) with the same number of neighbor points are grouped into the same group and named the same neighbor group. For any group of adjacent quantities, mark it as the group to be analyzed; for any single result point in the group to be analyzed, mark it as the point to be analyzed. Mark the points P(n,m) adjacent to the point to be analyzed as neighboring points, and number the H(n,m) of the neighboring points using the symbol T. i Let F be the index of T, where i is a positive integer. The H(n,m) of the points to be analyzed is labeled as F, and the result is obtained using the formula R. i =|FT i Calculate F and T i The absolute value of the difference, where R i For F and T i The absolute value of the difference; Arrange R in ascending order i Number them using the symbol E i This means that, with i as the X-axis, E i Establish a Cartesian coordinate system for the Y-axis, name it pixel feature map, and set E i Enter the pixel feature map according to i; Linear regression analysis is performed on the pixel feature map to obtain the slope of the regression line, which is named the gray value change amplitude of neighboring points. Each single fruit point in the same neighbor group is analyzed, and each same neighbor group is analyzed to obtain the gray value change amplitude of neighboring points of all single fruit points. A Cartesian coordinate system is established with H(n,m) of P(n,m) as the horizontal axis and the gray level change range of the neighboring points corresponding to P(n,m) as the vertical axis. This system is named the single fruit feature map. The gray level change range of P(n,m) and its corresponding neighboring points is recorded in the single fruit feature map. Each group of neighboring points has an independent single fruit feature map. Clustering analysis is performed on the single fruit feature map using a clustering algorithm to obtain different feature cluster sets. The range of the feature cluster set on the horizontal axis is marked as the cluster range. Feature cluster sets with overlapping cluster ranges are merged to obtain a merged cluster set. If a feature cluster set is not merged, it is marked as a merged cluster set. Obtain the range of any merged cluster set on the vertical axis and mark it as the feature range. Find the maximum value of the feature range and mark it as the fruit and vegetable image feature.

2. The image recognition method for the integrated fruit and vegetable production impurity removal machine based on video monitoring according to claim 1, characterized in that, Video monitoring of the fruit and vegetable production line for impurity removal and image extraction includes the following sub-steps: The integrated fruit and vegetable production cleaning machine is equipped with a high-resolution industrial camera for photographing the fruits and vegetables on the production line. The images of the fruit and vegetable production line that were captured were named "Fruit and Vegetable Images".

3. The image recognition method for the integrated fruit and vegetable production impurity removal machine based on video monitoring according to claim 2, characterized in that, Image preprocessing of fruit and vegetable images, resulting in preliminary images, includes the following sub-steps: Obtain an image of the fruit and vegetable production line when it is not running, and name it the production line image. The production line image does not contain any fruits or vegetables. Compare the production line image with the fruit and vegetable image, remove pixels with the same color value, and name the remaining fruit and vegetable image after removal as "no background image". The image without background is converted to grayscale to obtain a preliminary image.

4. The image recognition method for the integrated fruit and vegetable production impurity removal machine based on video monitoring according to claim 3, characterized in that, The process of constructing a fruit and vegetable anomaly recognition model involves screening normal and abnormal fruits and vegetables, extracting image features from both types of fruits and vegetables to obtain normal and abnormal features, and then training the fruit and vegetable anomaly recognition model based on these features. The process includes the following sub-steps: A fruit and vegetable anomaly identification model was constructed to screen normal and abnormal fruits and vegetables and extract their fruit and vegetable image features to obtain normal and abnormal fruit and vegetable features. The abnormal fruit and vegetable identification model is trained based on the extracted normal and abnormal fruit and vegetable features.

5. The image recognition method for the integrated fruit and vegetable production impurity removal machine based on video monitoring according to claim 4, characterized in that, Constructing a fruit and vegetable anomaly identification model involves screening normal and abnormal fruits and vegetables and extracting their image features to obtain normal and abnormal fruit and vegetable features. This process includes the following sub-steps: Construct a model for identifying anomalies in fruits and vegetables; Normal fruits and vegetables are screened from abnormal fruits and vegetables, and both normal and abnormal fruits and vegetables are screened from a first sample size. Extract the image features of normal and abnormal fruits and vegetables to obtain the normal features of a first sample number of fruits and vegetables and the abnormal features of a first sample number of fruits and vegetables.

6. The image recognition method for the integrated fruit and vegetable production impurity removal machine based on video monitoring according to claim 5, characterized in that, Training a fruit and vegetable anomaly recognition model based on the extracted normal and abnormal features of fruits and vegetables includes the following sub-steps: Find the maximum value among the normal characteristics of fruits and vegetables and mark it as the maximum normal characteristic of fruits and vegetables; find the minimum value among the abnormal characteristics of fruits and vegetables and mark it as the minimum abnormal characteristic of fruits and vegetables. The maximum normal feature of fruits and vegetables is compared with the minimum abnormal feature of fruits and vegetables. If the maximum normal feature of fruits and vegetables is less than the minimum abnormal feature of fruits and vegetables, the perfect training signal is output; otherwise, the accuracy test signal is output. If a perfect training signal is output, the recognition accuracy of the fruit and vegetable anomaly recognition model is marked as 100%, and the median of the maximum normal feature of fruits and vegetables and the minimum abnormal feature of fruits and vegetables is marked as the normal threshold of fruit and vegetable features. If the accuracy test signal is output, the number of abnormal fruit and vegetable features that are less than or equal to the maximum normal feature of fruit and vegetables is obtained and marked as A. The number of the first sample is marked as B. The recognition accuracy of the fruit and vegetable abnormality recognition model is calculated by A / B. At the same time, the minimum abnormal fruit and vegetable feature is marked as the normal threshold of fruit and vegetable features.

7. The image recognition method for a fruit and vegetable production impurity removal integrated machine based on video monitoring according to claim 6, characterized in that, After training, the abnormal fruit and vegetable identification model identifies abnormal fruits and vegetables in the fruit and vegetable production line, including the following sub-steps: The abnormal fruit and vegetable identification model is used to identify the fruits and vegetables in the fruit and vegetable production line. If the extracted fruit and vegetable image features are greater than the normal threshold of fruit and vegetable features, the corresponding fruits and vegetables are marked as abnormal fruits and vegetables. If abnormal fruits and vegetables are detected, they will be picked up by an abnormality handling container when they reach the end of the conveyor belt.

Images