A method for detecting eggshells in egg liquid

Abstract

The present invention belongs to the technical field of image processing, and specifically relates to a method for detecting eggshells in egg liquid. The method includes: obtaining the light area of the real-time egg liquid flow video according to the gray values and gray changes of the pixel points at the same position in different video frames of the real-time egg liquid flow video; obtaining the eggshell performance of each pixel point in each video frame according to the change of the optical flow field between adjacent video frames and the gray distribution of the pixel points; obtaining the suspected eggshell area of each video frame based on the eggshell performance of each pixel point in each video frame and the distance between each pixel point and the light area of the real-time egg liquid flow video; determining the video frame sequence in which the real eggshell exists according to the shape change and position change of the suspected eggshell area in consecutive video frames, and predicting the movement route of the eggshell to complete the fishing of the eggshell. The present invention improves the accuracy of eggshell detection in egg liquid and the timeliness of eggshell fishing.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing. More specifically, the present invention relates to a method for detecting eggshells in egg liquid. Background Art

[0002] Due to its rich nutritional value, eggs are one of the indispensable foods in residents' daily lives, and are thus also applied in food processing industries such as baked foods and meat products. However, due to their fragility, the egg liquid product industry has gradually developed. Egg liquid can better preserve the nutrition of eggs and is convenient for transportation and storage, and can efficiently provide egg raw materials for the food processing industry. The production process of egg liquid includes washing eggs, separating egg liquid and eggshells, separating egg white liquid and egg yolk liquid, removing broken eggshells, sterilizing, and filling and refrigerating. During the process of removing broken eggshells, sedimentation and filtration steps are often included. However, the sizes of broken eggshells are different, and there are also relatively large germ discs, vitelline membranes, chalazae, etc. in the egg white liquid and egg yolk liquid, resulting in difficulty in completely removing broken eggshells by sedimentation and filtration, so a more refined detection method is needed.

[0003] In order to detect whether there is invalid waste in effective food during the food processing process, machine vision technology and neural network models are currently often used to segment and classify effective food and invalid waste. In related technologies, for example, a method and system for online detection of buckwheat shelling parameters for a buckwheat shelling machine disclosed in a Chinese patent document with the authorization announcement number CN116703829B discloses classifying and segmenting unhulled grains, shelled buckwheat, and broken rice in an image through a neural network. A method for pixel-level detection of endogenous foreign objects in pecans based on hyperspectral and deep learning disclosed in a Chinese patent document with the authorization announcement number CN112132792B discloses establishing a classification model for pecan outer meat, inner meat, outer shell, and inner shell based on a neural network, so as to realize pixel-level detection of endogenous pecan shells in pecans.

[0004] However, the environment during the egg liquid processing process is complex. For example, the lighting will affect the presentation of the eggshell in the egg liquid image and affect the detection rate of the eggshell. For the sake of process efficiency, the egg liquid will also flow in real time to enter the subsequent process. The egg liquid structure and eggshell structure of the flowing egg liquid are not fixed, which will affect the accuracy of eggshell detection. If the egg liquid is detected statically, it will not only affect the production efficiency, but also miss detections due to the sinking of the eggshell. Summary of the Invention

[0005] To solve the technical problem of insufficient accuracy in detecting broken eggshells in egg liquid during the above-mentioned egg liquid production process, the present invention provides a method for detecting eggshells in egg liquid, including:

[0006] A real-time egg liquid flow video is obtained through a machine vision system, and the real-time egg liquid flow video includes several video frames; a light area of ​​the real-time egg liquid flow video is obtained according to the grayscale value and grayscale change of the pixel points at the same position in different video frames in the real-time egg liquid flow video; an eggshell performance of each pixel point in each video frame is obtained according to the change of the optical flow field of adjacent video frames and the grayscale distribution of the pixel points; a suspected eggshell area of ​​each video frame is obtained based on the eggshell performance of each pixel point in each video frame and the distance between each pixel point and the light area of ​​the real-time egg liquid flow video; a video frame sequence where a real eggshell exists is determined according to the shape change and position change of the suspected eggshell area of ​​consecutive video frames; a movement path of the eggshell is predicted according to the position change of the eggshell in the video frame sequence where the real eggshell exists; and the eggshell is scooped out based on the movement path of the eggshell.

[0007] The present invention obtains the light zone in the real-time egg liquid flow video in advance, thereby avoiding the influence of light reflection on eggshell detection in the egg liquid production environment. The present invention detects eggshells based on the differences in the changes in shape, position, etc. of eggshells and egg liquid during flow, and places the detection of eggshells in dynamic videos instead of simple static detection, thereby improving the accuracy of eggshell detection, extracting more abundant features in the movement of eggshells in egg liquid, and ensuring the detection rate of eggshells.

[0008] Preferably, the method of obtaining the light zone of the real-time egg liquid flow video comprises: establishing a three-dimensional coordinate system for the real-time egg liquid flow video, wherein the XOY plane in the three-dimensional coordinate system is parallel to the video frame plane, and the Z axis is the time axis; recording the pixels with the same X-axis and Y-axis coordinates as the same group of pixels; obtaining the light zone performance of each group of pixels according to the distribution of the number of pixels of each gray value of each group of pixels; clustering the light zone performance of all groups of the same group of pixels using a clustering algorithm to obtain a plurality of clusters of the same group of pixels, and recording the same group of pixel clusters whose mean value of the light zone performance is greater than the mean value of the light zone performance of all groups of the same group of pixels as the light zone of the real-time egg liquid flow video.

[0009] The present invention marks the light area in the real-time egg liquid flow video, so that the detection of the eggshell area avoids the interference of light reflection, and the detection of the eggshell is concentrated in the normal brightness area, thereby improving the accuracy of eggshell detection.

[0010] Preferably, the light area representation of the same group of pixels includes:

[0011] Obtain the grayscale values ​​of all pixels in the same group of pixels and count the number of pixels corresponding to each grayscale value; obtain the grayscale mean value of each pixel in the same group;

[0012] The light area performance of any group of pixels in the same group satisfies the expression:

[0013] ;

[0014] In the formula, represents the light area performance of the pixel points in the i-th group of the same group; S represents the maximum gray value; s represents the gray value; represents the number of pixel points with a gray value of s in the i-th group of the same group of pixel points; represents the average gray value of all pixel points in the same group.

[0015] Preferably, obtaining the eggshell performance of each pixel point in each video frame according to the optical flow field change of adjacent video frames and the gray distribution of pixel points includes: obtaining the flow regularity of any pixel point in continuously adjacent video frames according to the change of the optical flow vector of the pixel point in continuously adjacent video frames; obtaining the eggshell gray index of any pixel point in continuously adjacent video frames based on the number distribution of pixel points in the gray histogram and the gray change of pixel points; taking the ratio of the eggshell gray index to the flow regularity as the eggshell performance of the pixel point in continuously adjacent video frames.

[0016] The present invention combines the gray difference between the eggshell and the egg liquid and the morphological change difference in the flow process to obtain the eggshell performance of the pixel point, and accurately and comprehensively screens the pixel points belonging to the eggshell. And it analyzes the gray change and morphological change of the pixel point in adjacent video frames, avoiding the noise interference caused by only analyzing a single video frame.

[0017] Preferably, the flow regularity of any pixel point in continuously adjacent video frames satisfies the expression:

[0018] ;

[0019] In the formula, represents the flow regularity of the c-th pixel point in the z-th video frame in the T adjacent video frames on the right; T represents the number of preset adjacent video frames on the right; represents the difference in the moving distance along the X-axis of the c-th pixel point in the t-th and t + 1-th adjacent video frames on the right of the z-th video frame; represents the difference in the moving distance along the Y-axis of the c-th pixel point in the t-th and t + 1-th adjacent video frames on the right of the z-th video frame; represents the absolute value function; represents the exponential function with the natural constant as the base.

[0020] Preferably, the eggshell gray index of any pixel point in continuously adjacent video frames satisfies the expression:

[0021] ;

[0022] In the formula, represents the eggshell gray index of the c-th pixel point in the z-th video frame in the T adjacent video frames on the right; represents the c-th pixel point in the The number of pixel points of the grayscale value of the video frame in the corresponding grayscale histogram; Indicates the number of pixel points in the X-axis direction of the video frame, Indicates the number of pixel points in the Y-axis direction of the video frame, Indicates the number of pixel points of the video frame; T represents the number of preset adjacent video frames adjacent on the right; 、 Indicates the grayscale values of the c-th pixel point moving to the t-th and t + 1-th adjacent video frames adjacent on the right of the z-th video frame.

[0023] Preferably, obtaining the suspected eggshell area of each video frame based on the eggshell performance of each pixel point in each video frame and the distance between each pixel point and the light area of the real-time egg liquid flow video includes: clustering all pixel points of each video frame based on the distance and the eggshell performance to obtain several growth areas of each video frame; obtaining the minimum value of the distance between the centroid of each growth area and the centroid of all light areas of the real-time egg liquid flow video, and multiplying it by the number of pixel points in the corresponding growth area as the eggshell possibility of each growth area; recording the growth area with the eggshell possibility greater than the first threshold as the suspected eggshell area.

[0024] The present invention merges the pixel points belonging to a suspected eggshell area through clustering, so that the detected eggshell can be completely displayed, improving the possibility of successful fishing for the eggshell and avoiding the possibility of detection omission caused by incomplete eggshell detection.

[0025] Preferably, determining the video frame sequence where the real eggshell exists according to the shape change and position change of the suspected eggshell area of consecutive video frames includes: obtaining the pixel point set of each suspected eggshell area of each video frame, and obtaining the boundary pixel point sequence of each suspected eggshell area of each video frame; obtaining the possibility that any two suspected eggshell areas in any two adjacent video frames belong to the same eggshell based on the position relationship and shape relationship of the suspected eggshell areas in consecutive video frames; clustering the video frames belonging to the same eggshell through a clustering algorithm to obtain the video frame sequence where the suspected eggshell area exists, and recording the video frame sequence where the suspected eggshell area exists with the number of clustered video frames greater than the third threshold as the video frame sequence where the real eggshell exists.

[0026] The present invention merges all video frames where the same real eggshell exists, enabling the full-process analysis of the movement process of the real eggshell and improving the accuracy of predicting the position of the eggshell.

[0027] Preferably, the possibility that any two suspected eggshell areas in any two adjacent video frames belong to the same eggshell includes: obtaining the edit distance between the boundary pixel point sequence of the g-th suspected eggshell area of the z-th video frame and the boundary pixel point sequence of the g-th suspected eggshell area of the z + 1-th video frame, denoted as ; Compare the pixel intersection and union of the g-th suspected eggshell region in the z-th video frame with those in the g-th suspected eggshell region in the (z + 1)-th video frame, and multiply by the value obtained by negative correlation normalization with to obtain the possibility that the g-th suspected eggshell region in the z-th video frame and the g-th suspected eggshell region in the (z + 1)-th video frame belong to the same eggshell.

[0028] Preferably, predicting the movement route of the eggshell according to the position change of the eggshell in the video frame sequence where the real eggshell exists includes: obtaining the centroid coordinates of the eggshell region in each video frame in the video frame sequence where each real eggshell exists, and using the least squares method for spatial straight line fitting to obtain the centroid coordinate fitting straight line of each real eggshell, denoted as the movement route of the real eggshell.

[0029] The beneficial effects of the present invention are as follows:

[0030] (1) The present invention obtains the flow regularity of pixel points in continuously adjacent video frames through the change of the optical flow field, effectively differentiates the eggshell pixel points from the egg liquid pixel points, and improves the accuracy of eggshell detection;

[0031] (2) The present invention extracts the complete movement video of the eggshell on the surface of the egg liquid, improves the accuracy of eggshell detection, and improves the accuracy of predicting and fishing for the position of the eggshell;

[0032] (3) The present invention predicts the movement route of the eggshell, so that the time for fishing for the eggshell can be controlled more flexibly, and improves the convenience of eggshell fishing. Description of the Drawings

[0033] Figure 1 is a flowchart schematically showing a method for detecting an eggshell in egg liquid according to the present invention;

[0034] Figure 2 is a schematic diagram showing the flow of egg liquid;

[0035] Figure 3 is a schematic diagram comparing two video frames of a real-time egg liquid flow video. Detailed Embodiments

[0036] An embodiment of the present invention discloses a method for detecting an eggshell in egg liquid, referring to Figure 1 , including steps S1 - S5:

[0037] S1: Obtain a real-time egg liquid flow video through a machine vision system.

[0038] It should be noted that in order to timely retrieve the detected eggshells, the present invention considers installing a retrieval device, such as a robotic arm, above the egg liquid inspection tank, and then controlling the robotic arm through a control unit. In order to comprehensively obtain real-time images of the flowing egg liquid, a machine vision system, including an industrial camera and an image processing unit, should also be installed above the egg liquid inspection tank. The image processing unit processes the obtained real-time video of the flowing egg liquid, and the control unit controls the retrieval device based on the processing results of the real-time video of the flowing egg liquid to complete the real-time retrieval of the eggshells.

[0039] Specifically, the retrieval device is installed above the egg liquid inspection tank, and the machine vision system and the control unit are installed at the rear end of the retrieval device. The machine vision system collects the egg liquid video and performs grayscale processing to obtain a real-time video of the flowing egg liquid.

[0040] The retrieval device is a device capable of retrieving eggshells, such as a robotic arm or a net bag. The egg liquid inspection tank is a device that transports the egg liquid into the next process after separating the eggshells from the egg liquid, such as Figure 2 As shown in the schematic diagram of the flowing egg liquid, the arrow direction is the flowing direction of the egg liquid.

[0041] The machine vision system includes a high-resolution industrial camera and an image processing unit. The present invention determines the real eggshell area in the real-time video of the flowing egg liquid through the image processing unit.

[0042] The control unit controls the retrieval device according to the real eggshell area to complete the retrieval of the eggshells.

[0043] To avoid the accumulation of the real-time video of the flowing egg liquid being too long, resulting in redundancy, the duration of the real-time video of the flowing egg liquid can be limited. The duration can be set by the implementer according to the actual implementation situation, and there is no specific limitation. For example, dividing the length of the inspection tank by the flowing speed of the egg liquid is used as the duration of the real-time video of the flowing egg liquid.

[0044] So far, the real-time video of the flowing egg liquid has been obtained.

[0045] S2: According to the gray values and gray value changes of the same position pixel points in different video frames of the real-time video of the flowing egg liquid, obtain the light area of the real-time video of the flowing egg liquid.

[0046] It should be noted that, such as Figure 2, in the egg liquid production environment, when the ceiling light shines on the egg liquid, a highly bright light area will be generated. Due to the flow of the egg liquid, the shape of the light area changes, which may be detected as an eggshell. Therefore, the present invention first confirms the light area to avoid false detection in the follow-up. Since the position of the light is fixed, the area where the light reflects on the surface of the egg liquid is also fixed, and there are only small changes within a small range due to the inclination of the surface of the egg liquid. In addition, the light area shows the characteristic of high gray level in the real-time video of the flowing egg liquid due to its high brightness. Therefore, the light area of the real-time video of the flowing egg liquid can be confirmed according to the gray level values and gray level changes of the pixel points at the same position in different video frames.

[0047] Specifically, the real-time video of the flowing egg liquid includes several video frames. A three-dimensional coordinate system is established for the real-time video of the flowing egg liquid. The position of the first pixel point in the lower left corner of the first video frame is recorded as the origin. The direction to the right of the origin is recorded as the X-axis, the direction upward of the origin is recorded as the Y-axis, and the Z-axis is the time axis. The number of pixel points in the X-axis direction of the real-time video of the flowing egg liquid is recorded as M, and the number of pixel points in the Y-axis direction is recorded as N. The time distance between adjacent video frames depends on the frame rate of the industrial camera. For example, when the frame rate is 30 fps, the time distance between adjacent video frames is 1 / 30 second.

[0048] The pixel points with the same X-axis and Y-axis coordinates are recorded as the same group of pixel points, with a total of groups of the same group of pixel points. Obtain the gray level values of all pixel points of each group of the same group of pixel points and count the number of pixel points corresponding to each gray level value. It should be noted that the color of the egg yolk liquid or egg white liquid is relatively uniform, and the gray level values of each group of the same group of pixel points are relatively evenly distributed with the flow of the egg liquid. However, there will be more pixel points in the high gray level value area in the light area. Therefore, the more the number of high gray level value pixel points, the higher the light area performance of the same group of pixel points.

[0049] The light area performance of any group of the same group of pixel points satisfies the expression:

[0050] Obtain the gray level mean value of all groups of the same group of pixel points;

[0051] ;

[0052] In the formula, represents the light area performance of the i-th group of the same group of pixel points; S represents the maximum gray level value; s represents the gray level value; represents the number of pixel points with the gray level value of s in the i-th group of the same group of pixel points; represents the gray level mean value of all groups of the same group of pixel points. It should be noted that the gray level value range is from 0 to 255, so the maximum gray level value is 255.

[0053] In the formula, represents the gray level difference between the gray level value of s and the gray level mean value of all groups of the same group of pixel points. The larger this value is and the number of pixel points with the gray level value of s in the i-th group of the same group of pixel points The larger it is, the more pixels with larger gray values there are among the pixels in the i-th group of the same group, indicating that there are more cases of reflection at the positions of the pixels in the i-th group of the same group, and thus the light area performance is stronger.

[0054] Preferably, the K-means clustering algorithm is used to cluster the light area performances of all groups of pixels in the same group, obtaining several clusters of pixels in the same group. Calculate the average value of the light area performances of all groups of pixels in the same group and the average value of the light area performances of each cluster of pixels in the same group. Denote the cluster of pixels in the same group whose average value of the light area performance is greater than the average value of the light area performances of all groups of pixels in the same group as the light area of the real-time egg liquid flow video, and obtain several light areas of the real-time egg liquid flow video. It should be noted that the value of K in the K-means clustering algorithm is set by the implementer according to the actual implementation situation, and there is no specific limit. For example, the value of K can be set to 2.

[0055] Thus, several light areas of the real-time egg liquid flow video are obtained.

[0056] S3: According to the change of the optical flow field between adjacent video frames and the gray distribution of the pixels, obtain the eggshell performance of each pixel in each video frame; based on the eggshell performance of each pixel in each video frame and the distance between each pixel and the light area of the real-time egg liquid flow video, obtain the suspected eggshell area of each video frame.

[0057] It should be noted that after obtaining the light area of the egg liquid production environment projected on the real-time egg liquid flow video through the real-time egg liquid flow video, it is necessary to analyze frame by frame whether there is an eggshell in the video frame, such as Figure 3 As a schematic diagram of the comparison between two video frames of the real-time egg liquid flow video, the eggshell is within the white square. Since the boundaries between structures such as blastoderm, vitelline membrane, and chalaza in the egg liquid and the background egg liquid are not strong, and the eggshell is small, it is impossible to ensure the accuracy by using methods such as edge detection for image segmentation to obtain the eggshell area. However, there is an obvious density difference between the eggshell and the egg liquid. Therefore, the eggshell will sink, and the shape of the part exposed on the surface of the egg liquid will change, while the shape of the egg liquid changes less. Therefore, the movement of the pixels is considered for analysis.

[0058] It should be further noted that the optical flow method is a method for calculating the motion information of objects between adjacent frames by using the changes of pixels in the time domain of an image sequence and the correlation between adjacent frames. Therefore, the motion distance and direction of each pixel point in each video frame can be determined by the optical flow method, and the optical flow field corresponding to each time point of each video frame can be obtained. In the optical flow field, the motion distance and direction of the pixel points belonging to the egg liquid are more similar, while the motion distance and direction of the pixel points belonging to the eggshell are less similar. In addition, considering the fluidity of the egg liquid, relying solely on the motion of pixel points may affect the judgment of eggshell pixel points. Therefore, on the basis of shape change analysis, this invention adds the color dimension. There are also certain differences between the color of the eggshell and the color of the egg liquid. Through the high-frequency gray-scale distribution and low-frequency gray-scale distribution in the video frame, the accuracy of eggshell detection is further improved.

[0059] Specifically, the L-K algorithm is used to obtain the optical flow vectors of each pixel point of any two adjacent video frames. The optical flow vector of the pixel point is composed of the distances that the pixel point moves along the X-axis and Y-axis between adjacent video frames. Move the c-th pixel point from the position in the z-th video frame to the z + 1-th video frame, and record the distances moved along the X-axis and Y-axis as 、 , and record the optical flow vector of the c-th pixel point in the z-th and z + 1-th video frames as . The optical flow vectors of all pixel points in the z-th and z + 1-th video frames constitute the optical flow field of the z-th and z + 1-th video frames, denoted as .

[0060] It should be noted that in the process of the change of the optical flow field of consecutive adjacent video frames, the more consistent the optical flow vectors of the pixel points are, the more regular the flow of the egg liquid component at the corresponding position is, and the less likely it is to be the eggshell.

[0061] Preferably, according to the change of the optical flow vectors of pixel points in consecutive adjacent video frames, the flow regularity of any pixel point in consecutive adjacent video frames satisfies the expression:

[0062] Obtain the optical flow vectors and coordinates of the right adjacent preset T adjacent video frames of the c-th pixel point in the video frame. It should be noted that the number T of the preset adjacent video frames selected is set by the implementer according to the actual implementation situation. For example, the T value can be set to 15.

[0063] The optical flow vector is denoted as , where 、 、 are the coordinates of the c-th pixel point when it reaches the z + 1, z + 2, z + T respectively.

[0064] ;

[0065] In the formula, represents the flow regularity of the c-th pixel point in the T adjacent video frames adjacent to the right of the z-th video frame; T represents the number of preset adjacent video frames adjacent to the right; represents the distance difference of the c-th pixel point moving along the X-axis between the t-th and (t + 1)-th adjacent video frames adjacent to the right of the video frame; represents the distance difference of the c-th pixel point moving along the Y-axis between the t-th and (t + 1)-th adjacent video frames adjacent to the right of the video frame; represents the absolute value function; represents the exponential function with the natural constant as the base.

[0066] In the formula, represents the distance change of the c-th pixel point moving along the X-axis in the T adjacent video frames adjacent to the right. The larger this value is, the worse the consistency of the distance of the c-th pixel point moving along the X-axis, and the worse the flow regularity of the egg liquid; represents the distance change of the c-th pixel point moving along the Y-axis in the T adjacent video frames adjacent to the right. The larger this value is, the worse the consistency of the distance of the c-th pixel point moving along the Y-axis, and the worse the flow regularity of the egg liquid.

[0067] It should be noted that except for the pixel points in the light area, the egg liquid of several eggs is included in any video frame. The egg liquid components of these eggs are similar, so the gray-scale distribution is also similar. Therefore, in the gray-scale histogram of the video frame, there are a large number of pixel points for each gray-scale value belonging to the egg liquid, while the number of eggshells is small. Therefore, the number of pixel points for each gray-scale value belonging to the eggshell is small in the gray-scale histogram, and the pixel points belonging to the eggshell also have high stability in consecutive video frames. Thus, based on the number distribution of pixel points and the gray-scale change of pixel points in the gray-scale histogram, the gray-scale regularity of pixel points can be determined, and then the eggshell possibility of pixel points can be determined.

[0068] Preferably, the eggshell gray-scale index of any pixel point in consecutive adjacent video frames satisfies the expression:

[0069] Obtain the coordinates and gray-scale values of the c-th pixel point moving to the T adjacent video frames adjacent to the right of the z-th video frame;

[0070] ;

[0071] In the formula, represents the eggshell gray-scale index of the c-th pixel point in the T adjacent video frames adjacent to the right of the z-th video frame; represents the number of pixel points of the gray-scale value of the c-th pixel point in the z-th video frame in the corresponding gray-scale histogram; represents the number of pixel points in the X-axis direction of the video frame, represents the number of pixel points in the Y-axis direction of the video frame, represents the number of pixel points of the video frame; T represents the number of preset adjacent video frames adjacent on the right; , represents the gray values of the c-th pixel point moving to the t-th and t + 1-th adjacent video frames adjacent on the right of the z-th video frame.

[0072] In the formula, represents the ratio of the number of pixel points of the gray value corresponding to the c-th pixel point in the z-th video frame in the gray histogram of the z-th video frame to the total number of pixel points. This value represents the rarity of the gray value of the c-th pixel point. The larger this value is, the more common the gray value of the c-th pixel point is, and thus the smaller the possibility of the eggshell pixel point; represents the gray difference of the c-th pixel point in the t-th and t + 1-th video frames, represents the sum of the gray differences of the c-th pixel point in the T adjacent video frames adjacent on the right of the z-th video frame. The larger this value is, the more unstable the gray value of the c-th pixel point is, thus indicating less gray regularity, and further indicating a lower eggshell gray index.

[0073] It should be noted that the smaller the flow regularity of the pixel point in T consecutive adjacent video frames and the larger the eggshell gray index, the stronger the eggshell performance of the pixel point.

[0074] Preferably, the ratio of the eggshell gray index to the flow regularity is used as the eggshell performance of the pixel point. The eggshell performance of the c-th pixel point in the T adjacent video frames adjacent on the right of the z-th video frame is denoted as .

[0075] Thus, the eggshell performance of each pixel point in each video frame is obtained.

[0076] It should be noted that all pixel points in the suspected eggshell area have relatively high eggshell performance. Therefore, the pixel points can be clustered based on the eggshell performance. In addition, the pixel points in the suspected eggshell area are adjacent. Therefore, the pixel points can be clustered by the region growing algorithm.

[0077] Preferably, the region growing algorithm is used to cluster all pixel points in each video frame: all pixel points with an eggshell performance greater than 0.5 are used as growth seed points, and each is used as a growth region. If there are adjacent growth regions, the adjacent growth regions are merged into one growth region; the difference from the average eggshell performance of the growth region being less than 0.1 is used as the growth condition; for all pixel points adjacent to the growth region, the condition for stopping growth is that the difference from the average eggshell performance of the growth region is greater than or equal to 0.1; after the region growth is completed, several growth regions of each video frame are obtained.

[0078] It should be noted that in each growth region of each video frame, if the number of pixel points is too small or the distance from the light region is large, the growth region may be noise; if the number of pixel points is large and the distance from the light region is large, the growth region is more likely to be an eggshell. Therefore, the eggshell possibility of each growth region of each video frame can be obtained according to the size of the growth region and the distance from the light region.

[0079] Preferably, the eggshell possibility of each growth region of each video frame satisfies the expression:

[0080] ;

[0081] In the formula, represents the eggshell possibility of the hth growth region of the zth video frame; represents the number of pixel points in the hth growth region of the zth video frame; represents the set of distances between the centroid of the hth growth region of the zth video frame and the centroids of all light regions; represents the minimum value function; represents the normalization function.

[0082] Preferably, the growth regions with eggshell possibility greater than the first threshold are marked as suspected eggshell regions.

[0083] So far, several suspected eggshell regions of each video frame have been obtained.

[0084] S4: Determine the video frame sequence in which the real eggshell exists according to the shape change and position change of the suspected eggshell regions in consecutive video frames.

[0085] It should be noted that in S3, the suspected eggshell regions of each video frame are obtained from the perspective of a single pixel point and a single video frame. However, the characteristic of the real-time eggshell flow video is that the eggshell will move coherently in the egg liquid as a whole. For any video frame, an eggshell may be entirely on the surface, or mostly on the surface, or may have nearly completely fallen. Therefore, intercepting the eggshell from the start of its appearance to its complete fall can more accurately analyze the falling characteristics of the eggshell, so as to predict the subsequent position of the eggshell.

[0086] It should be further noted that as the eggshell moves, there is a certain position overlap and a certain shape overlap in the consecutive video frames. Therefore, based on the position relationship and shape relationship of the suspected eggshell regions in the consecutive video frames, the video frames belonging to the same eggshell region can be merged, and then the complete video frame sequence of the real eggshell can be obtained.

[0087] Specifically, obtain the set of pixel points of each suspected eggshell region in each video frame, and obtain the sequence of boundary pixel points of each suspected eggshell region in each video frame. The starting position of the sequence of boundary pixel points is the boundary pixel point with the smallest distance from the origin, and the sequence direction is the clockwise direction.

[0088] Preferably, the possibility that any two suspected eggshell regions in any two adjacent video frames belong to the same eggshell includes:

[0089] Obtain the edit distance between the sequence of boundary pixel points of the g-th suspected eggshell region in the z-th video frame and the sequence of boundary pixel points of the g-th suspected eggshell region in the (z + 1)-th video frame;

[0090] ;

[0091] In the formula, represents the possibility that the g-th suspected eggshell region in the z-th video frame and the g-th suspected eggshell region in the (z + 1)-th video frame belong to the same eggshell; represents the set of pixel points of the g-th suspected eggshell region in the z-th video frame; represents the set of pixel points of the k-th suspected eggshell region in the (z + 1)-th video frame; represents the edit distance between the sequence of boundary pixel points of the g-th suspected eggshell region in the z-th video frame and the sequence of boundary pixel points of the g-th suspected eggshell region in the (z + 1)-th video frame; represents the exponential function with the natural constant as the base.

[0092] In the formula, represents the ratio of the pixel intersection and pixel union of the g-th suspected eggshell region in the z-th video frame and the g-th suspected eggshell region in the (z + 1)-th video frame. This value represents the coincidence situation of the two suspected eggshell regions. The larger this value is, the more overlapping pixel points there are between the two suspected eggshell regions, indicating that the two suspected eggshell regions are more likely to belong to the same eggshell; the edit distance between the sequence of boundary pixel points of the g-th suspected eggshell region in the z-th video frame and the sequence of boundary pixel points of the g-th suspected eggshell region in the (z + 1)-th video frame , then represents the shape change of the two suspected eggshell regions. The larger this value is, the greater the boundary shape change of the two suspected eggshell regions, and the smaller the possibility of belonging to the same eggshell.

[0093] Preferably, any suspected eggshell area is marked as a target suspected eggshell area, and region growing is performed on the target suspected eggshell area: using the video frame where the target suspected eggshell area is located as the growing seed point; using the suspected eggshell areas in the left and right adjacent video frames that have intersecting pixel points and the probability of belonging to the same eggshell as the target suspected eggshell area being greater than a second threshold as the growing condition; using the suspected eggshell areas in the left and right adjacent video frames that do not have any intersecting pixel points as the termination growing condition; obtaining several consecutive video frames in which the target suspected eggshell area exists; and obtaining the video frame sequence in which the target suspected eggshell area exists. It should be noted that the second threshold is set by the implementer according to the actual implementation situation and is not specifically limited. For example, the second threshold can be set to 0.5.

[0094] It should be noted that only when there are many video frames in which the suspected eggshell area exists can it be determined as the change process of a real eggshell.

[0095] Preferably, the video frame sequence in which the suspected eggshell area exists and the number of video frames in the video frame sequence is greater than a third threshold is marked as the video frame sequence in which a real eggshell exists.

[0096] So far, several video frame sequences in which real eggshells exist have been obtained.

[0097] S5: Predict the movement route of the eggshell according to the position change of the eggshell in the video frame sequence in which the real eggshell exists; fish for the eggshell based on the movement route of the eggshell.

[0098] It should be noted that the eggshell may gradually sink, or may move on the surface of the egg liquid due to factors such as bubbles and buoyancy. Therefore, the positions for controlling the fishing device to fish for the eggshell are different. In addition, the time distance between the time when the eggshell is detected and the time of the latest video frame is different. If the time distance between the time when the eggshell is detected and the time of the latest video frame is short, then the change in the predicted position of the eggshell compared to the position where the eggshell is detected is relatively small. At the same time, if the displacement of the eggshell in the existing video frame sequence is relatively small, then the change in the predicted position for fishing for the eggshell compared to the position in the video frame sequence is also relatively small.

[0099] Specifically, in each video frame sequence in which a real eggshell exists, obtain the centroid coordinates of the eggshell area in each video frame, and use the least squares method for spatial straight line fitting to obtain the centroid coordinate fitting straight line of each real eggshell, which is denoted as the movement route of the real eggshell. On the movement route of the real eggshell, the centroid coordinates corresponding to the fishing time are denoted as the predicted coordinates of the corresponding real eggshell.

[0100] After obtaining the video sequence in which the real eggshell exists, the control center controls the retrieving device to reach the surface of the egg liquid. The time when the retrieving device reaches the surface of the egg liquid is recorded as the retrieving time. On the movement route of the real eggshell, the centroid coordinates corresponding to the retrieving time are recorded as the predicted coordinates of the corresponding real eggshell. The control center controls the retrieving device to reach the predicted coordinates to complete the retrieval of the eggshell.

[0101] Thus, the predicted coordinates of the real eggshell are obtained, and the retrieval of the eggshell is completed.

[0102] Although this specification has shown and described multiple embodiments of the present invention, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art will think of many changes, alterations, and alternative ways without departing from the idea and spirit of the present invention.

[0103] The above are all preferred embodiments of the present invention, and the protection scope of the present invention is not limited accordingly. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A method for detecting eggshell in egg liquid, characterized in that: include: Acquire a real-time egg liquid flow video through a machine vision system, where the real-time egg liquid flow video includes a number of video frames; According to the grayscale value and grayscale change of the pixel point at the same position in different video frames in the real-time egg liquid flow video, the light area of ​​the real-time egg liquid flow video is obtained; According to the change of the optical flow field of adjacent video frames and the grayscale distribution of the pixels, the eggshell performance of each pixel in each video frame is obtained, including: according to the change of the optical flow vector of the pixel in consecutive adjacent video frames, the flow regularity of any pixel in consecutive adjacent video frames is obtained, and the formula is: , For the The pixel at The right adjacent to the video frame The regularity of the flow of adjacent video frames, is the preset number of adjacent video frames to the right, For the The pixel at The video frame , The distance difference between the right adjacent video frames along the X axis, For the The pixel at The video frame , The difference in the distance along the Y axis between the right adjacent video frames, is the absolute value function; is an exponential function with a natural constant as the base; based on the number distribution of pixels in the grayscale histogram and the grayscale change of pixels, the eggshell grayscale index of any pixel in consecutive adjacent video frames is obtained, and the formula is: , For the The pixel at The right adjacent to the video frame The eggshell grayscale index of adjacent video frames, For the The pixel at The number of pixels in the grayscale histogram corresponding to the grayscale value of the video frame, is the number of pixels in the X-axis direction of the video frame, is the number of pixels in the Y-axis direction of the video frame, is the number of pixels in the video frame, , For the Move the pixel to The right adjacent , The grayscale value of adjacent video frames; the ratio of the eggshell grayscale index to the flow regularity is used as the eggshell representation of the pixel point in the consecutive adjacent video frames; Based on the eggshell performance of each pixel point in each video frame and the distance between each pixel point and the light area of ​​the real-time egg liquid flow video, the suspected eggshell area of ​​each video frame is obtained; Determine the video frame sequence where the real eggshell exists according to the shape change and position change of the suspected eggshell area in the continuous video frames; According to the position change of the eggshell in the video frame sequence where the real eggshell exists, the movement path of the eggshell is predicted; The eggshells are scooped up based on their movement paths.

2. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The light area for obtaining the real-time egg liquid flow video includes: A three-dimensional coordinate system is established for the real-time egg liquid flow video, in which the XOY plane is parallel to the video frame plane and the Z axis is the time axis; pixel points with the same X-axis and Y-axis coordinates are recorded as the same group of pixel points; According to the distribution of the number of pixels of each gray value of the pixels in the same group, the light area performance of each pixel in the same group is obtained; A clustering algorithm is used to cluster the light zone performance of all groups of pixels in the same group to obtain several clusters of pixels in the same group. The clusters of pixels in the same group whose mean light zone performance is greater than the mean light zone performance of all groups of pixels in the same group are recorded as the light zone of the real-time egg liquid flow video.

3. The method for detecting eggshell in egg liquid according to claim 2, characterized in that: The light area performance of the same group of pixels includes: Obtain the grayscale values ​​of all pixels in the same group of pixels and count the number of pixels corresponding to each grayscale value; obtain the grayscale mean value of each pixel in the same group; The light area performance of any group of pixels in the same group satisfies the expression: ； In the formula, Indicates The light area performance of the same group of pixels; Indicates the maximum gray value; Represents grayscale value; Indicates The gray value of the pixels in the same group is The number of pixels; Represents the grayscale mean of all pixels in the same group.

4. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The method of obtaining the suspected eggshell region of each video frame based on the eggshell performance of each pixel point in each video frame and the distance between each pixel point and the light area of ​​the real-time egg liquid flow video comprises: Based on the distance and eggshell performance, all the pixels of each video frame are clustered to obtain several growth regions of each video frame; The minimum value of the distance between the centroid of each growth area and the centroid of all light areas of the real-time egg liquid flow video is obtained, and multiplied by the number of pixels in the corresponding growth area to obtain the eggshell possibility of each growth area; The growth area with eggshell possibility greater than the first threshold is recorded as a suspected eggshell area.

5. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The method of determining the video frame sequence where the real eggshell exists according to the shape change and position change of the suspected eggshell area of ​​the continuous video frames comprises: Obtain a set of pixel points of each suspected eggshell region of each video frame, and obtain a sequence of boundary pixel points of each suspected eggshell region of each video frame; Based on the positional relationship and shape relationship of the suspected eggshell regions in the continuous video frames, the possibility that any two suspected eggshell regions in any two adjacent video frames belong to the same eggshell is obtained; The video frames belonging to the same eggshell are clustered by a clustering algorithm to obtain a video frame sequence with suspected eggshell areas, and the video frame sequence with suspected eggshell areas whose number of clustered video frames is greater than a third threshold is recorded as a video frame sequence with real eggshells.

6. The method for detecting eggshell in egg liquid according to claim 5, characterized in that: The possibility that any two suspected eggshell regions in any two adjacent video frames belong to the same eggshell includes: Get the The video frame The boundary pixel sequence of the suspected eggshell area is The video frame The edit distance of the boundary pixel sequence of the suspected eggshell area is denoted as ; For The video frame The suspected eggshell area The video frame The intersection and union of pixels in the suspected eggshell area are compared with Multiply the normalized value of the negative correlation by The video frame The suspected eggshell area and The video frame The probability that the suspected eggshell regions belong to the same eggshell.

7. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The method predicts the movement path of the eggshell according to the position change of the eggshell in the video frame sequence where the real eggshell exists, including: The centroid coordinates of the eggshell area in each video frame in the video frame sequence of each real eggshell are obtained, and the least squares method is used to perform spatial straight line fitting to obtain the centroid coordinate fitting line of each real eggshell, which is recorded as the movement path of the real eggshell.

Images