Method for detecting eggshells in egg liquid

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a method for detecting an eggshell in egg liquid, and the method comprises the steps: obtaining a light region of a real-time egg liquid flowing video according to the gray value and gray change of pixel points at the same position in different video frames in the real-time egg liquid flowing video; according to the optical flow field change of the adjacent video frames and the gray level distribution of the pixel points, obtaining the eggshell expression of each pixel point in each video frame; based on the eggshell expression of each pixel point in each video frame and the distance between each pixel point and the light area of the real-time egg liquid flowing video, obtaining a suspected eggshell area of each video frame; and according to the shape change and position change of the suspected eggshell area of the continuous video frames, determining a video frame sequence in which a real eggshell exists, and predicting the motion route of the eggshell to complete the extraction of the eggshell. The detection accuracy of the eggshell in the egg liquid and the timeliness of fishing the eggshell are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a method for detecting eggshells in egg liquid. Background Art

[0002] Eggs are one of the indispensable foods in people's daily life because of their rich nutritional value. Therefore, they are also used in food processing industries such as baked goods and meat products. However, due to their fragility, the egg liquid product industry has gradually developed. Egg liquid can better preserve the nutrition of eggs and is easy to transport and store, and can efficiently provide egg raw materials for the food processing industry. The production process of egg liquid includes washing eggs, separating egg liquid and eggshells, separating egg white liquid and egg yolk liquid, removing broken eggshells, sterilizing, filling and refrigerating. In the process of removing broken eggshells, precipitation and filtration steps are often included, but the broken eggshells are of different sizes, and there are also large embryonic discs, yolk membranes, ligaments and other structures in the egg white liquid and egg yolk liquid, which makes it difficult to completely remove the broken eggshells by precipitation and filtration, so more sophisticated detection methods are needed.

[0003] In order to detect whether the effective food contains invalid waste during food processing, machine vision technology and neural network models are often used to segment and classify effective food and invalid waste. In related technologies, for example, the Chinese patent document with authorization announcement number CN116703829B discloses an online detection method and system for buckwheat shelling parameters for buckwheat shelling machines, which discloses the classification and segmentation of unshelled grains, shelled buckwheat and broken rice in images through neural networks. The Chinese patent document with authorization announcement number CN112132792B discloses a pixel-level detection method for endogenous foreign matter in pecans based on hyperspectral and deep learning, which discloses the establishment of a classification model for pecan outer meat, inner meat, outer shell, and inner shell based on neural networks, thereby realizing pixel-level detection of endogenous pecan shells in pecans.

[0004] However, the egg liquid processing environment is complex. For example, the lighting will affect the presentation of the eggshell in the egg liquid image, affecting the eggshell detection rate. In order to improve the efficiency of the process, the egg liquid will flow in real time to enter the subsequent process. The egg liquid structure and eggshell structure of the flowing egg liquid are not fixed, which will affect the accuracy of eggshell detection. If the egg liquid is statically detected, it will not only affect production efficiency, but also miss detection due to the sinking of the eggshell. Summary of the invention

[0005] In order to solve the technical problem of insufficient accuracy in detecting broken eggshells in egg liquid during the above-mentioned egg liquid production process, the present invention provides a method for detecting eggshells in egg liquid, comprising: A real-time egg liquid flow video is obtained through a machine vision system, and the real-time egg liquid flow video includes several video frames; a light area of ​​the real-time egg liquid flow video is obtained according to the grayscale value and grayscale change of the pixel points at the same position in different video frames in the real-time egg liquid flow video; an eggshell performance of each pixel point in each video frame is obtained according to the change of the optical flow field of adjacent video frames and the grayscale distribution of the pixel points; a suspected eggshell area of ​​each video frame is obtained based on the eggshell performance of each pixel point in each video frame and the distance between each pixel point and the light area of ​​the real-time egg liquid flow video; a video frame sequence where a real eggshell exists is determined according to the shape change and position change of the suspected eggshell area of ​​consecutive video frames; a movement path of the eggshell is predicted according to the position change of the eggshell in the video frame sequence where the real eggshell exists; and the eggshell is scooped out based on the movement path of the eggshell.

[0006] The present invention obtains the light zone in the real-time egg liquid flow video in advance, thereby avoiding the influence of light reflection on eggshell detection in the egg liquid production environment. The present invention detects eggshells based on the differences in the changes in shape, position, etc. of eggshells and egg liquid during flow, and places the detection of eggshells in dynamic videos instead of simple static detection, thereby improving the accuracy of eggshell detection, extracting more abundant features in the movement of eggshells in egg liquid, and ensuring the detection rate of eggshells.

[0007] Preferably, the method of obtaining the light zone of the real-time egg liquid flow video comprises: establishing a three-dimensional coordinate system for the real-time egg liquid flow video, wherein the XOY plane in the three-dimensional coordinate system is parallel to the video frame plane, and the Z axis is the time axis; recording the pixels with the same X-axis and Y-axis coordinates as the same group of pixels; obtaining the light zone performance of each group of pixels according to the distribution of the number of pixels of each gray value of each group of pixels; clustering the light zone performance of all groups of the same group of pixels using a clustering algorithm to obtain a plurality of clusters of the same group of pixels, and recording the same group of pixel clusters whose mean value of the light zone performance is greater than the mean value of the light zone performance of all groups of the same group of pixels as the light zone of the real-time egg liquid flow video.

[0008] The present invention marks the light area in the real-time egg liquid flow video, so that the detection of the eggshell area avoids the interference of light reflection, and the detection of the eggshell is concentrated in the normal brightness area, thereby improving the accuracy of eggshell detection.

[0009] Preferably, the light area representation of the same group of pixels includes: Obtain the grayscale values ​​of all pixels in the same group of pixels and count the number of pixels corresponding to each grayscale value; obtain the grayscale mean value of each pixel in the same group; The light area performance of any group of pixels in the same group satisfies the expression: ; In the formula, It represents the light area performance of the pixels in the same group of the i-th group; S represents the maximum gray value; s represents the gray value; It represents the number of pixels with gray value s in the same group of pixels in the i-th group; Represents the grayscale mean of all pixels in the same group.

[0010] Preferably, the method of obtaining the eggshell performance of each pixel in each video frame according to the change of the optical flow field of adjacent video frames and the grayscale distribution of the pixels comprises: obtaining the flow regularity of any pixel in consecutive adjacent video frames according to the change of the optical flow vector of the pixel in consecutive adjacent video frames; obtaining the eggshell grayscale index of any pixel in consecutive adjacent video frames based on the number distribution of pixels and the grayscale change of pixels in the grayscale histogram; and taking the ratio of the eggshell grayscale index to the flow regularity as the eggshell performance of the pixel in consecutive adjacent video frames.

[0011] The present invention combines the grayscale difference between eggshell and egg liquid and the difference in morphological changes during the flow process to obtain the eggshell performance of the pixel points, accurately and comprehensively screen the pixel points belonging to the eggshell. In addition, the grayscale change and morphological change of the pixel points in adjacent video frames are analyzed, avoiding the noise interference caused by analyzing only a single video frame.

[0012] Preferably, the flow regularity of any pixel point in consecutive adjacent video frames satisfies the expression: ; In the formula, represents the flow regularity of the c-th pixel in the T adjacent video frames to the right of the z-th video frame; T represents the number of preset adjacent video frames to the right; represents the distance difference along the X axis of the c-th pixel in the t-th and t+1-th right adjacent video frames of the z-th video frame; represents the distance difference along the Y axis of the c-th pixel in the t-th and t+1-th right adjacent video frames of the z-th video frame; represents the absolute value function; Represents an exponential function with a natural constant as its base.

[0013] Preferably, the eggshell grayscale index of any pixel point in consecutive adjacent video frames satisfies the expression: ; In the formula, Represents the eggshell grayscale index of the c-th pixel in the T adjacent video frames to the right of the z-th video frame; Indicates that the cth pixel is in The number of pixels in the corresponding grayscale histogram of the grayscale value of the video frame; Indicates the number of pixels in the X-axis direction of the video frame. Indicates the number of pixels in the Y-axis direction of the video frame. represents the number of pixels in a video frame; T represents the number of preset adjacent video frames to the right; , Indicates the grayscale value of the c-th pixel moving to the t-th and t+1-th adjacent video frames to the right of the z-th video frame.

[0014] Preferably, the method of obtaining the suspected eggshell area of ​​each video frame based on the eggshell performance of each pixel in each video frame and the distance between each pixel and the light area of ​​the real-time egg liquid flow video includes: clustering all the pixels of each video frame based on the distance and the eggshell performance to obtain a number of growth areas of each video frame; obtaining the minimum value of the distance between the centroid of each growth area and the centroid of all light areas of the real-time egg liquid flow video, and multiplying it by the number of pixels in the corresponding growth area to obtain the eggshell possibility of each growth area; and recording the growth area with an eggshell possibility greater than a first threshold as a suspected eggshell area.

[0015] The present invention merges pixel points belonging to a suspected eggshell area through clustering, so that the detected eggshell can be fully displayed, the possibility of successfully catching the eggshell is improved, and the possibility of missing the eggshell due to incomplete eggshell detection is avoided.

[0016] Preferably, the method of determining a sequence of video frames in which a real eggshell exists based on shape changes and position changes of suspected eggshell areas in consecutive video frames comprises: obtaining a set of pixel points of each suspected eggshell area in each video frame, and obtaining a sequence of boundary pixel points of each suspected eggshell area in each video frame; obtaining the possibility that any two suspected eggshell areas in any two adjacent video frames belong to the same eggshell based on the positional relationship and shape relationship of the suspected eggshell areas in consecutive video frames; clustering the video frames belonging to the same eggshell by a clustering algorithm to obtain a sequence of video frames in which suspected eggshell areas exist, and recording a sequence of video frames in which suspected eggshell areas exist and the number of clustered video frames is greater than a third threshold as a sequence of video frames in which a real eggshell exists.

[0017] The present invention merges all video frames of a same real eggshell, so that the movement process of the real eggshell can be analyzed in its entirety, thereby improving the accuracy of predicting the position of the eggshell.

[0018] Preferably, the possibility that any two suspected eggshell regions in any two adjacent video frames belong to the same eggshell includes: obtaining the edit distance between the boundary pixel point sequence of the g-th suspected eggshell region of the z-th video frame and the boundary pixel point sequence of the g-th suspected eggshell region of the z+1-th video frame, recorded as ; Compare the pixel intersection and pixel union of the g-th suspected eggshell region of the z-th video frame with the g-th suspected eggshell region of the z+1-th video frame, and The negative correlation normalized value of is multiplied by the g-th suspected eggshell region in the z-th video frame and the g-th suspected eggshell region in the z+1-th video frame belong to the same eggshell.

[0019] Preferably, the method predicts the movement path of the eggshell based on the position change of the eggshell in the video frame sequence in which the real eggshell exists, including: obtaining the centroid coordinates of the eggshell area in each video frame in the video frame sequence in which each real eggshell exists, and using the least squares method to perform spatial straight line fitting to obtain the centroid coordinate fitting line of each real eggshell, which is recorded as the movement path of the real eggshell.

[0020] The beneficial effects of the present invention are: (1) The present invention obtains the flow regularity of pixels in consecutive adjacent video frames through the change of the optical flow field, effectively distinguishes eggshell pixels from egg liquid pixels, and improves the accuracy of eggshell detection; (2) The present invention extracts the complete motion video of the eggshell on the surface of the egg liquid, thereby improving the accuracy of eggshell detection and the accuracy of predicting and scooping the eggshell position; (3) The present invention predicts the movement path of the eggshell, so that the time for scooping the eggshell can be controlled more flexibly, thereby improving the convenience of scooping the eggshell. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart schematically showing a method for detecting eggshell in egg liquid in the present invention; Figure 2 Schematic diagram showing the flow of egg liquid; Figure 3 Schematic diagram showing the comparison of two video frames of real-time egg liquid flow video. DETAILED DESCRIPTION

[0022] The present invention discloses a method for detecting eggshell in egg liquid, referring to Figure 1 , comprising steps S1 to S5: S1: Obtain real-time egg liquid flow video through the machine vision system.

[0023] It should be noted that, in order to enable the detected eggshells to be picked up in time, the present invention considers placing a picking device, such as a mechanical arm, above the egg liquid inspection tank, and then controlling the mechanical arm through a control unit. In order to fully obtain the real-time egg liquid flow image, a machine vision system should also be placed above the egg liquid inspection tank, including an industrial camera and an image processing unit. The image processing unit processes the obtained real-time egg liquid flow video, and the control unit controls the picking device according to the real-time egg liquid flow video processing result to complete the real-time picking of the eggshell.

[0024] Specifically, the scooping device is placed above the egg liquid inspection tank, and the machine vision system and the control unit are placed at the rear end of the scooping device. The egg liquid video is collected by the machine vision system and grayscaled to obtain a real-time egg liquid flow video.

[0025] The scooping device is a device such as a mechanical arm or a net bag that can scoop egg shells. The egg liquid inspection tank is a device that separates the egg shells from the egg liquid and then transports the egg liquid to the next process, such as Figure 2 This is a schematic diagram of the flow of egg liquid, and the direction of the arrow is the direction of the egg liquid flow.

[0026] The machine vision system comprises a high-resolution industrial camera and an image processing unit. The present invention determines the real eggshell area in the real-time egg liquid flow video through the image processing unit.

[0027] The control unit controls the scooping device according to the real eggshell area to complete the scooping of the eggshell.

[0028] In order to avoid the real-time egg liquid flow video from accumulating too long and causing redundancy, the duration of the real-time egg liquid flow video can be limited. The duration can be set by the implementation personnel according to the actual implementation situation, and there is no specific restriction. For example, the length of the inspection slot is divided by the egg liquid flow speed to serve as the duration of the real-time egg liquid flow video.

[0029] At this point, real-time video of egg liquid flow is obtained.

[0030] S2: Obtain the light area of ​​the real-time egg liquid flow video according to the grayscale value and grayscale change of the pixel point at the same position in different video frames in the real-time egg liquid flow video.

[0031] It should be noted that if Figure 2 In the egg liquid production environment, when the top light shines on the egg liquid, a bright light area will be generated. Due to the flow of the egg liquid, the shape of the light area changes and may be detected as an eggshell. Therefore, the present invention first confirms the light area to avoid subsequent misdetection. Since the position of the light is fixed, the area where the reflection is formed on the surface of the egg liquid is also fixed, and only changes in a small range due to the inclination of the surface of the egg liquid. In addition, the light area shows a high grayscale characteristic in the real-time egg liquid flow video due to its high brightness. Therefore, the light area of ​​the real-time egg liquid flow video can be confirmed based on the grayscale value and grayscale change of the same position pixel point in different video frames.

[0032] Specifically, the real-time egg liquid flow video includes several video frames, and a three-dimensional coordinate system is established for the real-time egg liquid flow video, the first pixel position in the lower left corner of the first video frame is recorded as the origin, the direction to the right of the origin is recorded as the X axis, the direction upward from the origin is recorded as the Y axis, and the Z axis is the time axis, and the number of pixels in the X-axis direction of the real-time egg liquid flow video is recorded as M, and the number of pixels in the Y-axis direction is recorded as N. The time distance of adjacent video frames is determined according to the frame rate of the industrial camera, for example, when the frame rate is 30fps, the time distance of adjacent video frames is 1 / 30 second.

[0033] The pixels with the same X-axis and Y-axis coordinates are recorded as the same group of pixels. The pixels in the same group are grouped together, the grayscale values ​​of all the pixels in each group are obtained, and the number of pixels corresponding to each grayscale value is counted. It should be noted that the color of the egg yolk liquid or egg white liquid is relatively balanced, and the grayscale values ​​of the pixels in the same group are distributed evenly as the egg liquid flows, while the light area will have more pixels in the high grayscale value area, so the more high grayscale value pixels there are, the higher the light area performance of the pixels in the same group is.

[0034] The light area performance of any pixel in the same group satisfies the expression: Get the grayscale mean of all pixels in the same group; ; In the formula, It represents the light area performance of the pixels in the same group of the i-th group; S represents the maximum gray value; s represents the gray value; It represents the number of pixels with gray value s in the same group of pixels in the i-th group; Represents the grayscale mean of all pixels in the same group. It should be noted that the grayscale value ranges from 0 to 255, so the maximum grayscale value is 255.

[0035] In the formula, It represents the grayscale difference between the grayscale value s and the grayscale mean of all pixels in the same group. The larger the value, the more pixels in the same group of pixels have a grayscale value of s. The larger it is, the more pixels with larger grayscale values ​​there are in the i-th group of pixels, which means there are more reflections at the positions of the i-th group of pixels, and thus the light area performance is stronger.

[0036] Preferably, the light zone performance of all groups of pixels in the same group is clustered using a K-means clustering algorithm to obtain a number of pixel clusters in the same group, the light zone performance mean of all groups of pixels in the same group and the light zone performance mean of each pixel cluster in the same group are calculated, and the pixel clusters in the same group whose light zone performance mean is greater than the light zone performance mean of all groups of pixels in the same group are recorded as the light zone of the real-time egg liquid flow video, and a number of light zones of the real-time egg liquid flow video are obtained. It should be noted that the K value of the K-means clustering algorithm is set by the implementer according to the actual implementation situation, and is not specifically limited. For example, the K value can be set to 2.

[0037] So far, several light zones of the real-time egg liquid flow video have been obtained.

[0038] S3: According to the changes in the optical flow field of adjacent video frames and the grayscale distribution of the pixels, the eggshell performance of each pixel in each video frame is obtained; based on the eggshell performance of each pixel in each video frame and the distance between each pixel and the light area of ​​the real-time egg liquid flow video, the suspected eggshell area of ​​each video frame is obtained.

[0039] It should be noted that after obtaining the light area of ​​the egg liquid production environment projected on the real-time egg liquid flow video through the real-time egg liquid flow video, it is necessary to analyze whether there is an eggshell in the video frame by frame, such as Figure 3 This is a schematic diagram of the comparison of two video frames of the real-time egg liquid flow video. The white box is the eggshell. Since the boundaries between the embryonic disc, vitelline membrane, frenulum and other structures in the egg liquid and the background egg liquid are not strong, and the eggshell is small, the accuracy of obtaining the eggshell area through image segmentation using edge detection and other methods cannot be guaranteed. However, there is a significant density difference between the eggshell and the egg liquid, so the eggshell will sink, and the shape of the part exposed on the surface of the egg liquid will change, while the shape of the egg liquid changes less, so it is considered to analyze the movement of the pixel points.

[0040] It should be further explained that the optical flow method is a method for calculating the motion information of objects between adjacent frames using the change of pixels in the time domain in the image sequence and the correlation between adjacent frames. Therefore, the motion distance and direction of each pixel in each video frame can be determined by the optical flow method, and the optical flow field of each video frame corresponding to the time point is obtained. In the optical flow field, the motion distance and direction of the pixel points belonging to the egg liquid are more similar, while the motion distance and direction of the pixel points belonging to the eggshell are less similar. In addition, considering the fluidity of the egg liquid, relying only on the motion of the pixel points may affect the judgment of the eggshell pixel points. Therefore, on the basis of the shape change analysis, the present invention increases the color dimension, and there is also a certain difference between the color of the eggshell and the color of the egg liquid. Through the high-frequency grayscale distribution and low-frequency grayscale distribution in the video frame, the accuracy of eggshell detection is further improved.

[0041] Specifically, the LK algorithm is used to obtain the optical flow vector of each pixel of any two adjacent video frames. The optical flow vector of the pixel is composed of the distance the pixel moves along the X-axis and the Y-axis in the adjacent video frames. The c-th pixel is moved from the position of the z-th video frame to the z+1-th video frame, and the distance moved along the X-axis and the Y-axis is recorded as , , the optical flow vector of the c-th pixel in the z-th and z+1-th video frames is recorded as , the optical flow vectors of all pixels in the zth and z+1th video frames constitute the optical flow fields of the zth and z+1th video frames, recorded as .

[0042] It should be noted that, in the process of optical flow field changes of consecutive adjacent video frames, the more consistent the optical flow vectors of the pixel points are, the more regular the flow of the egg liquid components at the corresponding position is, and the less likely it is the eggshell.

[0043] Preferably, according to the change of the optical flow vector of the pixel point in the continuous adjacent video frames, the flow regularity of any pixel point in the continuous adjacent video frames satisfies the expression: Get the cth pixel in the The optical flow vectors and coordinates of T preset adjacent video frames to the right of the video frame. It should be noted that the number T of the selected preset adjacent video frames is set by the implementer according to the actual implementation situation, for example, the value of T can be set to 15.

[0044] The optical flow vector is recorded as ,in , , These are the coordinates of the cth pixel when it reaches the z+1, z+2, and z+T points respectively.

[0045] ; In the formula, represents the flow regularity of the c-th pixel in the T adjacent video frames to the right of the z-th video frame; T represents the number of preset adjacent video frames to the right; Indicates that the cth pixel is in The distance difference between the tth and t+1th right adjacent video frames of the video frame along the X axis; Indicates that the cth pixel is in The distance difference between the tth and t+1th right adjacent video frames of the video frame along the Y axis; represents the absolute value function; Represents an exponential function with a natural constant as its base.

[0046] In the formula, Indicates the change in the distance that the c-th pixel moves along the X-axis in the T adjacent video frames to the right. The larger the value, the worse the consistency of the distance that the c-th pixel moves along the X-axis, and the worse the regularity of the egg liquid flow. It represents the change in the distance that the c-th pixel moves along the Y axis in the T adjacent video frames to the right. The larger the value, the worse the consistency of the distance that the c-th pixel moves along the Y axis, and the worse the regularity of the egg liquid flow.

[0047] It should be noted that, excluding the pixels in the light zone, any video frame contains egg liquids of several eggs, and the egg liquid components of these eggs are similar, so the grayscale distribution is also similar. Therefore, in the grayscale histogram of the video frame, each grayscale value belonging to the egg liquid has a large number of pixels, while the number of eggshells is small, so the number of pixels belonging to each grayscale value of the eggshell in the grayscale histogram is small, and the pixels belonging to the eggshell also have high stability in continuous video frames. Therefore, based on the number distribution of pixels in the grayscale histogram and the grayscale change of pixels, the grayscale regularity of pixels can be determined, and then the eggshell possibility of pixels can be determined.

[0048] Preferably, the eggshell grayscale index of any pixel in consecutive adjacent video frames satisfies the expression: Get the coordinates and grayscale values ​​of the c-th pixel point moving to the right adjacent T adjacent video frames of the z-th video frame; ; In the formula, Represents the eggshell grayscale index of the c-th pixel in the T adjacent video frames to the right of the z-th video frame; Indicates the number of pixels in the corresponding grayscale histogram of the grayscale value of the c-th pixel in the z-th video frame; Indicates the number of pixels in the X-axis direction of the video frame. Indicates the number of pixels in the Y-axis direction of the video frame. represents the number of pixels in a video frame; T represents the number of preset adjacent video frames to the right; , Indicates the grayscale value of the c-th pixel moving to the t-th and t+1-th adjacent video frames to the right of the z-th video frame.

[0049] In the formula, The ratio of the number of pixels in the grayscale histogram of the zth video frame to the total number of pixels in the grayscale histogram of the cth pixel in the zth video frame indicates the rarity of the grayscale of the cth pixel. The larger the value, the more common the grayscale value of the cth pixel is, and the less likely it is an eggshell pixel. Indicates the grayscale difference of the c-th pixel in the t-th and t+1-th video frames, It represents the sum of the grayscale differences of the c-th pixel in the T adjacent video frames to the right of the z-th video frame. The larger the value is, the more unstable the grayscale of the c-th pixel is, which means the grayscale regularity is smaller, and then the eggshell grayscale index is lower.

[0050] It should be noted that the smaller the flow regularity of the pixel in T consecutive adjacent video frames and the larger the eggshell grayscale index, the stronger the eggshell performance of the pixel.

[0051] Preferably, the ratio of the eggshell gray index to the flow regularity is used as the eggshell performance of the pixel point, and the eggshell performance of the c-th pixel point in the right adjacent T adjacent video frames of the z-th video frame is recorded as .

[0052] At this point, the eggshell representation of each pixel in each video frame is obtained.

[0053] It should be noted that all pixels in the suspected eggshell area have a high eggshell performance, so the pixels can be clustered based on the eggshell performance. In addition, the pixels in the suspected eggshell area are adjacent, so the pixels can be clustered using the region growing algorithm.

[0054] Preferably, a region growing algorithm is used to cluster all pixel points of each video frame: all pixel points with eggshell performance greater than 0.5 are used as growth seed points, respectively as a growth region, and if there are adjacent growth regions, the adjacent growth regions are merged into one growth region; the difference between the average eggshell performance of the adjacent growth region and the growth region and less than 0.1 is used as a growth condition; the difference between all pixel points adjacent to the growth region and the average eggshell performance of the growth region is greater than or equal to 0.1 as a condition for stopping growth; complete regional growth, and obtain several growth regions of each video frame.

[0055] It should be noted that in each growth area of ​​each video frame, if the number of pixels is too small or the distance from the light area is large, the growth area may be noise; if the number of pixels is large and the distance from the light area is large, the growth area is more likely to be an eggshell. Therefore, the eggshell possibility of each growth area in each video frame can be obtained according to the size of the growth area and the distance from the light area.

[0056] Preferably, the eggshell probability of each growth area in each video frame satisfies the expression: ; In the formula, represents the eggshell probability of the h-th growth region in the z-th video frame; Represents the number of pixels in the hth growth area of ​​the zth video frame; represents the set of distances between the centroid of the hth growth region in the zth video frame and the centroids of all light regions; represents the minimum function; Represents the normalization function.

[0057] Preferably, the growth area whose eggshell possibility is greater than the first threshold is recorded as a suspected eggshell area.

[0058] So far, several suspected eggshell regions in each video frame have been obtained.

[0059] S4: Determine the video frame sequence where the real eggshell exists based on the shape change and position change of the suspected eggshell area in the continuous video frames.

[0060] It should be noted that S3 obtains the suspected eggshell area of ​​each video frame from the perspective of a single pixel and a single video frame. However, the characteristic of real-time eggshell flow video is that the eggshell as a whole will move coherently in the egg liquid. For any video frame, an eggshell may be completely on the surface, may be mostly on the surface, or may be close to falling completely. Therefore, intercepting the eggshell from its appearance to its complete fall can more accurately analyze the falling characteristics of the eggshell, thereby predicting the subsequent position of the eggshell.

[0061] It should be further explained that as the eggshell moves, there is a certain position overlap and a certain shape overlap of the eggshell in continuous video frames. Therefore, based on the positional relationship and shape relationship of the suspected eggshell area in continuous video frames, the video frames belonging to the same eggshell area can be merged to obtain a complete video frame sequence of the real eggshell.

[0062] Specifically, a set of pixel points of each suspected eggshell area of ​​each video frame is obtained, and a boundary pixel point sequence of each suspected eggshell area of ​​each video frame is obtained, wherein the starting position of the boundary pixel point sequence is the boundary pixel point with the smallest distance from the origin, and the sequence direction is clockwise.

[0063] Preferably, the possibility that any two suspected eggshell regions in any two adjacent video frames belong to the same eggshell includes: Obtaining the edit distance between the boundary pixel point sequence of the g-th suspected eggshell region of the z-th video frame and the boundary pixel point sequence of the g-th suspected eggshell region of the z+1-th video frame; ; In the formula, represents the possibility that the g-th suspected eggshell region in the z-th video frame and the g-th suspected eggshell region in the z+1-th video frame belong to the same eggshell; represents the pixel set of the g-th suspected eggshell region in the z-th video frame; represents the pixel set of the kth suspected eggshell region in the z+1th video frame; represents the edit distance between the boundary pixel sequence of the g-th suspected eggshell region in the z-th video frame and the boundary pixel sequence of the g-th suspected eggshell region in the z+1-th video frame; Represents an exponential function with a natural constant as its base.

[0064] In the formula, The ratio of the intersection and union of pixels of the g-th suspected eggshell region of the z-th video frame and the g-th suspected eggshell region of the z+1-th video frame, which indicates the overlap of the two suspected eggshell regions. The larger the value, the more overlapping pixels of the two suspected eggshell regions, which indicates that the possibility that the two suspected eggshell regions belong to the same eggshell is greater; the edit distance between the boundary pixel sequence of the g-th suspected eggshell region of the z-th video frame and the boundary pixel sequence of the g-th suspected eggshell region of the z+1-th video frame , it indicates the shape change of the two suspected eggshell regions. The larger the value, the greater the change in the boundary shape of the two suspected eggshell regions, and the smaller the possibility that they belong to the same eggshell.

[0065] Preferably, any suspected eggshell area is recorded as the target suspected eggshell area, and regional growth is performed on the target suspected eggshell area: the video frame where the target suspected eggshell area is located is used as the growth seed point; the suspected eggshell area with intersecting pixels in the left and right adjacent video frames, and the possibility that it belongs to the same eggshell as the target suspected eggshell area is greater than the second threshold value is used as the growth condition; the absence of any suspected eggshell area with intersecting pixels in the left and right adjacent video frames is used as the termination growth condition; several continuous video frames where the target suspected eggshell area exists are obtained; and a video frame sequence where the target suspected eggshell area exists is obtained. It should be noted that the second threshold value is set by the implementer according to the actual implementation situation, and there is no specific limitation. For example, the second threshold value can be set to 0.5.

[0066] It should be noted that only when there are more video frames with suspected eggshell areas can it be determined to be a real eggshell change process.

[0067] Preferably, a video frame sequence in which the number of video frames in the video frame sequence is greater than a third threshold and in which a suspected eggshell region exists is recorded as a video frame sequence in which a real eggshell exists.

[0068] So far, several video frame sequences with real eggshells have been obtained.

[0069] S5: predicting the movement path of the eggshell according to the position change of the eggshell in the video frame sequence where the real eggshell exists; and scooping the eggshell based on the movement path of the eggshell.

[0070] It should be noted that the eggshell may gradually sink, or may move on the surface of the egg liquid due to factors such as bubbles and buoyancy, so the position of the eggshell picked up by the control device is not the same, and the time distance of the eggshell detected is not the same as the time distance of the latest video frame. If the time distance of the eggshell detected is close to the time distance of the latest video frame, the predicted position of the eggshell and the position where the eggshell is detected will change slightly. At the same time, if the displacement of the eggshell in the existing video frame sequence is relatively small, the change of the predicted position of the eggshell picked up and the position in the video frame sequence will also be relatively small.

[0071] Specifically, the centroid coordinates of the eggshell area in each video frame in the video frame sequence in which each real eggshell exists are obtained, and the least square method is used to perform spatial straight line fitting to obtain the centroid coordinate fitting line of each real eggshell, which is recorded as the movement route of the real eggshell. On the movement route of the real eggshell, the centroid coordinates corresponding to the scooping time are marked as the predicted coordinates of the corresponding real eggshell.

[0072] After obtaining the video sequence of the real eggshell, the control center controls the scooping device to reach the surface of the egg liquid, and records the time when the scooping device reaches the surface of the egg liquid as the scooping time. On the movement route of the real eggshell, the center of mass coordinates corresponding to the scooping time are marked as the predicted coordinates corresponding to the real eggshell. The control center controls the scooping device to reach the predicted coordinates to complete the scooping of the eggshell.

[0073] At this point, the predicted coordinates of the real eggshell are obtained, and the eggshell is retrieved.

[0074] Although this specification has shown and described multiple embodiments of the present invention, it is obvious to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will think of many changes, modifications and alternatives without departing from the idea and spirit of the present invention.

[0075] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting eggshell in egg liquid, characterized in that: include: Acquire a real-time egg liquid flow video through a machine vision system, where the real-time egg liquid flow video includes a number of video frames; According to the grayscale value and grayscale change of the pixel point at the same position in different video frames in the real-time egg liquid flow video, the light area of ​​the real-time egg liquid flow video is obtained; According to the change of the optical flow field of adjacent video frames and the grayscale distribution of the pixels, the eggshell performance of each pixel in each video frame is obtained; based on the eggshell performance of each pixel in each video frame and the distance between each pixel and the light area of ​​the real-time egg liquid flow video, the suspected eggshell area of ​​each video frame is obtained; Determine the video frame sequence where the real eggshell exists according to the shape change and position change of the suspected eggshell area in the continuous video frames; According to the position change of the eggshell in the video frame sequence where the real eggshell exists, the movement path of the eggshell is predicted; The eggshells are scooped up based on their movement paths.

2. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The light area for obtaining the real-time egg liquid flow video includes: A three-dimensional coordinate system is established for the real-time egg liquid flow video, in which the XOY plane is parallel to the video frame plane and the Z axis is the time axis; pixel points with the same X-axis and Y-axis coordinates are recorded as the same group of pixel points; According to the distribution of the number of pixels of each gray value of the pixels in the same group, the light area performance of each pixel in the same group is obtained; A clustering algorithm is used to cluster the light zone performance of all groups of pixels in the same group to obtain several clusters of pixels in the same group. The clusters of pixels in the same group whose mean light zone performance is greater than the mean light zone performance of all groups of pixels in the same group are recorded as the light zone of the real-time egg liquid flow video.

3. The method for detecting eggshell in egg liquid according to claim 2, characterized in that: The light area performance of the same group of pixels includes: Obtain the grayscale values ​​of all pixels in the same group of pixels and count the number of pixels corresponding to each grayscale value; obtain the grayscale mean value of each pixel in the same group; The light area performance of any group of pixels in the same group satisfies the expression: ； In the formula, It represents the light area performance of the pixels in the same group of the i-th group; S represents the maximum gray value; s represents the gray value; It represents the number of pixels with gray value s in the same group of pixels in the i-th group; Represents the grayscale mean of all pixels in the same group.

4. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The method of obtaining the eggshell representation of each pixel in each video frame according to the change of the optical flow field of adjacent video frames and the grayscale distribution of the pixel, comprises: According to the change of the optical flow vector of the pixel point in the continuous adjacent video frames, the flow regularity of any pixel point in the continuous adjacent video frames is obtained; Based on the number distribution of pixels in the grayscale histogram and the grayscale change of pixels, the eggshell grayscale index of any pixel in consecutive adjacent video frames is obtained; The ratio of the eggshell grayscale index to the flow regularity is used as the eggshell representation of the pixel in consecutive adjacent video frames.

5. The method for detecting eggshell in egg liquid according to claim 4, characterized in that: The flow regularity of any pixel point in consecutive adjacent video frames satisfies the expression: ； In the formula, represents the flow regularity of the c-th pixel in the T adjacent video frames to the right of the z-th video frame; T represents the number of preset adjacent video frames to the right; represents the distance difference along the X axis of the c-th pixel in the t-th and t+1-th right adjacent video frames of the z-th video frame; represents the distance difference along the Y axis of the c-th pixel in the t-th and t+1-th right adjacent video frames of the z-th video frame; represents the absolute value function; Represents an exponential function with a natural constant as its base.

6. The method for detecting eggshell in egg liquid according to claim 4, characterized in that: The eggshell grayscale index of any pixel point in consecutive adjacent video frames satisfies the expression: ； In the formula, Represents the eggshell grayscale index of the c-th pixel in the T adjacent video frames to the right of the z-th video frame; Indicates that the cth pixel is in The number of pixels in the corresponding grayscale histogram of the grayscale value of the video frame; Indicates the number of pixels in the X-axis direction of the video frame. Indicates the number of pixels in the Y-axis direction of the video frame. Indicates the number of pixels in the video frame; T represents the number of preset adjacent video frames adjacent to the right; , Indicates the grayscale value of the c-th pixel moving to the t-th and t+1-th adjacent video frames to the right of the z-th video frame.

7. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The method of obtaining the suspected eggshell region of each video frame based on the eggshell performance of each pixel point in each video frame and the distance between each pixel point and the light area of ​​the real-time egg liquid flow video comprises: Based on the distance and eggshell performance, all the pixels of each video frame are clustered to obtain several growth regions of each video frame; The minimum value of the distance between the centroid of each growth area and the centroid of all light areas of the real-time egg liquid flow video is obtained, and multiplied by the number of pixels in the corresponding growth area to obtain the eggshell possibility of each growth area; The growth area with eggshell possibility greater than the first threshold is recorded as a suspected eggshell area.

8. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The method of determining the video frame sequence where the real eggshell exists according to the shape change and position change of the suspected eggshell area of ​​the continuous video frames comprises: Obtain a set of pixel points of each suspected eggshell region of each video frame, and obtain a sequence of boundary pixel points of each suspected eggshell region of each video frame; Based on the positional relationship and shape relationship of the suspected eggshell regions in the continuous video frames, the possibility that any two suspected eggshell regions in any two adjacent video frames belong to the same eggshell is obtained; The video frames belonging to the same eggshell are clustered by a clustering algorithm to obtain a video frame sequence with suspected eggshell areas, and the video frame sequence with suspected eggshell areas whose number of clustered video frames is greater than a third threshold is recorded as a video frame sequence with real eggshells.

9. The method for detecting eggshell in egg liquid according to claim 8, characterized in that: The possibility that any two suspected eggshell regions in any two adjacent video frames belong to the same eggshell includes: Get the edit distance between the boundary pixel sequence of the g-th suspected eggshell region in the z-th video frame and the boundary pixel sequence of the g-th suspected eggshell region in the z+1-th video frame, and record it as ; Compare the pixel intersection and pixel union of the g-th suspected eggshell region in the z-th video frame with the g-th suspected eggshell region in the z+1-th video frame, and The negative correlation normalized value of is multiplied by the g-th suspected eggshell region in the z-th video frame and the g-th suspected eggshell region in the z+1-th video frame belong to the same eggshell.

10. The method for detecting eggshell in egg liquid according to claim 1, characterized in that: The method predicts the movement path of the eggshell according to the position change of the eggshell in the video frame sequence where the real eggshell exists, including: The centroid coordinates of the eggshell area in each video frame in the video frame sequence of each real eggshell are obtained, and the least squares method is used to perform spatial straight line fitting to obtain the centroid coordinate fitting line of each real eggshell, which is recorded as the movement path of the real eggshell.

Images