Egg shape index extraction method and system based on binocular structured light

Abstract

The present application relates to the egg shape index extraction method and system based on binocular structured light. The egg shape index extraction method based on binocular structured light, the method comprises the following steps: projecting a plurality of groups of different frequency phase shift fringe patterns to the surface of the object to be measured; synchronously acquiring the fringe image data of the object surface through at least two image acquisition devices, and performing stereo correction to obtain corrected image data; calculating the wrapped phase of the fringe image in the image data based on the phase shift method, obtaining the absolute phase through the multi-frequency phase unwrapping algorithm, and reconstructing the three-dimensional point cloud data of the object combined with the stereo vision principle; the egg shape index extraction method and system based on binocular structured light, through the optimized binocular structured light technology and point cloud analysis algorithm, the precision, efficiency and robustness of the shape index detection of eggs and other ellipsoids are significantly improved, and the technical universality and industrial application potential are also considered.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of egg shape index detection, and particularly relates to an egg shape index extraction method and system based on binocular structured light. BACKGROUND

[0002] Egg quality detection is a key link in the process of egg commercialization, and the egg shape index, as an important indicator of egg shape, directly affects the sorting efficiency and product value. In the prior art, computer vision-based detection methods are mainly divided into two categories: two-dimensional image analysis and three-dimensional reconstruction.

[0003] In the field of two-dimensional image detection, the technical solution disclosed in patent CN202410219252 obtains a two-dimensional image of an egg through a single shot, and extracts the egg shape index based on image features. Although this method simplifies the operation process, it relies on two-dimensional projection information and is easily disturbed by factors such as camera shooting angle and uneven lighting, resulting in significant measurement errors. For example, when the egg is slightly tilted, the two-dimensional image cannot accurately reflect the true three-dimensional profile, and the egg shape index calculation result may have a deviation of more than 10%, which is difficult to meet the high-precision sorting requirements.

[0004] To overcome the limitations of two-dimensional technology, three-dimensional reconstruction methods have been gradually introduced. Patent CN202510158786 proposes a three-dimensional reconstruction scheme based on binocular vision, which obtains egg surface point cloud data through multi-view shooting and calculates the egg shape index. Although this method improves the measurement accuracy (error of about 0.5 mm), it requires multiple adjustments of the camera or egg position to collect multi-view images, resulting in a long detection time for a single sample and low efficiency, which cannot meet the high-throughput requirements of industrialized flow lines. In addition, multi-view shooting has high requirements for equipment synchronization and environmental stability, increasing the system complexity and cost.

[0005] In recent years, structured light technology has been applied to three-dimensional detection due to its high precision and non-contact characteristics. However, traditional binocular structured light systems usually need to project multiple high-frequency stripes and use complex phase unwrapping algorithms, which have the problems of large data processing amount and poor real-time performance. At the same time, existing schemes rely on fixed threshold filtering or simple geometric fitting in point cloud processing, which lack robustness to local depressions on the surface, environmental light interference and other abnormal situations, limiting their practical application range.

[0006] Therefore, there is an urgent need for a three-dimensional detection method that balances high precision, high efficiency and strong robustness, which can quickly reconstruct the surface topography of an egg under single-shot conditions and accurately extract the shape index, to meet the stringent requirements of agricultural sorting and industrial quality inspection. SUMMARY

[0007] The egg shape index extraction method and system based on binocular structured light provided by the present application have simple structure and reasonable design.

[0008] The present application achieves the above-mentioned purpose by the following technical solutions:

[0009] The first aspect of the present application provides an egg shape index extraction method based on binocular structured light, which comprises,

[0010] A plurality of groups of phase shift fringe patterns with different frequencies are projected onto the surface of the object to be measured.

[0011] The fringe image data of the object surface is synchronously acquired by at least two image acquisition devices, and stereoscopic correction is performed to obtain corrected image data.

[0012] The wrapped phase of the fringe image in the image data is calculated based on the phase shift method, the absolute phase is obtained by a multi-frequency phase unwrapping algorithm, and the three-dimensional point cloud data of the object is reconstructed in combination with the principle of stereovision.

[0013] A point cloud subset of the target region is extracted from the three-dimensional point cloud data, a reference plane is fitted through spatial geometric analysis, and the original point cloud is projected to a local coordinate system defined by the reference plane.

[0014] The candidate point set along the target direction is screened on the projection plane, the first shape dimension of the object is calculated, and after the projection plane is translated based on the normal vector direction of the reference plane, the second shape dimension is calculated.

[0015] The shape index of the object is generated according to the first shape dimension and the second shape dimension.

[0016] The shape index of the object includes an egg shape index.

[0017] As a further optimization scheme of the present application, the frequency range of the plurality of groups of phase shift fringe patterns is 1 / 70 to 1 / 45, and the adjacent frequency difference is not less than 1 / 100.

[0018] As a further optimization scheme of the present application, the plurality of groups of phase shift fringe patterns are three groups of fringe patterns, and the selected frequencies are 1 / 64, 1 / 56 and 1 / 49, respectively.

[0019] As a further optimization scheme of the present application, the light intensity expression of the phase shift fringe pattern is: ;

[0020] Wherein, N=12, n=0, 1, 2,…, N-1.

[0021] As a further optimization solution of the present invention, the image acquisition device is a CCD camera; the resolution of the CCD camera is 2592×1944 pixels.

[0022] As a further optimization solution of the present invention, the wrapping phase is calculated by the following formula: ;in, is the intensity value of the stripes in the nth group; N=12.

[0023] As a further optimization solution of the present invention, a point cloud subset of the target area is extracted from the three-dimensional point cloud data: Among them, Z i is the Z coordinate value of the point cloud, Z min is the minimum Z coordinate value, and t is the threshold value, which ranges from 0.03 to 0.07.

[0024] As a further optimization solution of the present invention, the spatial geometric analysis is PCA analysis, the normal vector of the reference plane is determined by the third principal component direction, and the original point cloud is aligned to the local coordinate system through a rotation matrix.

[0025] As a further optimization solution of the present invention, the target direction is the X-axis direction of the projection plane, and the screening condition of the candidate point set is the polar angle range:

[0026] θ i ∈[-θ tol ,θ tol ]∪[π-θ tol ,π+θ tol ]; where θ tol The value range is 3° to 10°.

[0027] The second aspect of the present application provides an egg shape index extraction system based on binocular structured light, which implements the above-mentioned egg shape index extraction method, including:

[0028] a structured light projection module for generating multi-frequency phase-shifted fringe patterns;

[0029] A multi-view image acquisition module, comprising at least two synchronously triggered industrial cameras;

[0030] The data processing module is configured to perform three-dimensional reconstruction, point cloud analysis, and shape index calculation.

[0031] The beneficial effects of the present invention are: through optimized binocular structured light technology and point cloud analysis algorithm, the present invention significantly improves the accuracy, efficiency and robustness of the detection of the shape indicators of eggs and other ellipsoids, while taking into account the technical versatility and industrial application potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic diagram of the steps of the egg shape index extraction method based on binocular structured light of the present application;

[0033] Figure 2 is a schematic diagram of the experimental device of the present application;

[0034] Figure 3 is a software calculation simulation view of the present application. DETAILED DESCRIPTION

[0035] The present application will be further described in detail below with reference to the accompanying drawings, and it is necessary to point out here that the following detailed description is only used to further illustrate the present application, and cannot be understood as limiting the scope of protection of the present application, and those skilled in the art can make some non-essential improvements and adjustments to the present application according to the above application content.

[0036] Example 1

[0037] Reference Figure 1 The method flow shown, the egg shape index extraction method based on binocular structured light, the method comprises,

[0038] projecting a plurality of groups of different frequency phase shift fringe patterns onto the surface of the object to be measured;

[0039] synchronously acquiring the fringe image data of the object surface by at least two image acquisition devices, and performing stereo correction to obtain corrected image data;

[0040] calculating the wrapped phase of the fringe image in the image data based on the phase shift method, obtaining the absolute phase by the multi-frequency phase unwrapping algorithm, and combining the stereo vision principle to reconstruct the three-dimensional point cloud data of the object;

[0041] extracting a point cloud subset of the target region from the three-dimensional point cloud data, fitting a reference plane through spatial geometric analysis, and projecting the original point cloud to the local coordinate system defined by the reference plane;

[0042] screening the candidate point set along the target direction on the projection plane, calculating the first shape dimension of the object, and after translating the projection plane based on the normal vector direction of the reference plane, calculating the second shape dimension;

[0043] generating the shape index of the object according to the first shape dimension and the second shape dimension;

[0044] The shape index of the object includes the egg shape index.

[0045] It should be noted that the method realizes high-precision and high-efficiency three-dimensional reconstruction by combining structured light and stereo vision, and is particularly suitable for detection of ellipsoidal or symmetrical objects (such as eggs and industrial parts) with smooth surfaces and no texture. In the present application, the egg shape index is understood in a broad sense, which can be understood as an egg shape index in practice, and is used to express the ellipticity or symmetry of an object. In the present embodiment, the object is taken as an egg in the egg category as an example for illustration.

[0046] In the present embodiment, the multiple sets of phase shift stripe patterns are three sets of stripe patterns, and the selected frequencies are 1 / 64, 1 / 56 and 1 / 49, respectively.

[0047] The light intensity expression of the phase shift stripe pattern is: ;

[0048] wherein N=12, n=0, 1, 2,…, N-1; wherein (x,y) represents the coordinates of a pixel in the stripe pattern; I(x,y) represents the light intensity on the pixel with coordinates (x,y); A(x,y) represents the background light intensity on the pixel with coordinates (x,y); and B(x,y) represents the modulation light intensity on the pixel with coordinates (x,y). is the wrapped phase to be solved in the phase unwrapping process; n represents the stripe pattern of the nth phase shift; and N is the number of phase shifts required by the phase shift method.

[0049] The image acquisition device is a CCD camera, and the left and right two CCD cameras synchronously acquire the pictures of the egg under the illumination mode of the stripe pattern, save the image data, then perform epipolar correction on the acquired pictures, and save the corrected images. In the present example, the camera shooting is controlled by a computer, and the stripe projection is controlled by a single-chip microcomputer connected with the computer. The image acquisition is completed in a completely light-shielded environment, the projector is the main light source, and light-absorbing materials are arranged inside the field of view of the projector and the camera to reduce the interference of ambient light on the decoding of the structured light. The egg is placed horizontally on the object table during acquisition; and the resolution of the CCD camera is 2592x1944Pixel.

[0050] Specifically, the wrapped phase is calculated by the phase shift method, the phase is unwrapped by the multi-frequency heterodyne method to obtain the absolute phase, and then the three-dimensional point cloud map is obtained according to the binocular structured light imaging principle. The wrapped phase is calculated by the following formula: ; wherein, is the intensity value of the nth set of stripes; and N=12.

[0051] The multi-frequency heterodyne method here is a multi-frequency phase unwrapping algorithm, which aims to eliminate the periodic jump of the wrapped phase and generate continuous absolute phase. Different frequency (stripe density) stripe patterns (such as high frequency, medium frequency and low frequency) are projected.

[0052] Phase is calculated by frequency difference:

[0053] High frequency fringe: provides high precision but is susceptible to noise;

[0054] Low frequency fringe: phase changes slowly, used to assist the unwrapping of high frequency phase;

[0055] Phase unwrapping formula: calculate absolute phase by Multi-Frequency Heterodyne: (k is determined by low frequency phase).

[0056] That is, if stripes with frequencies f1, f2, f3 are used, a virtual low frequency fringe f h =f1-f2, combined with f l =f1-f2, can gradually calculate the global phase.

[0057] Among them, the method for generating three-dimensional coordinates by binocular structured light imaging principle is prior art, which will not be repeated here.

[0058] Further, a point cloud subset of a target region is extracted from the three-dimensional point cloud data: ; Wherein, Z i is the point cloud Z coordinate value, Z min is the minimum Z coordinate value, and t is a threshold value, which is in the range of 0.03 to 0.07; In this embodiment, t is 0.05.

[0059] After the effective point cloud basePoints, PCA analysis is performed to obtain the eigenvector matrix M, wherein the third principal component is the plane normal vector n=(a, b, c), representing the vertical direction of the plane; The centroid C=(x c , y c , z c ) is the geometric center of the point cloud, which is used to locate the plane position, and the plane equation is fitted as: a(x-x c )+b(y-y c )+c(z-z c )=0.

[0060] Then a rotation matrix R is constructed to project the original point cloud to the fitted plane. The rotation matrix R is used to rotate the original coordinate system to a local coordinate system with the fitted plane as the reference, so that the normal vector of the plane is aligned with the Z axis of the new coordinate system. The rotation matrix R is composed of three principal components, whose column vectors are the X, Y and Z axis directions of the new coordinate system. The projected point cloud can be obtained by the following formula:

[0061] P proj = (P original - C)× R;

[0062] where P proj is the projected point cloud, P original is the original point cloud. Finally, set the Z coordinate of the projected point cloud to 0, ensuring that the result is a two-dimensional plane.

[0063] Figure 3 Simulate the view for software calculation. Afterwards, set the angle tolerance threshold θ tol , which defines the maximum angular deviation allowed to deviate from the theoretical X-axis direction. According to the actual measurement requirements, the threshold is set to 5°, and converted to radian value θ range = θ tol × π / 180. For all points p pro = (x i , y i ) on the projection plane and the projection geometric center c pro = (x c , y c ), calculate the relative coordinate offset: Δx i = x i - x c , Δy i = y i - y c ;

[0064] Calculate the polar angle: θ i = arctan2(Δy i , Δx i );

[0065] where θ i ∈ (-π, π], represents the directional angle of the point relative to the geometric center. Then filter the positive / negative X-axis direction candidate points, the positive X-axis direction region restricts the polar angle in the range [-θ tol , θ tol ], and the negative X-axis direction region restricts the polar angle in the range [π-θ tol , π+θ tol ]. Then calculate the maximum distance from the geometric center from the positive / negative direction candidate points:

[0066] .

[0067] Finally, take the larger value of the maximum radius of the positive / negative direction, calculate the final diameter D (short axis of the egg): D = 2*max(d 正 , d 负 ).

[0068] Based on the plane normal vector n and the pre-calculated maximum radius D / 2, the fitting plane is translated along the normal direction, and the translation distance is equal to the maximum radius of the target geometry, which ensures that all point clouds can be projected vertically to the target plane. Then, the original three-dimensional point cloud of the egg is projected to this plane, and the Y coordinates of all points in the projected point cloud P' are extracted , and the Y direction extreme difference Ay is calculated as the long axis of the egg: .

[0069] The method takes manual measurement as a sample, and the experimental comparison results are as follows:

[0070] Based on the above experimental results, it can be seen that the measurement accuracy of the method is high and consistent with manual measurement. The long axis measurement result of the method compared with manual measurement has an error mean of 0.18 mm (maximum error 0.54 mm), and t test (p=0.32) verifies that there is no significant difference. The short axis error mean is 0.15 mm (maximum error 0.58 mm), p=0.45, indicating that the measurement result has high reliability; the error standard deviation of the egg shape index is ≤0.02 (such as sample 1, both manual and system are 1.28), which verifies the reliability of the shape quantization index.

[0071] Embodiment 2

[0072] Referring to the structure shown in the figure, a double binocular structured light-based egg shape index extraction system implements the above egg shape index extraction method, comprising: Figure 2 A structured light projection module is configured to generate a multi-frequency phase shift fringe pattern.

[0073] A multi-view image acquisition module includes at least two synchronously triggered industrial cameras.

[0074] A data processing module is configured to perform three-dimensional reconstruction, point cloud analysis, and shape index calculation.

[0075] Among them, the structured light projection module is the projector 1 in

[0076] , and the multi-view image acquisition module is the right camera 2 and the left camera 3, Figure 2 in Figure 2 also includes a stage 4.

[0077] It should be noted that the double binocular structured light-based egg shape index extraction method and system significantly improve the accuracy, efficiency and robustness of the shape index detection of eggs and other ellipsoids by optimizing the double binocular structured light technology and the point cloud analysis algorithm, while taking into account the technical universality and industrial application potential.

[0078] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, different embodiments or examples described in the specification and the features of different embodiments or examples can be combined and combined by those skilled in the art without contradiction.

[0079] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

[0080] The above-described embodiments only express several implementation manners of the present application, which are described in detail and specifically, but should not be understood as limiting the scope of the patent of the present application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, which are within the protection scope of the present application.

Claims

1. A method for extracting an egg shape index based on binocular structured light, characterized in that, The method comprises, projecting a plurality of groups of phase-shifted fringe patterns with different frequencies onto the surface of the object to be measured; synchronously acquiring fringe image data of the surface of the object by at least two image acquisition devices, and performing stereoscopic correction to obtain corrected image data; calculating wrapped phases of fringe images in the image data based on the phase-shifting method, obtaining absolute phases by a multi-frequency phase-unwrapping algorithm, and combining the stereoscopic vision principle to reconstruct three-dimensional point cloud data of the object; extracting a point cloud subset of a target region from the three-dimensional point cloud data, fitting a reference plane through spatial geometric analysis, and projecting the original point cloud to a local coordinate system defined by the reference plane; screening a candidate point set along a target direction on the projection plane, calculating a first outer shape dimension of the object, and calculating a second outer shape dimension after translating the projection plane based on the normal vector direction of the reference plane; generating an outer shape index of the object according to the first outer shape dimension and the second outer shape dimension; the outer shape index of the object comprises an egg shape index; the first outer shape dimension is egg shape minor axis data, the second outer shape dimension is egg shape major axis data, and the egg shape index is a ratio of the egg shape major axis data; extracting a point cloud subset of a target region from the three-dimensional point cloud data: ; Wherein, Z i is the point cloud Z coordinate value, Z min is the minimum Z coordinate value, and t is a threshold value, the value range of which is 0.03 to 0.07; the spatial geometric analysis is PCA analysis, the normal vector of the reference plane is determined by the third principal component direction, and the original point cloud is aligned to the local coordinate system through a rotation matrix; the target direction is the X-axis direction of the projection plane, and the screening condition of the candidate point set is the polar angle range: θ i ∈[−θ tol , θ tol ∪ [π − θ tol , π + θ tol ; where θ tol ranges from 3° to 10°; then, the maximum distance from the candidate points in the positive and negative directions to the geometric center is calculated: ； finally, the larger value of the maximum radii in the positive and negative directions is taken as the final diameter D, i.e., the egg shape minor axis data: D = 2 * max(d 正 , d 负 ); Based on the plane normal vector n and the pre-calculated maximum radius D / 2, the fitting plane is translated along the normal direction, and the translation distance is equal to the maximum radius of the target geometry, which ensures that all point clouds can be projected vertically and horizontally to the target plane. Then, the original three-dimensional point cloud of the object to be measured is projected onto this plane, and the Y coordinates of all points in the projected point cloud P' are extracted The extreme difference Ay in the Y direction is calculated as the long axis data of the egg shape: 。 2. The binocular structured light based egg shape index extraction method according to claim 1, characterized in that: the frequency range of the plurality of groups of phase-shifted fringe patterns is 1 / 70 to 1 / 45, and the difference between adjacent frequencies is not less than 1 / 100.

3. The binocular structured light based egg shape index extraction method according to claim 2, characterized in that: The plurality of groups of phase-shifted fringe patterns are three groups of fringe patterns, and the selected frequencies are 1 / 64, 1 / 56 and 1 / 49 respectively.

4. The binocular structured light based egg shape index extraction method according to any one of claims 1-3, characterized in that: The light intensity expression of the phase-shifted fringe pattern is: ; where N = 12, n = 0, 1, 2, …, N-1; where (x, y) represents the coordinates of a pixel in the fringe pattern; I(x, y) represents the light intensity on the pixel with coordinates (x, y); A(x, y) represents the background light intensity on the pixel with coordinates (x, y); B(x, y) represents the modulation light intensity on the pixel with coordinates (x, y); φ(x, y) is the wrapped phase to be solved in the phase unwrapping process; n represents the nth phase-shifted fringe pattern; and N is the number of phase shifts required for the phase shifting method.

5. The binocular structured light based egg shape index extraction method according to claim 1, wherein: the image acquisition device is a CCD camera; and the resolution of the CCD camera is 2592*1944 Pixel.

6. The binocular structured light based egg shape index extraction method according to claim 4, characterized in that: The wrapped phase is calculated by the following formula: ； wherein, Intensity value of the nth stripe of the nth group; N = 12.

7. A binocular structured light based egg shape index extraction system, implementing the egg shape index extraction method as claimed in any one of claims 1-6, characterized in that, The system comprises: a structured light projection module for generating a multi-frequency phase-shifted fringe pattern; a multi-view image acquisition module comprising at least two synchronously triggered industrial cameras; a data processing module configured to perform three-dimensional reconstruction, point cloud analysis and outer shape index calculation.

Images