A fruit and vegetable laser three-dimensional marking method and device

Abstract

The application provides a fruit and vegetable laser three-dimensional marking method, which is based on the encoder of a conveying device servo motor to obtain conveying speed in real time, combines the geometric relationship between a field lens and the center of a fruit and vegetable contour to calculate a delay time; acquires and fits fruit and vegetable image data through a three-dimensional camera, combines marking information to judge the relationship between the height difference of a marking area and the focal depth of a laser, adopts different projection methods to generate fruit and vegetable laser three-dimensional marking instructions, and sends the instructions to a laser and a three-dimensional galvanometer for flight marking. The application also provides a fruit and vegetable laser three-dimensional marking device, which comprises an industrial computer, a three-dimensional camera, a conveying device, a laser, a three-dimensional galvanometer and a field lens, a servo motor is arranged on the conveying device, the three-dimensional camera and the laser are respectively arranged above the front and rear ends of the conveying device, and the three-dimensional camera, the conveying device, the laser and the three-dimensional galvanometer are connected with the industrial computer. The application can effectively improve the marking efficiency and precision, and is suitable for the automatic marking operation of fruit and vegetable products.

Description

Technical Field

[0001] This invention relates to laser marking technology, specifically a method and apparatus for three-dimensional laser marking of fruits and vegetables, which integrates three-dimensional surface reconstruction and flying marking technology, enabling high-precision and low-damage marking processing on the surface of fruits and vegetables. Background Technology

[0002] Traditional labeling of fresh produce often uses self-adhesive labels, which pose a risk of adhesive residue contamination, affecting the product's appearance, and the labels are prone to falling off or being damaged. During cold chain transportation, self-adhesive labels may detach due to surface condensation, directly impacting the efficiency of product traceability management.

[0003] Existing laser planar etching technology has insufficient tolerance for depth of focus. Fruits and vegetables have large radii of curvature, and when this depth of focus is exceeded, localized defocusing occurs on the surface, leading to pattern stretching distortion and blurring. Traditional static laser marking is inefficient, with processing times for a single fruit or vegetable reaching 3-5 seconds, which is insufficient to meet the speed requirements of large-scale automated marking. Therefore, this invention proposes a three-dimensional marking method combining machine vision and laser flying marking technology to improve the efficiency and accuracy of three-dimensional marking on fruit and vegetable surfaces, which has broad application prospects in the agricultural field. Summary of the Invention

[0004] The present invention aims to provide an efficient, accurate and environmentally friendly method and device for three-dimensional marking on the surface of fruits and vegetables, so as to improve production efficiency and marking quality, and solve the problems of low efficiency and low accuracy in traditional marking technology.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution.

[0006] A method for laser three-dimensional marking of fruits and vegetables includes: The two-dimensional pattern to be marked is input into the industrial control computer, and its shape and type are analyzed.

[0007] Three-dimensional data of the fruit and vegetable surface is acquired using a 3D camera, generating corresponding 3D point cloud data. When a change in height from low to high exceeds a fruit and vegetable height threshold, it is determined that the area has entered the fruit and vegetable zone.

[0008] For the acquired 3D point cloud, spherical fitting or freeform surface fitting methods are used. First, the surface is smoothed using the moving least squares method, and then the Gaussian curvature of each point is calculated. K With mean curvature H Statistical mean of the curvature of the entire surface m with standard deviation s If the standard deviation of curvature s K ≈0 and m H If stable, then perform spherical / ellipsoidal fitting; if... s K If the curvature is large and there are significant abrupt changes in the region, then freeform surface fitting is performed.

[0009] For spherical / ellipsoidal surfaces, the equations of the quadratic surfaces are solved using a non-iterative fast ellipsoid parameter estimation method with least-squares closed-fit fitting. Axe 2 + By 2 + Cz 2 + Dxy + Exz + Fyz + Gx + Gy + Hz + J =0, calculate the fitting residual. e .like e If the value is less than 0.05 mm, it is considered an effective ellipsoid fit.

[0010] For freeform surfaces, non-uniform B-spline fitting is used, the density of control points is adjusted based on curvature gradient weighting, and the surface smoothness is constrained by energy optimization method.

[0011] Calculate the depth of focus and determine the height difference of the marked area. Based on the laser spot diameter... 2h 0 ,wavelength l and the beam quality factor of the laser M 2 According to the formula Calculate the depth of focus DOF Extract the highest point of the point cloud data within the marked area. Z max and the lowest point Z min Calculate the height difference Δh=Z max - Z min .

[0012] like Δh≤DOF The local coordinate system projection method is adopted. The local coordinate system is directly established with the centroid of the marked area as the origin. The coordinates of the two-dimensional pattern are aligned with the corresponding points of the curved surface. The Z-axis coordinate height of the center point of the fruit and vegetable surface is fixed as the focal plane height to generate a static marking path.

[0013] like Δh>DOF First, the marked area is divided into meshes. Based on the changes in point cloud density and surface curvature, the marked area is divided into multiple sub-meshes, ensuring that the height difference Δ within each mesh is within a certain range. h i ≤DOFNext, the curvature adaptive normal vector is calculated. For each mesh vertex, a local plane is fitted with its neighborhood points, and the normal vector is extracted through the eigenvector corresponding to the minimum eigenvalue of the covariance matrix. n Cubic spline interpolation is performed at the center point of the grid to obtain a smooth surface normal vector field. Finally, dynamic focusing projection is performed along the normal vector. n Direction will be the two-dimensional pattern point (x,y) Project onto the curved surface, adjust the laser focus Z-axis position to... ,in, , Z 0 The location of the focus. d 0 To determine the distance between each position and the focal position, a three-dimensional galvanometer is used to ensure that the laser on the surface of the marking area is always at the focal plane position, thus generating a dynamic marking path.

[0014] When the 3D camera detects the outline of the first complete fruit or vegetable, a timing marker is triggered; this is combined with the real-time speed of the conveyor device. V Calculate the minimum bounding box of the fruit and vegetable outline, using the distance between the center of the bounding box and the field lens as... L Calculate the delay mark time T .

[0015] Based on the damage threshold of fruit and vegetable surfaces, the laser energy density parameters are adjusted by changing the scanning speed, scanning interval, number of scans, laser frequency, and laser energy. Combined with static / dynamic marking paths, three-dimensional laser marking instructions for fruits and vegetables are generated, and the laser and three-dimensional galvanometer are controlled to work together to achieve three-dimensional marking on the surface of fruits and vegetables.

[0016] A laser three-dimensional marking device for fruits and vegetables, comprising: The system includes an industrial computer, a 3D camera, a conveyor, a servo motor, a slide module, a laser, a 3D galvanometer, and a field lens. The industrial computer is equipped with a fruit and vegetable flying marking system. The conveyor is equipped with a servo motor. The 3D camera is located above the front end of the conveyor. The laser is located above the rear end of the conveyor. The field lens is connected to the 3D galvanometer and is mounted on the slide module. Together with the laser, the field lens forms an optical path system. The 3D galvanometer, laser, servo motor, and 3D camera are all connected to the industrial computer.

[0017] The industrial control computer is equipped with a fruit and vegetable flying marking system, which is responsible for processing the fruit and vegetable image information collected by the 3D camera, parsing the marking information and generating fruit and vegetable laser 3D marking instructions, which are then sent to the laser and 3D galvanometer.

[0018] The 3D camera is used to acquire 3D image data of fruits and vegetables and transmit the data to the industrial control computer.

[0019] The servo motor of the conveying device has an encoder, which is used to detect the transmission speed of the conveying device and feed it back to the industrial control computer to ensure that the fruits and vegetables accurately enter the marked area.

[0020] After acquiring the contour data of the first fruit and vegetable using a 3D camera, its minimum bounding box is calculated, and the coordinates of the center point of each fruit and vegetable's minimum bounding box on the conveyor device are recorded. The physical distance between the center point of the bounding box and the field lens is then used to determine the coordinates. L Compensation amount for projected fruit and vegetable size L 0 The speed of the conveyor V and system communication errors Δt The time it takes for the fruits and vegetables to move at a constant speed to the bottom of the flight marking system is calculated. T=(LL 0 ) / V+Δt Delay time T Then, the marking system will mark the fruits and vegetables that enter the field lens area.

[0021] Optionally, when encountering fruits and vegetables, if the height data of the 3D camera changes from low to high and the extreme difference between the high and low values ​​is greater than the preset fruit and vegetable height threshold, it indicates that the 3D camera has detected fruits and vegetables; if the height data of the 3D camera changes from high to low and the lowest value reaches the installation height of the 3D camera, it indicates that the 3D camera has acquired the first complete fruit and vegetable outline data.

[0022] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0023] Breakthroughs have been achieved through three-dimensional curved surface marking technology and high-speed flight marking technology. Based on the normal vector projection technology of three-dimensional curved surface and the block grid dynamic focusing technology, the focal depth limitation has been broken through, eliminating the pattern stretching caused by the curvature of fruits and vegetables, and realizing three-dimensional curved surface marking. By using dynamic flight focusing technology, combined with the real-time synchronization of three-dimensional galvanometer and conveying device, focus and position compensation is completed during the movement of fruits and vegetables, which improves efficiency and accuracy compared with traditional static marking, and realizes the customization and accuracy of information labeling. Attached Figure Description

[0024] Figure 1 The diagram shows the overall steps of the laser three-dimensional marking method for fruits and vegetables according to the present invention; Figure 2 The flowchart shown is a process for the laser three-dimensional marking method for fruits and vegetables according to the present invention. Figure 3 The diagram shown illustrates the process of projecting a two-dimensional pattern onto a three-dimensional curved surface according to the present invention. Figure 4 The diagram shown is a hardware assembly diagram of the fruit and vegetable laser three-dimensional marking device of the present invention; Figure reference numerals: 410-Laser, 420-3D galvanometer, 430-Field lens, 440-Conveying device, 450-Industrial computer, 460-3D camera, 470-Slide module, 480-Servo motor, L1-Projection method beyond the depth of focus, L2-Projection method within the depth of focus. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0026] Example 1 This embodiment provides a method for laser three-dimensional marking of fruits and vegetables, such as... Figure 1 As shown.

[0027] Step 110: The user enters the marking information as needed, including two-dimensional patterns on the surface of fruits and vegetables (such as vector graphics, bitmaps, text information, and QR codes).

[0028] Step 120: Activate the 3D camera to acquire 3D data of the fruit and vegetable surface. The 3D camera performs noise reduction on the point cloud data, statistically analyzes and removes outliers that deviate from the mean by more than two standard deviations; and performs coordinate alignment, registering continuous frame point clouds with the conveyor belt plane as the reference. In addition, the system also acquires image information such as the height, position, and movement speed of the fruits and vegetables. When a change in height exceeding a fruit and vegetable height threshold is detected, it is determined that the system has entered the fruit and vegetable area.

[0029] Step 130: After smoothing the surface of the acquired 3D point cloud using the moving least squares method, calculate the Gaussian curvature of each point. K With mean curvature H Statistical mean of the curvature of the entire surface m with standard deviation s If satisfied s K ≈0 and m H If stable, then perform spherical / ellipsoidal fitting; if... s K If the curvature is large and there are significant abrupt changes in the region, then freeform surface fitting is performed.

[0030] For spherical / ellipsoidal surfaces, the equations of the quadratic surfaces are solved using a non-iterative fast ellipsoid parameter estimation method with least-squares closed-fit fitting. Axe 2 + By 2 + Cz 2 + Dxy + Exz + Fyz + Gx + Gy + Hz + J =0, calculate the fitting residual. e .like e If the value is less than 0.05 mm, it is considered a valid ellipsoidal fit. For freeform surfaces, non-uniform B-spline fitting is used, the control point density is adjusted based on curvature gradient weighting, and the surface smoothness is constrained by energy optimization.

[0031] Step 140: Calculate the depth of focus and determine the height difference of the marked area. Based on the laser spot diameter... 2h 0 ,wavelength l and the beam quality factor of the laser M 2 According to the formula Calculate the depth of focus DOF Extract the highest point of the point cloud data within the marked area. Z max and the lowest point Z min Calculate the height difference Δh=Z max - Z min .

[0032] like Δh≤DOF The local coordinate system projection method is adopted, and a local coordinate system is directly established with the centroid of the marked area as the origin. The coordinates of the two-dimensional pattern are aligned with the corresponding points of the curved surface, and the z-axis coordinate is fixed as the focal plane height to generate a static marking path.

[0033] like Δh>DOF First, the marked area is meshed. Based on the changes in point cloud density and surface curvature, the marked area is divided into multiple sub-mesh (the smallest meshing unit has a side length of...). l satisfy , i (For the average inclination angle of the surface section), ensuring the height difference within each grid. Dh i ≤DOF Centered on the grid vertices, r Within the neighborhood of a 2mm sphere, select 50 points. For each grid vertex, use its neighboring points (radius) r=2*l By fitting a local plane to a set of interior points, we can find the minimum eigenvalue of the covariance matrix; the corresponding vector is the normal vector. n Cubic spline interpolation is performed at the center point of the grid to obtain a smooth surface normal vector field. Finally, dynamic focusing projection is performed along the normal vector. n Direction will be the two-dimensional pattern point (x,y) Project onto the curved surface, adjust the laser focus Z-axis position to... ,in, , Z 0 The location of the focus. d 0 To determine the distance between each position and the focal position, a three-dimensional galvanometer is used to ensure that the laser on the surface of the marking area is always at the focal plane position, thus generating a dynamic marking path.

[0034] Step 150: When the 3D camera detects the outline of the first complete fruit or vegetable, the timing is triggered; this is combined with the real-time speed of the conveyor device. V Calculate the minimum bounding box of the fruit and vegetable outline, using the distance between the center of the minimum bounding box of the fruit and vegetable and the field lens as... L Compensation amount for projected fruit and vegetable size L 0 System communication delay time Δt, Calculate the delay mark time T=(LL 0 ) / V+Δt For example, when marking a single apple, it is known that... L =1.2m, L 0 =0.04m, V =0.5m / s, Δt =0.07s, then T =(1.2-0.04) / 0.5+0.07=2.39s, meaning the system triggers the mark 2.39s after detecting the object leaving.

[0035] Step 160: Based on the damage threshold of fruit and vegetable surfaces, the laser energy density parameters are adjusted by changing the scanning speed, scanning interval, number of scans, laser frequency and laser energy. Combined with the static / dynamic marking path, three-dimensional laser marking instructions for fruits and vegetables are generated. The laser and three-dimensional galvanometer are controlled to work together to achieve three-dimensional marking on the surface of fruits and vegetables.

[0036] Optionally, in step 120, the surface of fruits and vegetables is monitored by a visual recognition system, and the positional changes of fruits and vegetables are judged by real-time image processing in order to adjust the marking angle of the galvanometer and avoid marking deviation caused by the natural movement of fruits and vegetables.

[0037] Optionally, in step 140, a suitable projection algorithm and path generation algorithm are selected by checking whether the height difference of the curved surface of the corresponding area of ​​the two-dimensional pattern on the fruit and vegetable surface is within the laser focal depth range. This process is as follows: Figure 2 As shown, it can ensure good scanning and reading performance even on fruit and vegetable surfaces with different curvatures and under different sizes of marking information.

[0038] Optionally, when the height difference of the curved surface of the two-dimensional pattern in the corresponding area on the fruit and vegetable surface is within the laser focal depth range, such as... Figure 3As shown in L2 of the 320 projection method; when the height difference of the curved surface of the two-dimensional pattern in the corresponding area on the fruit and vegetable surface exceeds the laser focal depth range, as... Figure 3 The L1 projection method is shown in the 310 projection.

[0039] Example 2 This embodiment provides a laser three-dimensional marking device for fruits and vegetables, including: Laser 410, 3D galvanometer 420, field lens 430, conveyor 440, industrial computer 450, 3D camera 460, slide module 470 and servo motor 480.

[0040] like Figure 4 As shown, the industrial control computer 450 is used to control the operation of the entire system, and it is equipped with a fruit and vegetable aerial marking system. The fruit and vegetable aerial marking system is responsible for processing the fruit and vegetable image information acquired by the 3D camera 460, parsing the marking information and generating fruit and vegetable laser 3D marking instructions, and then sending the instructions to the laser 410 and the 3D galvanometer 420.

[0041] A 3D camera 460 is mounted above the front end of the conveyor 440 to acquire 3D image data of the fruits and vegetables. It precisely measures the shape and position of the fruits and vegetables and transmits the acquired data to the industrial control computer 450 in real time.

[0042] The conveyor 440 is used to deliver fruits and vegetables to the marking area, ensuring their smooth movement. A servo motor 480 with a built-in encoder is mounted on it. This servo motor 480 drives the conveyor, and the encoder provides feedback on the conveyor's speed. The industrial computer 450 adjusts the operation of the marking system based on this information.

[0043] The laser 410 is located above the rear end of the conveying device 440 and is used to mark the surface of fruits and vegetables with QR codes, text, patterns, etc. The laser 410, in conjunction with the field lens 430 and the three-dimensional galvanometer 420, ensures that it can emit a stable laser beam.

[0044] The field mirror 430 is connected to the three-dimensional galvanometer 420, and together with the laser 410 via a reflector, they form a laser optical path system. The three-dimensional galvanometer 420 is responsible for precisely controlling the direction and focal length of the laser beam to ensure accurate marking on the surface of fruits and vegetables.

[0045] In this embodiment, the industrial control computer 450 receives the fruit and vegetable position and outline information transmitted by the 3D camera 460 and simultaneously processes the marking information input by the user. By analyzing the 3D image features and the user input information, the system generates a laser marking command containing 3D spatial coordinates. Simultaneously, the industrial control computer 450 calculates the minimum bounding box of the fruit and vegetable outline and, based on the real-time operating speed of the conveyor 440 and the geometric relationship between the bounding box center and the field mirror, calculates the laser delay marking time. Then, the generated laser 3D marking command and delay parameters are sent to the laser 410 and the 3D galvanometer 420, respectively. After the delay period ends, the system performs flight marking on the fruits and vegetables entering the field mirror area.

[0046] In practice, if the shape of fruits and vegetables is irregular or complex, the 3D camera 460 will detect changes in the height of the fruits and vegetables to determine whether complete fruit and vegetable data has been received.

[0047] Optionally, when the height data of the 3D camera 460 changes from low to high and the change exceeds the preset fruit and vegetable height threshold, the system determines that the 3D camera 460 has detected the fruit and vegetables.

[0048] Optionally, when the height data of the 3D camera 460 changes from high to low, and the lowest value reaches the installation height of the 3D camera 460, the system determines that the 3D camera 460 has successfully acquired the first complete fruit and vegetable image data, and begins processing, parsing, and marking.

[0049] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for laser three-dimensional marking of fruits and vegetables, characterized in that, The method includes: Obtain the two-dimensional pattern to be marked, input by the user; The system acquires 3D data of the fruit and vegetable surface using a 3D camera. When the height data changes from low to high and exceeds the fruit and vegetable height threshold, it is determined that the system has entered the fruit and vegetable area. After smoothing the surface using the moving least squares method, the Gaussian curvature of each point is calculated from the collected 3D point cloud data. K With mean curvature H Statistical mean of the curvature of the entire surface μ with standard deviation σ If the standard deviation of curvature σ K ≈0 and μ H If stable, perform spherical / ellipsoidal fitting; if σ K If the curvature is large and there are significant abrupt changes in the region, then freeform surface fitting is performed; Calculate the depth of focus based on the system's optical parameters. DOF Extract the Z-axis difference between the highest and lowest points of the marked area. Δh ,like Δh≤DOF Using a local coordinate system transformation method, the coordinates of the two-dimensional pattern are aligned with the corresponding area of ​​the fruit and vegetable curved surface, without changing the Z-axis coordinate, directly generating the marking command; if Δh>DOF The marking area is divided into grids, the surface normal vector of each grid point is extracted, the two-dimensional point is projected onto the three-dimensional surface along the direction of the surface normal vector, and the Z-axis position of the laser focus is adjusted according to the normal vector to generate a dynamic focusing path; When the 3D camera detects the outline of the first complete fruit or vegetable, a timing marker is triggered; this is combined with the real-time speed of the conveyor device. V Calculate the minimum bounding box of the fruit and vegetable outline, using the distance between the center of the bounding box and the field lens as... L Calculate the delay mark time T ; Based on the damage threshold of fruit and vegetable surfaces, the laser energy density parameters are adjusted by changing the scanning speed, scanning interval, number of scans, laser frequency, and laser energy. Combined with static / dynamic marking paths, three-dimensional laser marking instructions for fruits and vegetables are generated, and the laser and three-dimensional galvanometer are controlled to work together to achieve three-dimensional marking on the surface of fruits and vegetables.

2. The method according to claim 1, wherein, The surface fitting methods include least-squares closed-loop fitting for ellipsoidal surfaces and non-uniform B-spline fitting for freeform surfaces.

3. A laser three-dimensional marking device for fruits and vegetables, characterized in that, The device includes: The system includes an industrial computer, a 3D camera, a conveyor, a servo motor, a slide module, a laser, a 3D galvanometer, and a field lens. The industrial computer is equipped with a fruit and vegetable flying marking system. The conveyor is equipped with a servo motor. The 3D camera is located above the front end of the conveyor. The laser is located above the rear end of the conveyor. The field lens is connected to the 3D galvanometer and is mounted on the slide module. Together with the laser, the field lens forms an optical path system. The 3D galvanometer, laser, servo motor, 3D camera, and conveyor are all connected to the industrial computer.

4. The fruit and vegetable laser three-dimensional marking device according to claim 3, characterized in that: The industrial control computer is used to analyze the height data collected by the 3D camera. If the height data changes from low to high and the difference between the extreme high and low values ​​is greater than the preset fruit and vegetable height threshold, it means that the 3D camera has detected the fruit and vegetables. If the height data of the 3D camera changes from high to low and the lowest value reaches the installation height of the 3D camera, it means that the 3D camera has obtained the first complete fruit and vegetable outline data.

5. The fruit and vegetable laser three-dimensional marking device according to claim 3, characterized in that: The device establishes a dynamic machining coordinate system using a velocity-distance delay model, specifically satisfying... T=(LL 0 ) / V+Δt In the formula T For distance delay, L The physical distance between the center of the fruit / vegetable outline and the optical center of the galvanometer. L 0 This is the compensation amount for the projected size of fruits and vegetables. V For the speed of fruit and vegetable movement, Δt This refers to the system communication delay time.

Images