A 3D vision system and method for mold monitoring and detection

Through the 3D depth of field image fusion and multi-2D imaging component comparison detection of the 3D vision sensor system, the problem of low accuracy and efficiency of traditional mold detection is solved, high-precision, fast and non-contact mold detection is achieved, and production efficiency and product quality are improved.

CN118896556BActive Publication Date: 2024-12-03MAI XING (XIAMEN) ELECTRONICS CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411372812.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-29
Publication Date
2024-12-03
Estimated Expiration
2044-09-29

Smart Images

  • Figure CN118896556B_ABST
    Figure CN118896556B_ABST
Patent Text Reader

Abstract

The present invention relates to a 3D visual system for mold monitoring and detection, which comprises: the visual imaging device and a host computer connected to the visual imaging device circuit; the visual imaging device comprises a housing 1; a scanning mechanism is swingably provided on the inner side of the housing 1, which is used to project linear light to the mold to be tested and receive the reflected light and perform imaging; at least two 2D imaging components are provided in a circular array on the circumferential side of the scanning mechanism. The present invention fuses the point cloud imaged by the scanning mechanism with the plane images imaged by multiple 2D imaging components into a three-dimensional depth of field image, which can accurately complete the height profile detection of the product in the mold, thereby improving the precision and accuracy of the detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of mold detection, and in particular to a 3D vision system and method for mold monitoring and detection. Background Art

[0002] In the mold production process, it is very important to detect the height profile of the product in the mold and the detection of residues. Traditional detection methods often have problems of low accuracy and low efficiency. Therefore, a high-precision and high-efficiency mold monitoring and detection system is needed to meet production needs. Summary of the invention

[0003] The present invention provides a method for using a 3D vision sensor, so as to at least solve the problem in the prior art that when an angle exists during the imaging process of the 3D vision sensor, one end of the imaging angle is distorted, the detection is blurred, and the monitoring effect is affected.

[0004] To realize the above technical solution, the technical solution of the present invention is as follows: a 3D visual system for mold monitoring and detection, the 3D visual system for mold monitoring and detection comprises: the visual imaging device and a host computer connected to the visual imaging device circuit; the visual imaging device comprises a shell; a scanning mechanism is swingably provided inside the shell, which is used to project linear light to the mold to be tested and receive the reflected light and perform imaging; at least two 2D imaging components are provided in a circular array on the circumference of the scanning mechanism;

[0005] Wherein: the host computer compares the point cloud imaged by the scanning mechanism with the plane images imaged by the plurality of 2D imaging components through The three-dimensional depth of field image is fused to complete the height profile detection of the product in the mold; multiple 2D imaging components simultaneously complete image comparison to detect whether there is any residue.

[0006] Furthermore, the scanning mechanism comprises:

[0007] A projection assembly, rotatably disposed inside the housing, for projecting linear light onto the surface of the object to be measured; and

[0008] The receiving component is arranged at an angle with the projection component and is used to receive the reflected light and perform imaging.

[0009] Furthermore, the projection assembly includes a light source emitter horizontally arranged on the inner side of the shell; a rotating mechanism is rotatably provided along the optical path of the light projected by the light source emitter, which is used to change the refraction of the light to the surface of the object to be tested and move from the leftmost end to the rightmost end to complete the scanning.

[0010] Furthermore, the light source emitter is a laser emitter; and the rotating mechanism is one or more of a MEMS galvanometer or a rotatable plate-shaped refraction mirror.

[0011] Furthermore, the rotating mechanism is a MEMS galvanometer, and the light generated by the laser transmitter is projected onto the surface of the object to be measured through the MEMS galvanometer for scanning.

[0012] The present invention further discloses a method for mold monitoring and detection, which is applied to any of the above-mentioned 3D vision systems for mold monitoring and detection, and the method comprises:

[0013] S1, projecting a line of light to the mold to be tested through the scanning mechanism in the visual imaging device, and receiving the reflected light for imaging;

[0014] S2, using at least two 2D imaging components arranged in a circular array on the circumference of the scanning mechanism to simultaneously perform imaging;

[0015] S3, the host computer fuses the point cloud imaged by the scanning mechanism and the plane images imaged by the multiple 2D imaging components into a three-dimensional depth of field image to complete the height profile detection of the product in the mold;

[0016] S4. At the same time, image comparison detection is performed through multiple 2D imaging components to determine whether there is any product residue in the mold.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1) High-precision detection: The point cloud imaged by the scanning mechanism is fused with the plane images imaged by multiple 2D imaging components into a three-dimensional depth-of-field image, which can accurately complete the height profile detection of the product in the mold, thereby improving the precision and accuracy of the detection.

[0019] 2) Comprehensive inspection: Multiple 2D imaging components perform image comparison inspection simultaneously, which can effectively detect whether there is any residual product in the mold, ensuring the cleanliness of the mold and the quality of the product.

[0020] 3) Efficient inspection: The system can quickly complete the inspection of molds, improve production efficiency, and the efficiency can be as high as 3,600 pieces per hour, reducing inspection time and cost.

[0021] 4) Non-contact detection: It uses light projection and reception to detect without direct contact with the mold, thus avoiding damage to the mold and product.

[0022] 5) High reliability: The use of advanced laser emitters and MEMS galvanometer technologies ensures the stability and reliability of the system, enabling long-term stable operation.

[0023] 6) Strong flexibility: The scanning mechanism can be swung and the projection component can be rotated, which can adapt to the inspection needs of molds of different shapes and sizes, and has strong flexibility and versatility.

[0024] 7) Visual operation: The host computer presents the test results in the form of a three-dimensional depth-of-field image, so that the operator can intuitively understand the condition of the mold and facilitate analysis and judgment.

[0025] 8) Improve product quality: Through accurate inspection and monitoring of molds, problems can be discovered in time and measures can be taken to improve product quality and consistency.

[0026] 9) Reduce labor costs: The automated detection system reduces dependence on manual labor, reduces labor costs, and also reduces the impact of human factors on the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] To further illustrate the various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, a person of ordinary skill in the art should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0028] Figure 1 This is a schematic diagram of the structure of the 3D vision sensor of the present invention;

[0029] Figure 2 The 3D point cloud image of the present invention;

[0030] Figure 3 for Figure 2 Imaging actual physical picture.

[0031] Description of reference numerals: housing 1 , projection assembly 2 , receiving assembly 3 , receiving assembly 4 , light source transmitter 21 , rotating mechanism 22 . DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0034] Please see attached Figure 1 As shown: a 3D vision system for mold monitoring and detection, the 3D vision system for mold monitoring and detection comprises: the visual imaging device and a host computer connected to the visual imaging device circuit; the visual imaging device comprises a shell 1; a scanning mechanism is swingably provided inside the shell 1, which is used to project linear light to the mold to be tested and receive the reflected light and perform imaging; at least two 2D imaging components are provided in a circular array on the circumference of the scanning mechanism;

[0035] Wherein: the host computer compares the point cloud imaged by the scanning mechanism with the plane images imaged by the plurality of 2D imaging components through The three-dimensional depth of field image is fused to complete the height profile detection of the product in the mold; multiple 2D imaging components simultaneously complete image comparison to detect whether there is any residue.

[0036] Specifically: In actual shooting, there is often noise in the structured light image, which greatly affects the structured light stripe extraction effect. To address this problem, this paper uses two-dimensional Gaussian convolution to calculate the extracted structured light image, which can effectively reduce the negative impact of noise on the extraction of stripe centers. The calculation method is as follows:

[0037] (1)

[0038] (2)

[0039] In the formula, f(x, y) represents the original image, g(x, y) is the Gaussian convolution kernel, δ represents the mean square error of the Gaussian function, and its size satisfies , D is the stripe width.

[0040] After two-dimensional Gaussian convolution, the gradient distribution map of the stripe image can be obtained. The pixel-level center point obtained by the geometric center method is selected as the main point set for calculating the normal direction, which can quickly obtain the deformation of the laser stripe. Calibrate the grayscale value of the image point. Assuming (Gx, Gy) represents the gradient vector, the covariance matrix C of the gradient vector is formula (3):

[0041] (3)

[0042] Where Gx and Gy represent the gradient vectors of the image, and their expressions are (4):

[0043] (4)

[0044] The covariance matrix is ​​decomposed by principal component analysis, and the eigenvector corresponding to the maximum eigenvalue is the normal direction. The eigenvalue and the corresponding eigenvector are expressed as (5) to (8).

[0045] (5)

[0046] (6)

[0047] (7)

[0048] (8)

[0049] In formulas (5) to (8), W is the number of image points, and the eigenvalues ​​λ1 and λ2 correspond to eigenvectors v1 and v2, respectively. From the expression, we can see that λ1>λ2, so the corresponding eigenvector v1 is the normal direction of the stripe, and the normal vector is normalized to obtain the unit vector n=(nx,ny) that is consistent with the normal direction.

[0050] Finally, the grayscale centroid method is performed using equations (9) and (10).

[0051] (9)

[0052] (10)

[0053] In the formula, Gx (x, y) represents the pixel set in the x direction of the center point, and Gy (x, y) represents the pixel set in the y direction of the center point. xi and yi represent the final extracted structured light center point. By traversing all the center points obtained by the geometric center method, the center point is found to obtain the effective pixel point set of the sub-pixel point, and finally the sub-pixel center point can be obtained using the grayscale centroid method.

[0054] According to the constraint principle of the epipolar geometry of the binocular camera, a point X is set in space, and the projection points on the left and right cameras are m and m', which are called the matching points of the image and are recorded as ,but The epipolar line corresponding to m On, m is located The corresponding epipolar On, that is ,The relationship between the image points of the binocular camera and its epipolar lines can be expressed using the basic matrix. Let the two camera matrices be , the phase planes of the two cameras are expressed as ,but Backprojection epipolar The parametric equation is:

[0055] (11)

[0056] in is the generalized inverse of P, that is , C is the optical center of the first camera, that is , s is the polar normal, so we can get:

[0057] (12)

[0058] Right now , so the fundamental matrix between the two cameras is

[0059] Assume that the intrinsic matrix of the left camera is K, the memory matrix of the right camera is K', and the transformation relationship from the right camera to the left camera is [R, t]. Therefore, the coordinate transformation relationship between the right camera and the left camera is X'=RX+t, where R is the rotation matrix and t is the translation vector. The projection matrix of the left and right cameras can be expressed as:

[0060] (13)

[0061] The transformation matrix from the right camera to the left camera is

[0062] Where: is the rotation matrix parameter, is the translation vector parameter;

[0063] according to , is the epipolar line corresponding to the right camera, so:

[0064] (14)

[0065] Therefore, according to formula (14), we only need to know the intrinsic and extrinsic parameters of the two cameras to calculate the basic matrix, thereby determining the epipolar constraint relationship between the two cameras.

[0066] According to the spatial geometric relationship, the spatial point The three-dimensional coordinates of is the quantity to be determined, that is, the three-dimensional coordinates of the spatial point.

[0067] (15)

[0068] in , is the left and right camera focal length; , ; , are the principal point coordinates of the left and right cameras; is the principal point coordinate of the camera; are the left and right image coordinates of the detected laser point.

[0069] Based on the above embodiment, the scanning mechanism includes:

[0070] A projection assembly 2, rotatably disposed inside the housing 1, for projecting linear light onto the surface of the object to be measured; and

[0071] The receiving component 3 is arranged at an angle with the projection component 2 and is used to receive the reflected light and perform imaging.

[0072] Based on the above embodiment, the projection assembly 2 includes a light source emitter 21 horizontally arranged on the inner side of the shell 1; a rotating mechanism 22 is rotatably provided along the optical path of the light projected by the light source emitter 21, which is used to change the refraction of the light to the surface of the object to be tested and move from the leftmost end to the rightmost end to complete the scanning.

[0073] Based on the above embodiment, the light source emitter 21 is a laser emitter; the rotating mechanism 22 is one or more of a MEMS galvanometer or a rotatable plate-shaped refraction mirror.

[0074] Based on the above embodiment, the rotating mechanism 22 is a MEMS galvanometer, and the light generated by the laser transmitter is projected onto the surface of the object to be measured through the MEMS galvanometer for scanning.

[0075] The present invention also discloses a method for mold monitoring and detection, which is applied to any of the above-mentioned 3D vision systems for mold monitoring and detection, and the method comprises:

[0076] S1, projecting a line of light to the mold to be tested through the scanning mechanism in the visual imaging device, and receiving the reflected light for imaging;

[0077] S2, using at least two 2D imaging components arranged in a circular array on the circumference of the scanning mechanism to simultaneously perform imaging;

[0078] S3, the host computer fuses the point cloud imaged by the scanning mechanism and the plane images imaged by the multiple 2D imaging components into a three-dimensional depth of field image to complete the height profile detection of the product in the mold;

[0079] S4. At the same time, image comparison detection is performed through multiple 2D imaging components to determine whether there is any product residue in the mold.

[0080] By adopting the above scheme, the present invention has high-precision detection: the point cloud imaged by the scanning mechanism is fused with the plane images imaged by multiple 2D imaging components into a three-dimensional depth of field image, which can accurately complete the height profile detection of the product in the mold, improving the precision and accuracy of the detection; comprehensive detection: multiple 2D imaging components simultaneously perform image comparison detection, which can effectively detect whether there are residual products in the mold, ensuring the cleanliness of the mold and the quality of the product; efficient detection: the system can quickly complete the detection of the mold, improve production efficiency, and reduce detection time and cost; non-contact detection: detection is performed by light projection and reception, without direct contact with the mold, avoiding damage to the mold and the product; high reliability: advanced laser emission is adopted Technologies such as the projector and MEMS galvanometer ensure the stability and reliability of the system, and enable it to operate stably for a long time; Strong flexibility: The scanning mechanism can be swung and the projection component can be rotated, which can adapt to the detection needs of molds of different shapes and sizes, and has strong flexibility and versatility; Visual operation: The host computer presents the detection results in the form of a three-dimensional depth of field image, so that the operator can intuitively understand the condition of the mold, which is convenient for analysis and judgment; Improve product quality: Through accurate detection and monitoring of the mold, problems can be discovered in time and measures can be taken, thereby improving product quality and consistency; Reduce labor costs: The automated detection system reduces dependence on labor, reduces labor costs, and also reduces the impact of human factors on the detection results.

[0081] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not used to limit the present invention. Any person skilled in the art should be able to use the technical content disclosed above to make equivalent embodiments that are equivalent changes by slight changes or modifications without departing from the scope of the technical solution of the present invention. However, any brief modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A 3D vision system for mold monitoring and detection, the 3D vision system for mold monitoring and detection comprising: A visual imaging device and a host computer connected to the visual imaging device circuit; characterized in that the visual imaging device comprises a housing (1); a scanning mechanism is swingably provided inside the housing (1) for projecting linear light onto the mold to be tested and receiving reflected light to form an image; at least two 2D imaging components are provided in a circular array on the circumference of the scanning mechanism; Wherein: the host computer compares the point cloud imaged by the scanning mechanism with the plane images imaged by the plurality of 2D imaging components through The three-dimensional depth of field image is fused to complete the height profile detection of the product in the mold; multiple 2D imaging components simultaneously complete the image comparison to detect whether there is any residue; Where: , is the left and right camera focal length; , ; , are the principal point coordinates of the left and right cameras; is the principal point coordinate of the camera; are the left and right image coordinates of the detected laser point, is the rotation matrix parameter, is the translation vector parameter.

2. The 3D vision system for mold monitoring and detection according to claim 1, characterized in that: The scanning mechanism comprises: A projection assembly (2) is rotatably disposed inside the housing (1) and is used to project linear light onto the surface of the object to be measured; and The receiving component (3) is arranged at an angle with the projection component (2) and is used to receive the reflected light and perform imaging.

3. The 3D vision system for mold monitoring and detection according to claim 2, characterized in that: The projection assembly (2) comprises a light source emitter (21) arranged horizontally on the inner side of the housing (1); a rotating mechanism (22) is rotatably provided along the optical path of the light projected by the light source emitter (21) for changing the refraction of the light to the surface of the object to be measured and moving from the leftmost end to the rightmost end to complete scanning.

4. The 3D vision system for mold monitoring and detection according to claim 3, characterized in that: The light source emitter (21) is a laser emitter; the rotating mechanism (22) is one or more of a MEMS oscillating mirror and a rotatable plate-shaped refraction mirror.

5. The 3D vision system for mold monitoring and detection according to claim 4, characterized in that: The rotating mechanism (22) is a MEMS galvanometer, and the light generated by the laser transmitter is projected onto the surface of the object to be measured through the MEMS galvanometer for scanning.

6. A method for mold monitoring and detection, characterized in that: A 3D vision system for mold monitoring and detection as claimed in any one of claims 1 to 5, wherein the method comprises: S1, projecting linear light to the mold to be tested through the scanning mechanism in the visual imaging device, and receiving the reflected light for imaging; S2, using at least two 2D imaging components arranged in a circular array on the circumference of the scanning mechanism to simultaneously perform imaging; S3, the host computer fuses the point cloud imaged by the scanning mechanism and the plane images imaged by the multiple 2D imaging components into a three-dimensional depth of field image to complete the height profile detection of the product in the mold; S4. At the same time, image comparison detection is performed through multiple 2D imaging components to determine whether there is any product residue in the mold.

Citation Information

Patent Citations

  • High-reflectivity surface three-dimensional measurement method based on double-line structured light scanning

    CN111256587A

  • Method and equipment for determining processing track by visually scanning three-dimensional bonding surface

    CN113446961A

  • Portable unmanned aerial vehicle space positioning method and device in manned mode

    CN118011318A

  • camera for mounting on a conveyor and methods of inspection or identification

    DE102014114506A1

  • Method and apparatus for ascertaining a height profile image of a surface of an object

    WO2019170906A1