Three-dimensional measurement system, method and device based on pseudo-random speckles

Abstract

The present invention relates to a three-dimensional measurement system based on pseudo-random speckles, which includes an image acquisition device, an image preprocessing device, a system calibration device, a height calculation device, and a three-dimensional data processing device. Among them, the image acquisition device collects the speckle image of the reference datum plane and the speckle image of the surface of the object to be measured; the image preprocessing device performs image processing according to the speckle image of the reference datum plane and the speckle image of the surface of the object to be measured; the system calibration device obtains calibration parameters according to the speckle image of the reference datum plane; the height calculation device obtains the corresponding centroid offset according to the processed speckle image of the reference datum plane and the processed speckle image of the surface of the object to be measured; the three-dimensional data processing device obtains the three-dimensional contour information of the surface of the object to be measured according to the height values of each speckle point relative to the reference datum plane. The present invention also discloses a three-dimensional measurement method based on pseudo-random speckles and a three-dimensional measurement device based on pseudo-random speckles.

Description

Technical Field

[0001] The present application relates to the field of optical measurement technology, and in particular to a three-dimensional (3D) measurement system based on pseudo-random speckle, a three-dimensional measurement method based on pseudo-random speckle, and a three-dimensional measurement device based on pseudo-random speckle. Background Art

[0002] Compared with traditional two-dimensional image information, three-dimensional information can reflect objective objects more comprehensively and truly, and realize many detection requirements that traditional two-dimensional image information cannot meet, such as: measuring height, depth, thickness, flatness, warpage, wear and scratches, etc. With the continuous iteration of industrial development, many three-dimensional measurement technologies are becoming more and more mature. Among them, due to the characteristics of structured light three-dimensional measurement technology such as non-contact, high precision, high efficiency, and strong anti-interference, structured light three-dimensional measurement has played an increasingly important role in industrial automation and intelligent manufacturing in recent years. It has been widely used in the semiconductor industry (Printed Circuit Board (PCB) detection, chip detection, mobile phone industry, hardware industry and other fields.

[0003] According to the different light sources of the projectors, structured light 3D measurement technology can be roughly divided into: point structured light technology, line structured light technology and coded structured light technology. Among them, the point structured light technology has a simple algorithm, small calculation amount and high measurement accuracy, but it can only obtain the height information of a single contour point at a time, and requires the use of a precision translation stage to scan in the X and Y directions to complete the full-field measurement, which makes the measurement efficiency low. Line structured light technology can obtain the height information of a single contour line at a time, and requires the use of a precision translation stage to scan in the Y direction to complete the full-field measurement. Compared with point structured light technology, the algorithm of line structured light technology is more complex, the calculation amount is larger, and the measurement accuracy is lower. Coded structured light technology can obtain the height information of the entire surface contour at one time through the method of phase shift phase solution. Compared with point structured light technology, it does not require the use of a compact translation stage, but it needs to project multiple stripes at a time, resulting in complex algorithms, large calculation amounts, slow processing speeds and low measurement accuracy.

[0004] Therefore, how to quickly and efficiently obtain high-precision full-field surface profile height information in one go is an urgent problem that technicians need to solve in the current structured light three-dimensional measurement technology. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide a three-dimensional measurement system based on pseudo-random speckle, aiming to solve the problems of low measurement accuracy, slow processing speed, and low detection efficiency in obtaining full-field surface profile height information using the existing structured light three-dimensional measurement technology.

[0006] A three-dimensional measurement system based on pseudo-random speckles, comprising an image acquisition device, an image preprocessing device, a system calibration device, a height calculation device, and a three-dimensional data processing device. Among them, the image acquisition device is electrically connected to the image preprocessing device and is used to collect the speckle images of the reference benchmark plane and the speckle images of the surface of the object to be measured, and transmit the speckle images of the reference benchmark plane and the speckle images of the surface of the object to be measured to the image preprocessing device; the image preprocessing device is electrically connected to the system calibration device and the height calculation device. The image preprocessing device is used to perform image processing according to the speckle images of the reference benchmark plane and the speckle images of the surface of the object to be measured, transmit the processed speckle images of the reference benchmark plane to the system calibration device and the height calculation device, and transmit the processed speckle images of the surface of the object to be measured to the height calculation device; the system calibration device is electrically connected to the height calculation device. The system calibration device is used to obtain corresponding calibration parameters according to the speckle images of the reference benchmark plane and transmit the calibration parameters to the height calculation device; the height calculation device is electrically connected to the three-dimensional data processing device. The height calculation device is used to obtain the corresponding centroid offset according to the processed speckle images of the reference benchmark plane and the processed speckle images of the surface of the object to be measured, and calculate the height values of each speckle on the speckle image of the surface of the object to be measured relative to the reference benchmark plane by combining the centroid offset and the calibration parameters of each speckle, and transmit the obtained height values to the three-dimensional data processing device; the three-dimensional data processing device is used to obtain the three-dimensional contour information of the surface of the object to be measured according to the height values of each speckle relative to the reference benchmark plane, and analyze and detect the three-dimensional contour information of the surface of the object to be measured.

[0007] Optionally, the image acquisition device includes a first image acquisition circuit and a second image acquisition circuit. Among them, the first image acquisition circuit is electrically connected to the image preprocessing device. The first image acquisition circuit is used to collect the speckle images of the reference benchmark plane under different conditions and transmit the speckle images of the reference benchmark plane to the image preprocessing device; the second image acquisition circuit is electrically connected to the image preprocessing device. The second image acquisition circuit is used to collect the speckle images of the surface of the object to be measured under different conditions and transmit the speckle images of the surface of the object to be measured to the image preprocessing device.

[0008] Optionally, the image preprocessing device includes a first image preprocessing circuit and a second image preprocessing circuit. Among them, the first image preprocessing circuit is electrically connected to the first image acquisition circuit, the system calibration device, and the height calculation device. The first image preprocessing circuit is used to preprocess the speckle image of the reference datum plane to obtain the centroid coordinates of the speckle points, and transmit the preprocessed speckle image of the reference datum plane and the centroid coordinates of the speckle points to the system calibration device and the height calculation device. The second image preprocessing circuit is electrically connected to the second image acquisition circuit and the height calculation device. The second image preprocessing circuit is used to preprocess the speckle image of the surface of the target to be measured to obtain the centroid coordinates of the speckle points, and transmit the preprocessed speckle image of the surface of the target to be measured and the centroid coordinates of the speckle points to the height calculation device.

[0009] Optionally, the height calculation device includes a speckle image matching circuit and a height information acquisition circuit. Among them, the speckle image matching circuit is electrically connected to the first image preprocessing circuit, the second image preprocessing circuit, and the height information acquisition circuit. The speckle image matching circuit is used to match the preprocessed speckle image of the reference datum plane with the preprocessed speckle image of the surface of the target to be measured, and transmit the matching result to the height information acquisition circuit. The height information acquisition circuit is electrically connected to the system calibration device, the speckle image matching circuit, and the three-dimensional data processing device. The height information acquisition circuit is used to calculate the centroid offset according to the centroid coordinates of the speckle points, and calculate the height value of each speckle point relative to the reference datum plane according to the calibration parameters of each speckle point on the speckle image of the reference datum plane, and transmit the obtained height values of each speckle point relative to the reference datum plane to the three-dimensional data processing device.

[0010] Optionally, the image preprocessing device is further used to locate each speckle point on the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured respectively, and number each speckle point respectively according to the coordinate distribution of the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured.

[0011] Optionally, the system calibration device is further used to detect the repetition pattern of the speckle image of the reference datum plane transmitted by the image preprocessing device.

[0012] Optionally, the three-dimensional data processing device is further configured to perform three-dimensional reconstruction and data analysis on the height values of each speckle point on the speckle image of the surface of the target to be measured transmitted by the height calculation device, and perform combined stitching and local predetermined area mapping on the height values of the speckle points on the speckle images of all surfaces of the target to be measured, so as to obtain the three-dimensional contour information of the surface of the target to be measured.

[0013] Optionally, the centroid offset is the centroid offset of the speckle points in the speckle image of the surface of the target to be measured relative to the speckle image of the reference datum plane.

[0014] Optionally, the three-dimensional measurement system further includes a personal computer, a camera, a speckle generator, a calibration block, and a data cable. Among them, the personal computer, the camera, and the speckle generator are electrically connected through the data cable.

[0015] In summary, the image acquisition device acquires the speckle images of the reference datum plane and the surface of the target to be measured under different conditions. The image preprocessing device performs image processing according to the acquired speckle images of the reference datum plane and the surface of the target to be measured. The system calibration device processes and calculates the acquired speckle image of the reference datum plane to obtain corresponding calibration parameters. The height calculation device processes and calculates the processed speckle image of the reference datum plane and the processed speckle image of the surface of the target to be measured to obtain the centroid offset, and calculates the height values of each speckle point on the speckle image of the surface of the target to be measured relative to the reference datum plane by combining the centroid offset with the calibration parameters of each speckle point. The three-dimensional data processing device processes the height values of each speckle point on the speckle image of the surface of the target to be measured relative to the reference datum plane to obtain the three-dimensional contour information of the surface of the target to be measured, thereby completing the analysis and detection of the three-dimensional contour information of the surface of the target to be measured. Therefore, the three-dimensional measurement system based on pseudo-random speckles in the present application can quickly and efficiently obtain high-precision full-field surface contour height information at one time without relying on a precision displacement stage, thereby solving the problems of low measurement accuracy, slow processing speed, and low detection efficiency existing in the existing structured light three-dimensional measurement technology when obtaining full-field surface contour height information.

[0016] Based on the same inventive concept, the present application further provides a three-dimensional measurement method based on pseudo-random speckles, which is executed by the above three-dimensional measurement system. The three-dimensional measurement method includes: respectively collecting speckle images of a reference datum plane and a surface of a target to be measured; performing image processing on the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured to obtain centroid coordinates of corresponding speckle points; obtaining corresponding calibration parameters according to the speckle image of the reference datum plane; matching the preprocessed speckle image of the reference datum plane with the preprocessed speckle image of the surface of the target to be measured; calculating a centroid offset according to the centroid coordinates of the speckle points in the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured; calculating height values of respective speckle points relative to the reference datum plane according to the calibration parameters of respective speckle points on the speckle image of the reference datum plane and the centroid offset; and obtaining three-dimensional contour information of the surface of the target to be measured according to the height values of respective speckle points on the speckle image of the surface of the target to be measured relative to the reference datum plane.

[0017] Optionally, the respectively collecting speckle images of a reference datum plane and a surface of a target to be measured includes: collecting speckle images of the reference datum plane under different conditions; and collecting speckle images of the surface of the target to be measured under different conditions.

[0018] Optionally, the performing image processing on the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured to obtain centroid coordinates of corresponding speckle points includes: performing preprocessing on the speckle image of the reference datum plane to obtain centroid coordinates of speckle points and the preprocessed speckle image of the reference datum plane; and performing preprocessing on the speckle image of the surface of the target to be measured to obtain centroid coordinates of speckle points and the preprocessed speckle image of the surface of the target to be measured.

[0019] In summary, in the three-dimensional measurement method based on pseudo-random speckles, the speckle images of the reference datum plane and the surface of the target to be measured are respectively acquired; image processing is performed on the speckle images of the reference datum plane and the surface of the target to be measured to obtain the centroid coordinates of the corresponding speckle points; calibration parameters are obtained according to the speckle image of the reference datum plane; matching is performed between the pre-processed speckle image of the reference datum plane and the pre-processed speckle image of the surface of the target to be measured; the centroid offset is calculated according to the centroid coordinates of the speckle points in the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured; the height values of the respective speckle points relative to the reference datum plane are calculated according to the calibration parameters and the centroid offset of each speckle point on the speckle image of the reference datum plane; and the three-dimensional contour information of the surface of the target to be measured is obtained according to the height values of the respective speckle points on the speckle image of the surface of the target to be measured relative to the reference datum plane. Therefore, the three-dimensional measurement method based on pseudo-random speckles in the present application can quickly and efficiently obtain high-precision full-field surface contour height information at one time, thereby solving the problems of low measurement accuracy, slow processing speed, and low detection efficiency existing in the prior art structure light three-dimensional measurement technology when obtaining full-field surface contour height information.

[0020] Based on the same inventive concept, the present application also provides a three-dimensional measurement device based on pseudo-random speckles, which includes: at least one processor and a storage, at least one of the processors executes the computer execution instructions stored in the storage, and at least one of the processors executes the above three-dimensional measurement method based on pseudo-random speckles.

[0021] In summary, in the three-dimensional measurement device based on pseudo-random speckles, the speckle images of the reference datum plane and the speckle images of the surface of the target to be measured are respectively acquired; image processing is performed on the speckle images of the reference datum plane and the speckle images of the surface of the target to be measured to obtain the centroid coordinates of the corresponding speckle points; calibration parameters are obtained according to the speckle images of the reference datum plane; matching is performed between the preprocessed speckle images of the reference datum plane and the preprocessed speckle images of the surface of the target to be measured; the centroid offset is calculated according to the centroid coordinates of the speckle points in the speckle images of the reference datum plane and the speckle images of the surface of the target to be measured; the height values of the respective speckle points relative to the reference datum plane are calculated according to the calibration parameters and the centroid offset of the respective speckle points on the speckle images of the reference datum plane; and the three-dimensional contour information of the surface of the target to be measured is obtained according to the height values of the respective speckle points on the speckle images of the surface of the target to be measured relative to the reference datum plane. Therefore, the three-dimensional measurement device based on pseudo-random speckles in this application can quickly and efficiently obtain high-precision full-field surface contour height information at one time, thus solving the problems of low measurement accuracy, slow processing speed, and low detection efficiency in the existing structured light three-dimensional measurement technology for obtaining full-field surface contour height information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 Schematic diagram of the hardware composition of a three-dimensional measurement system based on pseudo-random speckles disclosed in an embodiment of the present application;

[0024] Figure 2 Schematic diagram of the structure of a three-dimensional measurement system based on pseudo-random speckles provided in an embodiment of the present application;

[0025] Figure 3 is Figure 2 Optical path schematic diagram of the three-dimensional measurement system based on pseudo-random speckles shown;

[0026] Fig. 4(a) is a schematic diagram of the surface fitting result of the target to be measured;

[0027] Fig. 4(b) is a schematic diagram of the local area mapping and supplement result of the surface of the target to be measured;

[0028] Figure 5 is Figure 2 Schematic diagram of the circuit structure of the three-dimensional measurement system based on pseudo-random speckles shown;

[0029] Figure 6 is a schematic diagram of the rough matching process;

[0030] Fig. 7(a) is a schematic diagram of the effect of the centroid offset of the scattered spots on the reference datum plane and the surface of the target to be measured;

[0031] Fig. 7(b) is a schematic diagram of the calculation of the height values of the scattered spots on the reference datum plane and the surface of the target to be measured;

[0032] Figure 8 is a schematic flowchart of a three-dimensional measurement method based on pseudo-random speckles disclosed in an embodiment of the present application;

[0033] Figure 9 is Figure 8 a schematic flowchart of step S10 in the three-dimensional measurement method based on pseudo-random speckles shown;

[0034] Figure 10 is Figure 8 a schematic flowchart of step S20 in the three-dimensional measurement method based on pseudo-random speckles shown;

[0035] Figure 11 is a schematic diagram of the hardware structure of a three-dimensional measurement device based on pseudo-random speckles disclosed in an embodiment of the present application. Detailed implementation manners

[0036] To facilitate the understanding of the present application, the present application will be described more comprehensively below with reference to the relevant drawings. Preferred embodiments of the present application are shown in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present application more thorough and comprehensive.

[0037] The descriptions of the following embodiments refer to the attached drawings for illustrating specific embodiments in which the present application can be implemented. The serial numbers assigned to the components herein, such as "first", "second", etc., are only used to distinguish the described objects and do not have any sequential or technical meanings. The terms "connection" and "coupling" used in the present application, unless otherwise specified, both include direct and indirect connections (couplings). The directional terms mentioned in the present application, such as "up", "down", "front", "rear", "left", "right", "inside", "outside", "side", etc., are only references to the directions in the attached drawings. Therefore, the directional terms used are for better and clearer illustration and understanding of the present application, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation to the present application.

[0038] In the description of the present application, it should be noted that unless otherwise clearly specified and defined, the terms "installed", "connected", and "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific situations. It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising", "may comprise", "including", or "may include" used in the present application indicate the existence of the corresponding functions, operations, elements, etc. disclosed, and do not limit one or more other functions, operations, elements, etc. In addition, the term "comprising" or "including" means the existence of the corresponding features, numbers, steps, operations, elements, components, or combinations thereof disclosed in the specification, and does not exclude the existence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, and is intended to cover non-exclusive inclusion.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. The terms used in the description of this application herein are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0040] Compared with traditional two-dimensional image information, three-dimensional information can more comprehensively and truly reflect objective objects, and meet many detection requirements that traditional two-dimensional image information cannot meet, such as measuring height, depth, thickness, flatness, warpage, wear and scratches, etc. With the advent of the Industrial 4.0 era, many detection equipment that can obtain and process three-dimensional information have been successfully developed, and many three-dimensional measurement technologies have become increasingly mature. Among them, due to the characteristics of structured light three-dimensional measurement technology such as non-contact, high precision, high efficiency, and strong anti-interference, structured light three-dimensional measurement has played an increasingly important role in industrial automation and intelligent manufacturing in recent years. It has been widely used in the semiconductor industry (Printed Circuit Boards (PCBs) Board, PCB) detection, chip detection, mobile phone industry, hardware industry and other fields. Structured light 3D measurement technology can be roughly divided into: point structured light technology, line structured light technology and coded structured light technology according to the different projector light sources. Among them, the point structured light technology has a simple algorithm, small amount of calculation, and high measurement accuracy, but it can only obtain the height information of a single contour point at a time, and requires the use of a precision translation stage to scan in the X and Y directions to complete the full-field measurement, which makes the measurement efficiency low. Line structured light technology can obtain the height information of a single contour line at a time, and requires the use of a precision translation stage to scan in the Y direction. It can complete full-field measurement. Compared with point structured light technology, line structured light technology has more complex algorithms, larger calculation amount and lower measurement accuracy. Coded structured light technology can obtain the height information of the entire surface contour at one time through phase shift and phase solution. Compared with point structured light technology, it does not require the use of a compact translation stage, but requires the projection of multiple stripes at one time, resulting in complex algorithms, large calculation amount, slow processing speed and low measurement accuracy. Therefore, how to quickly and efficiently achieve one-time acquisition of high-precision full-field surface contour height information without the help of a precision translation stage is a problem that technicians of structured light three-dimensional measurement technology urgently need to solve.

[0041] Based on this, in view of the shortcomings of existing structured light three-dimensional measurement technology in obtaining full-field surface profile height information, such as low measurement accuracy, slow processing speed, and low detection efficiency, the present application hopes to provide a three-dimensional measurement solution that can solve the above-mentioned technical problems, which can quickly and efficiently obtain high-precision full-field surface profile height information, thereby effectively improving the measurement efficiency, measurement accuracy and measurement speed of the three-dimensional measurement system based on pseudo-random speckle, and its details will be explained in subsequent embodiments.

[0042] It should be noted that structured light is a system structure composed of a projector and a camera. After the projector projects specific light information onto the object surface and the background, it is collected by the camera, and the position and depth of the object are calculated based on the changes in the light signals caused by the object, thereby restoring the entire three-dimensional space. The three-dimensional (3D) measurement technology of line structured light has been widely applied in semiconductor industries such as PCB board detection, Mini LED detection, and chip wafer detection, as well as in mobile phone industries such as screen thickness detection. In existing three-dimensional measurement systems, without the aid of a precise displacement stage, multiple fringe projections are required at one time, resulting in complex algorithms, large computational amounts, slow processing speeds, and low measurement accuracies.

[0043] A three-dimensional measurement system based on pseudo-random speckles disclosed in an embodiment of the present application can be applied to fields such as the semiconductor industry (PCB board detection, chip detection), the mobile phone industry (curved surface detection of mobile phone covers, screen thickness detection), and the hardware industry.

[0044] Please refer to Figure 1 and Figure 2 , Figure 1 which is a schematic diagram of the hardware composition of a three-dimensional measurement system based on pseudo-random speckles disclosed in an embodiment of the present application, Figure 2 and which is a schematic structural diagram of a three-dimensional measurement system based on pseudo-random speckles disclosed in an embodiment of the present application. As Figure 1 shown, the hardware part of the three-dimensional measurement system 100 provided in the embodiment of the present application mainly includes devices and equipment such as a personal computer (PC), a camera, a speckle generator, a calibration block, and a data cable. Among them, the personal computer, the camera, and the speckle generator are electrically connected through the data cable. In the embodiment of the present application, the camera can be a charge coupled device (CCD) camera or a complementary metal oxide semiconductor (CMOS) camera. The speckle generator can be a laser speckle transmitter.

[0045] As Figure 2As shown in the figure, an embodiment of the present application provides a three-dimensional measurement system 100 based on pseudo-random speckles, which at least includes an image acquisition device 110, an image preprocessing device 120, a system calibration device 130, a height calculation device 140, and a three-dimensional data processing device 150. Among them, the image acquisition device 110 is electrically connected to the image preprocessing device 120, the image preprocessing device 120 is electrically connected to both the system calibration device 130 and the height calculation device 140, the system calibration device 130 is electrically connected to the height calculation device 140, and the height calculation device 140 is electrically connected to the three-dimensional data processing device 150.

[0046] The image acquisition device 110 is configured to collect speckle images of a reference benchmark plane and a speckle image of the surface of a to-be-measured target under different conditions, and transmit the speckle images of the reference benchmark plane and the speckle image of the surface of the to-be-measured target to the image preprocessing device 120. Specifically, please refer to Figure 3 the optical path schematic diagram of the three-dimensional measurement system shown in the figure. The image acquisition device 110 can, based on a predetermined control algorithm, control the turning on or off of the speckle generator and the preset functions of the imaging unit through a control device, and collect speckle images of a reference benchmark plane and a speckle image of the surface of a to-be-measured target under different measurement requirements. Among them, the control device can be the personal computer, and the imaging unit can be a camera. In the embodiment of the present application, the image acquisition device 110 can include multiple camera units, and each camera unit can be a three-dimensional (3D) camera.

[0047] The image preprocessing device 120 is configured to perform image processing on the speckle images of the reference benchmark plane and the speckle image of the surface of the to-be-measured target acquired by the image acquisition device 110, and transmit the processed speckle images of the reference benchmark plane to the system calibration device 130 and the height calculation device 140, and at the same time transmit the processed speckle image of the surface of the to-be-measured target to the height calculation device 140.

[0048] Specifically, in the embodiments of the present application, the image preprocessing device 120 improves the image quality of the speckle image of the surface of the target to be measured acquired by the image acquisition device 110, thereby improving the accuracy when the height calculation device 140 performs processing, and locates each speckle on the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured respectively. According to the coordinate distribution of the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured, each speckle is numbered. The image preprocessing device 120 can also obtain the centroid information of each speckle on the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured, and eliminate or correct unreasonable speckles, thereby improving the calculation speed of the height calculation device 140 to facilitate the height calculation of each subsequent speckle.

[0049] The system calibration device 130 is configured to process and calculate corresponding calibration parameters according to the speckle image of the reference datum plane transmitted by the image preprocessing device 120, and transmit the processed and calculated calibration parameters to the height calculation device 140.

[0050] Specifically, in the embodiments of the present application, the system calibration device 130 detects according to the repetition pattern of the speckle image of the reference datum plane transmitted by the image preprocessing device 120, so as to ensure that the speckle distribution characteristics in each predetermined area on the speckle image of the reference datum plane are unique. At the same time, the system calibration device 130 can also calculate the calibration parameters of each speckle on the speckle image of the reference datum plane according to the speckle image of the reference datum plane transmitted by the image preprocessing device 120 for subsequent height calculation of each speckle on the speckle image of the surface of the target to be measured.

[0051] The height calculation device 140 is configured to process and calculate the centroid offset by using the processed speckle image of the reference datum plane and the processed speckle image of the surface of the target to be measured transmitted by the image preprocessing device 120; the height calculation device 140 is further configured to calculate the height value of each speckle on the speckle image of the surface of the target to be measured relative to the reference datum plane by combining the centroid offset with the calibration parameters of each speckle transmitted by the system calibration device 130, and transmit the obtained height values of each speckle relative to the reference datum plane to the three-dimensional data processing device 150. Wherein, the centroid offset is the centroid offset of the speckle in the speckle image of the surface of the target to be measured relative to the speckle image of the reference datum plane.

[0052] The three-dimensional data processing device 150 is configured to process the height values of each speckle on the speckle image of the surface of the target to be measured transmitted by the height calculation device 140 with respect to the reference datum plane to obtain the three-dimensional contour information of the surface of the target to be measured, and analyze and detect the three-dimensional contour information of the surface of the target to be measured.

[0053] In an embodiment of the present application, the three-dimensional data processing device 150 performs three-dimensional reconstruction and data analysis on the height values of each speckle on the speckle image of the surface of the target to be measured transmitted by the height calculation device 140 to obtain the three-dimensional contour information of the surface of the target to be measured, and analyzes and detects the three-dimensional contour information of the surface of the target to be measured. Please refer to the schematic diagram of the surface fitting result of the target to be measured shown in FIG. 4(a).

[0054] In an embodiment of the present application, the three-dimensional data processing device 150 also obtains the surface three-dimensional contour data information of the object to be measured by performing combined stitching and local predetermined area mapping on the height values of the speckles on the speckle images of all surfaces of the target to be measured. Please refer to the schematic diagram of the local area mapping supplement result of the surface of the target to be measured shown in FIG. 4(b).

[0055] Specifically, in an embodiment of the present application, according to the centroid coordinates of all the speckles on the speckle image of the surface of the target to be measured obtained by the image preprocessing device 120, the height values obtained by the height calculation device 140 are stitched together to obtain a height point cloud map, and then a local small area mapping operation is performed on the height point cloud map to establish an affine transformation between the speckles in the speckle image of the surface of the target to be measured in the local small area and the speckles in the speckle image of the reference datum plane, so as to supplement the height values in the blank area on the height point cloud map, and then perform corresponding smoothing and denoising operations on the supplemented height point cloud map to obtain the surface three-dimensional contour data information of the object to be measured.

[0056] In summary, the image acquisition device 110 acquires speckle images of the reference datum plane and the speckle images of the surface of the target to be measured under different conditions. The image preprocessing device 120 performs image processing based on the acquired speckle images of the reference datum plane and the speckle images of the surface of the target to be measured. The system calibration device 130 processes and calculates the acquired speckle images of the reference datum plane to obtain corresponding calibration parameters. The height calculation device 140 processes and calculates the processed speckle images of the reference datum plane and the processed speckle images of the surface of the target to be measured to obtain the centroid offset, and calculates the height values of the respective speckle points on the speckle image of the surface of the target to be measured relative to the reference datum plane by combining the centroid offset with the calibration parameters of each speckle point. The three-dimensional data processing device 150 processes the height values of the respective speckle points on the speckle image of the surface of the target to be measured relative to the reference datum plane to obtain the three-dimensional contour information of the surface of the target to be measured, thereby completing the analysis and detection of the three-dimensional contour information of the surface of the target to be measured. Therefore, the three-dimensional measurement system based on pseudo-random speckles according to the present application can, without relying on a precision displacement stage, quickly and efficiently achieve the one-time acquisition of high-precision full-field surface contour height information, thereby solving the problems of low measurement accuracy, slow processing speed, and low detection efficiency existing in the prior art structured light three-dimensional measurement technology when acquiring full-field surface contour height information.

[0057] Please refer to Figure 5 , which is Figure 2 a schematic circuit diagram of the three-dimensional measurement system based on pseudo-random speckles shown in Figure 5 As shown in

[0058] In an embodiment of the present application, the first image acquisition circuit 111 is configured to acquire speckle images of the reference datum plane under different conditions, and transmit the speckle images of the reference datum plane to the image preprocessing device 120.

[0059] Specifically, in an embodiment of the present application, the pseudo-speckle image projected onto the reference datum plane is acquired by the imaging unit, and the pseudo-speckle image is used as the pseudo-random speckle reference map. Among them, the light intensity of the speckle points in the pseudo-random speckle reference map can be adjusted by adjusting the exposure time and aperture of the imaging unit according to the power of the speckle generator and the reflection characteristics of the reference datum plane. In the specific implementation process, according to the power of the pseudo-random speckle projector and the reflection characteristics of the reference datum plane, the exposure time and aperture of the camera are adjusted to ensure that the light intensity of the speckle points on the acquired pseudo-random speckle reference map is appropriate.

[0060] Among them, the imaging unit can be a Charge Coupled Device (CCD) camera or a Complementary Metal-Oxide-Semiconductor (CMOS) camera.

[0061] The second image acquisition circuit 112 is used to collect speckle images on the surface of the target to be measured under different conditions, and transmit the speckle images on the surface of the target to be measured to the image preprocessing device 120.

[0062] Specifically, in the embodiment of the present application, a pseudo-speckle image projected onto the surface of the object to be measured is collected by the imaging unit, and the pseudo-speckle image is used as a pseudo-random speckle pattern. Among them, the light intensity of the speckle points in the pseudo-random speckle pattern can be adjusted by adjusting the exposure time and aperture of the imaging unit according to the power of the speckle generator and the reflection characteristics of the reference reference plane.

[0063] Please continue to refer to Figure 5 , the image preprocessing device 120 includes a first image preprocessing circuit 121 and a second image preprocessing circuit 122. Among them, the first image preprocessing circuit 121 is electrically connected to the first image acquisition circuit 111, the system calibration device 130, and the height calculation device 140 respectively, and the second image preprocessing circuit 122 is electrically connected to the second image acquisition circuit 112 and the height calculation device 140 respectively.

[0064] In the embodiment of the present application, the first image preprocessing circuit 121 is used to preprocess the speckle image of the reference reference plane transmitted by the first image acquisition circuit 111 to obtain the centroid coordinates of the speckle points, and transmit the preprocessed speckle image of the reference reference plane and the centroid coordinates of the speckle points to the system calibration device 130 and the height calculation device 140.

[0065] Specifically, in the embodiment of the present application, the first image preprocessing circuit 121 performs image filtering processing on the speckle image of the reference datum plane through a Gaussian filtering method (for example, performing 5*5 Gaussian filtering on the image), so as to eliminate the background noise on the speckle image of the reference datum plane, and obtain the speckle image of the reference datum plane after the image filtering processing. In the specific implementation process, according to the diameter size of the pseudo-random speckle points, an appropriate filtering window size is selected to ensure that the state of the speckle points will not be affected while eliminating the background noise as much as possible. Then, by statistically filtering the gray histogram of the speckle image of the reference datum plane after the image filtering processing, the gray distribution of the speckle image of the reference datum plane after the image filtering processing is understood, and a gray enhancement operation is performed on the speckle image of the reference datum plane after the image filtering processing to ensure that the speckle points in the darker areas of the image can be normally recognized, thereby obtaining a gray-enhanced image. Then, by using the Sobel differential operator, the image sharpening processing is performed on the speckle image of the reference datum plane, so that the edges of each speckle point on the speckle image of the reference datum plane become clearer, and a sharpened image is obtained. In the specific implementation process, according to the shape of the pseudo-random speckle points, an appropriate differential operator is selected to ensure that the edge shape of the speckle points will not be changed. Finally, by using the principle of the Otsu method for adaptive threshold calculation method, the optimal binary segmentation threshold of the sharpened image is determined, and the sharpened image is converted into a binary image by using this segmentation threshold. Then, relevant morphological processing is performed on the binary image to remove some messy points in the binary image, and a clean binary image is obtained. Then, the contour extraction is performed on the binary image after removing the messy points, and according to the evaluation of the geometric features such as the area and aspect ratio of each contour region, each speckle point on the binary image is segmented and marked. Finally, according to the marked area of each speckle point, the centroid coordinates of each speckle point on the reference image are calculated, and a speckle point positioning image is obtained.

[0066] In the embodiment of the present application, due to different situations of the speckle points, the centroid coordinates can refer to other centers in addition to the gray centroid, such as: centroid, energy peak center, Gaussian center, log center, etc. The present application does not make special limitations on this.

[0067] In the embodiment of the present application, the second image preprocessing circuit 122 is used to preprocess the speckle image of the surface of the target to be measured transmitted by the second image acquisition circuit 112 to obtain the centroid coordinates of the speckle points, and transmit the preprocessed speckle image of the surface of the target to be measured and the centroid coordinates of the speckle points to the height calculation device 140.

[0068] Specifically, in the embodiment of the present application, the second image preprocessing circuit 122 performs image filtering on the speckle image on the surface of the target to be measured by means of Gaussian filtering, so as to eliminate the background noise on the speckle image on the surface of the target to be measured, and obtain the speckle image on the surface of the target to be measured after image filtering. Then, by statistically analyzing the gray histogram of the speckle image on the surface of the target to be measured after image filtering, the gray distribution of the speckle image on the surface of the target to be measured after image filtering is understood, and a gray enhancement operation is performed on the speckle image on the surface of the target to be measured after image filtering, so as to ensure that the speckle points in some darker areas of the image can be normally recognized, thereby obtaining a gray-enhanced image. Then, by using the Sobel differential operator, image sharpening is performed on the speckle image of the reference datum plane, so that the edge of each speckle point on the speckle image of the reference datum plane becomes clearer, and a sharpened image is obtained. Then, by using the principle of the Otsu method for adaptive threshold calculation method, the optimal binary segmentation threshold of the sharpened image is determined, and the sharpened image is converted into a binary image by using this segmentation threshold. Then, relevant morphological processing is performed on the binary image to remove some messy points in the binary image, and a clean binary image is obtained. Then, contour extraction is performed on the binary image after removing the messy points, and based on the evaluation of geometric features such as the area and aspect ratio of each contour region, each speckle point on the binary image is segmented and marked. Finally, according to the marked area of each speckle point, the centroid coordinates of each speckle point on the reference image are calculated, and a speckle point positioning image is obtained.

[0069] In the embodiment of the present application, the preprocessing includes operations such as image filtering, gray enhancement, image sharpening, image binarization, speckle point positioning, and centroid extraction, which are not specifically limited in the present application.

[0070] Please continue to refer to Figure 5 , the height calculation device 140 includes a speckle image matching circuit 141 and a height information acquisition circuit 142. Among them, the speckle image matching circuit 141 is electrically connected to the first image preprocessing circuit 121, the second image preprocessing circuit 122, and the height information acquisition circuit 142 respectively, and the height information acquisition circuit 142 is electrically connected to the system calibration device 130, the speckle image matching circuit 141, and the three-dimensional data processing device 150.

[0071] In the embodiment of the present application, the speckle image matching circuit 141 is configured to match the speckle image of the preprocessed reference datum plane transmitted by the first image preprocessing circuit 121 with the speckle image of the preprocessed surface to be measured target transmitted by the second image preprocessing circuit 122, and transmit the matching result to the height information acquisition circuit 142.

[0072] Specifically, in the embodiment of the present application, the speckle image matching circuit 141 completes the preliminary matching between the speckle image of the preprocessed reference datum plane transmitted by the first image preprocessing circuit 121 and the speckle image of the preprocessed surface to be measured target transmitted by the second image preprocessing circuit 122 according to the image correlation algorithm, so as to realize the alignment of each effective speckle point on the speckle image of the reference datum plane and the speckle image of the surface to be measured target. In the specific implementation process, according to the speckle distribution of the pseudo-random speckle, appropriate image windows are sequentially selected on the reference image and the image to be measured for image correlation calculation. Since the speckle distribution of the pseudo-random speckle will present two situations of sparse and dense, it is necessary to adaptively adjust the size of the image window. Taking 20*100 as an example, ensure that there are at least 5 speckle points in the window area to ensure the accuracy of the image correlation calculation.

[0073] Among them, as Figure 6 shown, the rough matching process is to select the relative movement of the region of interest (ROI) (width is n, height is m) on the speckle image of the surface to be measured target and the speckle image of the reference datum plane. Each time the movement is made, the similarity C of the region of the speckle image of the reference datum plane is calculated. The formula is as follows:

[0074]

[0075] After the movement is completed, a similarity set {Ci} is obtained. Find the set with the largest similarity in the set and the corresponding position (i.e., the rough matching result), and obtain the corresponding numbered speckle points in this region.

[0076] In the embodiment of the present application, the height information acquisition circuit 142 is configured to calculate the centroid offset according to the centroid coordinates of the speckle points transmitted by the first image preprocessing circuit 121 and the second image preprocessing circuit 122, and calculate the height value of each speckle point relative to the reference datum plane according to the calibration parameters of each speckle point on the speckle image of the reference datum plane transmitted by the system calibration device 130, and transmit the obtained height value of each speckle point relative to the reference datum plane to the three-dimensional data processing device 150.

[0077] Specifically, in the embodiment of the present application, the centroid offset of the speckle points in the speckle image of the surface of the object to be measured relative to the speckle image of the reference reference plane is calculated according to the centroid coordinates of the speckle points transmitted by the first image preprocessing circuit 121 and the second image preprocessing circuit 122, and the calculation process is as follows:

[0078] By using the Otsu method, the segmentation threshold of the ROI image is determined, and the principle is as follows:

[0079] Calculate the cumulative mean M of the gray level K and the global mean MG of the image:

[0080]

[0081] Among them, P i is the probability value of the current gray level appearing. The final formula in this embodiment is:

[0082]

[0083] Among them, the gray level k that maximizes the formula is the calculated segmentation threshold, and the ROI image is converted into a binary image by using this segmentation threshold, and the hole target and the background plane are segmented out. Then, a morphological opening operation with a template of 5*5 is performed on the binary image to remove some messy points in the binary image. Then, contour extraction is performed on the binary image after removing the messy points, and the geometric features such as the area and aspect ratio of each contour region are evaluated to segment and mark each speckle point on the binary image. Finally, according to the marked area of each speckle point, the centroid coordinates of each speckle point on the reference image are calculated. The centroid formula:

[0084]

[0085] Among them, the target gray level is 255, and finally the centroid offset between the speckle points with corresponding numbers is calculated. It should be noted that the relative centroid offset is calculated only when the contour positions of the image to be measured and the reference image are both present.

[0086] Specifically, as shown in the schematic diagram of the effect of the centroid offset of the speckle points on the reference reference plane and the surface of the object to be measured in FIG. 7(a), and the schematic diagram of the calculation of the height value of the speckle points on the reference reference plane and the surface of the object to be measured in FIG. 7(b). In the embodiment of the present application, the height information acquisition circuit 142 is further configured to calculate the height value of each speckle point relative to the reference reference plane by using a calibration formula based on the calibration parameters and the centroid offset of each speckle point transmitted by the system calibration device 130. The calculation process is as follows:

[0087] Based on the set of centroid offsets obtained from formula (4) and the set of centroid offsets between the corresponding numbered speckle points in each ROI, construct the linear relationship between the lateral offset dx of the corresponding number and different height references. Among them, the abscissa is the different reference heights, and the ordinate is the set of lateral offsets dx of the corresponding number under different height references. The principle of linear fitting is as follows:

[0088]

[0089] The obtained K value through fitting is used to construct Δx = Kz, where z is the longitudinal displacement; this principle satisfies the triangulation principle. Similarly, the set of K values of each numbered speckle point and different references is obtained.

[0090] Then, according to the calibration data set (dx, dy, k), fit the plane equation with dx and dy as independent variables and K as the dependent variable. The least squares method for fitting the plane equation is as follows:

[0091] Among them, the constructed plane equation is: Z = a0 * x + a1 * y + a2, and the linear matrix equation obtained by the least squares principle is as follows:

[0092]

[0093] Among them, solving the matrix equation can obtain the values of a0, a1, and a2. From the above formula (6), the K value under any dx and dy within the calibration height range can be obtained, and the calibration relationship between the offset and height under any speckle correlation measurement can be obtained through the formula Δx = Kz.

[0094] Please refer to Figure 8 , which is a schematic flowchart of a three-dimensional measurement method based on pseudo-random speckles disclosed in an embodiment of the present application. The above Figures 1 to 7(b) The three-dimensional measurement system based on pseudo-random speckles in the shown embodiment measures the target to be measured through the following three-dimensional measurement method, effectively improving the measurement efficiency, measurement accuracy, and measurement speed of the three-dimensional measurement system based on pseudo-random speckles. As Figure 8 shown, the measurement method based on pseudo-random speckles at least includes the following steps.

[0095] S10. Respectively collect the speckle image of the reference reference plane and the speckle image of the surface of the target to be measured.

[0096] In this embodiment, please combine Figure 2 , and through the image acquisition device 110, collect the speckle images of the reference reference plane and the surface of the target to be measured under different conditions, and transmit the speckle image of the reference reference plane and the speckle image of the surface of the target to be measured to the image preprocessing device 120. Specifically, please refer to Figure 3Schematic diagram of the optical path of the three-dimensional measurement system shown. The image acquisition device 110 can, based on a predetermined control algorithm, control the turning on or off of the speckle generator and the preset functions of the imaging unit through the control device, and acquire the speckle images of the reference benchmark plane and the speckle images of the surface of the target to be measured under different circumstances according to the measurement requirements. Among them, the control device can be the personal computer, and the imaging unit can be a camera. In the embodiment of the present application, the image acquisition device 110 can include a plurality of camera units, and each camera unit can be a three-dimensional (3D) camera.

[0097] In the embodiment of the present application, please refer to Figure 9 and in combination with Figure 2 and Figure 5 , step S10 at least includes the following steps.

[0098] S11. Acquire the speckle images of the reference benchmark plane under different circumstances.

[0099] In this embodiment, the first image acquisition circuit 111 acquires the speckle images of the reference benchmark plane under different circumstances and transmits the speckle images of the reference benchmark plane to the image preprocessing device 120.

[0100] Specifically, in the embodiment of the present application, the pseudo-speckle image projected onto the reference benchmark plane is acquired through the imaging unit, and the pseudo-speckle image is used as the pseudo-random speckle reference map. Among them, the light intensity of the speckle points in the pseudo-random speckle reference map can be adjusted by adjusting the exposure time and aperture of the imaging unit according to the power of the speckle generator and the reflection characteristics of the reference benchmark plane. In the specific implementation process, according to the power of the pseudo-random speckle projector and the reflection characteristics of the reference benchmark plane, the exposure time and aperture of the camera are adjusted to ensure that the light intensity of the speckle points on the acquired pseudo-random speckle reference map is appropriate.

[0101] Among them, the imaging unit can be a charge-coupled device (CCD) camera or a complementary metal-oxide-semiconductor (CMOS) camera.

[0102] S12. Acquire the speckle images of the surface of the target to be measured under different circumstances.

[0103] In this embodiment, the second image acquisition circuit 112 acquires the speckle images of the surface of the target to be measured under different circumstances and transmits the speckle images of the surface of the target to be measured to the image preprocessing device 120.

[0104] Specifically, in the embodiments of the present application, the pseudo-speckle image projected onto the surface of the object to be measured is collected by the imaging unit, and the pseudo-speckle image is used as a pseudo-random speckle pattern. Among them, the light intensity of the speckle points in the pseudo-random speckle pattern can be adjusted by adjusting the exposure time and aperture of the imaging unit according to the power of the speckle generator and the reflection characteristics of the reference reference plane.

[0105] S20. Perform image processing on the speckle image of the reference reference plane and the speckle image of the surface of the object to be measured to obtain the centroid coordinates of the corresponding speckle points.

[0106] In this embodiment, please refer to Figure 1 and Figure 2 , the image preprocessing device 120 performs image processing on the speckle image of the reference reference plane and the speckle image of the surface of the object to be measured obtained by the image acquisition device 110, and transmits the processed speckle image of the reference reference plane to the system calibration device 130 and the height calculation device 140, and at the same time transmits the processed speckle image of the surface of the object to be measured to the height calculation device 140.

[0107] Specifically, in the embodiments of the present application, the image preprocessing device 120 improves the image quality of the speckle image of the surface of the object to be measured acquired by the image acquisition device 110, thereby improving the accuracy when the height calculation device 140 processes, and locates each speckle point on the speckle image of the reference reference plane and the speckle image of the surface of the object to be measured respectively, and numbers each speckle point according to the coordinate distribution of the speckle image of the reference reference plane and the speckle image of the surface of the object to be measured. The image preprocessing device 120 can also obtain the centroid information of each speckle point on the speckle image of the reference reference plane and the speckle image of the surface of the object to be measured, and eliminate or correct unreasonable speckle points, thereby improving the calculation speed of the height calculation device 140, so as to facilitate the height calculation of each subsequent speckle point.

[0108] In the embodiments of the present application, please refer to Figure 10 and combine with Figure 2 and Figure 5 , the step S20 at least includes the following steps.

[0109] S21. Perform preprocessing on the speckle image of the reference reference plane to obtain the centroid coordinates of the speckle points and the preprocessed speckle image of the reference reference plane.

[0110] Specifically, the first image preprocessing circuit 121 preprocesses the speckle image of the reference datum plane transmitted by the first image acquisition circuit 111 to obtain the centroid coordinates of the speckle points, and transmits the preprocessed speckle image of the reference datum plane and the centroid coordinates of the speckle points to the system calibration device 130 and the height calculation device 140.

[0111] In the embodiment of the present application, the first image preprocessing circuit 121 performs image filtering processing on the speckle image of the reference datum plane by means of a Gaussian filtering method (for example, performing 5*5 Gaussian filtering on the image), so as to eliminate the background noise on the speckle image of the reference datum plane, and obtain the speckle image of the reference datum plane after image filtering processing. In the specific implementation process, according to the diameter size of the pseudo-random speckle points, an appropriate filtering window size is selected to ensure that the state of the speckle points will not be affected while eliminating the background noise as much as possible. Then, by statistically filtering the gray histogram of the speckle image of the reference datum plane after the image filtering processing, the gray distribution of the speckle image of the reference datum plane after the image filtering processing is understood, and a gray enhancement operation is performed on the speckle image of the reference datum plane after the image filtering processing to ensure that the speckle points in the darker areas of the image can be normally recognized, so as to obtain a gray-enhanced image. Then, by using the Sobel differential operator, image sharpening processing is performed on the speckle image of the reference datum plane to make the edges of each speckle point on the speckle image of the reference datum plane clearer, and a sharpened image is obtained. In the specific implementation process, according to the shape of the pseudo-random speckle points, an appropriate differential operator is selected to ensure that the edge shape of the speckle points will not be changed. Finally, by using the Otsu method principle adaptive threshold calculation method, the optimal binary segmentation threshold of the sharpened image is determined, and the sharpened image is converted into a binary image by using this segmentation threshold. Then, relevant morphological processing is performed on the binary image to remove some clutter points in the binary image, and a clean binary image is obtained. Then, contour extraction is performed on the binary image after removing the clutter points, and the geometric features such as the area and aspect ratio of each contour region are evaluated, and each speckle point on the binary image is segmented and marked. Finally, according to the marked area of each speckle point, the centroid coordinates of each speckle point on the reference image are calculated to obtain a speckle point positioning map.

[0112] In the embodiment of the present application, due to different situations of the speckle points, the centroid coordinates may refer to other centers in addition to the gray centroid, such as: centroid, energy peak center, Gaussian center, log center, etc. The present application does not make special limitations on this.

[0113] S22. Preprocess the speckle image of the surface of the target to be measured to obtain the centroid coordinates of the speckle points and the speckle image of the surface of the target to be measured after preprocessing.

[0114] Specifically, the second image preprocessing circuit 122 preprocesses the speckle image of the surface of the target to be measured transmitted by the second image acquisition circuit 112 to obtain the centroid coordinates of the speckle points, and transmits the speckle image of the surface of the target to be measured after preprocessing and the centroid coordinates of the speckle points to the height calculation device 140.

[0115] Specifically, in the embodiment of the present application, the second image preprocessing circuit 122 performs image filtering processing on the speckle image of the surface of the target to be measured by means of Gaussian filtering method, so as to eliminate the background noise on the speckle image of the surface of the target to be measured, and obtain the speckle image of the surface of the target to be measured after image filtering processing. Then, by statistically analyzing the gray histogram of the speckle image of the surface of the target to be measured after image filtering processing, understand the gray distribution of the speckle image of the surface of the target to be measured after image filtering processing, and perform gray enhancement operation on the speckle image of the surface of the target to be measured after image filtering processing, so as to ensure that the speckle points in some darker areas on the image can be normally recognized, thereby obtaining a gray-enhanced image. Then, by using the Sobel differential operator, perform image sharpening processing on the speckle image of the reference datum plane, so that the edges of each speckle point on the speckle image of the reference datum plane become clearer, and obtain a sharpened image. Then, by using the principle of the Otsu method (Otsu method) adaptive threshold calculation method, determine the optimal binary segmentation threshold of the sharpened image, and use this segmentation threshold to convert the sharpened image into a binary image, and then perform relevant morphological processing on the binary image to remove some messy points in the binary image, and obtain a clean binary image. Then, perform contour extraction on the binary image after removing the messy points, and evaluate the geometric features such as the area and aspect ratio of each contour region, and segment and mark each speckle point on the binary image. Finally, according to the marked areas of each speckle point, calculate the centroid coordinates of each speckle point on the reference image, and obtain a speckle point positioning map.

[0116] In the embodiment of the present application, the preprocessing includes operations such as image filtering, gray enhancement, image sharpening, image binarization, speckle point positioning and centroid extraction, etc., and the present application does not make specific limitations here.

[0117] S30. Obtain corresponding calibration parameters according to the speckle image of the reference datum plane.

[0118] In this embodiment, please refer to Figure 1 and Figure 2, the system calibration device 130 processes and calculates the corresponding calibration parameters based on the speckle image of the reference datum plane transmitted by the image preprocessing device 120, and transmits the processed and calculated calibration parameters to the height calculation device 140.

[0119] Specifically, in the embodiment of the present application, the system calibration device 130 detects the repetition pattern of the speckle image of the reference datum plane transmitted by the image preprocessing device 120, so as to ensure that the speckle distribution characteristics in each predetermined area on the speckle image of the reference datum plane are unique. At the same time, the system calibration device 130 can also calculate the calibration parameters of each speckle on the speckle image of the reference datum plane based on the speckle image of the reference datum plane transmitted by the image preprocessing device 120, so as to be used for calculating the height of each speckle on the speckle image of the surface of the target to be measured subsequently.

[0120] S40. Match the speckle image of the preprocessed reference datum plane with the speckle image of the preprocessed surface of the target to be measured.

[0121] In the embodiment of the present application, please refer to Figure 5 , the speckle image matching circuit 141 matches the preprocessed speckle image of the reference datum plane transmitted by the first image preprocessing circuit 121 with the preprocessed speckle image of the surface of the target to be measured transmitted by the second image preprocessing circuit 122, and transmits the matching result to the height information acquisition circuit 142.

[0122] Specifically, in the embodiment of the present application, the speckle image matching circuit 141 completes the preliminary matching between the preprocessed speckle image of the reference datum plane transmitted by the first image preprocessing circuit 121 and the preprocessed speckle image of the surface of the target to be measured transmitted by the second image preprocessing circuit 122 according to the image correlation algorithm, and realizes the alignment of each effective speckle on the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured. In the specific implementation process, according to the speckle distribution of the pseudo-random speckles, appropriate image windows are sequentially selected on the reference image and the image to be measured for image correlation calculation. Since the speckle distribution of the pseudo-random speckles will present two situations of sparse and dense, it is necessary to adaptively adjust the size of the image window. Taking 20*100 as an example, ensure that there are at least 5 speckles in the window area to ensure the accuracy of the image correlation calculation.

[0123] S50. Calculate the centroid offset according to the centroid coordinates of the speckles in the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured.

[0124] In the embodiment of the present application, please refer toFigure 5 The height information acquisition circuit 142 calculates a centroid offset based on the centroid coordinates of the speckles transmitted by the first image preprocessing circuit 121 and the second image preprocessing circuit 122.

[0125] Specifically, in the embodiment of the present application, the centroid offset of the speckles in the speckle image of the surface of the target to be measured relative to the speckle image of the reference reference plane is calculated according to the centroid coordinates of the speckles transmitted by the first image preprocessing circuit 121 and the second image preprocessing circuit 122. The calculation process is as follows:

[0126] By using the Otsu method, the segmentation threshold of the ROI image is determined. The principle is as follows:

[0127] Calculate the cumulative mean M of the gray level K and the global mean MG of the image:

[0128]

[0129] where P i is the probability value of the current gray level. The final formula in this embodiment is:

[0130]

[0131] where the gray level k that maximizes the formula is the calculated segmentation threshold, and the ROI image is converted into a binary image by using this segmentation threshold, and the hole target and the background plane are segmented out. Then, a morphological opening operation with a template of 5*5 is performed on the binary image to remove some messy points in the binary image. Then, contour extraction is performed on the binary image after removing the messy points, and the geometric features such as the area and aspect ratio of each contour region are evaluated to segment and mark each speckle on the binary image. Finally, according to the marked area of each speckle, the centroid coordinates of each speckle on the reference image are calculated. The centroid formula:

[0132]

[0133] where the target gray level is all 255, and finally the centroid offset between the speckles with corresponding numbers is calculated. It should be noted that the relative centroid offset will only be calculated when the contour positions of the measured image and the reference image are both present.

[0134] S60. Calculate the height value of each speckle relative to the reference reference plane according to the calibration parameters and centroid offset of each speckle on the speckle image of the reference reference plane.

[0135] In the embodiment of the present application, the height information acquisition circuit 142 calculates the height values of each speckle relative to the reference datum plane based on the calibration parameters of each speckle on the speckle image of the reference datum plane transmitted by the system calibration device 130, and transmits the obtained height values of each speckle relative to the reference datum plane to the three-dimensional data processing device 150.

[0136] Specifically, as shown in the schematic diagram of the centroid offset of the speckles on the reference datum plane and the surface to be measured in Fig. 7(a), and the schematic diagram of the calculation of the height values of the speckles on the reference datum plane and the surface to be measured in Fig. 7(b). In the embodiment of the present application, the height information acquisition circuit 142 is further configured to calculate the height values of each speckle relative to the reference datum plane by using a calibration formula for the calibration parameters and centroid offsets of each speckle transmitted by the system calibration device 130. The calculation process is as follows:

[0137] According to the set of centroid offsets obtained from formula (4) and the set of centroid offsets between the speckles with corresponding numbers in each ROI, a linear relationship between the lateral offset dx with corresponding numbers and different height references is constructed. Among them, the abscissa is different reference heights, and the ordinate is the set of lateral offsets dx of the corresponding numbers under different height references. The principle of linear fitting is as follows:

[0138]

[0139] The obtained K value is used to construct Δx = Kz, where z is the longitudinal displacement; this principle satisfies the triangulation principle. Similarly, the set of K values of each numbered speckle and different references is obtained.

[0140] Then, according to the calibration data set (dx, dy, k), a plane equation with dx and dy as independent variables and K as the dependent variable is fitted. The plane equation fitted by the least squares method is as follows:

[0141] Among them, the constructed plane equation is: Z = a0 * x + a1 * y + a2, and the linear matrix equation obtained by the least squares principle is as follows:

[0142]

[0143] Among them, by solving the matrix equation, the values of a0, a1, and a2 can be obtained. From the above formula (6), the K value under any dx and dy within the calibration height range can be obtained, and the calibration relationship between the offset and height under any speckle correlation measurement can be obtained through the formula Δx = Kz.

[0144] S70. Obtain the three-dimensional contour information of the surface to be measured based on the height values of each speckle on the speckle image of the surface to be measured relative to the reference datum plane.

[0145] In an embodiment of the present application, please refer to Figure 1 and Figure 5 , the three-dimensional data processing device 150 processes the height values of each speckle on the speckle image of the surface of the target to be measured transmitted by the height calculation device 140 relative to the reference datum plane to obtain the three-dimensional contour information of the surface of the target to be measured, and analyzes and detects the three-dimensional contour information of the surface of the target to be measured.

[0146] In an embodiment of the present application, the three-dimensional data processing device 150 performs three-dimensional reconstruction and data analysis on the height values of each speckle on the speckle image of the surface of the target to be measured transmitted by the height calculation device 140, obtains the three-dimensional contour information of the surface of the target to be measured, and analyzes and detects the three-dimensional contour information of the surface of the target to be measured. Please refer to the schematic diagram of the surface fitting result of the target to be measured shown in Fig. 4(a).

[0147] In an embodiment of the present application, the three-dimensional data processing device 150 also obtains the surface three-dimensional contour data information of the object to be measured by combining and splicing the height values of the speckles on the speckle images of all surfaces of the target to be measured and mapping a local predetermined area. Please refer to the schematic diagram of the local area mapping and supplement result of the surface of the target to be measured shown in Fig. 4(b).

[0148] In summary, in the three-dimensional measurement method based on pseudo-random speckles, the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured are respectively acquired; image processing is performed on the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured to obtain the centroid coordinates of the corresponding speckles; the corresponding calibration parameters are obtained according to the speckle image of the reference datum plane; the preprocessed speckle image of the reference datum plane is matched with the preprocessed speckle image of the surface of the target to be measured; the centroid offset is calculated according to the centroid coordinates of the speckles in the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured; the height values of each speckle relative to the reference datum plane are calculated according to the calibration parameters and the centroid offset of each speckle on the speckle image of the reference datum plane; the three-dimensional contour information of the surface of the target to be measured is obtained according to the height values of each speckle on the speckle image of the surface of the target to be measured relative to the reference datum plane. Therefore, the three-dimensional measurement method based on pseudo-random speckles in the present application can quickly and efficiently obtain high-precision full-field surface contour height information at one time, thus solving the problems of low measurement accuracy, slow processing speed, and low detection efficiency in the existing structured light three-dimensional measurement technology for obtaining full-field surface contour height information.

[0149] Please refer to Figure 11, which is a schematic diagram of the hardware structure of a three-dimensional measurement device based on pseudo-random speckles disclosed in the embodiments of the present application. As Figure 11 shown, the three-dimensional measurement device 200 based on pseudo-random speckles provided in the embodiments of the present application includes at least one processor 201 and a memory 202. The three-dimensional measurement device 200 based on pseudo-random speckles further includes at least one bus 203. Among them, the processor 201 and the memory 202 are electrically connected through the bus 203. The three-dimensional measurement device 200 based on pseudo-random speckles may be a computer or a server, and the present application does not make a special limitation on this.

[0150] The three-dimensional measurement device 200 based on pseudo-random speckles may further include the three-dimensional measurement system based on pseudo-random speckles in the embodiments as described above Figures 1 to 7 shown. In a specific implementation process, at least one processor 201 executes the computer execution instructions stored in the memory 202, so that at least one processor 201 executes the three-dimensional measurement method of the three-dimensional measurement system based on pseudo-random speckles as described in Figures 8 - 10 the embodiments.

[0151] For the specific implementation process of the processor 201 provided in the embodiments of the present application, reference may be made to the three-dimensional measurement method embodiments of the three-dimensional measurement system based on pseudo-random speckles in the above Figures 8 - 10 described embodiments. The implementation principle and technical effects are similar, and will not be elaborated here in this embodiment.

[0152] It can be understood that the processor 201 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method provided in combination with the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0153] The memory 202 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM).

[0154] The bus 203 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. For ease of representation, the bus 203 in the drawings of the present application is not limited to only one bus or one type of bus.

[0155] It should be understood that the application of the present application is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present application.

Claims

1. A three-dimensional measurement system based on pseudo-random speckles, characterized in that The three-dimensional measurement system includes an image acquisition device, an image preprocessing device, a system calibration device, a height calculation device, and a three-dimensional data processing device. Among them, The image acquisition device is electrically connected to the image preprocessing device, and is used to collect the speckle images of the reference datum plane and the surface of the object to be measured, and transmit the speckle images of the reference datum plane and the surface of the object to be measured to the image preprocessing device; The image preprocessing device is electrically connected to the system calibration device and the height calculation device. The image preprocessing device is used to perform image processing based on the speckle images of the reference datum plane and the surface of the object to be measured, transmit the processed speckle image of the reference datum plane to the system calibration device and the height calculation device, and transmit the processed speckle image of the surface of the object to be measured to the height calculation device; The system calibration device is electrically connected to the height calculation device. The system calibration device is used to obtain corresponding calibration parameters according to the speckle image of the reference datum plane, and transmit the calibration parameters to the height calculation device; The height calculation device is electrically connected to the three-dimensional data processing device. The height calculation device is used to obtain the corresponding centroid offset according to the processed speckle image of the reference datum plane and the processed speckle image of the surface of the object to be measured, and calculate the height values of each speckle on the speckle image of the surface of the object to be measured relative to the reference datum plane by combining the centroid offset and the calibration parameters of each speckle, and transmit the obtained height values to the three-dimensional data processing device; The three-dimensional data processing device is used to obtain the three-dimensional contour information of the surface of the object to be measured according to the height values of each speckle relative to the reference datum plane, and analyze and detect the three-dimensional contour information of the surface of the object to be measured.

2. The three-dimensional measurement system based on pseudo-random speckles according to claim 1, characterized in that The image acquisition device includes a first image acquisition circuit and a second image acquisition circuit. Among them, The first image acquisition circuit is electrically connected to the image preprocessing device. The first image acquisition circuit is used to collect the speckle images of the reference datum plane under different conditions, and transmit the speckle images of the reference datum plane to the image preprocessing device; The second image acquisition circuit is electrically connected to the image preprocessing device. The second image acquisition circuit is used to collect the speckle images of the surface of the object to be measured under different conditions, and transmit the speckle images of the surface of the object to be measured to the image preprocessing device.

3. The three-dimensional measurement system based on pseudo-random speckles according to claim 2, characterized in that The image preprocessing device includes a first image preprocessing circuit and a second image preprocessing circuit. Among them, The first image preprocessing circuit is electrically connected to the first image acquisition circuit, the system calibration device, and the height calculation device. The first image preprocessing circuit is used to preprocess the speckle image of the reference datum plane to obtain the centroid coordinates of the speckles, and transmit the preprocessed speckle image of the reference datum plane and the centroid coordinates of the speckles to the system calibration device and the height calculation device; The second image preprocessing circuit is electrically connected to both the second image acquisition circuit and the height calculation device. The second image preprocessing circuit is configured to preprocess the speckle image on the surface of the object to be measured to obtain the centroid coordinates of the speckle points, and transmit the preprocessed speckle image on the surface of the object to be measured and the centroid coordinates of the speckle points to the height calculation device.

4. The three-dimensional measurement system based on pseudo-random speckles according to claim 3, characterized in that The height calculation device includes a speckle image matching circuit and a height information acquisition circuit, where The speckle image matching circuit is electrically connected to the first image preprocessing circuit, the second image preprocessing circuit, and the height information acquisition circuit. The speckle image matching circuit is configured to match the preprocessed speckle image of the reference datum plane with the preprocessed speckle image of the surface of the object to be measured, and transmit the matching result to the height information acquisition circuit; The height information acquisition circuit is electrically connected to the system calibration device, the speckle image matching circuit, and the three-dimensional data processing device. The height information acquisition circuit is configured to calculate the centroid offset according to the centroid coordinates of the speckle points, and calculate the height values of the respective speckle points relative to the reference datum plane according to the calibration parameters of the respective speckle points on the speckle image of the reference datum plane, and transmit the obtained height values of the respective speckle points relative to the reference datum plane to the three-dimensional data processing device.

5. The three-dimensional measurement system based on pseudo-random speckles according to claim 4, characterized in that The image preprocessing device is further configured to locate each speckle point on the speckle image of the reference datum plane and the speckle image on the surface of the object to be measured, and number each speckle point respectively according to the coordinate distribution of the speckle image of the reference datum plane and the speckle image on the surface of the object to be measured.

6. The three-dimensional measurement system based on pseudo-random speckles according to claim 4, characterized in that The system calibration device is further configured to detect the repetition pattern of the speckle image of the reference datum plane transmitted by the image preprocessing device.

7. The three-dimensional measurement system based on pseudo-random speckles according to claim 4, characterized in that The three-dimensional data processing device is further configured to perform three-dimensional reconstruction and data analysis on the height values of the respective speckle points on the speckle image of the surface of the object to be measured transmitted by the height calculation device, and perform combined stitching and local predetermined area mapping on the height values of the speckle points on the speckle images of all surfaces of the objects to be measured, so as to obtain the three-dimensional contour information of the surface of the object to be measured.

8. The three-dimensional measurement system based on pseudo-random speckles according to any one of claims 1-7, characterized in that The centroid offset is the centroid offset of the speckle points in the speckle image of the surface of the object to be measured relative to the centroid of the speckle image of the reference datum plane.

9. The three-dimensional measurement system based on pseudo-random speckles according to any one of claims 1-7, characterized in that The three-dimensional measurement system further includes a personal computer, a camera, a speckle generator, a calibration block, and a data cable. Among them, the personal computer, the camera, and the speckle generator are electrically connected through the data cable.

10. A three-dimensional measurement method based on pseudo-random speckles, which is executed by the three-dimensional measurement system according to any one of claims 1-9, characterized in that The three-dimensional measurement method includes: Respectively acquire the speckle image of the reference datum plane and the speckle image of the surface of the object to be measured; Perform image processing on the speckle image of the reference datum plane and the speckle image of the surface of the object to be measured to obtain the centroid coordinates of the corresponding speckle points; Obtain the corresponding calibration parameters according to the speckle image of the reference datum plane; Match the speckle image of the reference datum plane after preprocessing with the speckle image of the surface of the target to be measured after preprocessing; Calculate the centroid offset according to the centroid coordinates of the speckle points in the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured; Calculate the height value of each speckle point relative to the reference datum plane according to the calibration parameters of each speckle point on the speckle image of the reference datum plane and the centroid offset; Obtain the three-dimensional contour information of the surface of the target to be measured according to the height values of each speckle point on the speckle image of the surface of the target to be measured relative to the reference datum plane.

11. The three-dimensional measurement method based on pseudo-random speckles according to claim 10, wherein, The separately collecting the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured includes: Collect the speckle images of the reference datum plane under different conditions; Collect the speckle images of the surface of the target to be measured under different conditions.

12. The three-dimensional measurement method based on pseudo-random speckles according to claim 11, wherein, The performing image processing on the speckle image of the reference datum plane and the speckle image of the surface of the target to be measured to obtain the centroid coordinates of the corresponding speckle points includes: Perform preprocessing on the speckle image of the reference datum plane to obtain the centroid coordinates of the speckle points and the speckle image of the reference datum plane after preprocessing; Perform preprocessing on the speckle image of the surface of the target to be measured to obtain the centroid coordinates of the speckle points and the speckle image of the surface of the target to be measured after preprocessing.

13. A three-dimensional measurement device based on pseudo-random speckles, wherein, Includes: At least one processor and a storage, at least one of the processors executes the computer execution instructions stored in the storage, and at least one of the processors executes the three-dimensional measurement method based on pseudo-random speckles according to any one of claims 10 to 12.

Images