To-be-measured object surface multi-angle data output device, method and system and medium

Abstract

The invention discloses a to-be-measured object surface multi-angle data output device, method and system and a medium, and the device comprises at least one camera which is installed above a to-be-measured object; the first spectroscope is mounted outside the visual field of the camera and is used for dividing the laser beam into at least two sub-beams; the beam splitter is mounted on a propagation path of the sub-beams and is used for splitting the sub-beams and then enabling the sub-beams to reach the first reflectors at different positions, so that reflected light of the first reflectors is modulated through corresponding gratings to form stripe light, and the stripe light is projected to the surface of an object to be measured; the second spectroscope is mounted between the first reflector and the corresponding grating; according to the invention, the pointing stability of each projection light path is maintained in real time through active optical feedback compensation, the system drift error is fundamentally suppressed, long-term and reliable multi-angle phase deflection measurement is realized, and the pollution of low signal-to-noise ratio data is effectively suppressed by introducing a self-adaptive weighted fusion mechanism based on pixel-level contrast.

Description

Technical Field

[0001] This invention belongs to the field of machine vision, and in particular relates to a device, method, system and medium for outputting multi-angle data from the surface of an object under test. Background Technology

[0002] Surface defect detection is a key technology in industrial manufacturing and quality control, aiming to automatically identify scratches, dents, stains, unevenness, and other defects on product surfaces. Phase deflection, due to its high-precision optical measurement capabilities, has been widely applied to the three-dimensional morphology measurement and defect detection of precision components, optical mirrors, electronic screens, and other products. Its basic principle involves projecting a series of structured fringe lights with specific phase shift patterns onto the surface of the object under test using a single display screen or projector. A camera then captures the fringe images modulated by the surface morphology, and phase calculations are used to reconstruct the three-dimensional height information of the surface.

[0003] However, existing technologies primarily rely on fringe projection from a single observation angle. This method has a fundamental limitation: due to the complex geometry (such as steep slopes, depressions, and edges) and diverse optical properties (such as specular reflection, highlights, and low reflectivity areas) of the surface under test, a single fixed angle of projection can produce severe shadows or obstructions in certain areas, or result in insufficient reflected energy of the fringe light due to an unfavorable incident angle. These factors directly manifest as significant differences in fringe contrast (i.e., the amplitude of brightness variation) among pixel regions at different spatial locations in the acquired fringe image. In low-contrast regions, the signal-to-noise ratio of phase calculation is significantly reduced, leading to poor accuracy and high noise in the final height data of that region, and even causing phase resolution errors and data loss, severely affecting the integrity and reliability of the entire surface measurement.

[0004] To address the issue of incomplete coverage from a single viewpoint, an intuitive approach is to introduce multiple displays to project data from different angles to obtain more comprehensive surface information. However, existing technologies face challenges such as how to effectively integrate measurement data from multiple different angles, how to construct a multi-angle light source observation system, and how to resolve errors caused by pose deviations within the observation system.

[0005] Therefore, this invention provides a device, method, system, and medium for outputting multi-angle data of the surface of the object under test. The device maintains the pointing stability of each projection optical path in real time through active optical feedback compensation, fundamentally suppressing system drift error and realizing long-term reliable multi-angle phase deflection measurement. Furthermore, it introduces an adaptive weighted fusion mechanism based on pixel-level contrast, which effectively suppresses the contamination of low signal-to-noise ratio data. Summary of the Invention

[0006] The purpose of this invention is to overcome the above-mentioned problems in the prior art and to provide a device, method, system and medium for outputting multi-angle data of the surface of the object to be tested.

[0007] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0008] A multi-angle data output device for the surface of an object under test, comprising:

[0009] At least one camera is mounted above the object being measured;

[0010] The first beam splitter, installed outside the camera's field of view, is used to split the laser beam into at least two beams;

[0011] A beam splitter is installed on the propagation path of the sub-beams to split the sub-beams so that they reach the first reflecting mirrors at different positions. The reflected light from the first reflecting mirrors is then modulated by the corresponding grating to form striped light, which is then projected onto the surface of the object to be measured.

[0012] The second beam splitter is installed between the first reflector and the corresponding grating, and is used to split the reflected light from the first reflector to the corresponding grating and the corresponding detector respectively.

[0013] The processor, connected to the camera, is used to analyze the current offset of the reflected light from the first reflector corresponding to the currently activated grating at the current incident position of the detector relative to the reference point, and to control and adjust the first reflector so that the current offset is less than or equal to a preset offset threshold. Based on phase deflection analysis, the processor adjusts the detection data corresponding to the currently activated grating and outputs it.

[0014] The reference point is confirmed by the detector synchronously collecting the reflected light from the first reflector at the corresponding collection position when calibrating each grating and the corresponding camera.

[0015] Furthermore, the output device further includes:

[0016] The second reflector, installed between the beam splitter and the first beam splitter, is used to change the propagation path of the sub-beams, so that the sub-beams propagate in the vertical direction.

[0017] Furthermore, the output device further includes:

[0018] The third reflector, installed below the beam splitter, is used to directly reflect the beam from the beam splitter to the corresponding first reflector.

[0019] Furthermore, the grating is a liquid crystal spatial light modulator.

[0020] A method for outputting multi-angle data from the surface of an object under test involves projecting fringe light from at least two locations with different poses onto the surface of the object and outputting corresponding detection data based on phase deflection. The output method includes:

[0021] The laser beam is split into at least two sub-beams, which pass through at least one beam splitter and reach the corresponding first reflector. The reflected light from the first reflector is then modulated by the corresponding grating to form striped light, which is then projected onto the surface of the object to be tested.

[0022] Each grating and its corresponding camera are calibrated so that the camera can collect the stripe light corresponding to different gratings and the detector can simultaneously collect the reflected light from the first mirror, thereby confirming the reference point based on the collection position.

[0023] The current incident position of the light reflected by the first mirror corresponding to the currently activated grating is analyzed at the detector, and the current offset is calculated based on the reference point.

[0024] Adjust the first reflector so that the current offset is less than or equal to the preset offset threshold, and then adjust and output the detection data corresponding to the currently activated grating based on phase deflection analysis.

[0025] Furthermore, the output method also includes:

[0026] Determine if an activated grating exists on the propagation path corresponding to the current output of the beam splitter:

[0027] If yes, increase the current output splitting ratio to a preset high value; otherwise, decrease the current output splitting ratio to a preset low value.

[0028] Furthermore, the output method also includes:

[0029] A mapping model between the angle adjustment of the first reflector and the current offset is constructed, and the angle adjustment under different current offsets is collected to fit the parameters of the mapping model.

[0030] Furthermore, the output method also includes:

[0031] The detection data is collected and output to obtain the original height data for all pixel positions corresponding to different gratings;

[0032] The brightness variation of neighboring pixels at each pixel position is analyzed in the stripe patterns corresponding to different gratings captured by the camera. The weight of the corresponding original height data is determined based on the brightness variation, and then the weighted average of the original height data is used to obtain the final height data.

[0033] A multi-angle data output system for the surface of an object under test includes:

[0034] The laser control module is used to control the laser beam to split into at least two sub-beams, which pass through at least one beam splitter and reach the corresponding first reflector. The reflected light from the first reflector is then modulated by the corresponding grating to form striped light, which is then projected onto the surface of the object to be tested.

[0035] The reference calibration module is used to calibrate each grating and its corresponding camera, so that the camera can collect the stripe light corresponding to different gratings and simultaneously collect the reflected light of the first mirror through the detector, thereby confirming the reference point based on the collection position.

[0036] The offset analysis module is used to analyze the reflected light from the first mirror corresponding to the currently activated grating at the current incident position of the detector, and thus calculate the current offset based on the reference point.

[0037] The output adjustment module is used to adjust the first reflector so that the current offset is less than or equal to the preset offset threshold, thereby adjusting the detection data corresponding to the currently activated grating based on phase deflection analysis and outputting it.

[0038] A computer-readable storage medium includes a computer program that, when executed by a processor, implements the above-described output method.

[0039] The beneficial effects of this invention are:

[0040] (1) In this invention, a beam of laser light is intelligently split into multiple independent optical paths by a first beam splitter and a beam splitter. Each optical path is guided by a first reflector to a programmable grating to generate striped structured light, thereby realizing multi-angle striped light source illumination to obtain multi-angle phase deflection data. A second beam splitter and a detector are embedded in each sub-optical path to sample the position of the reflected light spot of the first reflector in real time. The processor compares this position with the reference point established during system calibration and drives the adjustment of the first reflector to eliminate the deviation. The first reflector is corrected in real time by the reference point confirmed during the initial system calibration so that the optical path accurately reaches the grating to generate striped light with a preset intensity distribution. This makes multi-angle phase deflection measurement free from dependence on an ultra-stable mechanical environment. The pointing stability of each projection optical path is maintained in real time through active optical feedback compensation, which fundamentally suppresses system drift error and realizes high-precision and high-stability static multi-angle striped light source illumination in phase deflection measurement.

[0041] (2) In this invention, the laser beam is split into at least two sub-beams, which pass through at least one beam splitter and reach the corresponding first reflector. The reflected light from the first reflector is modulated by the corresponding grating to form striped light, which is then projected onto the surface of the object to be measured. Multiple projection light paths with different poses are generated by a single laser light source to cooperate with the corresponding camera for multi-angle phase deflection measurement. This achieves co-source, stable, and programmable multi-angle striped light projection, providing a unified optical reference and variable illumination conditions for multi-angle phase deflection measurement.

[0042] By calibrating each grating and its corresponding camera, the camera can collect the fringe light corresponding to different gratings, and the detector can simultaneously collect the reflected light from the first mirror. The reference point is then determined based on the collection position. The calibration process determines the spatial pose of the corresponding grating so that the camera can capture the fringe light. Based on this, the collection position of the reflected light spot of the first mirror on the detector is recorded as the calibration reference point. This establishes an accurate initial reference for subsequent online monitoring and feedback control, ensuring that the geometric relationship of each projected light path strictly corresponds to the reference point, so as to optimize the pose of the first mirror through closed-loop control.

[0043] By analyzing the reflected light from the first reflector corresponding to the currently activated grating at the current incident position on the detector, the current offset is calculated based on the reference point. The offset of the current incident position of the light spot on the detector is analyzed in real time based on the calibrated reference point. By monitoring whether the direction of each projected light path has shifted in real time, it is determined whether the current optical path state still conforms to the geometric relationship at the time of calibration. This achieves real-time monitoring of the stability of the optical path and provides data support for subsequent active correction.

[0044] By adjusting the first reflector to make the current offset less than or equal to the preset offset threshold, the detection data corresponding to the currently activated grating is adjusted based on phase deflection analysis and output. This allows for rapid and accurate correction of the optical path direction based on the monitored offset. This closed-loop control ensures that the geometric relationship of the projected fringes is always strictly consistent with the calibration state, thus guaranteeing the accuracy and reliability of the phase deflection data.

[0045] (3) In this invention, high-precision original phase data with multiple complementary perspectives is obtained by projecting stripes at multiple angles, ensuring that each surface region obtains high-quality observations at least from one or more perspectives, thus overcoming the coverage blind spots and information loss of a single light source from the data source. On this basis, an adaptive weighted fusion mechanism based on pixel-level contrast is further introduced to optimize and fuse multi-angle phase data at the pixel level, effectively suppressing the pollution of low signal-to-noise ratio data, significantly improving reconstruction accuracy and consistency, and finally achieving high-precision and robust three-dimensional surface morphology reconstruction to overcome the problems of decreased data accuracy caused by shadows, occlusions, and uneven pixel-level contrast caused by a single light source angle in phase deflection measurement. Attached Figure Description

[0046] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0047] Figure 1 This is a schematic diagram of the output device in this invention;

[0048] Figure 2 This is a flowchart of the output method in this invention;

[0049] Figure 3 This is a block diagram of the output system structure in this invention;

[0050] In the diagram: 1. Detector; 2. First beam splitter; 3. Beam splitter; 4. First reflector; 5. Grating; 6. Second beam splitter; 7. Second reflector; 8. Third reflector. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] like Figure 1 As shown, the present invention first provides a multi-angle data output device for the surface of an object to be measured, comprising:

[0053] At least one camera is mounted above the object being measured;

[0054] The first beam splitter 2 is installed outside the camera's field of view and is used to split the laser beam into at least two beams;

[0055] Beam splitter 3 is installed on the propagation path of the sub-beams to split the sub-beams and send them to the first reflecting mirrors 4 at different positions. The reflected light from the first reflecting mirrors 4 is then modulated by the corresponding grating 5 to form striped light, which is then projected onto the surface of the object to be measured.

[0056] The second beam splitter 6 is installed between the first reflector 4 and the corresponding grating 5, and is used to split the reflected light from the first reflector 4 to the corresponding grating 5 and the corresponding detector 1 respectively.

[0057] The processor, connected to the camera, is used to analyze the current offset of the reflected light from the first reflector 4 corresponding to the currently activated grating 5 at the current incident position of the detector 1 relative to the reference point, and to control and adjust the first reflector 4 so that the current offset is less than or equal to a preset offset threshold. Based on the phase deflection analysis, the processor adjusts the detection data corresponding to the currently activated grating 5 and outputs it.

[0058] The reference point is confirmed by the detector 1 synchronously collecting the reflected light from the first reflector 4 at the corresponding collection position when calibrating each grating 5 and the corresponding camera.

[0059] In this embodiment, the camera type can be any camera type that satisfies the phase deflection fringe image acquisition, such as an area scan monochrome camera, an area scan color camera, a line scan monochrome camera, or a line scan color camera. When the camera type is a color camera, the color information of the object under test can also be acquired simultaneously. When the camera type is a line scan camera, it needs to maintain a relative motion state with the object under test in order to acquire pixel rows frame by frame to stitch together a complete image of the object under test.

[0060] In this embodiment, the number of cameras is the same as the number of corresponding gratings 5 ​​and is located on the reflected light path of the stripe light generated by the corresponding grating 5 incident on the surface of the object to be tested, and is used to collect the image of the object to be tested projected by the corresponding stripe light.

[0061] In this embodiment, the same light source can be split into different stripe light source outputs by the first beam splitter 2 and the beam splitter 3. This ensures the homogeneity and coherence between multiple angle optical paths, and ensures that each projection optical path has consistent wavelength, phase noise and time stability. This fundamentally eliminates the comparison error caused by the performance difference and drift inconsistency of multiple independent stripe light sources, and improves the phase consistency and measurement accuracy of multi-angle data.

[0062] In this embodiment, the function of the second beam splitter 6 is to split the light so that a portion of the light can be used for the pose adjustment of the first reflector 4 to control the offset adjustment, thereby improving the accuracy of stripe image acquisition and image quality.

[0063] In this embodiment, the two beam splits of the light source serve different purposes. The first beam split is used to form a multi-angle striped light source, and the second beam split is used to ensure the input quality of the multi-angle striped light source, so as to improve the quality of subsequent phase deflection data acquisition.

[0064] In this embodiment, the offset between the projection incident position of the second portion of light onto the corresponding detector 1 and the reference point is used to determine whether the incident angle or position of the other portion of light entering the corresponding grating 5 meets the usage requirements.

[0065] In this embodiment, the position of the reference point needs to be obtained through calibration in order to determine whether the offset exceeds a preset offset threshold.

[0066] In this embodiment, the preset offset threshold can be set based on experience or accurately obtained based on a specific calculation formula. It is used to determine whether the offset is too large compared with the calibrated reference point position, thereby affecting the incident angle or position of the corresponding grating 5, causing the striped light to fail to be generated according to the predetermined light intensity distribution.

[0067] In this invention, a laser beam is intelligently split into multiple independent optical paths by a first beam splitter 2 and a beam splitter 3. Each optical path is guided by a first reflector 4 to a programmable grating 5 to generate striped structured light, realizing multi-angle striped light source illumination to obtain multi-angle phase deflection data. A second beam splitter 6 and a detector 1 are embedded in each sub-optical path for real-time sampling of the reflected light spot position of the first reflector 4. The processor compares this position with the reference point established during system calibration and drives the adjustment of the first reflector 4 to eliminate deviation. The first reflector 4 is corrected in real time with the reference point confirmed during the initial system calibration, so that the optical path accurately reaches the grating 5 to generate striped light with a preset intensity distribution. This frees multi-angle phase deflection measurement from dependence on an ultra-stable mechanical environment. By actively optically feedback compensation, the pointing stability of each projection optical path is maintained in real time, fundamentally suppressing system drift error and realizing high-precision and high-stability static multi-angle striped light source illumination in phase deflection measurement.

[0068] In some embodiments, to allow for a more flexible and compact arrangement of the initial optical path from the light source to the beam splitter 3, adapting to different device structural designs and ensuring that the optical path propagates along a preset path, the output device further includes:

[0069] The second reflector 7 is installed between the beam splitter 3 and the first beam splitter 2 to change the propagation path of the sub-beam so that the sub-beam propagates in the vertical direction.

[0070] In this embodiment, by adding a second reflector 7 to change the propagation path of the sub-beam, the optical path space is folded, allowing the light source and beam splitting components to be arranged in a more flexible side position, optimizing the structural layout of the entire device, reducing the size of the equipment, and enhancing its engineering practicality.

[0071] In some embodiments, to enable the beam splitter 3 to utilize its beam splitting advantage, the multiple sub-beams output by the beam splitter 3 are precisely and reliably guided to the respective first reflecting mirrors 4, which may be located in different spatial directions. The output device further includes:

[0072] The third reflector 8 is installed below the beam splitter 3 and is used to directly reflect the light beam from the beam splitter 3 to the corresponding first reflector 4.

[0073] In this embodiment, the third reflector 8 configured for each sub-optical path serves the same purpose as the second reflector 7: to achieve precise and efficient directional transmission of the light beam. However, the two reflectors are placed at different positions in the beam splitter 3. The second reflector 7 is located upstream of the beam splitter 3 to reflect the light beam to the beam splitter 3 for beam splitting. The third reflector 8 is located downstream of the beam splitter 3. When the beam splitter 3 is in the transmission function, the propagation of the light path can be controlled through the third reflector 8. The arrangement of the second reflector 7 and the third reflector 8 ensures that each sub-beam can independently and accurately reach its corresponding first reflector 4, realizing the separation and collimation of the multi-angle stripe light source in physical space.

[0074] In some implementations, in order to achieve fast, flexible, and programmable control of the projected stripe pattern (such as period, direction, phase) to adapt to the needs of different measurement modes or high-precision phase calculation, the grating 5 is a liquid crystal spatial light modulator.

[0075] In this embodiment, by selecting a liquid crystal spatial light modulator as the grating 5, the generation method of stripe light is upgraded from a fixed physical grating 5 to a fully electronically controlled and programmable method. Stripe patterns with different parameters can be generated and switched in real time through software instructions, which greatly enhances the measurement flexibility and functionality of the system and facilitates the implementation of more complex phase deflection algorithms.

[0076] like Figure 2 As shown, the present invention further provides a method for outputting multi-angle data from the surface of a test object, which utilizes striped light from at least two different poses projected onto the surface of the test object, and outputs corresponding detection data based on phase deflection. The output method includes:

[0077] The laser beam is split into at least two sub-beams, which pass through at least one beam splitter 3 and reach the corresponding first reflector 4. The reflected light from the first reflector 4 is modulated by the corresponding grating 5 to form striped light, which is then projected onto the surface of the object to be tested.

[0078] Each grating 5 is calibrated with its corresponding camera, so that the camera collects the stripe light corresponding to different gratings 5, and the detector 1 synchronously collects the reflected light of the first reflector 4, thereby confirming the reference point based on the collection position.

[0079] The current incident position of the reflected light from the first reflector 4 corresponding to the currently activated grating 5 is analyzed at the detector 1, and the current offset is calculated based on the reference point.

[0080] Adjust the first reflector 4 so that the current offset is less than or equal to the preset offset threshold, and then adjust and output the detection data corresponding to the currently activated grating 5 based on phase deflection analysis.

[0081] In this embodiment, the laser beam is split into at least two sub-beams, which pass through at least one beam splitter 3 and reach the corresponding first reflector 4. The reflected light from the first reflector 4 is modulated by the corresponding grating 5 to form striped light, which is then projected onto the surface of the object under test. Multiple projection optical paths with different poses are generated by a single laser light source to cooperate with the corresponding camera for multi-angle phase deflection measurement. This achieves co-source, stable, and programmable multi-angle striped light projection, providing a unified optical reference and variable illumination conditions for multi-angle phase deflection measurement.

[0082] By calibrating each grating 5 and its corresponding camera, the camera can collect the stripe light corresponding to different gratings 5, and the detector 1 can simultaneously collect the reflected light from the first reflector 4. The reference point is then determined based on the collection position. The calibration process determines the spatial pose of the corresponding grating 5 so that the camera can capture the stripe light. Based on this, the collection position of the reflected light spot of the first reflector 4 on the detector 1 is recorded as the calibration reference point. This establishes an accurate initial reference for subsequent online monitoring and feedback control, ensuring that the geometric relationship of each projected light path strictly corresponds to the reference point, so as to optimize the pose of the first reflector 4 through closed-loop control.

[0083] By analyzing the reflected light from the first reflector 4 corresponding to the currently activated grating 5 at the current incident position of the detector 1, the current offset is calculated based on the reference point. The offset of the current incident position of the light spot on the detector 1 is analyzed in real time based on the calibrated reference point. By monitoring whether the direction of each projected light path has shifted in real time, it is determined whether the current optical path state still conforms to the geometric relationship at the time of calibration. This achieves real-time monitoring of the stability of the optical path and provides data support for subsequent active correction.

[0084] By adjusting the first reflector 4 to make the current offset less than or equal to the preset offset threshold, the detection data corresponding to the currently activated grating 5 is adjusted and output based on the phase deflection analysis. This allows for rapid and accurate correction of the optical path direction based on the monitored offset. This closed-loop control ensures that the geometric relationship of the projected fringes is always strictly consistent with the calibration state, thus guaranteeing the accuracy and reliability of the phase deflection data.

[0085] In some implementations, in order to dynamically and intelligently allocate limited light source energy in a multi-path beam splitting system according to the currently used projection optical path, so as to avoid energy waste and improve the quality of the effective signal, the output method further includes:

[0086] Determine if an activated grating 5 exists on the propagation path corresponding to the current output of beam splitter 3:

[0087] If yes, increase the current output splitting ratio to a preset high value; otherwise, decrease the current output splitting ratio to a preset low value.

[0088] In this embodiment, the beam splitting ratio of the beam splitter 3 is dynamically adjusted by judging the usage status of the optical path at the back end of the beam splitter 3, so as to realize the intelligent on-demand allocation of optical energy according to actual needs, and to concentrate the laser energy to supply the currently working optical path, significantly improving the intensity and signal-to-noise ratio of the stripe light of the path, thereby effectively improving the data quality of the current measurement angle without changing the total power of the light source.

[0089] In some implementations, to make the feedback control process of the first reflector 4 faster and more precise, avoiding repeated trial and error or oscillation, thereby improving the response speed and control accuracy of the entire self-calibration system, the output method further includes:

[0090] A mapping model between the angle adjustment amount of the first reflector 4 and the current offset is constructed, and the angle adjustment amount under different current offsets is collected to fit the parameters of the mapping model.

[0091] In this embodiment, by establishing a quantitative mapping model between offset and adjustment and fitting its parameters, a precise mathematical model and feedforward compensation basis are provided for feedback control. This enables the control system to directly calculate the required precise correction amount based on the monitored offset, thereby achieving fast, overshoot-free precise positioning, significantly shortening the optical path stabilization time, and improving the overall dynamic performance and stability accuracy of the system.

[0092] In some embodiments, in order to effectively fuse the phase deflection data acquired by the camera under the transmission of fringe light sources at various angles, the output method further includes:

[0093] The detection data is collected and output to obtain the original height data for all pixel positions corresponding to different gratings 5;

[0094] The brightness variation of neighboring pixels at each pixel position is analyzed in the stripe patterns corresponding to different gratings 5 ​​captured by the camera. The weight of the corresponding original height data is determined based on the brightness variation, and the weighted average of each original height data is then used to obtain the final height data.

[0095] In this embodiment, the degree of brightness change represents the contrast of the neighboring pixels at each pixel location, and the neighboring pixels at each pixel location represent the corresponding area at each pixel location.

[0096] In this embodiment, the fusion method includes:

[0097] Control at least two displays to generate striped light that is projected onto the surface of the object under test, such that each display projects at least three striped images with different phase shifts in two orthogonal directions;

[0098] Traverse the gray values ​​of all pixel positions in each stripe image and calculate the contrast of the region corresponding to the same pixel position in stripe images with different phase shifts on the same display screen to obtain the average contrast value. Then, use the average contrast value that meets the preset threshold range as the weight corresponding to the current pixel position.

[0099] Analyze the stripe images corresponding to each display screen, calculate the detection data corresponding to each display screen based on phase deflection, and then obtain the different original height data corresponding to all pixel positions based on each detection data;

[0100] Extract the different original height data corresponding to all pixel positions, and then calculate the final height data corresponding to each pixel position by weighted averaging according to the weight corresponding to each pixel position.

[0101] If the average contrast value does not meet the preset threshold range, the weight corresponding to the current pixel position is set to zero.

[0102] In this embodiment, the method for determining the preset threshold range of contrast can be either by directly setting a fixed range based on experience, or by setting a corresponding range based on the region corresponding to each pixel position, specifically including:

[0103] A1. Calculate the mean contrast (mean1) and first standard deviation (std1) within the corresponding regions of all pixel locations;

[0104] A2. Calculate the deviation β between the current pixel brightness I(x,y) and the mean contrast value:

[0105] β=|I(x,y)-mean1|

[0106] If β > k × std1 (k is a constant, usually set according to the actual situation to adjust the threshold range), then the contrast of the region corresponding to the pixel position does not meet the preset threshold range.

[0107] In this invention, stripes are projected from multiple angles to obtain high-precision raw phase data with complementary multi-view perspectives, ensuring that each surface region receives high-quality observations from at least one or more viewpoints, thus overcoming the coverage blind spots and information gaps of a single light source from the data source. Building upon this, an adaptive weighted fusion mechanism based on pixel-level contrast is further introduced to optimize and fuse multi-angle phase data at the pixel level. This effectively suppresses contamination from low signal-to-noise ratio data, significantly improving reconstruction accuracy and consistency, ultimately achieving high-precision and robust 3D surface topography reconstruction. This overcomes the data accuracy degradation problems caused by shadows, occlusion, and uneven pixel-level contrast due to a single light source angle in phase deflection measurements.

[0108] In this invention, at least two displays are controlled to project striped light onto the surface of the object under test. Each display projects at least three striped images with different phase shifts in two orthogonal directions, ensuring that each pixel area on the surface of the object under test has multiple viewing angles. Complete phase modulation information of the surface of the object under test is obtained from different illumination angles, providing data support for multi-view data fusion, solving the coverage blind zone problem of a single viewpoint, and detecting different defect types on the surface of the object under test by using striped images with different phase shifts in two orthogonal directions, thereby improving the defect detection efficiency of the object under test.

[0109] By iterating through the grayscale values ​​of all pixel positions in each stripe image and calculating the contrast of the corresponding region at the same pixel position in stripe images with different phase shifts on the same display screen, the average contrast value is obtained. The average contrast value that meets the preset threshold range is used as the weight corresponding to the current pixel position, so as to introduce local contrast as an evaluation standard for measurement data quality. The high signal-to-noise ratio data corresponding to high contrast pixel regions are used to participate in the phase calculation result fusion with higher weight, thereby improving data reliability. Invalid data caused by complete shadow or overexposure is filtered out using the preset threshold range to further improve data reliability. The physical measurement quality is transformed into mathematical fusion weight, providing a scientific basis for subsequent weighted fusion.

[0110] By analyzing the stripe images corresponding to each display screen, the detection data corresponding to each display screen is obtained based on phase deflection calculation. Then, based on the detection data, the original height data corresponding to all pixel positions is obtained. Stripe images from multiple perspectives are projected onto the object under test to acquire images of the object under test from different light source perspectives. The phase deflection algorithm is applied to the image of the object under test to calculate the phase distribution map under each channel. Then, through pre-calibrated system parameters, the phase values ​​are converted into the original height data of the object surface. Each pixel position in the image of the object under test contains the original height data from different light source perspectives to overcome the data accuracy problem caused by blind zone coverage under a single perspective.

[0111] By extracting different original height data corresponding to all pixel positions, and then weighting and averaging the data according to the weight of each pixel position, the final height data corresponding to each pixel position is obtained. This aims to integrate the original height data from multiple light source perspectives to select the best and eliminate the worst, effectively suppressing the error in the final three-dimensional shape accuracy caused by pixel-level contrast differences under different perspectives. Based on the actual optical response characteristics of each pixel on the object surface from different light source perspectives, the contribution of the data corresponding to each perspective is dynamically adjusted. By giving higher signal-to-noise ratio data higher weights, the system uses more reliable data, significantly improving the measurement accuracy of the final three-dimensional shape data.

[0112] In this embodiment, the weight of the corresponding pixel position whose average contrast does not meet the preset threshold range is set to zero. The reason is that when a pixel is in shadow at a certain viewpoint, the average contrast will be too low. When specular reflection saturation occurs at a certain viewpoint, the average contrast will be too high. When both of these situations exceed the reasonable preset threshold range, the data is determined to be invalid. The weight of the corresponding viewpoint of the pixel is set to zero so that it does not participate in the final weighted average calculation. This eliminates the pollution of the fusion result by measurement data with poor quality and unreliability, thereby improving the fusion quality of the final height data.

[0113] In some implementations, in order to clarify the specific stripe type required to realize phase measurement profilometry and ensure the theoretical basis and accuracy of phase calculation, the stripe light is sinusoidal stripe light or cosine stripe light, that is, the gray value of the stripe light along the stripe change direction is sinusoidal or cosine distributed.

[0114] In this embodiment, the intensity distribution of the projected stripes is limited to follow a sine or cosine function to meet the data input requirements for achieving high-precision phase calculation (such as phase shift method).

[0115] In this embodiment, satisfying the phase shift method means that the acquired channel data is preprocessed before phase deflection to make the data satisfy the phase shift method calculation. The specific preprocessing steps need to be set according to the actual situation.

[0116] In this embodiment, sine or cosine refers to the pattern of light intensity variation. The light intensity distribution of sinusoidal striped light, along the direction perpendicular to the stripes, strictly follows the law of a sine (or cosine) function, which can be specifically expressed by the following formula:

[0117] I(x,y)=A+Bcos(2πfx+φ0)

[0118] Where: I(x,y) is the intensity of a point on the image plane, A is the average light intensity (background light), B is the modulation degree (contrast) of the stripes, f is the spatial frequency of the stripes (how many cycles per unit distance), x is the position coordinate on the image plane, and φ0 is the initial phase.

[0119] In some implementations, in order to provide the most commonly used and reliable phase shifting scheme and to optimally resolve the phase from the stripe image, each display screen projects four stripe images with different phase shifts in two orthogonal directions, and the phase shifts of the four stripe images are 0°, 90°, 180°, and 270°, respectively.

[0120] In this embodiment, a standard four-step phase-shifting method is adopted with a phase-shifting interval of 90 degrees to effectively eliminate the influence of background light intensity and surface reflectivity non-uniformity. This method is used to obtain the wrapping phase through high-precision and high-robustness calculations, providing the most reliable input for subsequent phase unfolding and height calculations.

[0121] In some implementations, in order to determine the size of the local region for local contrast, so that the contrast value can stably reflect the stripe quality within a small neighborhood, rather than the random fluctuations of a single pixel, the region corresponding to the same pixel position is an N×N array centered on the current pixel position, where N is an odd number.

[0122] In this embodiment, by setting the region corresponding to the pixel position as a window of odd size (such as 3x3, 5x5) centered on the pixel, missing data for edge pixels is filled in by mirroring. The contrast of the data in the neighborhood centered on the corresponding pixel is determined as the contrast of the pixel, thereby improving data reliability.

[0123] In this embodiment, the value of N determines the error accuracy of the collected data. A larger value of N indicates higher error accuracy of the acquired data, while a smaller value of N indicates lower error accuracy of the acquired data. The specific value of N needs to be set according to the actual situation to achieve a balance between error accuracy and computing resources.

[0124] In some implementations, to provide a specific, normalized, and physically clear standard formula for calculating contrast, and to ensure the consistency and comparability of weight calculations, the formula for calculating contrast is as follows:

[0125]

[0126] Among them, C (x,y) I represents the contrast at pixel location (x, y). max I is the maximum gray value in an N×N array centered at pixel position (x,y). min It is the minimum gray value in an N×N array centered at pixel position (x,y).

[0127] In this embodiment, the contrast calculation formula is used to calculate the relative difference between the maximum and minimum gray levels within a local window, and the result is normalized to the [0,1] interval, so that the result only reflects the modulation depth and is independent of the absolute light intensity. This achieves the standardization of the contrast value, making the contrast from different viewing angles and different absolute brightness comparable, and laying a mathematical foundation for setting a unified preset threshold interval and performing weighted fusion.

[0128] In this embodiment, in order to remove abnormal pixels that are either too bright or too dark in the N×N array centered at pixel position (x,y) to avoid abnormal pixels contaminating the contrast calculation results, the specific filtering method includes:

[0129] S1. Calculate the second brightness mean2 and the second standard deviation std2 of the entire image;

[0130] S2. Set the brightness threshold to mean2±3std2. Pixels with brightness values ​​higher than mean2+3std2 are considered abnormally bright, and pixels with brightness values ​​lower than mean2-3std2 are considered abnormally dark.

[0131] S3. When a pixel that is not within the brightness threshold is detected in the N×N array centered at pixel position (x,y), the pixel is marked as invalid and is not included in the contrast calculation formula.

[0132] In some implementations, to combine adaptive weights with the original height data to provide a specific, operable fusion operator, the weighted average is calculated using the following formula:

[0133]

[0134] Among them, H (x,y) h represents the final height data at pixel position (x, y). (x,y)_i ω represents the original height data at pixel position (x, y) in the detection data corresponding to the i-th display screen. (x,y)_i Let M be the weight of the pixel position (x, y) in the detection data corresponding to the i-th display screen, and M be the number of display screens.

[0135] In this embodiment, the weighted average formula is used to limit the final height data to a weighted average of the original height data from each viewpoint, and contrast is used as an evaluation index for data reliability. By using high-quality data to dominate the results, accuracy is improved.

[0136] In some implementations, in order to filter out and correct significant gross errors or invalid values ​​caused by outliers in the original height data due to phase calculation errors, after obtaining different original height data corresponding to all pixel positions, it is determined whether the current original height data meets the preset height threshold range: if yes, the next step is continued; if no, the current original height data is replaced with the original height data corresponding to any pixel in the adjacent position that meets the preset height threshold range.

[0137] In this embodiment, by setting a reasonable physical height range, i.e., a preset height threshold interval, obviously unreasonable abnormal data is filtered out and replaced with effective neighboring values ​​to achieve data repair, thereby improving the cleanliness of the data source and preventing these abnormal data from having an adverse effect on the final result in the weighted average, and further enhancing the robustness of the entire fusion method.

[0138] In some implementations, to simplify system calibration and data alignment, all stripe images are captured by the same camera.

[0139] In this embodiment, by limiting the use of a single camera to acquire stripe images projected by all displays, all raw data are in the same camera coordinate system, avoiding complex extrinsic parameter calibration and image registration between multiple cameras. This greatly simplifies the system structure and data processing flow, requiring only the calibration of the positional relationship between a single camera and multiple displays. This reduces the difficulty of system integration and computational cost, making the multi-angle fusion scheme easier to implement and practical.

[0140] like Figure 3 As shown, the present invention also provides a multi-angle data output system for the surface of an object under test, comprising:

[0141] The laser control module is used to control the laser beam to split into at least two sub-beams, which pass through at least one beam splitter and reach the corresponding first reflector. The reflected light from the first reflector is then modulated by the corresponding grating to form striped light, which is then projected onto the surface of the object to be tested.

[0142] The reference calibration module is used to calibrate each grating and its corresponding camera, so that the camera can collect the stripe light corresponding to different gratings and simultaneously collect the reflected light of the first mirror through the detector, thereby confirming the reference point based on the collection position.

[0143] The offset analysis module is used to analyze the reflected light from the first mirror corresponding to the currently activated grating at the current incident position of the detector, and thus calculate the current offset based on the reference point.

[0144] The output adjustment module is used to adjust the first reflector so that the current offset is less than or equal to the preset offset threshold, thereby adjusting the detection data corresponding to the currently activated grating based on phase deflection analysis and outputting it.

[0145] Finally, the present invention also provides a computer-readable storage medium including a computer program that, when executed by a processor, implements the above-described output method.

[0146] In practical applications, a computer-readable storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0147] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0148] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0149] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0150] In the description of this specification, references to terms such as "an embodiment," "example," and "specific example" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0151] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A device for outputting multi-angle data of the surface of an object to be measured, characterized in that, include: At least one camera is mounted above the object being measured; The first beam splitter, installed outside the camera's field of view, is used to split the laser beam into at least two beams; A beam splitter is installed on the propagation path of the sub-beams to split the sub-beams so that they reach the first reflecting mirrors at different positions. The reflected light from the first reflecting mirrors is then modulated by the corresponding grating to form striped light, which is then projected onto the surface of the object to be measured. The second beam splitter is installed between the first reflector and the corresponding grating, and is used to split the reflected light from the first reflector to the corresponding grating and the corresponding detector respectively. The processor, connected to the camera, is used to analyze the current offset of the reflected light from the first reflector corresponding to the currently activated grating at the current incident position of the detector relative to the reference point, and to control and adjust the first reflector so that the current offset is less than or equal to a preset offset threshold. Based on phase deflection analysis, the processor adjusts the detection data corresponding to the currently activated grating and outputs it. The reference point is confirmed by the detector synchronously collecting the reflected light from the first reflector at the corresponding collection position when calibrating each grating and the corresponding camera.

2. The multi-angle data output device for the surface of an object under test according to claim 1, characterized in that, Also includes: The second reflector, installed between the beam splitter and the first beam splitter, is used to change the propagation path of the sub-beams, so that the sub-beams propagate in the vertical direction.

3. The multi-angle data output device for the surface of an object under test according to claim 1, characterized in that, Also includes: The third reflector, installed below the beam splitter, is used to directly reflect the beam from the beam splitter to the corresponding first reflector.

4. The multi-angle data output device for the surface of an object under test according to claim 1, characterized in that, The grating is a liquid crystal spatial light modulator.

5. A method for outputting multi-angle data from the surface of an object under test, comprising projecting fringe light from at least two different poses onto the surface of the object under test, and outputting corresponding detection data based on phase deflection, characterized in that, The output method includes: The laser beam is split into at least two sub-beams, which pass through at least one beam splitter and reach the corresponding first reflector. The reflected light from the first reflector is then modulated by the corresponding grating to form striped light, which is then projected onto the surface of the object to be tested. Each grating and its corresponding camera are calibrated so that the camera can collect the stripe light corresponding to different gratings and the detector can simultaneously collect the reflected light from the first mirror, thereby confirming the reference point based on the collection position. The current incident position of the light reflected by the first mirror corresponding to the currently activated grating is analyzed at the detector, and the current offset is calculated based on the reference point. Adjust the first reflector so that the current offset is less than or equal to the preset offset threshold, and then adjust and output the detection data corresponding to the currently activated grating based on phase deflection analysis.

6. The method for outputting multi-angle data of the surface of an object under test according to claim 5, characterized in that, Also includes: Determine if an activated grating exists on the propagation path corresponding to the current output of the beam splitter: If yes, increase the current output splitting ratio to a preset high value; otherwise, decrease the current output splitting ratio to a preset low value.

7. The method for outputting multi-angle data of the surface of an object under test according to claim 5, characterized in that, Also includes: A mapping model between the angle adjustment of the first reflector and the current offset is constructed, and the angle adjustment under different current offsets is collected to fit the parameters of the mapping model.

8. The method for outputting multi-angle data of the surface of an object under test according to claim 5, characterized in that, Also includes: The detection data is collected and output to obtain the original height data for all pixel positions corresponding to different gratings; The brightness variation of neighboring pixels at each pixel position is analyzed in the stripe patterns corresponding to different gratings captured by the camera. The weight of the corresponding original height data is determined based on the brightness variation, and then the weighted average of the original height data is used to obtain the final height data.

9. A multi-angle data output system for the surface of an object under test, characterized in that, include: The laser control module is used to control the laser beam to split into at least two sub-beams, which pass through at least one beam splitter and reach the corresponding first reflector. The reflected light from the first reflector is then modulated by the corresponding grating to form striped light, which is then projected onto the surface of the object to be tested. The reference calibration module is used to calibrate each grating and its corresponding camera, so that the camera can collect the stripe light corresponding to different gratings and simultaneously collect the reflected light of the first mirror through the detector, thereby confirming the reference point based on the collection position. The offset analysis module is used to analyze the reflected light from the first mirror corresponding to the currently activated grating at the current incident position of the detector, and thus calculate the current offset based on the reference point. The output adjustment module is used to adjust the first reflector so that the current offset is less than or equal to the preset offset threshold, thereby adjusting the detection data corresponding to the currently activated grating based on phase deflection analysis and outputting it.

10. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the output method as described in any one of claims 5-8.

Images