3D camera and debugging method, device, storage medium, program product

Abstract

Embodiments of the present application provide a 3D camera and a debugging method and device, a storage medium and a program product. An industrial computer acquires a point cloud diagram collected by a laser line scanning structured light camera (3D camera) based on a current laser projection angle and a corresponding depth diagram. In response to a laser debugging operation, the industrial computer determines a target laser projection angle according to a to-be-debugged position marked in the depth diagram and camera calibration parameters, sends control information including the target laser projection angle to the laser line scanning structured light camera to control the laser line scanning structured light camera to project a laser line based on the target laser projection angle, so that the laser line can cover the to-be-debugged position, and the point cloud imaging effect of an area where the to-be-debugged position is located can be accurately improved. The user does not need to manually and repeatedly adjust camera parameters, the efficiency and accuracy of 3D camera debugging can be improved, and thus the point cloud imaging effect is improved.

Description

Technical Field

[0001] This application relates to the field of three-dimensional imaging technology, and in particular to a 3D camera and its debugging method, device, storage medium, and program product. Background Technology

[0002] Laser line scanning structured light 3D imaging technology is widely used in industrial inspection, 3D modeling, automated production, medical imaging, and robot navigation. In industrial scenarios, such as dimensional inspection of precision parts, body contour measurement in automobile manufacturing, or package volume recognition in logistics sorting, laser line scanning structured light cameras project laser lines and capture their deformation on the surface of objects to generate high-precision depth maps and point cloud data, thereby achieving 3D reconstruction of the target object.

[0003] However, in practical applications, factors such as ambient light interference, laser line projection angle deviation, improper camera parameter settings, or differences in the reflective properties of object surfaces often lead to missing or distorted point cloud data, affecting the subsequent measurement accuracy and detection reliability.

[0004] When the point cloud effect is not as expected during the imaging process of a laser line-scanning structured light camera, users need to manually adjust the camera parameters to optimize the point cloud imaging effect. However, the existing adjustment methods rely on manual experience, and users need to repeatedly adjust the parameters and scan multiple times to observe the effect. The correlation between parameters and imaging results is unclear, resulting in low adjustment efficiency and easy omission of key issues. Summary of the Invention

[0005] This application provides a 3D camera and debugging method, device, storage medium, and program product to solve the problems of low debugging efficiency and easy omission of key issues in existing 3D camera debugging methods.

[0006] In a first aspect, embodiments of this application provide a 3D camera debugging method, applied to an industrial control computer, the method comprising:

[0007] Acquire the point cloud map and its corresponding depth map collected by the laser line scan structured light camera based on the current laser projection angle;

[0008] In response to a laser debugging operation, a target laser projection angle is determined based on the location to be debugged marked in the depth map and camera calibration parameters, wherein the target laser projection angle is used to ensure that the laser line projected by the laser line scan structured light camera includes the location to be debugged.

[0009] Control information, including the target laser projection angle, is sent to the laser line scan structured light camera.

[0010] In one possible implementation, after acquiring the point cloud map and its corresponding depth map collected by the laser line-scan structured light camera based on the current laser projection angle, and before determining the target laser projection angle according to the position to be debugged and the camera calibration parameters in response to the laser debugging operation, the method further includes:

[0011] The visualization interface displays the point cloud map and its corresponding depth map acquired by the laser line scan structured light camera based on the current laser projection angle;

[0012] In response to the user's operation of marking the location to be debugged in the depth map displayed on the visualization interface, the pixel coordinates of the location to be debugged are determined.

[0013] In one possible implementation, the camera calibration parameters include camera intrinsic parameters and galvanometer zero-position angle values. Determining the target laser projection angle based on the location to be calibrated marked in the depth map and the camera calibration parameters includes:

[0014] Based on the pixel coordinates of the location to be debugged marked in the depth map and the camera intrinsic parameters, determine the spatial point coordinates of the location to be debugged in the camera coordinate system.

[0015] The target laser projection angle is determined based on the spatial coordinates of the position to be debugged in the camera coordinate system and the zero-position angle value of the galvanometer.

[0016] In one possible implementation, sending control information including the target laser projection angle to the laser line-scanning structured light camera includes:

[0017] In response to the point cloud image acquisition operation, control information including the target laser projection angle is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to generate a point cloud image and its corresponding depth map based on the laser line image acquired at the target laser projection angle.

[0018] In one possible implementation, the method further includes:

[0019] The target laser projection angle and the laser line image acquired by the laser line scanning structured light camera are displayed on the visualization interface;

[0020] In response to the user's adjustment operation on the displayed laser projection angle in the visualization interface, the target laser projection angle is adjusted;

[0021] In response to the laser line image acquisition operation, control information including the adjusted target laser projection angle is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to emit a laser line and acquire a laser line image based on the adjusted target laser projection angle.

[0022] The laser line image displayed on the visualization interface is updated based on the laser line image currently acquired by the laser line scanning structured light camera.

[0023] In one possible implementation, the method further includes:

[0024] The center line of the laser stripes in the laser line image is superimposed on the laser line image displayed in the visualization interface.

[0025] In one possible implementation, the method further includes:

[0026] The visualization interface displays the imaging parameters of the laser line scan structured light camera, including at least one of single-line exposure time, camera gain, and laser brightness.

[0027] The imaging parameters are adjusted in response to the user's adjustment operation on the displayed imaging parameters in the visualization interface.

[0028] In one possible implementation, the step of sending control information containing the adjusted target laser projection angle to the laser line scanning structured light camera in response to the laser line image acquisition operation specifically includes:

[0029] In response to the laser line image acquisition operation, control information including the adjusted target laser projection angle and the adjusted imaging parameters is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to emit a laser line and acquire a laser line image based on the adjusted target laser projection angle and the adjusted imaging parameters.

[0030] In one possible implementation, after sending control information including the target laser projection angle to the laser line-scanning structured light camera, the method further includes:

[0031] Acquire the point cloud image and its corresponding depth map captured by the laser line scan structured light camera in this instance;

[0032] The visualization interface displays the point cloud map and its corresponding depth map collected this time, as well as at least one point cloud map and its corresponding depth map collected in the past.

[0033] In response to the user's selection, the selected point cloud map is output as the final point cloud map.

[0034] In one possible implementation, the method further includes:

[0035] The laser projection angle corresponding to the most recently output point cloud map is used as the current laser projection angle and stored.

[0036] In one possible implementation, the method further includes:

[0037] In response to a user marking a location to be debugged in the depth map displayed on the visualization interface, a marker line for the location to be debugged is displayed in the depth map displayed on the visualization interface. The marker line is used to indicate the position of the laser line projected onto the location to be debugged.

[0038] Secondly, embodiments of this application provide a 3D camera, including:

[0039] Receive control information including the target laser projection angle;

[0040] The laser projection angle of the 3D camera is adjusted according to the target laser projection angle.

[0041] In one possible implementation, the target laser projection angle is determined based on the location to be debugged marked in a historically acquired depth map and camera calibration parameters.

[0042] Thirdly, embodiments of this application provide a 3D camera debugging device, including: a memory and a processor;

[0043] The memory stores computer-executed instructions;

[0044] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0045] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0046] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0047] This application provides a 3D camera and debugging method, device, storage medium, and program product. In this method, an industrial control computer can acquire a point cloud map and its corresponding depth map collected by a laser line-scanning structured light camera (3D camera) based on the current laser projection angle. In response to a laser debugging operation, the industrial control computer determines a target laser projection angle based on the location to be debugged marked in the depth map and the camera calibration parameters. The target laser projection angle is used to ensure that the laser line projected by the laser line-scanning structured light camera includes the location to be debugged, thereby accurately associating the laser projection angle of the 3D camera with the location to be debugged. The industrial control computer sends control information including the target laser projection angle to the laser line-scanning structured light camera to control the laser line-scanning structured light camera to project a laser line based on the target laser projection angle, so that the laser line can cover the location to be debugged. This can accurately improve the point cloud imaging effect in the area where the location to be debugged is located, eliminating the need for the user to manually and repeatedly adjust camera parameters, improving the efficiency and accuracy of 3D camera debugging, and thus improving the point cloud imaging effect. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] Figure 1 A schematic diagram of the system architecture to which this application applies;

[0050] Figure 2 A flowchart illustrating a 3D camera debugging method provided for an exemplary embodiment of this application;

[0051] Figure 3 A detailed flowchart illustrating the debugging process of a 3D camera provided for an exemplary embodiment of this application;

[0052] Figure 4 An example of a debugging interface provided for embodiments of this application. Figure 1 ;

[0053] Figure 5 An example of a debugging interface provided for embodiments of this application. Figure 2 ;

[0054] Figure 6 An example of a debugging interface provided for embodiments of this application. Figure 3 ;

[0055] Figure 7 An example of a debugging interface provided for embodiments of this application. Figure 4 ;

[0056] Figure 8 A schematic diagram of a debugging interface provided in an embodiment of this application. Figure 5 ;

[0057] Figure 9 This is a schematic diagram of the structure of the 3D camera debugging device provided in this application.

[0058] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0059] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0060] First, let me explain the terms used in this application:

[0061] Structured light camera: A device used to measure the three-dimensional features of an object, also called a structured light 3D camera. It includes an optical engine and a camera that cooperate with each other and are fixed in position. The optical engine projects a grating stripe pattern onto the surface of the object being measured, and then the camera takes pictures of the pattern on the surface of the object. Based on pre-encoded rules, the captured pattern data is decoded and processed to obtain a high-precision three-dimensional (3D) point cloud of the object being measured.

[0062] When the point cloud effect is not as expected during the imaging process of a laser line-scanning structured light camera, users need to manually adjust the camera parameters to optimize the point cloud imaging effect. However, the existing adjustment methods rely on manual experience, and users need to repeatedly adjust the parameters and scan multiple times to observe the effect. The correlation between parameters and imaging results is unclear, resulting in low adjustment efficiency and easy omission of key issues.

[0063] To address the aforementioned technical problems, this application provides a 3D camera debugging method. An industrial control computer can acquire a point cloud map and its corresponding depth map collected by a laser line-scanning structured light camera (3D camera) based on the current laser projection angle. In response to a laser debugging operation, the industrial control computer determines a target laser projection angle based on the location to be debugged marked in the depth map and the camera calibration parameters. The target laser projection angle is used to ensure that the laser line projected by the laser line-scanning structured light camera includes the location to be debugged, thereby accurately associating the laser projection angle of the 3D camera with the location to be debugged. The industrial control computer sends control information, including the target laser projection angle, to the laser line-scanning structured light camera.

[0064] The laser line scan structured light camera (3D camera) receives control information including the target laser projection angle; it adjusts the laser projection angle of the laser line scan structured light camera according to the target laser projection angle. The laser line projected by the laser line scan structured light camera (3D camera) based on the target laser projection angle can cover the area to be debugged, which can accurately improve the point cloud imaging effect of the area to be debugged. This eliminates the need for users to manually and repeatedly adjust camera parameters, improving the efficiency and accuracy of 3D camera debugging, thereby enhancing the point cloud imaging effect.

[0065] Figure 1 A schematic diagram of the system architecture to which this application applies, such as Figure 1 As shown, the system architecture includes an industrial control computer and a 3D camera. The industrial control computer and the 3D camera have a communication connection for data transmission.

[0066] The 3D camera can be a laser line-scanning structured light camera. When acquiring a point cloud image of the target object, the 3D camera projects a laser line (also called a laser stripe or laser beam) onto the target object based on the current laser projection angle, acquires the laser line image, and generates a point cloud image and its corresponding depth map based on the laser line image. The 3D camera can send at least one of the acquired laser line image, point cloud image, and corresponding depth map data to the industrial control computer as needed by the system.

[0067] The industrial control computer can analyze the point cloud map and its corresponding depth map that have been acquired by the 3D camera. If the point cloud map is not good (such as missing points, local distortion / noise, etc.), it can adaptively adjust the laser projection angle of the 3D camera according to the position to be debugged marked in the depth map and the camera calibration parameters of the 3D camera.

[0068] The 3D camera re-captures point cloud maps and their corresponding depth maps based on the adjusted laser projection angle.

[0069] Furthermore, the above debugging process can be iterated multiple times to better improve the point cloud imaging effect.

[0070] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0071] Figure 2 This is a flowchart illustrating a 3D camera debugging method provided as an exemplary embodiment of this application. The execution entity in this embodiment is an industrial control computer. For example... Figure 2 As shown, the specific steps of this method are as follows:

[0072] S201. Obtain the point cloud map and its corresponding depth map collected by the laser line scan structured light camera based on the current laser projection angle.

[0073] After acquiring a point cloud map and its corresponding depth map based on the current laser projection angle, the laser line-scanning structured light camera sends the acquired point cloud map and its corresponding depth map to the industrial control computer. The industrial control computer receives the point cloud map and its corresponding depth map acquired based on the current laser projection angle from the laser line-scanning structured light camera.

[0074] The industrial control computer can provide a visual interface for parameter debugging, also known as a debugging interface. After acquiring the point cloud map and its corresponding depth map collected by the laser line-scanning structured light camera based on the current laser projection angle, the industrial control computer can display the point cloud map and its corresponding depth map in the visual interface. The point cloud map and depth map can be displayed in different areas of the debugging interface.

[0075] For example, the debugging interface supports various interaction methods, such as mouse interaction and keyboard interaction. Users can zoom in, zoom out, rotate and perform other operations on the point cloud map and / or depth map displayed in the debugging interface, which makes it easier for users to carefully view the missing details of the point cloud map and / or depth map.

[0076] Optionally, the debugging interface can also output guidance prompts for debugging steps. For example, when a user enters the debugging interface for the first time, the debugging interface can automatically pop up guidance prompts for debugging steps, which can be reopened by clicking the prompt icon on subsequent visits.

[0077] Users can zoom in, zoom out, rotate, and perform other operations on the point cloud map and / or depth map in the debugging interface, so as to accurately locate areas where the point cloud is missing or the point cloud effect does not meet the user's expectations, and mark the locations to be debugged in the point cloud map and / or depth map.

[0078] Because the point cloud map is 3D, users need to perform operations such as scaling, rotating, and dragging to accurately locate areas with missing points or where the point cloud effect does not meet their expectations when marking the locations to be debugged on the voltage map. The depth map, being 2D, allows users to accurately locate areas with missing points or where the point cloud effect does not meet their expectations through simple scaling operations, and to mark the locations to be debugged, making it much easier for users to operate. Therefore, in this embodiment, the user is guided to mark the locations to be debugged on the depth map displayed in the debugging interface. In response to the user's operation of marking the locations to be debugged on the depth map displayed in the visualization interface, the industrial control computer determines the pixel coordinates of the locations marked by the user on the depth map.

[0079] In one optional implementation, in response to a user marking a location to be debugged in the depth map displayed on the visualization interface, the industrial control computer displays a marker line for the location to be debugged in the depth map displayed on the visualization interface. The marker line for the location to be debugged is used to indicate the position of the laser line projected onto the location to be debugged.

[0080] For example, in a debugging scenario for a laser line scanning structured light camera that emits a vertical laser line, the marker line for the location to be debugged can be a vertical straight line containing the location to be debugged. Furthermore, if there is a debugging scenario for a structured light camera that emits a horizontal (or diagonal) laser line, the marker line for the location to be debugged can be a horizontal (or diagonal) straight line containing the location to be debugged.

[0081] In one example scenario, when it is determined based on the acquired point cloud map and / or depth map that there are missing point clouds or the point cloud effect does not meet the user's expectations, the user can activate the debug mode function and select to enter the debug interface for debugging. After entering the debug interface, the industrial control computer can automatically load and display the currently acquired point cloud map and its corresponding depth map, without the need for re-acquisition, thus reducing waiting time.

[0082] In another example scenario, after acquiring the collected point cloud map and / or depth map, the industrial control computer can use a hole detection algorithm to automatically detect missing points in the point cloud and automatically mark the locations to be debugged based on these missing points; and / or, the industrial control computer can use a data integrity assessment algorithm to automatically assess the integrity of the point cloud data and detect missing points, automatically marking the locations to be debugged based on these missing points. Furthermore, other algorithms / models that can automatically detect missing points or areas with poor point cloud quality can also be used; this embodiment does not specifically limit the specific algorithms / models used.

[0083] After marking the location to be debugged in the depth map, laser debugging operations can be performed in the debugging interface to trigger the industrial control computer to adjust the laser projection angle of the laser line scan structured light camera based on the marked location in the depth map. For example, the laser debugging operation can be the user clicking on controls such as "Start Debugging" or "Confirm" in the debugging interface, or other similar operations, which are not specifically limited here.

[0084] S202. In response to the laser debugging operation, determine the target laser projection angle based on the location to be debugged marked in the depth map and the camera calibration parameters, wherein the target laser projection angle is used to ensure that the laser line projected by the laser line scanning structured light camera includes the location to be debugged.

[0085] In response to the laser debugging operation, the industrial control computer adaptively determines the target laser projection angle based on the location to be debugged marked in the depth map and the camera calibration parameters of the laser line scanning structured light camera, so that the laser line projected by the laser line scanning structured light camera at the target laser projection angle includes the location to be debugged.

[0086] For example, the industrial control computer can use a camera projection angle calculation algorithm to determine the target laser projection angle based on the pixel coordinates of the location to be debugged marked in the depth map and the camera calibration parameters of the laser line scan structured light camera.

[0087] Specifically, the camera calibration parameters include camera intrinsic parameters and galvanometer zero-position angle values. Using a camera projection angle calculation algorithm, the target laser projection angle is determined based on the location to be calibrated marked on the depth map and the camera calibration parameters. This can be achieved in the following way:

[0088] Based on the pixel coordinates of the location to be debugged marked in the depth map and the camera intrinsic parameters, determine the spatial coordinates of the location to be debugged in the camera coordinate system; based on the spatial coordinates of the location to be debugged in the camera coordinate system and the zero-position angle value of the galvanometer, determine the target laser projection angle.

[0089] First, the intrinsic parameters of the structured light 2D camera after calibration in a laser line-scan structured light camera are known:

[0090] .

[0091] Where K is the intrinsic parameter matrix, For focal length in the image Pixel representation along the axial direction (corresponding to the width direction of the image), For focal length in the image Pixel representation along the axis (corresponding to the height direction of the image).

[0092] Point cloud coordinates are unified in the 2D camera coordinate system. The galvanometer calibration was completed during camera manufacturing. Calibration parameters include: galvanometer rotation center: The initial direction of laser emission when the galvanometer is in the zero position: Unit vector of the galvanometer rotation axis: Zero-position angle value of the galvanometer: .

[0093] Therefore, at any angle The laser below can be considered to be from The emitted rays have the following direction: ,in Refers to the axis Rotation Rotation matrix of the angle.

[0094] The pixel coordinates of the location to be debugged marked by the user in the depth map can be represented as: The target position in the camera coordinate system is determined by using camera intrinsic parameters and distortion coefficients. This refers to the spatial coordinates of the location to be adjusted in the camera coordinate system. The process is as follows:

[0095] First, the pixel coordinates of the location to be debugged are transformed to the normalized camera coordinate system using the camera distortion model: , .

[0096] Then, distortion removal is performed using an optical distortion model to obtain the normalized planar camera coordinates after distortion removal: , .

[0097] in This is a distortion removal function. After applying it, the corresponding position in the normalized plane after distortion removal can be obtained.

[0098] Finally, the spatial coordinates of the location to be debugged in the camera coordinate system can be obtained using the pinhole model: Where Z is the pixel coordinate. Corresponding depth value .

[0099] At this point, the spatial coordinates of the target position in the camera coordinate system are obtained. This provides a basis for determining the target laser projection angle.

[0100] Since the point cloud and the galvanometer rotation center are in the same coordinate system, the vector between them can be expressed as: .

[0101] Normalized representation: .

[0102] Since rotation about an axis only affects the components perpendicular to the axis, the vector is projected onto a plane perpendicular to the axis of rotation: , .

[0103] Then, normalize the units: , .

[0104] Calculate the angle between the initial position of the galvanometer and the currently marked position to be adjusted: , , .

[0105] Therefore, the galvanometer angle (i.e., the target laser projection angle) corresponding to the currently marked position to be adjusted is: .

[0106] S203. Send control information, including the target laser projection angle, to the laser line scanning structured light camera.

[0107] After determining the target laser projection angle, the industrial control computer sends control information, including the target laser projection angle, to the laser line scanning structured light camera to control the laser line scanning structured light camera to project a laser line based on the target laser projection angle, acquire the laser line image, and generate a point cloud map and its corresponding depth map.

[0108] In this embodiment, the industrial control computer can acquire the point cloud map and its corresponding depth map collected by the laser line scanning structured light camera (3D camera) based on the current laser projection angle. In response to the laser debugging operation, the industrial control computer determines the target laser projection angle based on the location to be debugged marked in the depth map and the camera calibration parameters. The target laser projection angle is used to ensure that the laser line projected by the laser line scanning structured light camera includes the location to be debugged, thereby accurately associating the laser projection angle of the laser line scanning structured light camera with the location to be debugged. The industrial control computer sends control information including the target laser projection angle to the laser line scanning structured light camera, so that the laser line projected by the laser line scanning structured light camera based on the target laser projection angle can cover the location to be debugged. This can accurately improve the point cloud imaging effect in the area where the location to be debugged is located, eliminating the need for the user to manually and repeatedly adjust the camera parameters. This improves the efficiency and accuracy of 3D camera debugging, thereby improving the point cloud imaging effect.

[0109] In one optional embodiment, after receiving control information including the target laser projection angle, the laser line scanning structured light camera projects a laser line according to the target laser projection angle, acquires a laser line image, and generates a point cloud map and its corresponding depth map based on the laser line image. The laser line scanning structured light camera can send the acquired laser line image to an industrial control computer.

[0110] The industrial control computer can display the target laser projection angle and the laser line image acquired by the laser line scanning structured light camera on a visual interface (i.e., the debugging interface). Users can observe whether there are any abnormalities in the laser line image displayed on the debugging interface (such as broken / discontinuous, diffused, or split laser lines). If an abnormality is confirmed in the laser line image, the user can adjust the currently displayed laser projection angle in the debugging interface.

[0111] Optionally, the industrial computer can perform line-lifting processing on the laser line image to obtain the center line of the laser stripes. The center line of the laser stripes is then overlaid on the laser line image displayed on the debugging interface. By displaying the center line, users can determine whether the laser line is projected onto the area to be debugged, and users can also use observation of the center line to help determine if there are any abnormalities in the laser line image.

[0112] Optionally, to ensure the accuracy of the user's adjustment of the laser projection angle, the debugging interface can support fine-tuning operations with preset step sizes to ensure that the laser line can be accurately projected onto the location containing the area to be adjusted. The preset step size can be set and adjusted according to actual application needs; for example, the preset step size can be 0.01°, and no specific limitation is made here.

[0113] In response to the user's adjustment of the displayed laser projection angle on the visual interface (debugging interface), the industrial control computer can synchronously adjust the target laser projection angle displayed on the debugging interface.

[0114] After each adjustment of the displayed target laser projection angle, the user can perform a laser line image acquisition operation in the debugging interface to trigger the laser line scanning structured light camera to project a laser line based on the adjusted target laser projection angle and acquire a laser line image. This laser line image acquisition operation can be performed by the user triggering a control in the debugging interface used to acquire laser line images, such as clicking the "Acquire Laser Line Image" button in the debugging interface.

[0115] In response to the laser line image acquisition operation, the industrial control computer sends control information containing the adjusted target laser projection angle to the laser line scanning structured light camera. This controls the laser line scanning structured light camera to emit a laser line and acquire a laser line image based on the adjusted target laser projection angle. The industrial control computer updates the laser line image displayed on the visualization interface based on the currently acquired laser line image from the laser line scanning structured light camera. This allows for real-time display of the laser line image re-acquired based on the adjusted target laser projection angle, enabling users to observe the laser line image acquired based on the adjusted target laser projection angle in real time.

[0116] If the laser line image acquired based on the adjusted target laser projection angle still shows anomalies, the user can continue to adjust the current laser projection angle in the debugging interface until the laser line image acquired based on the adjusted target laser projection angle is normal. Then, the user can perform a point cloud image acquisition operation in the debugging interface. This point cloud image acquisition operation can be triggered by the user using controls in the debugging interface, such as clicking the "Acquire Point Cloud Image" button.

[0117] In response to the point cloud image acquisition operation, the industrial control computer sends control information, including the adjusted target laser projection angle, to the laser line scanning structured light camera to control the laser line scanning structured light camera to generate a point cloud image and its corresponding depth map based on the laser line image acquired at the adjusted target laser projection angle.

[0118] In one example scenario, the industrial control computer displays the target laser projection angle and the laser line image acquired by the laser line scanning structured light camera on the debugging interface. The user observes that the laser line image displayed on the debugging interface shows no abnormalities and has not adjusted the target laser projection angle. In this case, the user can directly perform point cloud image acquisition operations within the debugging interface.

[0119] In response to the point cloud image acquisition operation, the industrial control computer sends control information, including the target laser projection angle, to the laser line scanning structured light camera to control the laser line scanning structured light camera to generate a point cloud image and its corresponding depth map based on the laser line image acquired at the target laser projection angle.

[0120] In an optional embodiment, the industrial control computer can also display the imaging parameters of the laser line-scanning structured light camera on the debugging interface. These imaging parameters include at least one of single-line exposure time, camera gain, and laser brightness. The user can adjust the currently displayed imaging parameters in the debugging interface. In response to the user's adjustment of the displayed imaging parameters on the debugging interface, the industrial control computer adjusts the imaging parameters.

[0121] After adjusting the displayed imaging parameters, the user can perform laser line image acquisition operations in the debugging interface to trigger the laser line scanning structured light camera to acquire laser line images based on the adjusted imaging parameters.

[0122] For example, the imaging parameters of the laser line scanning structured light camera displayed on the debugging interface can be the imaging parameters currently used by the laser line scanning structured light camera obtained synchronously, and the debugging interface supports users to manually fine-tune various imaging parameters.

[0123] In one example, the industrial computer can display the target laser projection angle and imaging parameters on the same debugging interface. The user can adjust the target laser projection angle and / or imaging parameters in the debugging interface, and then acquire laser line images after adjustment.

[0124] In response to the laser line image acquisition operation, the industrial control computer sends control information to the laser line scanning structured light camera, including the adjusted target laser projection angle and the adjusted imaging parameters, so as to control the laser line scanning structured light camera to emit a laser line and acquire a laser line image based on the adjusted target laser projection angle and the adjusted imaging parameters.

[0125] Furthermore, the industrial control computer updates the laser line image displayed on the visualization interface based on the laser line image currently acquired by the laser line scanning structured light camera. This allows for the real-time display of the laser line image re-acquired based on the adjusted target laser projection angle and / or adjusted imaging parameters, enabling users to observe the laser line image acquired based on the adjusted target laser projection angle and / or adjusted imaging parameters in real time.

[0126] If the laser line image acquired based on the adjusted target laser projection angle and / or adjusted imaging parameters still has anomalies, the user can continue to adjust the current laser projection angle and / or imaging parameters in the debugging interface until the laser line image acquired based on the adjusted target laser projection angle and / or adjusted imaging parameters is normal. Then, the user can perform point cloud map acquisition operation in the debugging interface.

[0127] In response to the point cloud image acquisition operation, the industrial control computer sends control information, including the adjusted target laser projection angle and / or the adjusted imaging parameters, to the laser line scanning structured light camera to generate a point cloud image and its corresponding depth map based on the laser line image acquired at the adjusted target laser projection angle and / or the adjusted imaging parameters.

[0128] Based on any of the foregoing embodiments, the laser line images acquired by the laser line scanning structured light camera may include laser line images acquired by the left camera and the right camera. When displaying the laser line images on the visualization interface, the laser line images from the left camera and the right camera can be displayed in two separate areas. When the laser line images are updated, the laser line images from the left camera and the right camera displayed on the visualization interface are updated synchronously.

[0129] In one optional embodiment, after the industrial control computer sends control information including the target laser projection angle to the laser line scanning structured light camera, it acquires the point cloud map and its corresponding depth map collected by the laser line scanning structured light camera this time, and displays the point cloud map and its corresponding depth map collected this time, as well as at least one point cloud map and its corresponding depth map collected in the past in the visualization interface.

[0130] Among them, at least one point cloud map and its corresponding depth map collected in history can be the point cloud map and its corresponding depth map collected in the previous one or two previous collections, or it can be the point cloud map and its corresponding depth map obtained after any round (or multiple rounds) of debugging that the user selected to save during the previous debugging process.

[0131] By simultaneously displaying the point cloud image and its corresponding depth map of the current acquisition, as well as at least one point cloud image and its corresponding depth map of the previous acquisition, in the visualization interface, users can easily compare and observe the point cloud images and their corresponding depth maps of the current and previous acquisitions to select the optimal point cloud image as the final point cloud image. In response to the user's selection, the industrial control computer outputs the user-selected point cloud image as the final point cloud image.

[0132] Optionally, the industrial control computer can store the laser projection angle corresponding to the most recently output point cloud image as the current laser projection angle. For example, in response to a user's operation of saving debugging parameters to a parameter group on the debugging interface, the industrial control computer can write and store the laser projection angle and imaging parameters corresponding to the most recently output point cloud image into the camera's default loaded parameter group. This parameter group is automatically called upon when the camera starts or images subsequently, eliminating the need for repeated debugging. This is suitable for rapid on-site deployment and maintenance scenarios, making it more practical.

[0133] In addition, after the industrial control computer writes and stores the laser projection angle and imaging parameters corresponding to the most recently output point cloud map into the default parameter group, it can automatically exit the debugging mode. Optionally, the industrial control computer can exit the debugging mode in response to the user's exit debugging mode operation.

[0134] This embodiment's solution, through visual marking of the area to be debugged and a guided process, allows users to accurately locate the area, avoiding blind trial and error. The graphical parameter adjustment, combined with intuitive comparisons of old and new point cloud maps and depth maps, significantly shortens the debugging cycle and greatly improves debugging efficiency. Furthermore, it supports fine-tuning of the target laser projection angle and imaging parameters, along with real-time comparison of point cloud effects. Combined with point cloud completion algorithms, it can quickly achieve optimal point cloud imaging results. The debugging interface provides mouse interaction, step-by-step guidance, and real-time parameter adjustment functions, lowering the operational threshold for on-site technicians. Debugging can be completed without professional training, making it more convenient to operate.

[0135] This application provides a 3D camera that can receive control information including the target laser projection angle and adjust the laser projection angle of the 3D camera according to the target laser projection angle.

[0136] In one example scenario, the 3D camera can be a laser line scanning structured light camera. It receives control information from an industrial control computer, including the target laser projection angle. It adjusts the laser projection angle of the 3D camera according to the target laser projection angle, projects a laser line according to the adjusted target laser projection angle, acquires laser line images, and generates point cloud maps and their corresponding depth maps.

[0137] In other scenarios, users can manually configure / adjust the target laser projection angle of the 3D camera using other 3D camera configuration / adjustment devices, and send control information including the target laser projection angle to the 3D camera.

[0138] In this embodiment, the target laser projection angle can be determined based on the location to be debugged marked in the historically acquired depth map and the camera calibration parameters, which can be executed by the industrial control computer or the control module of the 3D camera.

[0139] The determination is based on the location to be debugged marked in the historical depth map and the camera calibration parameters, and the specific implementation method is as follows:

[0140] Acquire the point cloud map and its corresponding depth map collected by the laser line scanning structured light camera based on the current laser projection angle; in response to the laser debugging operation, determine the target laser projection angle according to the position to be debugged marked in the depth map and the camera calibration parameters, wherein the target laser projection angle is used to ensure that the laser line projected by the laser line scanning structured light camera includes the position to be debugged; send control information including the target laser projection angle to the laser line scanning structured light camera.

[0141] In an optional implementation, after acquiring the point cloud map and its corresponding depth map collected by the laser line-scan structured light camera based on the current laser projection angle, and before determining the target laser projection angle according to the position to be debugged and the camera calibration parameters in response to the laser debugging operation, the method further includes:

[0142] The visualization interface displays the point cloud map and its corresponding depth map acquired by the laser line scan structured light camera based on the current laser projection angle; in response to the user's operation of marking the position to be debugged in the depth map displayed in the visualization interface, the pixel coordinates of the position to be debugged are determined.

[0143] Optionally, in response to the user's operation of marking the location to be debugged in the depth map displayed on the visualization interface, a marker line for the location to be debugged is displayed in the depth map displayed on the visualization interface. The marker line is used to indicate the position of the laser line projected onto the location to be debugged.

[0144] In one optional implementation, the camera calibration parameters include camera intrinsic parameters and galvanometer zero-position angle values. The target laser projection angle is determined based on the location to be calibrated marked on the depth map and the camera calibration parameters, including:

[0145] Based on the pixel coordinates of the location to be debugged marked in the depth map and the camera intrinsic parameters, determine the spatial point coordinates of the location to be debugged in the camera coordinate system.

[0146] The target laser projection angle is determined based on the spatial coordinates of the position to be debugged in the camera coordinate system and the zero-position angle value of the galvanometer.

[0147] In one optional implementation, control information including the target laser projection angle is sent to the laser line-scanning structured light camera, including:

[0148] In response to the point cloud image acquisition operation, control information including the target laser projection angle is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to generate a point cloud image and its corresponding depth map based on the laser line image acquired at the target laser projection angle.

[0149] In one optional implementation, the target laser projection angle and the laser line image acquired by the laser line scanning structured light camera are displayed on a visualization interface. The target laser projection angle is adjusted in response to the user's adjustment operation on the displayed laser projection angle in the visualization interface.

[0150] In response to the laser line image acquisition operation, control information containing the adjusted target laser projection angle is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to emit a laser line and acquire a laser line image based on the adjusted target laser projection angle; the laser line image displayed on the visualization interface is updated according to the laser line image currently acquired by the laser line scanning structured light camera.

[0151] In one alternative implementation, the center line of the laser stripes in the laser line image is superimposed on the laser line image displayed on the visualization interface.

[0152] In one optional implementation, imaging parameters of the laser line-scanning structured light camera are displayed on a visualization interface. These imaging parameters include at least one of single-line exposure time, camera gain, and laser brightness. The imaging parameters are adjusted in response to user adjustments made to the displayed imaging parameters on the visualization interface.

[0153] In one optional implementation, in response to the laser line image acquisition operation, control information including the adjusted target laser projection angle is sent to the laser line scanning structured light camera, specifically including:

[0154] In response to the laser line image acquisition operation, control information containing the adjusted target laser projection angle and adjusted imaging parameters is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to emit a laser line and acquire a laser line image based on the adjusted target laser projection angle and adjusted imaging parameters.

[0155] In one optional implementation, after sending control information including the target laser projection angle to the laser line-scanning structured light camera, the method further includes:

[0156] The system acquires the point cloud image and its corresponding depth map from the current laser line-scanning structured light camera. It displays the currently acquired point cloud image and its corresponding depth map, as well as at least one previously acquired point cloud image and its corresponding depth map, in a visualization interface. In response to user selection, it outputs the user-selected point cloud image as the final point cloud image.

[0157] In one optional implementation, the laser projection angle corresponding to the most recently output point cloud map is used as the current laser projection angle and stored.

[0158] The specific implementation principle and technical effects of this embodiment are described in the relevant content of the foregoing embodiments, and will not be repeated here.

[0159] For example, Figure 3 This is a detailed flowchart illustrating the 3D camera debugging process provided in this embodiment. In this example, a laser line-scanning structured light camera is used as the execution entity for parameter debugging. The laser line-scanning structured light camera includes a left camera, a right camera, a laser line-scanning structured light source, a control module, and a display terminal. The laser line-scanning structured light source projects structured light (i.e., a laser line) and supports adjusting the laser line's position and brightness according to control signals. The left and right cameras acquire the laser line-scanning structured light reflection map to obtain the laser line image and generate a depth map and point cloud map. The control module is the core execution unit, responsible for laser line-scanning structured light control, algorithm processing, and data storage. The display terminal presents the debugging interface, supports user interaction, and displays the imaging effect.

[0160] like Figure 3 As shown, the complete process for parameter tuning of a laser line-scan structured light camera in an example scenario includes:

[0161] S1. The user sets the camera's imaging parameters, such as 2D / 3D exposure time and gain. The camera uses the default laser projection angle.

[0162] S2. The camera uses the current laser projection angle and imaging parameters to acquire point cloud maps.

[0163] S3. Determine if the point cloud map is complete.

[0164] In this step, the user can determine whether there are missing points in the currently acquired point cloud map, or the industrial control computer or camera control module can use a hole detection algorithm or a data integrity assessment algorithm to automatically determine whether there are missing points in the point cloud map, thereby determining whether the point cloud map is complete.

[0165] If the point cloud is complete, end the debugging process. If the point cloud is incomplete, proceed to step S4 and subsequent steps.

[0166] S4. Click the point cloud missing prompt button.

[0167] In this example, when a user discovers a missing point cloud, they can enter debug mode by clicking the "Point Cloud Missing Notification" button on the main interface of the display terminal. Additionally, in other example scenarios, users can enter debug mode by clicking the "Point Cloud Debugging Tool" button in the "More Tools" module on the main interface. This embodiment does not specifically limit the method by which the user enters debug mode.

[0168] S5. Enter debug mode.

[0169] S6. Automatically jumps to the debugging interface, displaying the point cloud map and depth map captured by the camera.

[0170] After the user enters debug mode, the terminal automatically loads the debug interface, which displays the point cloud map and depth map of the last collection.

[0171] S7. Prompt the user: Please use the mouse to click on the missing part, and the laser will automatically move to the corresponding position.

[0172] For example, Figure 4 An example of a debugging interface provided in this embodiment. Figure 1 ,like Figure 4 As shown, the debugging interface has four display areas. The upper left display area displays the previously acquired depth map, the upper right display area displays the previously acquired point cloud map, and the lower left and lower right display areas display the laser line images from the left and right cameras, respectively. In this step, the previously acquired point cloud map and depth map can be displayed in their respective display areas. At this time, the lower left and lower right display areas are blank (the laser line image to be acquired).

[0173] like Figure 4 As shown, when a user enters debug mode for the first time, the debug interface will automatically display a guide to the debug steps: 1. Click the location to be debugged (within the missing area of ​​the point cloud) on the depth map. After the user marks the location to be debugged, the debug interface will display a guide to the next step: 2. The laser line position (estimated result) of the location to be debugged. The user can exit the guide by using the "Exit" control on the debug interface.

[0174] S8. The user marks the location to be debugged.

[0175] Users can click on the depth map displayed in the debugging interface to select the missing point cloud area (e.g., ...). Figure 4 The point in the area indicated by the middle arrow is marked as the location to be debugged. For example, the user can zoom in / out of the depth map as needed to accurately mark the location to be debugged.

[0176] For example, such as Figure 4 As shown, marker lines for the location to be debugged can be overlaid on the depth map to indicate to the user the position of the laser line projected onto the location to be debugged.

[0177] In addition, users can zoom in, zoom out, rotate, and drag the point cloud image displayed on the debugging interface to carefully observe the point cloud effect.

[0178] S9. The camera control module acquires the pixel coordinates of the position to be debugged, automatically determines the target laser projection angle, projects a laser line to the corresponding position based on the target laser projection angle, and displays the target laser projection angle and imaging parameters on the debugging interface.

[0179] For example, such as Figure 5 As shown, the parameter adjustment area on the right side of the debugging interface displays the current laser projection angle, as well as imaging parameters such as the single-line laser line exposure time, single-line laser line gain, and laser brightness. The debugging interface allows users to adjust the displayed laser projection angle and / or imaging parameters. Specifically, the single-line exposure time is used to adjust the exposure time of the laser line image. The single-line laser line gain is used to adjust the gain of the laser line image. The laser brightness is used to adjust the laser projection brightness.

[0180] like Figure 5 As shown, the parameter adjustment area also displays whether the camera's laser enable parameter is enabled. Laser Enable: When enabled, the laser line scan structured light source projects a laser line onto the object. When the user performs a laser line map acquisition operation or a point cloud map acquisition operation, the camera's laser line enable automatically turns on and performs a scan from left to right. After the scan is completed, it turns on again, and the laser line stops at the marked position to be adjusted, to help the user clarify the laser line scanning position and the corresponding position of the workpiece.

[0181] S10. Prompt the user: Prompt the user to adjust the target laser projection angle and / or imaging parameters in the rear parameter adjustment area.

[0182] S11. The user adjusts the target laser projection angle and / or imaging parameters.

[0183] Users can Figure 5 In the debugging interface shown, adjust the camera's target laser projection angle and / or imaging parameters. After adjusting the parameters, the user can click... Figure 5 The “Acquire Laser Line Image” button shown triggers the camera to project a laser line based on the adjusted target laser projection angle and / or imaging parameters, and acquires a laser line image.

[0184] S12. In response to the laser line image acquisition operation, the camera projects a laser line based on the adjusted target laser projection angle and / or imaging parameters, and acquires a laser line image, which is then displayed in the debugging interface.

[0185] In response to the laser line image acquisition operation, the control module disables the camera's laser enable, controls the left and right cameras to acquire laser line images based on the adjusted target laser projection angle and / or imaging parameters, and displays the laser line images of the left and right cameras in the corresponding areas after acquisition is completed, while simultaneously re-enabling the laser enable.

[0186] For example, such as Figure 6 As shown, the laser line images from the left and right cameras acquired in this study are displayed in the lower left and lower right display areas, respectively.

[0187] Users can observe the most recently acquired laser line image. If the laser line image is found to be abnormal (such as overexposure or underexposure), steps S11-S12 can be repeated until the most recently acquired laser line image is confirmed to be normal. Then, the user can click... Figure 6 The "Acquire Point Cloud Map" button shown triggers the camera to project a laser line based on the adjusted target laser projection angle and / or imaging parameters, acquire the laser line map, and generate a point cloud map and a depth map.

[0188] S13. In response to the point cloud map acquisition operation, the camera acquires a point cloud map and a depth map based on the adjusted target laser projection angle and / or imaging parameters, and displays the acquired point cloud map and depth map in the debugging interface.

[0189] For example, such as Figure 7 As shown, after the camera acquires point cloud and depth maps based on the adjusted target laser projection angle and / or imaging parameters, the point cloud and depth maps displayed in the debugging interface can be updated to the latest acquired point cloud and depth maps. This allows the user to observe whether the latest acquired point cloud map is complete, i.e., whether the completeness of the point cloud map meets the usage requirements. If the latest acquired point cloud map is determined to be incomplete, the user can repeat steps S8-S13 until a complete point cloud map is acquired, i.e., a point cloud map whose completeness meets the usage requirements.

[0190] In one example, the debugging interface can simultaneously display the point cloud map and its corresponding depth map (i.e., the point cloud map and depth map acquired based on the adjusted parameters), as well as the optimal point cloud map and its corresponding depth map acquired in the past, so that users can compare the completeness of the new and old point cloud maps.

[0191] For example, Figure 8 This is a schematic diagram of another debugging interface provided in this embodiment. (See diagram below.) Figure 8 The debugging interface shown displays historical depth and point cloud maps in the upper two areas, while the lower two areas display the current depth and point cloud maps (i.e., point cloud and depth maps acquired based on the adjusted parameters). Users can zoom in and perform other operations on the point cloud maps and / or depth maps displayed on this debugging interface to better compare the completeness of the new and old point cloud maps, enabling real-time comparison of parameter adjustments and imaging effects, and improving debugging accuracy and efficiency.

[0192] For example, such as Figure 8 As shown, laser line images (such as...) can be overlaid on the latest acquired depth map and point cloud map. Figure 8 The white rectangular area on the latest depth map and point cloud map (as shown in the image) and its lifting results (i.e., the center line of the laser stripe, such as...) Figure 8(The blue lines shown in the image) are used to help users observe the adjusted laser line projection effect.

[0193] Users can select a point cloud map that meets the usage requirements in the debugging interface as the final point cloud map.

[0194] S14. In response to the user's selection operation, output the point cloud map selected by the user as the final point cloud map; write the laser projection angle and imaging parameters corresponding to the final point cloud map into the default parameter group and store them.

[0195] This embodiment's solution, through visual marking of the area to be debugged and a guided process, allows users to accurately locate the area, avoiding blind trial and error and reducing the difficulty of user operation. The interface-based parameter adjustment, combined with intuitive comparison of old and new point cloud maps and depth maps, significantly shortens the debugging cycle and greatly improves debugging efficiency. Furthermore, it supports fine-tuning of the target laser projection angle and imaging parameters, along with real-time comparison of point cloud effects. Combined with point cloud completion algorithms, it can quickly achieve optimal point cloud imaging results. The debugging interface provides mouse interaction, step-by-step guidance, and real-time parameter adjustment functions, lowering the operational threshold for on-site technicians. Debugging can be completed without professional training, making it more convenient to operate.

[0196] Figure 9 This is a schematic diagram of the structure of the 3D camera debugging device provided in this application. Figure 9 As shown, the 3D camera debugging device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0197] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0198] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0199] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0200] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0201] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0202] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described above for an industrial control computer or a 3D camera.

[0203] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described above by the industrial control computer or 3D camera.

[0204] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0205] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0206] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0207] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0208] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0209] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0210] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0211] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A 3D camera debugging method, characterized in that, Applied to industrial control computers, the method includes: Acquire the point cloud map and its corresponding depth map collected by the laser line scan structured light camera based on the current laser projection angle; In response to a laser debugging operation, a target laser projection angle is determined based on the location to be debugged marked in the depth map and camera calibration parameters, wherein the target laser projection angle is used to ensure that the laser line projected by the laser line scan structured light camera includes the location to be debugged. Control information, including the target laser projection angle, is sent to the laser line scan structured light camera.

2. The method according to claim 1, characterized in that, After acquiring the point cloud map and its corresponding depth map collected by the laser line-scan structured light camera based on the current laser projection angle, in response to the laser debugging operation, before determining the target laser projection angle based on the location to be debugged and the camera calibration parameters, the process further includes: The visualization interface displays the point cloud map and its corresponding depth map acquired by the laser line scan structured light camera based on the current laser projection angle; In response to the user's operation of marking the location to be debugged in the depth map displayed on the visualization interface, the pixel coordinates of the location to be debugged are determined.

3. The method according to claim 1, characterized in that, The camera calibration parameters include camera intrinsic parameters and galvanometer zero-position angle values. Determining the target laser projection angle based on the location to be calibrated marked on the depth map and the camera calibration parameters includes: Based on the pixel coordinates of the location to be debugged marked in the depth map and the camera intrinsic parameters, determine the spatial point coordinates of the location to be debugged in the camera coordinate system. The target laser projection angle is determined based on the spatial coordinates of the position to be debugged in the camera coordinate system and the zero-position angle value of the galvanometer.

4. The method according to claim 1, characterized in that, Sending control information, including the target laser projection angle, to the laser line-scanning structured light camera includes: In response to the point cloud image acquisition operation, control information including the target laser projection angle is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to generate a point cloud image and its corresponding depth map based on the laser line image acquired at the target laser projection angle.

5. The method according to claim 4, characterized in that, The method further includes: The target laser projection angle and the laser line image acquired by the laser line scanning structured light camera are displayed on the visualization interface; In response to the user's adjustment operation on the displayed laser projection angle in the visualization interface, the target laser projection angle is adjusted; In response to the laser line image acquisition operation, control information including the adjusted target laser projection angle is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to emit a laser line and acquire a laser line image based on the adjusted target laser projection angle. The laser line image displayed on the visualization interface is updated based on the laser line image currently acquired by the laser line scanning structured light camera.

6. The method according to claim 5, characterized in that, The method further includes: The center line of the laser stripes in the laser line image is superimposed on the laser line image displayed in the visualization interface.

7. The method according to claim 5, characterized in that, The method further includes: The visualization interface displays the imaging parameters of the laser line scan structured light camera, including at least one of single-line exposure time, camera gain, and laser brightness. The imaging parameters are adjusted in response to the user's adjustment operation on the displayed imaging parameters in the visualization interface.

8. The method according to claim 7, characterized in that, The step of sending control information containing the adjusted target laser projection angle to the laser line scanning structured light camera in response to the laser line image acquisition operation specifically includes: In response to the laser line image acquisition operation, control information including the adjusted target laser projection angle and the adjusted imaging parameters is sent to the laser line scanning structured light camera to control the laser line scanning structured light camera to emit a laser line and acquire a laser line image based on the adjusted target laser projection angle and the adjusted imaging parameters.

9. The method according to any one of claims 1-8, characterized in that, After sending control information including the target laser projection angle to the laser line scan structured light camera, the method further includes: Acquire the point cloud image and its corresponding depth map captured by the laser line scan structured light camera in this instance; The visualization interface displays the point cloud map and its corresponding depth map collected this time, as well as at least one point cloud map and its corresponding depth map collected in the past. In response to the user's selection, the selected point cloud map is output as the final point cloud map.

10. The method according to claim 9, characterized in that, The method further includes: The laser projection angle corresponding to the most recently output point cloud map is used as the current laser projection angle and stored.

11. The method according to claim 2, characterized in that, The method further includes: In response to a user marking a location to be debugged in the depth map displayed on the visualization interface, a marker line for the location to be debugged is displayed in the depth map displayed on the visualization interface. The marker line is used to indicate the position of the laser line projected onto the location to be debugged.

12. A 3D camera, characterized in that, include: Receive control information including the target laser projection angle; The laser projection angle of the 3D camera is adjusted according to the target laser projection angle.

13. The 3D camera according to claim 12, characterized in that, The target laser projection angle is determined based on the location to be debugged marked in the historically acquired depth map and the camera calibration parameters.

14. A 3D camera debugging device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-11.

16. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-11.

Images