Tactile scanning imaging method and related device

Abstract

The embodiment of the invention provides a tactile scanning imaging method and a related device. The method is applied to the tactile scanning imaging robot, the tactile scanning imaging robot comprises a visual module, a tactile detection module, a wireless network bridge and a computer, and the method comprises the steps that the visual module is adopted to conduct optical surface contour reconstruction on a target object, and multiple pieces of target point cloud data are obtained, the target point cloud data comprises position information and normal information; sorting the multiple pieces of target point cloud data by using a greedy algorithm to obtain a measurement sequence; a scanning path of a mechanical arm of the tactile scanning imaging robot is planned according to the measurement sequence, so that the tactile detection module conducts tactile scanning detection on a target curved surface according to the scanning path, and a measurement result is obtained; and imaging the measurement result to obtain a touch tomography image of the target object. Based on this, the embodiment of the invention can realize tactile scanning imaging of a complex curved surface.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of robot tactile technology, and in particular to a tactile scanning imaging method and related devices. Background Art

[0002] In research on tactile softness perception and internal imaging, a 3D force scanning imaging system has been successfully used to investigate the factors influencing tactile softness and to perform 3D imaging of internal structures using differences in indentation depth. However, due to structural limitations, existing 3D force scanning imaging systems can only perform tactile scanning imaging on small objects, and these objects must have a flat surface. Therefore, existing 3D force scanning imaging systems are unable to perform tactile scanning imaging on complex curved surfaces. Summary of the Invention

[0003] The embodiments of the present invention provide a tactile scanning imaging method and related devices, which can realize tactile scanning imaging of complex curved surfaces.

[0004] In a first aspect, an embodiment of the present invention provides a tactile scanning imaging method, which is applied to a tactile scanning imaging robot. The tactile scanning imaging robot includes a vision module, a tactile detection module, a wireless bridge, and a computer. The method includes:

[0005] Reconstructing the optical surface contour of the target object using the vision module to obtain a plurality of target point cloud data, wherein the target point cloud data includes position information and normal information;

[0006] Use the greedy algorithm to sort multiple target point cloud data to obtain the measurement sequence;

[0007] Planning a scanning path of the mechanical arm of the tactile scanning imaging robot according to the measurement sequence, so that the tactile detection module performs tactile scanning detection on the target surface according to the scanning path to obtain a measurement result;

[0008] The measurement results are imaged to obtain a tactile tomographic image of the target object.

[0009] In some embodiments, the optical surface contour reconstruction of the target object using the vision module to obtain a plurality of target point cloud data includes:

[0010] Using the visual module to perform a three-dimensional visual scan of the target object to obtain a point cloud result of surface contour reconstruction;

[0011] Truncating the point cloud scene in the point cloud result and performing voxel downsampling to obtain target point cloud data;

[0012] Normal estimation is performed on the target point cloud data according to the position information to obtain the normal information.

[0013] In some embodiments, the vision module includes a depth camera and a Raspberry Pi. The Raspberry Pi reads the image frames of the depth camera, generates a depth map and a color map through the official decoding library of the depth camera, and then uses a UDP video stream to send the depth map and the color map to the computer through the wireless bridge. The computer combines the depth map and the color map with the posture of the depth camera to reconstruct the three-dimensional scene.

[0014] In some embodiments, the tactile detection module includes a robotic arm, a pressure sensor, and a probe. The target point to be measured is selected in the three-dimensional scene generated by the vision module. The computer sends the position of the target point to the robotic arm via the Modbus-TCP protocol, and the robotic arm moves to the position of the target point to perform tactile scanning detection.

[0015] In some embodiments, the scan path is expressed as follows:

[0016]

[0017] Wherein, H is the scanning path of the robot arm, L is the length of the probe, h is the distance between the measurement points, and θ is the angle between the normals of the measurement points.

[0018] In some embodiments, the method of sorting the plurality of target point cloud data using a greedy algorithm to obtain a measurement sequence includes:

[0019] Calculate the scanning path H of the robot arm between all the target point cloud data, and select the one with the largest H as the preset starting point P0 and the preset end point P n ;

[0020] Randomly select one of the remaining target point cloud data and insert it into P0 and P n In the gap, compare the sum of H values of different methods and select the one with the smallest sum of H value as the target insertion method;

[0021] The remaining target point cloud data are inserted in sequence according to the target insertion method until all the target point cloud data are sorted.

[0022] In some embodiments, the method further comprises:

[0023] Converting the normal information in the measurement sequence into a posture to obtain a rotation matrix;

[0024] The rotation matrix is converted into a rotation vector using the Loglider rotation formula.

[0025] In a second aspect, an embodiment of the present invention further provides a tactile scanning imaging robot, the robot comprising:

[0026] A vision module is used to reconstruct the optical surface contour of the target object to obtain a plurality of target point cloud data, wherein the target point cloud data includes position information and normal information;

[0027] The sorting module is used to sort multiple target point cloud data using a greedy algorithm to obtain a measurement sequence;

[0028] a planning module, configured to plan a scanning path of a mechanical arm of the tactile scanning imaging robot according to the measurement sequence;

[0029] a tactile detection module, configured to perform tactile scanning detection on the target curved surface according to the scanning path to obtain a measurement result;

[0030] The imaging module is used to image the measurement results to obtain a tactile tomographic image of the target object.

[0031] In a third aspect, an embodiment of the present invention further provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the tactile scanning imaging method as described in the first aspect when executing the computer program.

[0032] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the tactile scanning imaging method as described in the first aspect.

[0033] According to an embodiment of the present invention, a tactile scanning imaging method and related apparatus are provided, wherein the tactile scanning imaging method is applied to a tactile scanning imaging robot. The tactile scanning imaging robot includes a vision module, a tactile detection module, a wireless bridge, and a computer. The method comprises: using the vision module to reconstruct the optical surface contour of a target object to obtain multiple target point cloud data, wherein the target point cloud data includes position information and normal information; using a greedy algorithm to sort the multiple target point cloud data to obtain a measurement sequence; planning a scanning path for the tactile scanning imaging robot's robotic arm based on the measurement sequence, so that the tactile detection module performs tactile scanning and detection on the target curved surface according to the scanning path to obtain a measurement result; and imaging the measurement result to obtain a tactile tomographic image of the target object. In this embodiment of the present invention, the vision module is first used to reconstruct the optical surface contour of a target object with a complex curved surface, simultaneously estimating the normal of the surface contour, and outputting the position and normal information of the points to be measured. The greedy algorithm is then used to sort the points to be measured and plan the scanning path of the robotic arm. The tactile detection module is then used to perform tactile scanning and detection on the target curved surface according to the scanning path to obtain a measurement result. Finally, the measurement results are imaged to obtain a tactile tomographic image of the target object. Based on this, the embodiment of the present invention can achieve tactile scanning imaging of complex curved surfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of a tactile scanning imaging method provided by one embodiment of the present invention;

[0035] Figure 2 This is a three-dimensional reconstruction effect diagram of a depth camera provided by an embodiment of the present invention;

[0036] Figure 3 This is an effect diagram of optical surface contour reconstruction of a human thigh model provided by one embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of path planning provided by an embodiment of the present invention;

[0038] Figure 5 This is a tactile tomography image provided by an embodiment of the present invention;

[0039] Figure 6 is a structural diagram of a tactile scanning imaging device provided by an embodiment of the present invention;

[0040] Figure 7 FIG. 1 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, used in the specification, claims, and accompanying drawings are used to distinguish similar items and are not necessarily intended to describe a specific sequence or precedence.

[0043] In the embodiments of the present invention, words such as "further," "exemplarily," or "optionally" are used to indicate examples, illustrations, or explanations and should not be interpreted as being more preferred or advantageous over other embodiments or designs. The use of words such as "further," "exemplarily," or "optionally" is intended to present related concepts in a concrete manner.

[0044] In order to more conveniently describe the working principle of the embodiment of the present invention later, an introduction to relevant technical scenarios is first given below.

[0045] In research on tactile softness perception and internal imaging, a 3D force scanning imaging system has been successfully used to investigate the factors influencing tactile softness and to perform 3D imaging of internal structures using differences in indentation depth. However, due to structural limitations, existing 3D force scanning imaging systems can only perform tactile scanning imaging on small objects, and these objects must have a flat surface. Therefore, existing 3D force scanning imaging systems are unable to perform tactile scanning imaging on complex curved surfaces.

[0046] Based on this, the present invention provides a tactile scanning imaging method and related apparatus based on an unsupervised industrial large-scale model. The tactile scanning imaging method is applied to a tactile scanning imaging robot, which includes a vision module, a tactile detection module, a wireless network bridge, and a computer. The method comprises: using the vision module to reconstruct the optical surface contour of a target object to obtain multiple target point cloud data, wherein the target point cloud data includes position information and normal information; using a greedy algorithm to sort the multiple target point cloud data to obtain a measurement sequence; planning a scanning path for the tactile scanning imaging robot's robotic arm based on the measurement sequence, so that the tactile detection module performs tactile scanning and detection of the target surface according to the scanning path to obtain measurement results; and imaging the measurement results to obtain a tactile tomographic image of the target object. In this embodiment of the present invention, the vision module first reconstructs the optical surface contour of a target object with a complex curved surface, simultaneously estimating the normal of the surface contour, and outputting the position and normal information of the points to be measured. The greedy algorithm is then used to sort the points to be measured and plan the robotic arm's scanning path. The tactile detection module then performs tactile scanning and detection of the target surface according to the scanning path to obtain measurement results. Finally, the measurement results are imaged to obtain a tactile tomographic image of the target object. Based on this, the embodiment of the present invention can achieve tactile scanning imaging of complex curved surfaces.

[0047] The embodiments of the present invention are further described below with reference to the accompanying drawings.

[0048] like Figure 1 As shown, Figure 1 This is a flowchart of a tactile scanning imaging method provided by an embodiment of the present invention. The tactile scanning imaging method is applied to a tactile scanning imaging robot. The tactile scanning imaging robot includes a visual module, a tactile detection module, a wireless bridge and a computer. The tactile scanning imaging method may include but is not limited to steps S101 to S104.

[0049] Step S101: reconstructing the optical surface contour of the target object using a visual module to obtain a plurality of target point cloud data, wherein the target point cloud data includes position information and normal information;

[0050] Step S102: sorting multiple target point cloud data using a greedy algorithm to obtain a measurement sequence;

[0051] Step S103: planning a scanning path of the mechanical arm of the tactile scanning imaging robot according to the measurement sequence, so that the tactile detection module performs tactile scanning detection on the target surface according to the scanning path to obtain a measurement result;

[0052] Step S104: imaging the measurement results to obtain a tactile tomographic image of the target object.

[0053] It is understandable that the tactile scanning imaging robot is mainly composed of a robotic arm (UR10e, including a control box), an S-type pressure sensor (Suzhou-Ouluda AT8301), a probe, a depth camera (Intel realsense D435i), a Raspberry Pi (raspberrypi5), a wireless bridge (VONETS VAR1200-L 5G) and a computer.

[0054] The above components can be divided into two functional modules, the vision module and the tactile detection module.

[0055] The vision module consists of a depth camera, a Raspberry Pi, a wireless bridge, and a computer. The Raspberry Pi reads image frames from the depth camera and generates a depth map and a color map using the camera's official decoding library. These images are then sent to the computer via the wireless bridge using a UDP video stream. The computer combines the depth and color maps with the camera's pose to reconstruct the 3D scene.

[0056] The tactile testing module primarily consists of a robotic arm, pressure sensor, probe, wireless bridge, and computer. Within the 3D scene generated by the vision module, a point to be measured is selected. The computer then transmits the position of that point to the robotic arm via the Modbus-TCP protocol. The robotic arm then moves to that position and executes the test steps. The entire system organically connects the computer and the two functional modules via a wireless bridge, enabling remote control of the robotic arm for mechanical testing.

[0057] It is understood that in the present invention, a depth camera is used to detect the surface contours of complex curved surfaces. The depth camera consists of two global shutter infrared cameras (left / right) with a baseline distance of approximately 55mm, which are used to capture infrared images of the scene at a frame rate of up to 90Hz; an infrared laser transmitter (active light source) that emits invisible near-infrared (NIR) speckle patterns (wavelength of approximately 860nm) to artificially add texture to low-texture scenes (such as white walls and monochrome objects) to assist in stereo matching; and an RGB camera that provides color images that are aligned with the depth map for color point cloud generation.

[0058] It is understandable that when the depth camera is capturing images, the infrared laser emitter in the middle of the depth camera continuously emits a set of square scattered point matrices, while the two infrared cameras continuously capture the scattered point matrices. Since the positions of the two infrared cameras are different, the positions of the captured scattered point matrices are also different. The positional deviation of the same laser point in the two infrared cameras is called parallax. The size of the parallax is inversely proportional to the distance from the laser point to the camera. Therefore, as long as the parallax of a certain point is known, the depth information of that point can be known. By calculating the parallax of all laser points, a depth map can be obtained, such as Figure 2 As shown in a), the color map is as follows Figure 2 As shown in b).

[0059] After aligning the color and depth images, they can be combined into an RGB-D image. Each pixel in the color image is stored in a computer as the three primary colors (R, G, B). Adding the depth value to the color data creates an (R, G, B, D) image, known as an RGB-D image. The dimensions of an RGB-D image are (image width × image length × 4).

[0060] The depth of the obtained RGB-D image is relative to the camera's coordinate system. To obtain the coordinates of the image in reality, you need to multiply the depth term of the RGB-D image by the camera's pose, as shown in the following formula:

[0061]

[0062] In the above formula, (X, Y, Z) is the actual coordinate of the sample in space; T is the position of the camera; (W, H, D) is the depth component of RGB-D. The effect of 3D reconstruction is as follows Figure 2 As shown in (c), the coordinate system in the figure is the origin coordinate system of the robotic arm. It should be noted that when 3D reconstruction is performed using an RGB-D image captured at a single angle, the reconstructed model is incomplete. Therefore, to fully reconstruct the model, images must be captured from multiple angles. In other words, reconstructing a scene using a single set of color, depth, and pose data is incomplete. This can be addressed by capturing images from multiple angles and combining them for reconstruction.

[0063] It is understandable that, taking the target object as a human thigh model, as an example, Figure 3 As shown in (a), a human thigh model is visually reconstructed in 3D. The reconstruction result is shown in Figure 3 As shown in b), the human thigh model was successfully reconstructed on the computer. However, during the reconstruction process, the desktop was also reconstructed. In the subsequent tactile detection, the desktop is not desirable. Therefore, the point cloud scene needs to be truncated and voxel downsampling is performed to reduce the number of measurement points. The result is as follows: Figure 3 In order to perform tactile measurement on a point, in addition to knowing the measured position, we also need to know the normal of the measured point. Therefore, it is also necessary to estimate the normal of the point cloud model. The result is as follows Figure 3 As shown in (d).

[0064] It is understandable that after obtaining the point cloud data and its normal vector of the target object, these point cloud data cannot be directly used for tactile detection. Because these point cloud data are disordered, using these disordered data for tactile detection is very likely to cause the robot arm to interfere with the sample and collide, resulting in damage to the sample and the robot arm. Therefore, it is necessary to sort these point cloud data and plan the path for the robot arm's scanning. When the robot arm moves between any two measurement points, its movement path can be simplified to Figure 4 As shown in a), the mathematical relationship is shown in the following formula:

[0065]

[0066] In the above formula, H is the approximate movement trajectory of the robot arm, L is the length of the probe, h is the distance between the measurement points, and θ is the angle between the normals of the measurement points.

[0067] In order to prevent the target object from interfering with the robotic arm, this can be achieved as long as the distance H between any two adjacent measurement points is small enough.

[0068] At the same time, it is necessary to perform tactile measurements on all measurement points without duplication. This problem can be classified as solving the Hamiltonian Path problem. The distance H between two adjacent measurement points is small enough, that is, only the local optimum is considered. In this study, it was decided to use a greedy algorithm to arrange the measurement points. Its logic is as follows Figure 4 As shown in b).

[0069] First, the approximate moving trajectory H of the robot arm is calculated between all the points to be measured, and the one with the largest H is selected as the preset starting point P0 and the preset end point P n Then randomly select one of the remaining measurement points and insert it into P0 and P n In the gap, such as Figure 4 As shown in (b), there are three possible insertion methods for the first insertion. The sum of the H values of the three methods is compared, and the method with the smallest H value is selected as the final insertion method. The above steps are then repeated for the remaining measurement points until all measurement points are sorted.

[0070] It's understandable that the data for a single point in the sorted sequence consists of its position and its normal vector (X, Y, Z, NX, NY, NZ), while the data required by the robotic arm is pose data consisting of its position and posture (X, Y, Z, RX, RY, RZ). Therefore, the normal vectors in the sequence also need to be converted into posture data.

[0071] It can be understood that since the probe needs to move in a straight line along the direction of the normal vector, and the probe coincides with the Z axis of the end-of-arm posture, according to the definition of the rotation matrix, the rotation matrix of this posture is as follows:

[0072]

[0073] Where (NX, NY, NZ) is the representation of the normal vector of the point in the base coordinate system.

[0074] The rest of the matrix is unknown, which means there are countless poses that meet the measurement requirements, and a reasonable pose needs to be selected as the final data. In order to prevent entanglement between connecting lines, it is hoped that the depth camera is always facing outward. In mathematical terms, it is hoped that the projection of the Y axis of the pose on the XY plane of the base coordinate system coincides with the projection of the position on the XY plane of the base coordinate system, and the direction is from the center of the circle to the outside. The calculation is shown in the following formula:

[0075]

[0076] Solving the above equations yields the Y-axis component of the posture. The remaining X-axis component can be obtained by cross-multiplying the Y-axis component with the Z-axis component. Finally, using the Loglider rotation formula, the rotation matrix is converted into a rotation vector, which is the final data format.

[0077] Taking the human thigh model as an example, the human thigh model was measured according to the above sequence, and all measurement steps were successfully completed without interference between the robot arm and the sample. The measurement results were imaged to obtain the tactile tomography image of the human thigh model as shown below: Figure 5 As shown in a) to j), the indentation tomography of the target object was successfully achieved.

[0078] Based on this, for target objects with complex curved surfaces, the present invention first uses a vision module to reconstruct the target object's optical surface contour, simultaneously estimating the surface contour normal and outputting the position and normal information of the test points. A greedy algorithm is then used to sort the test points and plan the robotic arm's scanning path. The tactile detection module then performs tactile scanning of the target surface based on the scanning path, obtaining measurement results. Finally, the measurement results are imaged to produce a tactile tomographic image of the target object, thus enabling tactile scanning imaging of complex curved surfaces.

[0079] In addition, if Figure 6 As shown, one embodiment of the present invention further discloses a tactile scanning imaging robot, the robot comprising:

[0080] The vision module 110 is used to reconstruct the optical surface contour of the target object to obtain a plurality of target point cloud data, wherein the target point cloud data includes position information and normal information;

[0081] A sorting module 120 is used to sort the multiple target point cloud data using a greedy algorithm to obtain a measurement sequence;

[0082] a planning module 130 for planning a scanning path of a robotic arm of the tactile scanning imaging robot according to the measurement sequence;

[0083] The tactile detection module 140 is used to perform tactile scanning detection on the target surface according to the scanning path to obtain a measurement result;

[0084] The imaging module 150 is used to image the measurement results to obtain a tactile tomographic image of the target object.

[0085] The tactile scanning imaging device of the embodiment of the present invention is used to execute the tactile scanning imaging method of the above embodiment. Its specific processing process is the same as that of the tactile scanning imaging method of the above embodiment, and will not be described in detail here.

[0086] In addition, if Figure 7 As shown, an embodiment of the present invention further discloses an electronic device, comprising: at least one processor 210; at least one memory 220 for storing at least one program; when the at least one program is executed by the at least one processor 210, a tactile scanning imaging method as in any of the previous embodiments is implemented.

[0087] In addition, an embodiment of the present invention further discloses a computer-readable storage medium, in which computer-executable instructions are stored. The computer-executable instructions are used to execute the tactile scanning imaging method in any of the above embodiments.

[0088] The system architecture and application scenarios described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are equally applicable to similar technical problems.

[0089] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0090] In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0091] As used in this specification, the terms "component," "module," "system," and the like are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components can reside in a process or execution thread, and a component can be located on one computer or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, through local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, or a network, such as the Internet interacting with other systems via signals).

Claims

1. A tactile scanning imaging method, applied to a tactile scanning imaging robot, wherein the tactile scanning imaging robot includes a vision module, a tactile detection module, a wireless bridge, and a computer. The method comprises: Reconstructing the optical surface contour of the target object using the vision module to obtain a plurality of target point cloud data, wherein the target point cloud data includes position information and normal information; Use the greedy algorithm to sort multiple target point cloud data to obtain the measurement sequence; Planning a scanning path of the mechanical arm of the tactile scanning imaging robot according to the measurement sequence, so that the tactile detection module performs tactile scanning detection on the target surface according to the scanning path to obtain a measurement result; The measurement results are imaged to obtain a tactile tomographic image of the target object.

2. The method according to claim 1, characterized in that The optical surface contour reconstruction of the target object is performed using the visual module to obtain a plurality of target point cloud data, including: Using the visual module to perform a three-dimensional visual scan of the target object to obtain a point cloud result of surface contour reconstruction; Truncating the point cloud scene in the point cloud result and performing voxel downsampling to obtain target point cloud data; Normal estimation is performed on the target point cloud data according to the position information to obtain the normal information.

3. The method according to claim 1, characterized in that The visual module includes a depth camera and a Raspberry Pi. The Raspberry Pi reads the image frames of the depth camera, generates a depth map and a color map through the official decoding library of the depth camera, and then uses a UDP video stream to send the depth map and the color map to the computer through the wireless bridge. The computer combines the depth map and the color map with the posture of the depth camera to reconstruct the three-dimensional scene.

4. The method according to claim 3, characterized in that The tactile detection module includes a robotic arm, a pressure sensor and a probe. The target point to be measured is selected in the three-dimensional scene generated by the vision module. The computer sends the position and posture of the target point to the robotic arm via the Modbus-TCP protocol. The robotic arm moves to the position and posture of the target point to perform tactile scanning detection.

5. The method according to claim 4, characterized in that The expression of the scan path is as follows: Wherein, H is the scanning path of the robot arm, L is the length of the probe, h is the distance between the measurement points, and θ is the angle between the normals of the measurement points.

6. The method according to claim 5, characterized in that The method of using a greedy algorithm to sort multiple target point cloud data to obtain a measurement sequence includes: Calculate the scanning path H of the robot arm between all the target point cloud data, and select the one with the largest H as the preset starting point P0 and the preset end point P n ; Randomly select one of the remaining target point cloud data and insert it into P0 and P n In the gap, compare the sum of H values of different methods and select the one with the smallest sum of H value as the target insertion method; The remaining target point cloud data are inserted in sequence according to the target insertion method until all the target point cloud data are sorted.

7. The method according to claim 6, characterized in that The method further comprises: Converting the normal information in the measurement sequence into a posture to obtain a rotation matrix; The rotation matrix is converted into a rotation vector using the Loglider rotation formula.

8. A tactile scanning imaging robot, characterized in that: The robot comprises: A vision module is used to reconstruct the optical surface contour of the target object to obtain a plurality of target point cloud data, wherein the target point cloud data includes position information and normal information; The sorting module is used to sort multiple target point cloud data using a greedy algorithm to obtain a measurement sequence; a planning module, configured to plan a scanning path of a mechanical arm of the tactile scanning imaging robot according to the measurement sequence; a tactile detection module, configured to perform tactile scanning detection on the target curved surface according to the scanning path to obtain a measurement result; The imaging module is used to image the measurement results to obtain a tactile tomographic image of the target object.

9. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the tactile scanning imaging method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the tactile scanning imaging method according to any one of claims 1 to 7.

Images