A mechanical arm assisted free scanning virtual array photoacoustic three-dimensional imaging method and system

Abstract

The application discloses a kind of mechanical arm auxiliary free scanning virtual array photoacoustic three-dimensional imaging method and system, it is related to three-dimensional imaging technical field, the method includes: by photoacoustic probe array free scanning target area, acquisition multiple frames original signal and real-time record each frame corresponding six degrees of freedom pose.According to time synchronization mechanism, signal and pose are matched, the geometric coordinates of each array element in probe coordinate system are combined, the spatial position of array element in world coordinate system is calculated by rigid body transformation, and it is mapped to uniform reconstruction coordinate system, and a virtual multi-array sensor array is constructed.Finally, using the spatial position of each array element in the array and its original signal, three-dimensional photoacoustic image reconstruction is carried out, and the three-dimensional image of target area is obtained.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional imaging technology, and in particular relates to a robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging method and system. Background Technology

[0002] Photoacoustic imaging is a novel biomedical imaging technique that combines the contrast of optical imaging with the spatial resolution of ultrasound imaging. This technique uses pulsed laser light to irradiate biological tissue; the tissue absorbs the light energy and undergoes transient thermoelastic expansion, thereby generating an ultrasonic signal. This signal is then received by an ultrasound transducer to reconstruct the image. Due to its combination of high contrast and deep tissue penetration capability, photoacoustic imaging has broad application prospects in fields such as vascular imaging, tumor detection, functional imaging, and intraoperative navigation.

[0003] To obtain three-dimensional structural information of a target region, three-dimensional photoacoustic imaging technology has gradually become a research hotspot. Existing three-dimensional photoacoustic imaging methods mainly include the following categories: (1) Three-dimensional imaging method based on two-dimensional array probe. This method can realize real-time three-dimensional imaging by directly acquiring three-dimensional data through a two-dimensional array. However, the two-dimensional array probe has a complex structure, high cost, and limited number of array elements, making it difficult to balance the imaging field of view and resolution.

[0004] (2) Three-dimensional imaging methods based on mechanical scanning. This method typically uses a motor or scanning mechanism to drive a one-dimensional array probe to perform linear, rotational, or circular scanning along a preset trajectory, and achieves three-dimensional reconstruction by stitching together multiple frames of data. However, this type of method relies on a fixed scanning trajectory, has high requirements for mechanical structure, poor scanning flexibility, and is difficult to adapt to complex or irregular target areas.

[0005] With the development of robotics technology, robotic arm systems can provide high-precision spatial positioning capabilities and output six-DOF pose information of the end effector in real time, including three-dimensional spatial position and orientation information. Combining robotic arm systems with photoacoustic imaging systems holds promise for accurately recording the probe's spatial pose under free scanning conditions, thus providing reliable geometric constraints for 3D reconstruction. However, most existing technologies use robotic systems to actively drive the scanning path, lacking a technical solution for passively recording pose under free scanning conditions using robotic systems, and further combining element spatial mapping and virtual array construction for 3D photoacoustic reconstruction. Furthermore, existing methods lack mechanisms to optimize and guide the scanning path based on the reconstruction results, addressing issues such as uneven sampling and insufficient viewpoint coverage during free scanning. Therefore, a new photoacoustic 3D imaging method is urgently needed, capable of achieving high-precision pose recording using a robotic arm system under free scanning conditions, and constructing a virtual array based on this pose information to achieve high-quality 3D photoacoustic image reconstruction, further improving scanning efficiency and imaging quality. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a robotic arm-assisted free-scanning virtual array photoacoustic three-dimensional imaging method, comprising: The target area is freely scanned by an array of photoacoustic probes to collect multiple frames of raw photoacoustic signals, and the six-degree-of-freedom pose information corresponding to each frame of raw photoacoustic signal is recorded in real time. According to the time synchronization mechanism, the original photoacoustic signal of each frame is matched with the corresponding six-degree-of-freedom pose information. Based on the geometric coordinates of each element in the photoacoustic probe array in the probe coordinate system and the matched six-degree-of-freedom pose information, the spatial position of each element in the photoacoustic probe array in the world coordinate system is calculated by rigid body coordinate transformation and mapped to a unified three-dimensional reconstruction coordinate system to construct a virtual multi-element sensor array. Based on the spatial position of each element in the virtual multi-element sensor array and its corresponding original photoacoustic signal, a three-dimensional photoacoustic image reconstruction is performed on the target area to obtain a three-dimensional photoacoustic image of the target area.

[0007] Furthermore, before freely scanning the target area using the photoacoustic probe array, the process includes: mounting the photoacoustic probe array on the end effector of the robotic arm system, wherein the robotic arm system is in a passive pose recording mode during the scanning process, recording only the pose information of the end effector without performing active scanning motion.

[0008] Furthermore, the six-degree-of-freedom pose information includes: the three-dimensional translation vector and attitude information of the photoacoustic probe array in the world coordinate system.

[0009] Furthermore, it also includes: during photoacoustic scanning or 3D photoacoustic image reconstruction, identifying areas with insufficient imaging information based on the acquired partial 3D reconstruction results, and generating scanning path suggestions to guide operators in adjusting the probe scanning position and orientation.

[0010] Furthermore, matching the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism includes: The robotic arm system acquires the three-dimensional translation vector and attitude information of the end effector in a preset reference coordinate system. Using the laser trigger signal as a unified timing reference, the photoacoustic acquisition and the robotic arm pose recording are synchronized in time. For the The original photoacoustic signal of the frame, its trigger time is denoted as In the robotic arm pose recording sequence, with Two adjacent sampling times are denoted as follows: and And satisfy ,when When the sampling time is inconsistent with the robotic arm pose recording sequence, calculate the time interpolation coefficient. : in, For the first The triggering time of the original frame-optical signal. The first position in the robotic arm pose recording sequence Each sampling time, The first position in the robotic arm pose recording sequence Each sampling time; Linear interpolation is used to obtain the time step of the three-dimensional translation vector. Interpolation 3D translation vector at the location : in, For a moment The three-dimensional translation vector at that location, For a moment The three-dimensional translation vector at that location, For matrix transpose, For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm; When attitude information is represented using quaternions, spherical linear interpolation is used to apply the attitude information: in, For a moment The interpolation unit quaternion at the location, and They are time points and The unit quaternion at the location, for and The angle between them; When attitude information is represented using Euler angles, linear interpolation is used for the attitude information; The interpolated pose information is matched with the corresponding frame's original photoacoustic signal, and the first... The interpolated pose information corresponding to the original photoacoustic signal is uniformly represented as a homogeneous transformation matrix: in, For the first The homogeneous transformation matrix of the pose information after interpolation corresponding to the original photoacoustic signal. For the first The rotation matrix obtained by converting the pose information after interpolation of the original photoacoustic signal. It is a three-dimensional zero column vector.

[0011] This invention also proposes a robotic arm-assisted free-scanning virtual array photoacoustic three-dimensional imaging system, comprising: The acquisition module is used to freely scan the target area through the photoacoustic probe array, acquire multiple frames of raw photoacoustic signals, and record the six-degree-of-freedom pose information corresponding to each frame of raw photoacoustic signals in real time. A virtual multi-element sensor array module is constructed to match the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism. Based on the geometric coordinates of each element in the photoacoustic probe array in the probe coordinate system and the matched six-degree-of-freedom pose information, the spatial position of each element in the photoacoustic probe array in the world coordinate system is calculated through rigid body coordinate transformation and mapped to a unified three-dimensional reconstruction coordinate system to construct a virtual multi-element sensor array. The three-dimensional reconstruction module is used to reconstruct a three-dimensional photoacoustic image of the target area based on the spatial position of each element in the virtual multi-element sensor array and its corresponding original photoacoustic signal, thereby obtaining a three-dimensional photoacoustic image of the target area.

[0012] Furthermore, before freely scanning the target area using the photoacoustic probe array, the process includes: mounting the photoacoustic probe array on the end effector of the robotic arm system, wherein the robotic arm system is in a passive pose recording mode during the scanning process, recording only the pose information of the end effector without performing active scanning motion.

[0013] Furthermore, the six-degree-of-freedom pose information includes: the three-dimensional translation vector and attitude information of the photoacoustic probe array in the world coordinate system.

[0014] Furthermore, it also includes: during photoacoustic scanning or 3D photoacoustic image reconstruction, identifying areas with insufficient imaging information based on the acquired partial 3D reconstruction results, and generating scanning path suggestions to guide operators in adjusting the probe scanning position and orientation.

[0015] Furthermore, matching the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism includes: The robotic arm system acquires the three-dimensional translation vector and attitude information of the end effector in a preset reference coordinate system. Using the laser trigger signal as a unified timing reference, the photoacoustic acquisition and the robotic arm pose recording are synchronized in time. For the The original photoacoustic signal of the frame, its trigger time is denoted as In the robotic arm pose recording sequence, with Two adjacent sampling times are denoted as follows: and And satisfy ,when When the sampling time is inconsistent with the robotic arm pose recording sequence, calculate the time interpolation coefficient. : in, For the first The triggering time of the original frame-optical signal. The first position in the robotic arm pose recording sequence Each sampling time, The first position in the robotic arm pose recording sequence Each sampling time; Linear interpolation is used to obtain the time step of the three-dimensional translation vector. Interpolation 3D translation vector at the location : in, For a moment The three-dimensional translation vector at that location, For a moment The three-dimensional translation vector at that location, For matrix transpose, For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm; When attitude information is represented using quaternions, spherical linear interpolation is used to apply the attitude information: in, For a moment The interpolation unit quaternion at the location, and They are time points and The unit quaternion at the location, for and The angle between them; When attitude information is represented using Euler angles, linear interpolation is used for the attitude information; The interpolated pose information is matched with the corresponding frame's original photoacoustic signal, and the first... The interpolated pose information corresponding to the original photoacoustic signal is uniformly represented as a homogeneous transformation matrix: in, For the first The homogeneous transformation matrix of the pose information after interpolation corresponding to the original photoacoustic signal. For the first The rotation matrix obtained by converting the pose information after interpolation of the original photoacoustic signal. It is a three-dimensional zero column vector.

[0016] Compared with the prior art, the present invention has the following advantages and technical effects: (1) High scanning flexibility: The robotic arm system is in passive pose recording mode during the scanning process. The operator can move the probe freely according to the structure of the target area without relying on a preset fixed scanning trajectory. It can adapt to complex target areas of any shape, significantly improving scanning flexibility.

[0017] (2) High spatial positioning accuracy of array elements: The six-degree-of-freedom pose information is recorded in real time by the robotic arm system, and each physical array element is accurately mapped to a unified world coordinate system through time synchronization and coordinate transformation, which ensures the positioning accuracy of each array element in the virtual array.

[0018] (3) High quality of 3D reconstruction: The virtual multi-element sensor array is composed of all physical elements in multiple frames of scan data. The total number of elements is equal to the product of the number of elements in a single frame and the number of effective scan frames. Data filtering, density weighting and various reconstruction algorithms ensure the reconstruction quality.

[0019] (4) Strong imaging integrity: The path optimization module evaluates the reconstruction quality in real time during the scanning process, identifies areas with insufficient imaging information and generates scanning suggestions to guide operators to supplement the insufficient coverage areas, thereby improving the spatial sampling integrity and angular coverage. Attached Figure Description

[0020] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a system structure diagram of Embodiment 2 of the present invention. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0023] In this invention, the photoacoustic probe array is mounted on the end effector of the robotic arm system. During the scanning process, the robotic arm system does not actively drive the probe to execute the preset trajectory movement, but is in a passive pose recording mode to output the six-degree-of-freedom pose information corresponding to the probe in real time; the laser emits pulsed laser towards the target to be tested.

[0024] Example 1 Before starting imaging, perform advanced extrinsic parameter calibration: In this embodiment, the photoacoustic probe array can be a linear ultrasonic transducer array, an arc array, or other one-dimensional array forms. Assume a single-frame probe array includes... The first physical array element, the... The fixed geometric coordinates of each array element in the probe coordinate system are: Among them, superscript Indicates the probe coordinate system. , , The first The coordinate components of each array element along the three axes of the probe coordinate system are indicated by the superscript. This represents the matrix transpose. The coordinates of the probe coordinate system of each array element are pre-calibrated and kept fixed.

[0025] Before imaging begins, the coordinate system of the robotic arm's end effector must first be established. With the probe coordinate system Spatial extrinsic parameter calibration between the coordinate systems yields the extrinsic parameter transformation matrix of the probe coordinate system relative to the end effector coordinate system. in, for Homogeneous transformation matrix, For the probe coordinate system To the end effector coordinate system of Rotation matrix, For the corresponding three-dimensional translation vector, This is a three-dimensional zero column vector. This extrinsic parameter is used to accurately map the probe element coordinates to the world coordinate system later.

[0026] In this embodiment, the external parameter calibration can be performed using a rigid fixing fixture method: several marker points are fixed on the probe array and the end effector of the robotic arm, respectively. The coordinates of the marker points in each coordinate system are simultaneously obtained using an optical tracking system or ultrasonic echo method, and the rotation matrix is ​​solved using the least squares method. Translation vector This minimizes the sum of the mapping residuals of each marker point between the two coordinate systems. Specifically, let the point be the first marker point in the probe coordinate system. The coordinates of the marker points are The corresponding coordinates in the end effector coordinate system are Then the optimal extrinsic parameters satisfy: ,in This represents the total number of marker points. After calibration, it should be verified that the mapping residual of each marker point does not exceed the preset accuracy threshold.

[0027] like Figure 1 As shown in the figure, this embodiment proposes a robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging method, which specifically includes the following steps: Step S1: The target area is freely scanned by the photoacoustic probe array to collect multiple frames of original photoacoustic signals, and the six-degree-of-freedom pose information corresponding to each frame of the original photoacoustic signal is recorded in real time. Specifically, before freely scanning the target area using the photoacoustic probe array, the process further includes: mounting the photoacoustic probe array on the end effector of the robotic arm system. During the scanning process, the robotic arm system is in a passive pose recording mode, recording only the pose information of the end effector without performing active scanning motion.

[0028] Specifically, the six-degree-of-freedom pose information includes: the three-dimensional translation vector and attitude information of the photoacoustic probe array in the world coordinate system.

[0029] Preferably, during imaging, the operator manually holds the end of the robotic arm or drags it with zero force to allow the photoacoustic probe array to freely scan the target area. During scanning, the laser emits pulsed laser light at a set repetition frequency to the target tissue. After absorbing the light energy, the tissue undergoes transient thermoelastic expansion and emits ultrasonic waves, which are received by the photoacoustic probe array. For the first... The laser trigger is recorded at the moment of triggering. The acquired multi-channel raw photoacoustic signals are as follows: in, For the first Frame multi-channel photoacoustic raw signal vector, For the first The first frame The photoacoustic time-domain signal received by each array element , This represents the total number of scan frames.

[0030] At the same time, the robotic arm controller or external pose recording module outputs the six-degree-of-freedom pose information of the end effector at a sampling rate no lower than the photoacoustic sampling trigger frequency.

[0031] Step S2: Match the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism. Based on the geometric coordinates of each element in the photoacoustic probe array in the probe coordinate system and the matched six-degree-of-freedom pose information, calculate the spatial position of each element in the photoacoustic probe array in the world coordinate system through rigid body coordinate transformation, and map it to a unified three-dimensional reconstruction coordinate system to construct a virtual multi-element sensor array. Specifically, matching the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism includes: The robotic arm system acquires the three-dimensional translation vector and attitude information of the end effector in a preset reference coordinate system. Using the laser trigger signal as a unified timing reference, the photoacoustic acquisition and the robotic arm pose recording are synchronized in time. Due to the timing of photoacoustic signal acquisition The timing of the robotic arm pose sampling is usually not completely coincident, therefore time synchronization and pose interpolation are required. Using the laser trigger signal as a unified timing reference, for the... The original photoacoustic signal of the frame, its trigger time is denoted as In the robotic arm pose recording sequence, with Two adjacent sampling times are denoted as follows: and And satisfy ,when When the sampling time is inconsistent with the robotic arm pose recording sequence, calculate the time interpolation coefficient. : in, For the first The triggering time of the original frame-optical signal. The first position in the robotic arm pose recording sequence Each sampling time, The first position in the robotic arm pose recording sequence Each sampling time; Linear interpolation is used to obtain the time step of the three-dimensional translation vector. Interpolation 3D translation vector at the location : in, For a moment The three-dimensional translation vector at that location, For a moment The three-dimensional translation vector at that location, For matrix transpose, For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm; When attitude information is represented using quaternions, spherical linear interpolation is used to apply the attitude information: in, For a moment The interpolation unit quaternion at the location, and They are time points and The unit quaternion at the location, for and The angle between them, when In this case, one of the quaternions is inverted to ensure interpolation is performed along the shortest path. If the attitude change is small, normalized linear interpolation can also be used as an approximation. When the attitude information uses Euler angles... and When representing the attitude information, linear interpolation is used: ; When attitude information is represented using Euler angles, linear interpolation is used for the attitude information; The interpolated quaternion Convert to Rotation matrix Combined with translation interpolation results , will the The frame end effector pose is uniformly represented as Homogeneous transformation matrix: in, For the first The homogeneous transformation matrix of the pose information after interpolation corresponding to the original photoacoustic signal. For the first The rotation matrix obtained by converting the pose information after interpolation of the original photoacoustic signal. For a three-dimensional zero column vector, Record Clearly indicate the coordinate system of the end effector To the world coordinate system The transformation relationship.

[0032] Specifically, the spatial position of each element in the photoacoustic probe array in the world coordinate system is calculated through rigid body coordinate transformation and mapped to a unified three-dimensional reconstruction coordinate system to construct a virtual multi-element sensor array, including: Combined with extrinsic transformation matrix Calculate the transformation matrix of the probe coordinate system relative to the world coordinate system: in, For the first Frame probe coordinate system To the world coordinate system of Homogeneous transformation matrix, its rotation matrix components Translation vector components They are respectively: in, and They are respectively (Right now The rotation matrix components and translation vector components in ).

[0033] For the The first frame Each array element has the following spatial position in the world coordinate system: in, For the first Frame number The three-dimensional coordinate column vector of each element in the world coordinate system, with superscript... Indicates the world coordinate system. Note: The fixed geometric coordinates of the probe array elements, and the frame number. Irrelevant; The reason for the frame variation is that each frame corresponds to a different pose transformation matrix. .

[0034] When the total collection When processing frame data, a virtual multi-element sensor array is constructed based on the spatial locations of all array elements in all frames: in, This is a virtual multi-element sensor array, containing the world coordinate system positions corresponding to all frames and all elements. The total number of elements in the virtual multi-element sensor array is: in, The total number of elements in a virtual multi-element sensor array is expanded into a large-scale virtual element set through free scanning, which is originally a limited number of physical elements in a single frame. This improves the spatial sampling capability and observation angle coverage capability during 3D reconstruction.

[0035] Preferably, to reduce the impact of abnormal sampling on the reconstruction results, the virtual array data is filtered after constructing the virtual multi-element sensor array. This is done using a continuous index. Renumber the virtual array elements, let the first one be numbered. The actual spatial location of each virtual array element is The corresponding reference space location is Then its translational deviation is: in, For translational deviation, Let be the Euclidean norm. If the attitude is represented by a rotation matrix, let the be... The actual rotation matrix of the frame corresponding to each virtual element is: The corresponding reference rotation matrix is Then the attitude deviation angle is: in, The attitude deviation angle is... Let be the trace of the matrix. When or At that time, the virtual array metadata was determined to be abnormal pose data and was removed. The translation deviation threshold, This is the attitude deviation angle threshold.

[0036] Reference position and reference rotation matrix By examining the first The pose information of each virtual array element corresponding to a frame and several adjacent frames is obtained by moving average filtering. Specifically, let the window length be... ,but ,in For the first The frame centered on the virtual array element The set of element indices within the frame neighborhood; reference rotation matrix It is obtained by the average rotation of each rotation matrix in the neighborhood of the corresponding frame (calculated by the mean in the logarithmic mapping domain).

[0037] For the reserved first The signal-to-noise ratio (SNR) of the photoacoustic signals corresponding to each virtual array element is determined. Let the average power within the signal window be: in, The average power within the signal window. For the first The first virtual array element photoacoustic signal One sampling point, Let be the length of the signal window. Assume the average power within the noise window is: in, The average power within the noise window. For the first The virtual array element corresponds to the noise signal of the first... One sampling point, Let be the length of the noise window. Then the ... The signal-to-noise ratio of the photoacoustic signal corresponding to each virtual element for: when When the time comes, the corresponding signal will be discarded, among which The threshold is set for the signal-to-noise ratio.

[0038] For the retained valid virtual array elements, let the total number of retained valid virtual array elements after filtering be . Weighting coefficients are constructed based on local spatial density. The local spatial density at each array element is calculated using kernel density estimation: in, For the first Local spatial density at each virtual array element For bandwidth kernel function, For the first The spatial locations of each effective virtual element. The weighting coefficient is inversely proportional to the local density: in, For the first The weight coefficients of each virtual array element. To prevent the regularization constant from being zero in the denominator, the corresponding weighted photoacoustic signal is: in, For the first The photoacoustic time-domain signal corresponding to each virtual array element This is a weighted photoacoustic signal, thereby reducing the excessive contribution of overly dense array elements to the reconstruction results.

[0039] Step S3: Based on the spatial position of each element in the virtual multi-element sensor array and its corresponding original photoacoustic signal, perform three-dimensional photoacoustic image reconstruction on the target area to obtain a three-dimensional photoacoustic image of the target area.

[0040] Specifically, this also includes: during photoacoustic scanning or 3D photoacoustic image reconstruction, identifying areas with insufficient imaging information based on the acquired partial 3D reconstruction results, and generating scanning path suggestions to guide operators in adjusting the probe scanning position and orientation.

[0041] Preferably, a three-dimensional voxel mesh is established in a unified three-dimensional reconstruction coordinate system: in, It is a three-dimensional voxel mesh. The total number of voxels. For the first The central coordinates of the individual element , , These are the coordinate components of the voxel center along the three axes.

[0042] For the There are 10 effective virtual array elements, and their spatial locations are: The speed of sound in the medium is Then the first The sound wave propagation time from an individual pixel to this array element is: in, For the first Individual element to the first The sound wave propagation time of each effective virtual array element For the first The spatial positions of each effective virtual element in the three-dimensional reconstructed coordinate system. This is considering the non-uniform distribution of sound velocity in the medium. The propagation time can be expressed as a path integral: ,in voxels arrive The sound wave propagation path.

[0043] This embodiment preferably uses a time-delayed stacking method for 3D reconstruction. For any voxel... Its reconstruction strength value is: in, For the first Reconstruction intensity value of individual pixels, For the first The weight coefficients of each effective virtual array element. For the first The propagation time of the photoacoustic signal of each virtual array element The sampled value at that location. If If the value is not an integer, linear interpolation is used to obtain the signal value. in, The integer sample point index is the one rounded down. For the decimal part, The sampling time interval for the photoacoustic signal. and The first The first virtual array element photoacoustic signal and the A discrete sampling point.

[0044] Besides the time-delayed superposition method, back-projection algorithms, iterative reconstruction algorithms based on system matrices, total variation constraint reconstruction algorithms, or sparse prior-based reconstruction algorithms can also be used as needed. Those skilled in the art will recognize that the aforementioned virtual array geometric constraints can be applied to different 3D reconstruction frameworks without departing from the core concept of this invention.

[0045] Preferably, in a further improvement of this embodiment, the system performs real-time or staged reconstruction during the scanning process and optimizes and guides the subsequent scanning path based on the current reconstruction results. Let the currently reconstructed 3D image be... For the position of each voxel Construct an imaging quality evaluation function: in, Spatial location Image quality evaluation value at the location, This is a signal strength indicator (the amplitude of the photoacoustic signal at this location). This is a structural clarity index (taking the local gradient magnitude at this location). For angular coverage indicators, , , These are the weighting coefficients for the three indicators. An image quality evaluation value below a preset threshold will be considered... The set of voxels is defined as the region with insufficient imaging information: in, For the set of voxels in the region with insufficient imaging information, This is the preset imaging quality threshold.

[0046] For any voxel in the region with insufficient imaging information , define the first The unit observation direction vector of each effective virtual element for this voxel is: in, For the first Each effective virtual element pairs voxel position The unit observation direction vector. Angular coverage index. Measured by the degree of dispersion among the observation direction vectors of each array element: in, This represents the total number of valid virtual array elements currently participating in the calculation. For the first Each effective virtual element pairs voxel position The unit observation direction vector, For the first Each effective virtual element pairs voxel position The unit observation direction vector. The larger this index is, the more dispersed the observation directions are and the more comprehensive the angular coverage.

[0047] Let the next candidate scan pose be The corresponding set of newly added virtual array element positions is Then, for each voxel in the region with insufficient imaging information, a path optimization objective function is established: in, Candidate pose The corresponding path optimization objective function value, voxel position The weighting coefficients, Candidate pose voxel position Information gain. From the candidate scan pose set. Solving for the optimal candidate pose: in, This is the optimal candidate scan pose. Based on the optimal candidate pose... Calculate the translation adjustment amount based on the difference between the current probe pose and the current probe pose: in, This is the probe translation adjustment amount. Let be the translation vector corresponding to the optimal candidate pose. This is the translation vector corresponding to the current probe pose; simultaneously, the corresponding attitude adjustment amount is calculated. The system outputs these guidance amounts to the operator in a graphical or textual prompt format to guide them in adjusting the probe's scanning position, direction, or attitude.

[0048] In this embodiment, the information gain function Defined as candidate pose After the introduction of the new virtual array elements, voxels Increment in coverage at angle: in, To add to the current virtual array After all the newly added array elements are added, the voxels Angular coverage at that location This represents the current angular coverage. The candidate scan pose set. It consists of a finite number of candidate poses obtained by uniformly sampling in the translation and rotation directions with a preset step size, centered on the current probe pose. Voxel weights. It can be taken as the prior structural importance of the region where the voxel is located, or it can be directly set as a constant.

[0049] Through the above methods, this embodiment achieves high-precision passive recording of probe pose using a robotic arm system under free scanning conditions, and completes high-quality three-dimensional photoacoustic imaging by combining array element spatial mapping, virtual array construction, data filtering and weighting, and three-dimensional reconstruction algorithms; at the same time, it can also optimize and guide the scanning path in real time based on partial reconstruction results, thereby further improving imaging integrity and reconstruction quality.

[0050] Example 2 like Figure 2 As shown, this embodiment proposes a robotic arm-assisted free-scanning virtual array photoacoustic three-dimensional imaging system, which specifically includes the following modules: The acquisition module is used to freely scan the target area through the photoacoustic probe array, acquire multiple frames of raw photoacoustic signals, and record the six-degree-of-freedom pose information corresponding to each frame of raw photoacoustic signals in real time. A virtual multi-element sensor array module is constructed to match the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism. Based on the geometric coordinates of each element in the photoacoustic probe array in the probe coordinate system and the matched six-degree-of-freedom pose information, the spatial position of each element in the photoacoustic probe array in the world coordinate system is calculated through rigid body coordinate transformation and mapped to a unified three-dimensional reconstruction coordinate system to construct a virtual multi-element sensor array. The three-dimensional reconstruction module is used to reconstruct a three-dimensional photoacoustic image of the target area based on the spatial position of each element in the virtual multi-element sensor array and its corresponding original photoacoustic signal, thereby obtaining a three-dimensional photoacoustic image of the target area.

[0051] Since the system technical solution of this embodiment 2 is based on the technical solution of embodiment 1, it will not be described again.

Claims

1. A robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging method, characterized in that, include: The target area is freely scanned by an array of photoacoustic probes to collect multiple frames of raw photoacoustic signals, and the six-degree-of-freedom pose information corresponding to each frame of raw photoacoustic signal is recorded in real time. According to the time synchronization mechanism, the original photoacoustic signal of each frame is matched with the corresponding six-degree-of-freedom pose information. Based on the geometric coordinates of each element in the photoacoustic probe array in the probe coordinate system and the matched six-degree-of-freedom pose information, the spatial position of each element in the photoacoustic probe array in the world coordinate system is calculated by rigid body coordinate transformation and mapped to a unified three-dimensional reconstruction coordinate system to construct a virtual multi-element sensor array. Based on the spatial position of each element in the virtual multi-element sensor array and its corresponding original photoacoustic signal, a three-dimensional photoacoustic image reconstruction is performed on the target area to obtain a three-dimensional photoacoustic image of the target area.

2. The robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging method as described in claim 1, characterized in that, Before freely scanning the target area using the photoacoustic probe array, the process further includes: mounting the photoacoustic probe array on the end effector of the robotic arm system, wherein the robotic arm system is in a passive pose recording mode during the scanning process, recording only the pose information of the end effector without performing active scanning motion.

3. The robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging method as described in claim 1, characterized in that, The six-degree-of-freedom pose information includes: the three-dimensional translation vector and attitude information of the photoacoustic probe array in the world coordinate system.

4. The robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging method as described in claim 1, characterized in that, Also includes: During photoacoustic scanning or 3D photoacoustic image reconstruction, areas with insufficient imaging information are identified based on the acquired partial 3D reconstruction results, and scanning path suggestions are generated to guide operators in adjusting the probe scanning position and orientation.

5. The robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging method as described in claim 1, characterized in that, Matching the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism includes: The robotic arm system acquires the three-dimensional translation vector and attitude information of the end effector in a preset reference coordinate system. Using the laser trigger signal as a unified timing reference, the photoacoustic acquisition and the robotic arm pose recording are synchronized in time. For the The original photoacoustic signal of the frame, its trigger time is denoted as In the robotic arm pose recording sequence, with Two adjacent sampling times are denoted as follows: and And satisfy ,when When the sampling time is inconsistent with the robotic arm pose recording sequence, calculate the time interpolation coefficient. : in, For the first The triggering time of the original frame-optical signal. The first position in the robotic arm pose recording sequence Each sampling time, The first position in the robotic arm pose recording sequence Each sampling time; Linear interpolation is used to obtain the time step of the three-dimensional translation vector. Interpolation 3D translation vector at the location : in, For a moment The three-dimensional translation vector at that location, For a moment The three-dimensional translation vector at that location, For matrix transpose, For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm; When attitude information is represented using quaternions, spherical linear interpolation is used to apply the attitude information: in, For a moment The interpolation unit quaternion at the location, and They are time points and The unit quaternion at the location, for and The angle between them; When attitude information is represented using Euler angles, linear interpolation is used for the attitude information; The interpolated pose information is matched with the corresponding frame's original photoacoustic signal, and the first... The interpolated pose information corresponding to the original photoacoustic signal is uniformly represented as a homogeneous transformation matrix: in, For the first The homogeneous transformation matrix of the pose information after interpolation corresponding to the original photoacoustic signal. For the first The rotation matrix obtained by converting the pose information after interpolation of the original photoacoustic signal. It is a three-dimensional zero column vector.

6. A robotic arm-assisted free-scanning virtual array photoacoustic three-dimensional imaging system, characterized in that, include: The acquisition module is used to freely scan the target area through the photoacoustic probe array, acquire multiple frames of raw photoacoustic signals, and record the six-degree-of-freedom pose information corresponding to each frame of raw photoacoustic signals in real time. A virtual multi-element sensor array module is constructed to match the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism. Based on the geometric coordinates of each element in the photoacoustic probe array in the probe coordinate system and the matched six-degree-of-freedom pose information, the spatial position of each element in the photoacoustic probe array in the world coordinate system is calculated through rigid body coordinate transformation and mapped to a unified three-dimensional reconstruction coordinate system to construct a virtual multi-element sensor array. The three-dimensional reconstruction module is used to reconstruct a three-dimensional photoacoustic image of the target area based on the spatial position of each element in the virtual multi-element sensor array and its corresponding original photoacoustic signal, thereby obtaining a three-dimensional photoacoustic image of the target area.

7. The robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging system as described in claim 6, characterized in that, Before freely scanning the target area using the photoacoustic probe array, the process further includes: mounting the photoacoustic probe array on the end effector of the robotic arm system, wherein the robotic arm system is in a passive pose recording mode during the scanning process, recording only the pose information of the end effector without performing active scanning motion.

8. The robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging system as described in claim 6, characterized in that, The six-degree-of-freedom pose information includes: the three-dimensional translation vector and attitude information of the photoacoustic probe array in the world coordinate system.

9. The robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging system as described in claim 6, characterized in that, Also includes: During photoacoustic scanning or 3D photoacoustic image reconstruction, areas with insufficient imaging information are identified based on the acquired partial 3D reconstruction results, and scanning path suggestions are generated to guide operators in adjusting the probe scanning position and orientation.

10. The robotic arm-assisted free scanning virtual array photoacoustic three-dimensional imaging system as described in claim 6, characterized in that, Matching the original photoacoustic signal of each frame with the corresponding six-degree-of-freedom pose information according to the time synchronization mechanism includes: The robotic arm system acquires the three-dimensional translation vector and attitude information of the end effector in a preset reference coordinate system. Using the laser trigger signal as a unified timing reference, the photoacoustic acquisition and the robotic arm pose recording are synchronized in time. For the The original photoacoustic signal of the frame, its trigger time is denoted as In the robotic arm pose recording sequence, with Two adjacent sampling times are denoted as follows: and And satisfy ,when When the sampling time is inconsistent with the robotic arm pose recording sequence, calculate the time interpolation coefficient. : in, For the first The triggering time of the original frame-optical signal. The first position in the robotic arm pose recording sequence Each sampling time, The first position in the robotic arm pose recording sequence Each sampling time; Linear interpolation is used to obtain the time step of the three-dimensional translation vector. Interpolation 3D translation vector at the location : in, For a moment The three-dimensional translation vector at that location, For a moment The three-dimensional translation vector at that location, For matrix transpose, For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm For a moment The x-axis coordinate of the robotic arm For a moment The y-axis coordinate of the robotic arm For a moment The z-axis coordinate of the robotic arm; When attitude information is represented using quaternions, spherical linear interpolation is used to apply the attitude information: in, For a moment The interpolation unit quaternion at the location, and They are time points and The unit quaternion at the location, for and The angle between them; When attitude information is represented using Euler angles, linear interpolation is used for the attitude information; The interpolated pose information is matched with the corresponding frame's original photoacoustic signal, and the first... The interpolated pose information corresponding to the original photoacoustic signal is uniformly represented as a homogeneous transformation matrix: in, For the first The homogeneous transformation matrix of the pose information after interpolation corresponding to the original photoacoustic signal. For the first The rotation matrix obtained by converting the pose information after interpolation of the original photoacoustic signal. It is a three-dimensional zero column vector.

Images