An automatic foot scanning ultrasonic photoacoustic imaging system and method thereof

Abstract

The application discloses a kind of foot automatic scanning ultrasonic photoacoustic imaging system and method thereof, it is related to foot underwater photoacoustic imaging technical field, comprising: in response to start scanning instruction;Control foot scanning mechanical arm moves, and the surface image of patient foot is acquired, to generate foot three-dimensional point cloud model;Based on foot three-dimensional point cloud model, the scanning path information of probe is planned;During scanning process, based on foot scanning path information, the predicted water temperature data corresponding to three-dimensional space path point position of next time is acquired in real time, and the temperature adjusting device arranged at water immersion imaging cabin controls water temperature fluctuation on foot scanning path;Control probe emits ultrasonic and laser, and receive the ultrasonic signal and photoacoustic signal of back transmission, according to ultrasonic signal and photoacoustic signal after back transmission, reconstruct the ultrasonic image and photoacoustic image corresponding to the patient foot.The method can realize the full automatic, high repeatability and safe and reliable high-precision noninvasive imaging of foot.

Description

Technical Field

[0001] This invention generally relates to the field of underwater photoacoustic imaging technology for the foot, and specifically to an automatic scanning ultrasonic photoacoustic imaging system and method for the foot. Background Technology

[0002] Currently, foot diseases (such as diabetic foot, plantar fasciitis, and peripheral vascular disease) are common and serious chronic illnesses, especially in diabetic patients, where foot complications can lead to amputation or even death. Therefore, early and accurate diagnosis is crucial for preventing the condition from worsening.

[0003] Current foot imaging primarily relies on manual operation by doctors holding ultrasound probes, which suffers from poor repeatability, unstable image quality, and difficulty in standardization. Some studies have attempted to apply robotic arms to medical imaging and utilize photoacoustic imaging in immersion environments to reflect the functional state of the foot, such as microcirculation; however, since most robotic arms are only suitable for trunk contact scanning, they cannot adapt to foot scanning conditions in immersion environments affected by complex factors such as temperature and buoyancy, thus causing distortion in ultrasound and photoacoustic images.

[0004] Therefore, there is an urgent need for a system and method that can automatically scan the foot using ultrasound and photoacoustic imaging in a water immersion environment, so as to achieve fully automatic, highly repeatable, multimodal, safe and reliable non-invasive imaging of the foot. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an automatic foot scanning ultrasonic photoacoustic imaging system and method thereof.

[0006] In a first aspect, the present invention provides an automated foot scanning ultrasonic photoacoustic imaging method, wherein the imaging method is applied to an automated foot scanning system, the automated foot scanning system comprising at least: a foot scanning robotic arm and a probe disposed on the foot scanning robotic arm; the method comprises the following steps: In response to the command to start scanning; The foot scanning robotic arm is controlled to move to the scanning starting position inside the immersion imaging chamber and acquire a surface image of the patient's foot to generate a three-dimensional point cloud model of the foot; the immersion imaging chamber contains a coupling medium and is the immersion area of ​​the patient's foot. Based on the three-dimensional point cloud model of the foot, the scanning path information of the probe is planned; the scanning path information includes at least the three-dimensional spatial path point position of the probe at each time point and the probe remains perpendicular to the surface of the patient's foot during the scanning process; During the scanning process, based on the scanning path information, the foot scanning robotic arm is controlled to drive the probe to move according to the scanning path information, and the normal attitude of the probe is adjusted in real time. The predicted water temperature data in the water immersion imaging chamber is acquired in real time, and the water temperature fluctuation in the water immersion imaging chamber is eliminated by a temperature adjustment device arranged in the water immersion imaging chamber. The probe is controlled to emit ultrasound and laser, and to receive the returned ultrasound and photoacoustic signals. Based on the returned ultrasound and photoacoustic signals, an ultrasound image and a photoacoustic image corresponding to the patient's foot are reconstructed. At the same time, the reconstructed photoacoustic image is superimposed on the grayscale background of the ultrasound image in pseudo-color form to display the anatomical structure and functional information of the foot.

[0007] According to the technical solution provided by the present invention, the foot scanning robotic arm is equipped with a visual sensor; The generation of the 3D point cloud model of the foot includes: The refractive compensation parameters corresponding to the coupling medium are obtained in advance through underwater calibration, and the imaging model of the vision sensor is refractively corrected based on the refractive compensation parameters. The foot scanning robotic arm is controlled to drive the vision sensor to acquire images from multiple viewpoints around the patient's foot according to the preset scanning angle sequence of the robotic arm, thereby obtaining local point clouds from multiple viewpoints; The iterative nearest point algorithm is used to register the local point clouds corresponding to adjacent viewpoints to generate a complete 3D point cloud model of the foot.

[0008] According to the technical solution provided by the present invention, the scanning path information further includes: the scanning step size of the probe at each time and the normal orientation that the probe should maintain; The planning of the probe's scanning path information specifically includes: The scanning step length is dynamically calculated based on the local curvature of the foot's three-dimensional point cloud model. The effective field of view of the probe is obtained, and the interval between adjacent scanning sub-paths is set according to the effective field of view, so that the overlapping area of ​​adjacent scanning sub-paths satisfies the first preset condition. Using the size of the foot 3D point cloud model as the boundary constraint, and the scanning step size and the interval between adjacent scanning sub-paths as the execution condition constraint, multiple 3D spatial path point positions are generated. Based on the positions of multiple three-dimensional spatial path points, the normal vector of the patient's foot surface at different times is calculated. Based on each surface normal vector, the normal orientation of the probe at each time is set so that the probe remains perpendicular to the patient's foot surface during the scanning process.

[0009] According to the technical solution provided by the present invention, the method further includes: The transformation matrix between the coordinate system of the visual sensor and the end coordinate system of the foot scanning robot arm is determined. At the same time, the fixed transformation relationship between the base coordinate system of the foot scanning robot arm and the coordinate system of the water immersion imaging chamber is calibrated so as to transform the foot three-dimensional point cloud model to the end coordinate system and the base coordinate system in sequence, and unify it to the coordinate system of the water immersion imaging chamber to achieve reference system alignment.

[0010] According to the technical solution provided by the present invention, the method further includes: For each target link of the foot scanning robotic arm located in the coupling medium, the equivalent disturbance torque generated by buoyancy at each rotating joint is calculated in real time based on the immersion volume of the target link; The equivalent disturbance torque is used as a feedforward term and superimposed on the joint torque command of the foot scanning robot arm to control and adjust the drive of the foot scanning robot arm.

[0011] According to the technical solution provided by the present invention, the method further includes: when controlling the movement of the foot scanning robotic arm, using a seven-segment S-shaped velocity curve to constrain the maximum end velocity and maximum acceleration of the foot scanning robotic arm.

[0012] According to the technical solution provided by the present invention, the temperature adjustment device includes: a plurality of heating pads and cooling pads arranged in an array, each element being independently controlled, forming several independent temperature control zones on the immersion imaging chamber; The real-time acquisition of predicted water temperature data within the immersion imaging chamber, and the elimination of water temperature fluctuations within the chamber via a temperature adjustment device located at the immersion imaging chamber, includes: Based on the obtained infrared thermal image distribution of the patient's foot surface and the current temperature of each temperature control zone, a pre-established local temperature field change model inside the immersion imaging chamber is input, and the predicted water temperature corresponding to each temperature control zone is obtained. If any of the predicted water temperatures exceeds the preset water temperature range, the heating pads or cooling pads of the corresponding temperature control zone will be controlled to operate until the water temperature in the corresponding temperature control zone is within the preset water temperature range.

[0013] According to the technical solution provided by the present invention, the probe is connected to the end of the foot scanning robotic arm via a torque sensor; the method further includes: real-time acquisition of feedback data from the torque sensor, wherein the feedback data is used to reflect the real-time interaction force between the probe and the end of the foot scanning robotic arm; Based on the real-time force, calculate the real-time contact force between the probe and the surface of the patient's foot; When the real-time contact force is determined to be greater than a preset safety threshold, the foot scanning robotic arm is controlled to pause its movement or retract in the reverse direction along the normal.

[0014] Secondly, the present invention provides an automatic scanning ultrasound photoacoustic imaging system for the foot, wherein the imaging device is used in conjunction with a water immersion imaging chamber, the water immersion imaging chamber having a coupling medium, and the water immersion imaging chamber being the immersion area of ​​the patient's foot; the imaging device includes: A foot scanning robotic arm, wherein the end of the foot scanning robotic arm is equipped with a probe and sensor assembly, and the probe integrates ultrasonic imaging and photoacoustic imaging functions; The foot scanning robotic arm is a high-rigidity serial robotic arm, and its rotary joint housing is made of high-strength aluminum alloy or carbon fiber composite material; each rotary joint adopts a dynamic sealing structure combining magnetohydrodynamic seal and O-ring; the sensor assembly includes at least: a vision sensor, which is a binocular structured light camera, encapsulated in a waterproof housing, and the waterproof housing is made of planar optical glass and coated with a hydrophobic coating. A temperature adjustment device, comprising: a plurality of heating pads and cooling pads arranged in an array, wherein the heating pads and cooling pads are arranged in a preset partition on the immersion imaging chamber; A control drive module is connected to the temperature adjustment device and the foot scanning robotic arm via signals, and is used to drive the foot scanning robotic arm to move and acquire water temperature data collected by the temperature adjustment device. The main control module is at least signal-connected to the probe, the sensor assembly, and the control drive module.

[0015] According to the technical solution provided by the present invention, the probe includes an ultrasonic transducer array and a light guide device; The ultrasonic transducer array is used to receive the transmitted ultrasonic and photoacoustic signals. The light guiding device includes an optical fiber bundle that extends along the axial direction of the ultrasonic transducer array housing. The optical fiber bundle is located at the center or side of the acoustic field of the ultrasonic transducer array and is used to transmit laser light into the detection field of view of the ultrasonic transducer array.

[0016] In summary, this technical solution specifically discloses an automatic foot scanning ultrasound photoacoustic imaging system and method; wherein, the method includes: responding to a scan start command; controlling the foot scanning robotic arm to move to the scanning start position within the immersion imaging chamber, and acquiring a surface image of the patient's foot to generate a three-dimensional point cloud model of the foot; the immersion imaging chamber contains a coupling medium, and the immersion imaging chamber is the area where the patient's foot is immersed; based on the three-dimensional point cloud model of the foot, planning the scanning path information of the probe; the scanning path information includes at least the three-dimensional spatial path point positions of the probe at each time point, and the probe remains perpendicular to the surface of the patient's foot during the scanning process; during the scanning process, Based on the scanning path information, the foot scanning robotic arm is controlled to move the probe according to the scanning path information, and the normal attitude of the probe is adjusted in real time. The predicted water temperature data in the immersion imaging chamber is acquired in real time, and the water temperature fluctuation in the immersion imaging chamber is eliminated by a temperature adjustment device placed in the immersion imaging chamber. The probe is controlled to emit ultrasound and laser, and receive the returned ultrasound and photoacoustic signals. Based on the returned ultrasound and photoacoustic signals, the ultrasound image and photoacoustic image corresponding to the patient's foot are reconstructed. At the same time, the reconstructed photoacoustic image is superimposed on the grayscale background of the ultrasound image in pseudo-color form to display the anatomical structure and functional information of the foot.

[0017] Beneficial effects: This invention automatically controls the movement of the robotic arm in response to the start scanning command, acquires images of the foot surface, and generates a three-dimensional point cloud model, achieving full automation of foot scanning and avoiding the problems of manual operation differences and unstable image quality. In addition, by planning the scanning path based on the three-dimensional point cloud model and ensuring that the probe is always perpendicular to the foot surface, the optimal imaging angle can be obtained in complex curved areas, thereby improving the acquisition quality of ultrasound and photoacoustic images. At the same time, the water temperature at the next step of the path is monitored in real time during the scanning process, and the water temperature fluctuation is actively controlled by the temperature adjustment device, which effectively suppresses local water temperature changes caused by robotic arm movement or foot heat radiation, and prevents sound velocity fluctuations from interfering with image reconstruction. Finally, high-fidelity, distortion-free ultrasound and photoacoustic images are reconstructed, providing a reliable and repeatable standardized imaging method for the early and accurate diagnosis of foot diseases. Attached Figure Description

[0018] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of a process for an automated foot scanning ultrasound photoacoustic imaging method.

[0019] Figure 2 This is a schematic diagram of step S200 in an automatic foot scanning ultrasound photoacoustic imaging method.

[0020] Figure 3This is a schematic diagram of step S300 in an automatic foot scanning ultrasound photoacoustic imaging method.

[0021] Figure 4 This is a schematic diagram of step S400 in an automatic foot scanning ultrasound photoacoustic imaging method.

[0022] Figure 5 This is a schematic diagram of an automatic foot scanning ultrasonic photoacoustic imaging system.

[0023] Labels in the diagram: 1. Immersion imaging chamber; 2. Probe; 3. Light guide device; 4. Visual sensor; 5. First waterproof joint motor; 6. Second waterproof joint motor; 7. Foot scanning robotic arm; 8. Coupling medium; 9. Temperature control unit; 10. Temperature adjustment device; 11. Cable bundle; 12. Control drive module; 13. Main control module; 14. Temperature sensor; 15. Side platform. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] Example 1 To make the technical solutions of the embodiments of the present invention clearer and easier to understand, the application background of the embodiments of the present invention will be introduced below.

[0027] Currently, foot diseases (such as diabetic foot, plantar fasciitis, peripheral vascular disease, etc.) are common and serious chronic diseases in clinical practice. Especially in diabetic patients, foot complications may lead to amputation or even death. Early and accurate diagnosis is crucial to prevent the condition from worsening.

[0028] Clinically, existing foot imaging methods mainly include conventional ultrasound, computed tomography (CT), magnetic resonance imaging (MRI), and photoacoustic imaging. Among these, CT angiography (CTA) and digital subtraction angiography (DSA) require the injection of contrast agents and are invasive examinations; CT also poses radiation risks. MRI examinations are time-consuming and the equipment is expensive. Although photoacoustic imaging can reflect vascular morphology, blood oxygenation, metabolic function, and other functional states, it requires a water immersion environment to ensure sound wave coupling, and most robotic arms lack waterproofing, limiting its application in photoacoustic systems.

[0029] Existing equipment suffers from the following common drawbacks: it cannot achieve fully automated scanning, being mostly static or semi-automatic, and lacks intelligent path planning capabilities; it also lacks temperature control, as human infrared thermal radiation and robotic arm movement can cause changes in the water temperature around the feet, and these temperature fluctuations affect the speed of sound, resulting in distortion of ultrasound and photoacoustic images. Although some studies have attempted to apply robotic arms to medical imaging to replace manual handheld probes, most are limited to torso contact scanning and are ill-suited to the immersion environment required for ultrasound / photoacoustic imaging. Therefore, there is a need for a dedicated foot-based automated scanning and imaging robot with underwater adaptive motion control, dynamic temperature field compensation, intelligent path planning, and multimodal compatibility to address these clinical challenges.

[0030] In view of this, the present invention proposes an automatic foot scanning ultrasound-photoacoustic imaging method, comprising: responding to a scan start command; controlling a foot scanning robotic arm to move to a scanning start position within a water immersion imaging chamber and acquiring a surface image of the patient's foot to generate a three-dimensional point cloud model of the foot; the water immersion imaging chamber contains a coupling medium and is the immersion area of ​​the patient's foot; based on the three-dimensional point cloud model of the foot, planning the scanning path information of the probe so that the probe remains perpendicular to the surface of the patient's foot during the scanning process; the scanning path information includes at least the three-dimensional spatial path point positions of the probe at each time step; during the scanning process, based on the foot scanning path information, real-time monitoring of the water temperature data corresponding to the three-dimensional spatial path point positions at the next time step, and controlling the water temperature fluctuations on the foot scanning path through a temperature adjustment device arranged in the water immersion imaging chamber; controlling the probe to emit ultrasound and laser, and receiving the returned ultrasound and photoacoustic signals, and reconstructing the ultrasound and photoacoustic images corresponding to the patient's foot based on the returned ultrasound and photoacoustic signals.

[0031] As can be seen, this invention automatically controls the movement of the robotic arm, acquires images of the foot surface, and generates a three-dimensional point cloud model in response to the start scanning command, achieving full automation of foot scanning and avoiding the problems of manual operation differences and unstable image quality. In addition, the scanning path is planned based on the three-dimensional point cloud model and the probe is always kept perpendicular to the foot surface, thereby improving the acquisition quality of ultrasound and photoacoustic images. At the same time, the water temperature at the next step of the path is monitored in real time during the scanning process, and the water temperature fluctuation is actively controlled by the temperature adjustment device, which effectively suppresses local water temperature changes caused by robotic arm movement or foot heat radiation, and prevents the interference of sound velocity fluctuations on image reconstruction. Finally, high-fidelity, distortion-free ultrasound and photoacoustic images are reconstructed, providing a reliable and repeatable standardized imaging method for the early and accurate diagnosis of foot diseases.

[0032] Please refer to the following. Figure 1 The flowchart shown in this embodiment illustrates an automatic foot scanning ultrasound photoacoustic imaging method. The execution entity of this embodiment can be an automatic foot scanning system or its main control module; no specific limitation is made here. Specifically, the steps of this invention will be further explained, and the method includes the following steps: S100, responding to the scan start command; The start scan command can be an instruction triggered by the operator to activate the automatic foot scanning system. For example, the system's main control module provides an operating interface. After completing preparations such as fixing the patient's foot and setting the water temperature, the operator can issue a start scan command to the automatic foot scanning system by clicking the start scan button, and then begin executing subsequent steps to scan the patient's foot.

[0033] S200: Control the foot scanning robotic arm to move to the scanning starting position inside the immersion imaging chamber and acquire the surface image of the patient's foot to generate a three-dimensional point cloud model of the foot; the immersion imaging chamber contains a coupling medium and is the immersion area of ​​the patient's foot. In this embodiment of the invention, the foot scanning robotic arm (also referred to as a robotic arm) is a high-rigidity, waterproof, multi-degree-of-freedom serial robotic arm. Its end effector can be equipped with a visual sensor and an ultrasound-photoacoustic composite probe (i.e., the probe in this embodiment of the invention), enabling precise movement in a water-immersion environment. The water immersion imaging chamber serves as a container holding the coupling medium, into which the patient's foot is immersed to achieve acoustic coupling of ultrasound and photoacoustic signals. The coupling medium is typically deionized water or warm water, whose acoustic impedance is close to that of human soft tissue, effectively transmitting ultrasound and photoacoustic signals and reducing signal attenuation.

[0034] Specifically, see Figure 2 The process of generating a 3D point cloud model of the foot includes: S201. Obtain the refractive compensation parameters corresponding to the coupling medium in advance through underwater calibration, and perform refractive correction on the imaging model of the vision sensor based on the refractive compensation parameters; The vision sensor uses a binocular structured light camera and is mounted on the foot-scanning robotic arm by being encapsulated in a waterproof housing. The window of the waterproof housing is made of flat optical glass and coated with a hydrophobic coating to reduce the impact of water droplets on imaging.

[0035] Because the refractive index of light differs between water and air, and the vision sensor is encapsulated in a waterproof housing, light must pass through the housing window glass and coupling medium sequentially to reach the target object. If camera parameters calibrated in air are used directly, refraction will cause depth measurement errors. Therefore, this embodiment of the invention introduces refraction compensation parameters to correct the refraction of the vision sensor's imaging model.

[0036] For example, the refraction compensation parameters can be obtained as follows: A standard-sized grid calibration plate is placed in a water immersion imaging chamber, maintaining the water temperature consistent with that during scanning (e.g., 32°C). A foot-mounted scanning robotic arm drives a binocular structured light camera to photograph the calibration plate from different distances and angles, recording the image coordinates of known feature points on the calibration plate. Subsequently, a multi-layer refraction imaging model (air-glass-water) is established based on Snell's law. The equivalent refraction plane parameters (such as glass thickness, refractive index, and equivalent plane position) in the model are solved using nonlinear least squares optimization and used as refraction compensation parameters. In this way, during subsequent actual scanning, for each pixel, the actual propagation direction of light underwater is calculated based on the refraction compensation parameters, and the true three-dimensional coordinates of the object surface are deduced, thereby eliminating refraction distortion.

[0037] S202. Control the foot scanning robotic arm to drive the vision sensor to collect images from multiple viewpoints around the patient's foot according to the preset scanning angle sequence of the robotic arm, and obtain local point clouds from multiple viewpoints. Specifically, based on the single-view effective field of view of the binocular structured light camera (approximately 150 mm horizontally, 120 mm vertically, and a working distance of 200-350 mm), combined with the three-dimensional dimensions of the foot (generally 200-280 mm long, 80-120 mm wide, and 60-100 mm high), the system will pre-generate a scanning view sequence using spherical uniform sampling or an adaptive sampling method based on model curvature. The scanning view sequence includes multiple poses, and each pose includes the spatial position and orientation of the visual sensor, ensuring that all views combined can completely cover the surface of the foot, and that the overlap area between adjacent views is not less than 30%, to ensure that subsequent point cloud registration has a sufficient feature matching basis. Among them, multiple poses usually include directions such as sole, instep, medial side, lateral side, heel, and toes, totaling 8-16 viewpoints.

[0038] Due to the complex shape of the foot surface (including arch depressions, heel protrusions, and gaps between toes), a single viewpoint cannot simultaneously capture all areas. Therefore, a robotic arm is needed to drive a vision sensor to sequentially capture images from multiple viewpoints, obtaining a local point cloud from each viewpoint. Specifically, after obtaining the generated scanning viewpoint sequence, the foot scanning robotic arm drives a binocular structured light camera to sequentially acquire images around the patient's foot according to the preset scanning viewpoint sequence. At each viewpoint, a structured light projector projects an coded pattern, and the left and right cameras of the binocular structured light camera simultaneously acquire images. Based on the principle of binocular parallax and structured light decoding, the 3D point corresponding to each pixel is calculated, generating the local point cloud for each viewpoint.

[0039] S203. The iterative nearest point algorithm is used to register the local point clouds corresponding to adjacent viewpoints to generate a complete 3D point cloud model of the foot.

[0040] The Iterative Closest Point (ICP) algorithm is a classic point cloud registration algorithm. It finds the optimal rotation matrix and translation vector between two point clouds through repeated iterations, so that they are aligned as much as possible. Registration refers to transforming local point clouds from different viewpoints (in different coordinate systems) to the same global coordinate system (such as the coordinate system of a water immersion imaging cabin), so that they are stitched together into a complete and continuous point cloud model.

[0041] Specifically, after obtaining the local point clouds from each viewpoint, this embodiment of the invention first extracts the Fast Point Feature Histogram (FPFH) of each local point cloud. Initial transformation matrices (coarse alignment) between adjacent viewpoints are obtained through feature matching to avoid ICP getting trapped in local optima. Subsequently, the ICP algorithm is used for iterative optimization: for two local point clouds P and Q, the algorithm continuously searches for the closest point in P to the nearest point in Q, calculates the rigid body transformation (rotation R, translation t) that minimizes the sum of squared distances between point pairs, and then applies the transformation to P, repeating until the error is less than a threshold or the maximum number of iterations is reached. Finally, since pairwise registration of adjacent viewpoints will produce cumulative errors (such as gaps between the first and last viewpoints), this invention also eliminates cumulative errors through global graph optimization. Then, using a pre-defined foot placement range, background noise points outside the range are removed. Finally, a complete 3D point cloud model of the foot surface is generated, ensuring a point cloud density of at least 10 points per square millimeter.

[0042] For example, the global graph optimization method is as follows: by constructing a graph with point clouds from each viewpoint as nodes and viewpoint edges as registration relationships, the global pose of each viewpoint point cloud (i.e., the rotation matrix and translation vector of each point cloud in the global coordinate system) is adjusted using an optimization algorithm to minimize the sum of squared distances between all registered point pairs.

[0043] To improve the motion accuracy and repeatability of the foot scanning robotic arm during movement and to avoid positioning deviations or collision risks caused by inconsistencies in coordinate systems, in a preferred embodiment, the present invention further includes the following steps: The system pre-determines the transformation matrix between the coordinate system of the vision sensor and the end-effector coordinate system of the foot scanning robotic arm through hand-eye calibration, and simultaneously calibrates the fixed transformation relationship between the base coordinate system of the foot scanning robotic arm and the coordinate system of the immersion imaging chamber, so as to sequentially transform the three-dimensional point cloud model of the foot to the end-effector coordinate system and the base coordinate system, and unify it to the coordinate system of the immersion imaging chamber, thereby achieving reference system alignment. The specific process will not be elaborated here; refer to the current coordinate transformation theory. Under the reference system alignment, the foot scanning robotic arm can be controlled to drive the probe to move according to the scanning path information planned by the system, and the normal attitude of the probe can be adjusted in real time.

[0044] Therefore, this invention achieves precise registration of the foot's 3D point cloud model and the robotic arm's motion space within the same reference system by uniformly calibrating and transforming the visual sensor coordinate system, the robotic arm's end effector coordinate system, the robotic arm's base coordinate system, and the immersion imaging chamber coordinate system. Based on this, the robotic arm can precisely drive the probe along the foot surface and maintain perpendicularity according to the planned 3D spatial path point positions and the probe's normal orientation. This improves the tracking accuracy and repeatability of the planned scanning path, avoids positioning deviations or collision risks caused by inconsistent coordinate systems, and ensures that the acquisition positions of ultrasound and photoacoustic images are completely consistent with the preset path, providing a reliable spatial reference for subsequent high-quality image reconstruction.

[0045] S300. Based on the three-dimensional point cloud model of the foot, plan the scanning path information of the probe; the scanning path information includes at least the three-dimensional spatial path point position of the probe at each time point and the probe remains perpendicular to the surface of the patient's foot during the scanning process; The scanning path information is a set of position and orientation data corresponding to the probe when performing the scanning task. It includes at least a series of three-dimensional spatial path point positions as the planned scanning path. The scanning path information may also include parameters such as the scanning step size of the probe at each time, the normal orientation that the probe should maintain, and the movement speed of the probe (foot scanning robotic arm). These are not specifically limited here. The three-dimensional spatial path point positions can be understood as a sequence of spatial coordinate points that the probe center needs to pass through in sequence. These points are usually located at a fixed vertical distance (e.g., 0~5 mm) above the foot surface to guide the probe to traverse the entire scanning area.

[0046] It should be noted that the ultrasonic / photoacoustic signal is perpendicularly incident when the imaging surface of the probe coincides with the normal vector of the tangent plane at that point on the foot surface, resulting in the highest echo reception efficiency and the best image quality. Therefore, the spatial orientation of the probe needs to be controlled at each path point. This spatial orientation can be determined by two degrees of freedom: the pitch angle (rotation around the horizontal axis) and the rotation angle (rotation around the vertical axis) to ensure that the imaging surface is aligned with the surface normal vector.

[0047] Further, see Figure 3 The scanning path information for the probe in step S300 specifically includes the following steps: S301. Dynamically calculate the scanning step length based on the local curvature of the three-dimensional point cloud model of the foot; Since the surface of the foot is not flat, there are areas with drastic curvature changes such as the arch (concave), heel (convex), and toes (complex curved surfaces). If a fixed step size is used for sampling, details are easily lost in areas with large curvature, while too much redundant data is collected in areas with small curvature, reducing scanning efficiency. Therefore, this invention determines the corresponding scanning step size by using the local curvature of the three-dimensional point cloud model of the foot.

[0048] Specifically, in this step, the system first performs curvature analysis on the 3D point cloud model of the foot. In this embodiment of the invention, for each point, a local curvature value (such as Gaussian curvature or average curvature) is obtained by fitting a quadratic surface to a set of neighboring points or calculating the rate of change of the normal vector. Then, a mapping function is established based on a preset step size range (e.g., 0.5 mm to 2 mm) and a curvature threshold: the larger the curvature, the smaller the step size; the smaller the curvature, the larger the step size. For example, when the curvature is higher than the upper threshold, the step size is 0.5 mm; when the curvature is lower than the lower threshold, the step size is 2 mm. Linear interpolation is used in the intermediate region to obtain the dynamic scanning step size, which is stored in the scanning path information.

[0049] S302. Obtain the effective field of view of the probe, and set the interval between adjacent scanning sub-paths according to the effective field of view, so that the overlapping area of ​​adjacent scanning sub-paths meets the first preset condition. The probe also has an imaging width (i.e., effective field of view width, for example, 10 mm) perpendicular to the scanning direction. If the interval between two adjacent scanning sub-paths is too large, some areas of the foot surface will not be covered; if the interval is too small, a large amount of overlap will occur, reducing efficiency. Therefore, in this step, the system will read the effective field of view width parameter of the probe and set the overlap rate of the adjacent scanning sub-paths to meet the first preset condition. The first preset condition is set, for example, that the overlap rate of the adjacent scanning sub-paths should be greater than 30%, that is, at least 30% overlap area should be retained between adjacent paths to ensure the continuity and integrity of image stitching. The specific value of the overlap rate can be adjusted according to the imaging requirements and the positioning accuracy of the robotic arm (e.g., 30%~50%), and no special limitation is imposed.

[0050] S303. Using the size of the foot 3D point cloud model as the boundary constraint and the scanning step size and the interval between adjacent scanning sub-paths as the execution condition constraint, generate multiple 3D spatial path point positions. After determining the scanning step size and the interval between adjacent scanning sub-paths, it is necessary to actually calculate all the three-dimensional spatial path points that the probe needs to traverse during the scanning task, as the final complete planned scanning path. The specific method used is as follows: First, based on the bounding box (minimum / maximum x, y, z coordinates) of the foot's three-dimensional point cloud model, determine the main scanning direction (e.g., the foot length direction, usually the x-axis) and the secondary direction (the foot width direction, usually the y-axis). Then, in the secondary direction, generate a series of parallel scanning planes (or curves) at the set intervals of the scanning sub-paths. For each scanning sub-path, sample the point cloud surface along the main direction with a dynamic step size to obtain the three-dimensional point coordinates on that path. At the same time, using the size of the point cloud model as a boundary constraint, limit the position of the three-dimensional spatial path points of the robotic arm and probe, ensuring no interference between the robotic arm and probe and non-scanning areas such as the immersion imaging chamber and the patient's foot, avoiding collision risks, and finally generating a three-dimensional path point array composed of the three-dimensional spatial path point positions of the probe at each time point, which is the planned scanning path.

[0051] S304. Based on the positions of multiple three-dimensional spatial path points, calculate the normal vector of the patient's foot surface at different times, and set the normal orientation that the probe should maintain at each time according to each surface normal vector.

[0052] Each 3D path point corresponds to a specific location on the foot surface. To achieve the optimal signal-to-noise ratio when the probe transmits / receives signals at that point, the probe's imaging plane needs to be perpendicular to the foot surface. Therefore, the surface normal vector at that point must be calculated, and the probe's spatial orientation determined accordingly. A common method for calculating the normal vector is as follows: For each path point, take several point cloud points in its neighborhood (e.g., all points within a radius of 1-2 mm), and fit a plane using the least squares method. The normal direction of this plane is the surface normal vector. Alternatively, principal component analysis (PCA) can be used to obtain the eigenvector corresponding to the smallest eigenvalue among the three principal directions of the neighborhood point set as the normal vector. After obtaining the normal vector, the inverse kinematics of the robotic arm is used to convert the normal vector into pitch and rotation angle commands for the probe joints, which are then stored in the scanning path information.

[0053] Finally, based on the above scanning path information and the alignment of the reference frame, the foot scanning robotic arm module drives the foot scanning robotic arm to move and the joint motors on it to rotate according to the information in the scanning path information. This corrects the position and posture of the imaging probe in real time, ensuring that the probe can always maintain a posture perpendicular to the surface of the imaged object during the scanning process. This ensures that relatively high image quality can be obtained, providing more accurate image results for subsequent analysis, research and pathological information.

[0054] In a preferred embodiment, considering that when the robotic arm is partially submerged in water, the underwater links are subjected to buoyancy and fluid resistance, these external forces will generate equivalent disturbance torque at the rotary joints. If the rotary joint structure has insufficient rigidity or a large transmission clearance, the disturbance torque will be transmitted to the end effector of the robotic arm through joint elastic deformation and transmission backlash, causing deviation in the end effector's pose. Therefore, this invention increases the bending stiffness of the joint by employing a high-rigidity joint structure; at the same time, it uses a pre-tensioned transmission mechanism to control the transmission clearance within ±0.01°. Under these conditions, most of the disturbance torque generated by buoyancy and resistance on the underwater section is borne by the joint structure itself, reducing the residual disturbance amplitude transmitted to the end effector, thereby effectively ensuring the consistency of motion between the above-water and underwater sections, and maintaining the end effector trajectory accuracy within ±0.1mm even when the submersion depth changes.

[0055] Meanwhile, to address the aforementioned issues, this method further includes: for each target link of the foot scanning robotic arm located within the coupling medium, calculating in real time the equivalent disturbance torque generated by buoyancy at each rotary joint based on the immersion volume of the target link; and using the equivalent disturbance torque as a feedforward term, superimposing it into the joint torque command of the foot scanning robotic arm to control and adjust the drive of the foot scanning robotic arm.

[0056] Specifically, for the i-th target link of the robotic arm, its submerged volume in water is... (The density of water varies with joint angle q) The acceleration due to gravity is Then the buoyancy vector The size is as follows (1): (1); The point of application of buoyancy is located at the geometric center of the link, and the equivalent disturbance torque it generates at the rotational joint of the corresponding target link is... The following formula (2): (2); In the formula, Let be the Jacobian matrix of the point of application of buoyancy relative to joint i.

[0057] The control system adds the equivalent disturbance torque as a feedforward term to the rotary joint torque command. Specifically, the formula is as follows (3): (3); In the formula, For feedback control quantities (such as PID output). The equivalent disturbance torque serves as the feedforward compensation amount. Therefore, this feedforward compensation allows the rotary joint to preemptively counteract the buoyancy effect during non-horizontal motion, preventing end-effector attitude drift.

[0058] Furthermore, considering that water resistance is proportional to the square of velocity, in order to reduce the disturbance of the water body around the foot caused by the movement of the robotic arm, the system adopts a seven-segment S-shaped velocity curve in trajectory planning and limits the maximum end velocity; therefore, in this embodiment of the invention, the method further includes: when controlling the movement of the foot scanning robotic arm, using a seven-segment S-shaped velocity curve to constrain the maximum end velocity and maximum acceleration of the foot scanning robotic arm.

[0059] First, the characteristic that water resistance is proportional to the square of the velocity is specifically shown in the following formula (4): Formula (4); In the formula, The drag coefficient, For the area facing the flow, The velocity of the end effector of the foot-scanning robotic arm relative to the water; The density of the coupling medium (water); This represents resistance to flow.

[0060] Specifically, the acceleration function a(t) of the seven-segment S-shaped velocity curve is composed of three segments of uniform acceleration, one segment of uniform velocity, and three segments of uniform deceleration. Therefore, the velocity function v(t) and the displacement function s(t) are obtained by integrating the acceleration as follows (5): (5); In the formula, Let be the integral variable, and represent time. Using this S-curve, the acceleration at the end of the foot-scanning robotic arm is limited to a finite range, and fluid resistance... rate of change This also changes continuously, avoiding the impact and disturbance to the water body caused by sudden changes in resistance. In this embodiment of the invention, the maximum end-effector velocity of the foot-scanning robotic arm is ≤0.01 m / s and the maximum acceleration is ≤0.05 m / s². 2 When the foot scanning robotic arm's movement causes a peak velocity disturbance of the water around the foot of ≤0.002 m / s, the interference with imaging is negligible. Therefore, the above values ​​can be used to constrain the maximum end velocity and maximum acceleration of the foot scanning robotic arm.

[0061] S400: During the scanning process, based on the scanning path information, the foot scanning robotic arm is controlled to drive the probe to move according to the scanning path information, and the normal attitude of the probe is adjusted in real time. The predicted water temperature data in the immersion imaging chamber is also acquired in real time, and the water temperature fluctuation in the immersion imaging chamber is eliminated by the temperature adjustment device arranged in the immersion imaging chamber. During the scanning process of an automated foot scanning system, the movement of the foot scanning robotic arm and the thermal radiation from the foot itself can alter the local water temperature. These temperature fluctuations can cause changes in the speed of sound, leading to distortion in both ultrasonic and photoacoustic images. Therefore, this invention addresses this issue by predicting the water temperature change trend at a specific path point before the foot scanning robotic arm reaches that point to counteract the temperature fluctuations. Specifically, a temperature adjustment device (an array of heating pads and cooling plates) arranged within the immersion imaging chamber actively compensates for the local temperature, ensuring that the water temperature along the planned scanning path remains stable (temperature error ≤ ±0.5°C).

[0062] Specifically, the automatic foot scanning system of the present invention includes a temperature adjustment device, which includes multiple heating pads and cooling pads arranged in an array. Each element is independently controlled, forming several independent temperature control zones on the immersion imaging chamber. In order to cooperate with the temperature adjustment device, a temperature sensor array (such as a 2×2 or 3×3 thermistor array) and an infrared spectrum collector are also installed at the bottom of the probe of the foot scanning robotic arm facing the foot to monitor the local water temperature and the changes in water surface heat distribution caused by the foot in real time. The sampling frequency is not less than 20Hz.

[0063] Further, see Figure 4 The above step S400 includes the following steps: S401. Based on the obtained infrared thermal image distribution of the patient's foot surface and the current temperature of each temperature control zone, a pre-established local temperature field change model inside the immersion imaging chamber is input to obtain the predicted water temperature corresponding to each temperature control zone. S402. If any predicted water temperature exceeds the preset water temperature range, control the operation of the heating pad or cooling pad of the corresponding temperature control zone until the water temperature in the corresponding temperature control zone is within the preset water temperature range. The local temperature field variation model is a mathematical model describing the spatial and temporal changes in water temperature within the imaging chamber. This scheme employs a hybrid lumped parameter and finite difference model: the immersion imaging chamber is divided into several spatial grids (e.g., 10mm × 10mm × 10mm), and the temperature change of each grid is jointly determined by thermal conduction (heat exchange between adjacent grids), thermal convection (forced convection caused by the movement of the foot scanning robotic arm), and thermal radiation (infrared radiation from the foot). The inputs to the local temperature field variation model include: the thermal disturbance intensity calculated from the positions and speeds of all three-dimensional spatial path points of the foot scanning robotic arm obtained from the scanning path information for the next 2-5 seconds; the current water temperature data of each temperature control zone; and the infrared thermal image distribution of the foot surface. Finally, by using the fact that the convective heat transfer coefficient is proportional to the square of the foot scanning robotic arm's movement speed (calibrated through offline CFD simulation), the local temperature field variation model can predict the temperature change of each grid at future time points in real time.

[0064] It should be noted that the infrared thermal image distribution of the foot surface is obtained by scanning the patient's foot with an infrared image acquisition device to obtain the temperature distribution at various points on the foot surface. This distribution is used as a boundary condition input into the temperature field model to calculate the spatial distribution of heat radiated from the foot to the water. Current water temperature data for each temperature-controlled zone: The temperature-controlled zones are determined by the arrangement of multiple heating pads and cooling plates in an array within the temperature adjustment device. Each zone has one or more temperature sensors. The system reads the temperature values ​​of all sensors at a frequency of no less than 20 Hz, which serves as the initial state and feedback quantity for the local temperature field change model.

[0065] Specifically, for each future three-dimensional spatial path point position, based on the movement speed of the foot scanning robotic arm... Calculate the intensity of thermal disturbance For details, please refer to the following formula (6): (6); In the formula, The velocity of the end effector of the foot-scanning robotic arm relative to the water. For time; is the thermal diffusion time constant.

[0066] Based on formula (6), the predicted water temperature of each spatial grid in the immersion imaging chamber is obtained by accumulating the current water temperature of each temperature control zone and the spatial distribution data of heat radiated from the feet to the water. Grids with predicted water temperatures exceeding the preset water temperature range (e.g., 32±0.5℃) are marked as areas to be pre-adjusted. For example, if the predicted water temperature is 32.8℃, which exceeds the preset water temperature range, the spatial grid with a predicted water temperature of 32.8℃ is marked as an area to be pre-adjusted, and the overall water temperature of the temperature control zone where the area to be pre-adjusted is located is calculated for temperature adjustment (generally, there will not be a single point with excessive temperature rise, so each adjustment is basically done on a temperature control zone basis).

[0067] It should be noted that for some areas where the foot scanning robotic arm will not be disturbed, the intensity of thermal disturbance does not need to be considered. At the same time, if the area to be pre-adjusted is found to be in the scanning path information, the present invention also eliminates water temperature fluctuations on the probe scanning path in advance by calculating the movement speed of the foot scanning robotic arm.

[0068] Specifically, in control applications, each spatial grid of the region to be pre-tuned corresponds to a temperature control zone. The system maps the pre-tuned region to the specific temperature control zone and calculates the target pre-tuned temperature based on the predicted disturbance intensity. For details, please refer to the following formula (7): (7); in, This represents the predicted disturbance temperature rise corresponding to the i-th temperature control zone, and its value can be positive or negative. This is a compensation coefficient, ranging from 0.8 to 1.2, used to correct the deviation between model predictions and actual thermal disturbances; The target temperature is set to, for example, 32°C, and this data matches the preset water temperature range setting. The preset target temperature is the dynamic target temperature set by the feedforward.

[0069] Based on the estimated time for the robotic arm to reach the temperature-controlled zone. The temperature control unit activates the heating pads or cooling pads of the temperature control zone in advance, with the activation time being... ,in, The preset advance time is 3-5 seconds, which is pre-calibrated based on the system's temperature control response capability under the most unfavorable operating conditions, and is sufficient to adjust the water temperature from the maximum deviation to the target value. After startup, the power is regulated by a PID control law, as shown in the following formula (8): (8); Let be the measured water temperature of the i-th temperature-controlled zone. The PID controller uses... In order to track the target, due to The water temperature is pre-increased or decreased based on predicted disturbances, effectively introducing feedforward compensation. This ensures that the actual water temperature stabilizes near the target temperature before the robotic arm arrives, thus offsetting localized water temperature disturbances caused by robotic arm movement and foot thermal radiation. This provides a stable water temperature environment for ultrasonic and photoacoustic imaging. The aforementioned prediction and adjustment process is updated in real-time at a frequency of 10-20 Hz, ensuring the uniformity of the water temperature field throughout the dynamic scanning process. This keeps the temperature error within ±0.5°C, reducing temperature variation interference caused by robotic arm movement and foot infrared thermal radiation, and guaranteeing that image distortion due to water temperature changes will not occur during image reconstruction.

[0070] The S500 controls the probe to emit ultrasound and laser, and receives the returned ultrasound and photoacoustic signals. Based on the returned ultrasound and photoacoustic signals, it reconstructs the ultrasound and photoacoustic images corresponding to the patient's foot. At the same time, the reconstructed photoacoustic image is superimposed on the grayscale background of the ultrasound image in pseudo-color form to display the anatomical structure and functional information of the foot.

[0071] The foot scanning robotic arm moves the probe along the path planned in the scanning path information. Under the premise of maintaining a stable water temperature, it sequentially or alternately acquires ultrasonic and photoacoustic signals, and reconstructs multimodal images reflecting the structure and function of the foot through signal processing algorithms. Specifically, the main control module controls the drive module to trigger imaging acquisition at equal spatial intervals (i.e., every scanning step) during the movement of the robotic arm. The trigger signal simultaneously activates the ultrasonic emission circuit and the laser (which can operate separately or alternately). To ensure a one-to-one correspondence between the spatial positions of the ultrasonic and photoacoustic images, the system records the precise pose of the robotic arm's end effector at each trigger moment and associates it with the acquired signal frames.

[0072] The ultrasound imaging process includes: an ultrasound transducer array inside the probe emits short pulses to the foot tissue and receives reflected echoes from the tissue interface. The echo signals from each channel are amplified with low noise and converted from analog to digital before being sent to the host computer for processing, forming multi-channel raw echo data. The host computer performs beamforming, envelope detection, logarithmic compression, and grayscale mapping on the multi-channel data to reconstruct ultrasound images, which are used to display the soft tissue layers, tendons, fascia, blood vessels, and other structures of the foot.

[0073] Next, the photoacoustic imaging process includes: irradiating the foot surface with a pulsed laser through an optical fiber guide device, while chromophores in the tissue (such as hemoglobin) absorb the laser energy, causing an instantaneous temperature rise and thermoelastic expansion, which excites a broadband ultrasonic signal (photoacoustic signal). The same ultrasonic transducer array receives the photoacoustic signal. Finally, the received raw photoacoustic signal is processed, such as by filtering and deconvolution. Finally, a two-dimensional or three-dimensional photoacoustic image of light absorption distribution and tissue morphology is generated by using time-delay superposition, back projection algorithm, or model-based reconstruction methods.

[0074] Based on the aforementioned ultrasound and photoacoustic imaging, the system generates one ultrasound image and one photoacoustic image for each scanning location. Since both are acquired using the same probe and in the same coordinate system, they can be directly superimposed and fused to form a composite image of structure and function. For example, the ultrasound image displays the morphology of the plantar fascia, while the photoacoustic image shows the distribution of surrounding microvessels and blood oxygen levels. Fusion of these two images helps in the accurate diagnosis of areas of inflammation or ischemia. Simultaneously, because the probe acquires data point-by-point or line-by-line along the scanning path, a series of two-dimensional image frames are obtained (each three-dimensional spatial path point corresponds to one or more frames). The main control module stitches these two-dimensional images into a complete three-dimensional ultrasound / photoacoustic image dataset according to the spatial position corresponding to each frame, achieving three-dimensional visualization of the overall foot structure.

[0075] In a preferred embodiment, the probe is connected to the end of the foot scanning robotic arm via a torque sensor; the method further includes: real-time acquisition of feedback data from the torque sensor, the feedback data being used to reflect the real-time force between the probe and the end of the foot scanning robotic arm; Calculate the real-time contact force between the probe and the patient's foot surface based on the real-time force. When the real-time contact force is determined to be greater than the preset safety threshold, the foot scanning robotic arm is controlled to pause its movement or retract in the reverse direction along the normal.

[0076] Specifically, during the scanning process, the foot scanning robotic arm moves the probe along a planned path. A torque sensor installed between the probe and the end effector of the robotic arm continuously collects torque data at a fixed sampling frequency. This torque data serves as feedback data, reflecting the real-time force between the probe and the end effector of the foot scanning robotic arm. Because the probe itself has a certain weight and is subject to buoyancy in water, and also experiences inertial forces during acceleration and deceleration, the torque sensor readings cannot be directly taken as the contact force between the probe and the foot surface. The system performs dynamic compensation; for example, based on the probe's mass, the buoyancy it experiences while submerged in water, and the acceleration generated by the robotic arm's movement, the system subtracts the influence of the probe's own weight, buoyancy, and inertial forces from the sensor readings to obtain the true contact force between the probe and the foot surface. In particular, the system extracts the normal contact force component perpendicular to the foot surface (real-time contact force), as this component is crucial information for patient foot safety.

[0077] After obtaining the real-time contact force, it is compared with a preset safety threshold (e.g., 5N). When the real-time normal contact force exceeds the preset safety threshold, it indicates excessive pressure between the probe and the foot surface, potentially causing patient discomfort. In this case, the control drive module can send an emergency stop command to the foot scanning robotic arm, halting all joint motors and keeping the probe in its current position. Alternatively, the system can calculate the opposite direction of the foot surface normal vector (i.e., the direction away from the foot surface) based on the current probe posture, and then control the foot scanning robotic arm to retract a preset distance (e.g., 2-5 mm) along that direction until the real-time contact force drops below the safety threshold. This step ensures absolute patient safety during the fully automated scanning process, improving the reliability of clinical applications and patient comfort. It also protects the imaging probe from mechanical damage, extending the equipment's lifespan.

[0078] In summary, the automatic foot scanning ultrasonic photoacoustic imaging system and method proposed in this invention firstly automatically acquires images of the foot surface and generates a high-density three-dimensional point cloud model using a robotic arm equipped with a binocular structured light camera. Combined with an intelligent path planning algorithm that dynamically adjusts step length and path interval based on curvature, the probe remains perpendicular to the foot surface throughout the scanning process. This completely replaces the irregular operation of traditional manual handheld probes, effectively improving the consistency and repeatability of imaging. This provides a reliable means for standardized screening and long-term follow-up of diseases such as diabetic foot and peripheral vascular disease. Secondly, considering the water immersion environment necessary for photoacoustic imaging, this solution designs a high-rigidity, low-transmission-backlash waterproof robotic arm. It employs magnetohydrodynamic sealing and buoyancy feedforward compensation, along with a seven-segment S-shaped velocity curve to limit end-effector velocity and jerk, ensuring both the accuracy of underwater movement and effectively suppressing the impact and disturbance of the water body caused by the robotic arm's movement. Furthermore, this solution constructs an active temperature control system based on a local temperature field change model. It uses a moving temperature sensor at the bottom of the probe and a fixed sensor inside the chamber to sense the water temperature. Utilizing a zone-independent temperature adjustment device, it predicts and actively adjusts the water temperature in the path the robotic arm will traverse, keeping temperature fluctuations within a controlled range. This avoids image distortion caused by changes in sound velocity, achieving geometrically accurate ultrasound and photoacoustic images without additional sound velocity correction. Finally, the system integrates multiple safety mechanisms, including water temperature over-limit protection, collision detection and emergency retreat, and foot fixation pressure monitoring, ensuring reliability for clinical use. In summary, this invention achieves fully automated, high-precision, high-fidelity, and high-safety foot ultrasound and photoacoustic imaging, demonstrating advantages in motion control, path planning, temperature compensation, and multimodal imaging consistency, providing an innovative solution for the early and accurate diagnosis of foot diseases.

[0079] The above text combined Figures 1-4 The automatic foot scanning ultrasonic photoacoustic imaging method provided in the embodiments of the present invention has been described in detail below, and will be discussed in conjunction with the accompanying drawings. Figure 5 An automatic foot scanning ultrasonic photoacoustic imaging system provided by an embodiment of the present invention will be introduced.

[0080] Specifically, in combination Figure 5 This automatic foot scanning ultrasound photoacoustic imaging system is used in conjunction with a water immersion imaging chamber 1. The water immersion imaging chamber 1 contains a coupling medium 8, and the water immersion imaging chamber 1 is the immersion area of ​​the patient's foot. The imaging system includes: Foot scanning robotic arm 7, with probe 2 and sensor assembly at the end of the foot scanning robotic arm, probe 2 integrating ultrasonic imaging and photoacoustic imaging functions; Temperature adjustment device 10 includes: multiple heating pads and cooling pads arranged in an array, and the heating pads and cooling pads are arranged in preset zones on the water immersion imaging chamber. The control drive module 12 is connected to the temperature adjustment device 10 and the foot scanning robotic arm 7 by signal, and is used to drive the foot scanning robotic arm 7 to move and acquire the water temperature data collected by the temperature adjustment device 10. The main control module 13 is at least connected to the probe 2, the sensor assembly, and the control drive module 12.

[0081] The automatic foot scanning ultrasound-photoacoustic imaging system provided by this invention is used in conjunction with a water immersion imaging chamber 1. The water immersion imaging chamber contains a coupling medium (usually deionized water or physiological saline, with the temperature controlled at approximately 32°C). The patient immerses their foot in the chamber, and the coupling medium provides a low-attenuation propagation path for ultrasound and photoacoustic signals. The imaging system mainly includes: a foot scanning robotic arm 7, a temperature adjustment device 10, a control and drive module 12, and a main control module 13. Specifically, the foot scanning robotic arm 7 is a high-rigidity, waterproof, tandem robotic arm with a probe 2 and sensor assembly installed at its end. The probe 2 is an ultrasonic and photoacoustic composite probe, integrating an ultrasonic transducer array and a fiber optic light guide. It can both emit ultrasonic waves and receive echoes, and transmit laser pulses via fiber optics to excite photoacoustic signals, achieving dual-modal imaging. Simultaneously, the probe can use a planar array transducer so that its imaging surface normal aligns with the foot surface normal. The sensor assembly includes a vision sensor 4, a temperature sensor array 14, and an infrared image acquisition unit, used to acquire three-dimensional information of the foot surface and local water temperature data. The foot scanning robotic arm 7 is driven by a control drive module, enabling precise movement in three-dimensional space, guiding the probe 2 along a planned scanning path across the foot surface.

[0082] The temperature adjustment device 10 consists of multiple heating pads and thermoelectric coolers (TECs) arranged in an array and installed in preset zones on the bottom and side walls of the immersion imaging chamber. The heating / cooling elements within each zone can be independently controlled, forming several independent temperature-controlled zones (e.g., a 6×4 grid, with each zone approximately 50 mm × 50 mm in size). This device is used to actively regulate the local water temperature within the imaging chamber, suppressing water temperature fluctuations caused by robotic arm movement and foot heat radiation, and maintaining the water temperature stable within a target range of ±0.5°C.

[0083] The control drive module 12 is connected to the foot scanning robotic arm 7 and the temperature adjustment device 10 via signals. On one hand, it receives scanning path instructions sent by the main control module 13, drives the joint motors of the foot scanning robotic arm 7 to move along the planned trajectory, and provides real-time feedback on the end-effector pose of the robotic arm. On the other hand, it integrates a temperature control unit 9 and an infrared unit. The temperature control unit 9 collects real-time water temperature data of each temperature control zone and in front of the probe 2 through a temperature sensor. The infrared unit acquires the infrared thermal image distribution of the patient's foot surface through an infrared spectrum collector and transmits it to the main control module 13. At the same time, it adjusts the power of the heating or cooling elements in each temperature control zone according to the temperature control instructions.

[0084] The main control module 13 is the core of the entire system's control and processing. It is connected to the probe 2, vision sensor 4, control drive module 12, and temperature control unit 9 via cable bundle 11. The main control module 13 is responsible for at least the following: receiving the start scan command, controlling the vision sensor 4 to acquire foot images, and generating a three-dimensional point cloud model of the foot; automatically planning the scanning path and probe posture based on the three-dimensional point cloud model of the foot; coordinating the timing of robotic arm movement, temperature adjustment, and image acquisition; and receiving the ultrasonic and photoacoustic signals transmitted back from the probe 2, reconstructing the ultrasonic and photoacoustic images, and displaying them.

[0085] Accordingly, through the coordinated operation of the above four parts, this device achieves fully automated, highly repeatable, and high-image-quality foot ultrasound and photoacoustic imaging.

[0086] In a preferred embodiment, the sensor assembly includes at least: a vision sensor, which is a binocular structured light camera, encapsulated in a waterproof housing, the waterproof housing being made of planar optical glass and coated with a hydrophobic coating.

[0087] In this device, the vision sensor employs a binocular structured light camera, consisting of two industrial cameras (left and right) and a structured light projector. This camera projects an coded grating pattern and calculates the three-dimensional coordinates of the foot surface using the binocular parallax principle. For proper operation within a water-immersion imaging chamber, the entire camera is encapsulated in a waterproof housing made of corrosion-resistant 316L stainless steel or engineering plastic.

[0088] The optical windows of the waterproof housing are made of flat optical glass with high light transmittance (≥95% transmittance in the visible and near-infrared bands) and good mechanical strength, capable of withstanding the water pressure inside the chamber. It should be noted that the outer surface of the glass is coated with a hydrophobic coating (e.g., fluorosilane or nano-silica coating). This coating causes water droplets to form spherical shapes on the glass surface, making them easy to roll off, thus reducing the interference of water droplet adhesion on projection and imaging.

[0089] In a preferred embodiment, the foot scanning robotic arm is a high-rigidity serial robotic arm, and its rotary joint housing is made of high-strength aluminum alloy or carbon fiber composite material; each rotary joint adopts a dynamic sealing structure combining magnetohydrodynamic seal and O-ring.

[0090] The shell material for each rotating joint is made of high-strength aluminum alloy (such as 7075-T6) or carbon fiber composite material. The high rigidity design ensures that most of the disturbance torque generated by buoyancy and fluid resistance on the underwater linkage is borne by the shell itself, and the end-point trajectory accuracy can still be maintained within ±0.1 mm when the immersion depth changes.

[0091] Furthermore, a dynamic sealing scheme combining magnetohydrodynamic (MHD) seals and O-rings is employed between the shaft and housing of each rotating joint. Specifically, the MHD seal involves injecting a magnetic fluid between the permanent magnet and the pole shoe of the joint shaft to form a liquid ring. This ring rotates with the shaft and dynamically fills the gap, effectively preventing water molecules from entering. MHD seals offer advantages such as zero leakage, low friction, and long lifespan, making them suitable for high-speed underwater rotational dynamic sealing. The O-ring auxiliary seal involves adding a nitrile rubber O-ring to the outside or inside of the MHD seal as a static backup seal to prevent leakage during extreme pressure fluctuations or MHD aging.

[0092] In a preferred embodiment, the probe includes an ultrasonic transducer array and a light guide device 3; The ultrasonic transducer array is used to receive the transmitted ultrasonic and photoacoustic signals. The light guiding device 3 includes an optical fiber bundle that extends along the axial direction of the ultrasonic transducer array housing. The optical fiber bundle is positioned at the center or side of the acoustic field of the ultrasonic transducer array and is used to transmit laser light into the detection field of view of the ultrasonic transducer array.

[0093] The ultrasonic transducer array is used to transmit and receive ultrasonic signals, as well as photoacoustic signals. The light guiding device includes an optical fiber bundle extending axially along the ultrasonic transducer array housing and fixedly installed at the center or side of the acoustic field of the ultrasonic transducer array. This bundle transmits laser light into the detection field of view of the ultrasonic transducer array. When the optical fiber bundle is located at the center of the acoustic field, the laser irradiation direction coincides with the center line of the acoustic field, achieving collinear photoacoustic acquisition and high positioning accuracy. When located at the side, the laser irradiates at an angle, reducing surface reflection interference. A miniature heat dissipation structure is also provided inside the probe housing to prevent overheating caused by prolonged laser operation. This integrated design ensures that the laser irradiation area is always within the effective field of view of the ultrasonic transducer, eliminating the need for additional optical path adjustments, simplifying the system structure, and improving the efficiency of photoacoustic signal reception.

[0094] Based on this, the working principle of the automatic foot scanning ultrasonic photoacoustic imaging system proposed in this embodiment of the invention is as follows: First, warm water is added to the immersion imaging chamber 1 as a sound propagation coupling medium. The temperature adjustment device 10 sets the corresponding water temperature as needed and heats the immersion imaging chamber through heating pads. The patient places their foot on the side platform 15 inside the immersion imaging chamber 1 and fixes their ankle to ensure that the foot does not wobble. Multiple light guide devices 3 are dispersed and fixed on one side of the probe 2. The probe is an ultrasonic photoacoustic multimodal probe. The probe 2, temperature sensor 14, and vision sensor 4 are fixed on the first waterproof joint motor 5. The foot scanning robotic arm 7 drives the vision sensor 4 to take pictures of the patient's foot from multiple angles, such as the sole, back, and sides of the foot, to obtain a full-coverage image of the foot and generate a three-dimensional point cloud model of the foot. The three-dimensional point cloud model of the foot is identified using a preset algorithm, and the coordinate systems of the immersion imaging chamber 1, the foot, and the robotic arm 7 are unified to determine the size of the foot area within the immersion imaging chamber 1, thereby limiting the position of the three-dimensional spatial path points of the foot scanning robotic arm 7 and the probe 2. Simultaneously, based on the effective field of view of probe 2 and the set scanning step length range, a collision-free, high-coverage scanning path is automatically generated to ensure that probe 2 maintains a perpendicular posture to the foot surface throughout the scanning process. The first waterproof joint motor controls the pitch posture of probe 2, and the second waterproof joint motor 6 controls the rotation posture of probe 2. After the main control module 13 clicks "Start Scanning," it controls the drive module 12 to drive the foot scanning robotic arm 7 and the two joint motors to move along the automatically generated scanning path and scanning step length in three axes. The joint motors will adjust their pitch and rotation angles in real time according to the angle information within the scanning path. The temperature control unit 9 predicts the movement path based on the planned path position of the foot scanning robotic arm 7 and probe 2. For areas prone to disturbance along the path, a zoned independent heating / cooling strategy is adopted. The water temperature near the area is pre-adjusted before the foot scanning robotic arm 7 approaches to compensate for the water temperature interference caused by the movement of the foot scanning robotic arm 7. The temperature control unit 9 updates the prediction and adjustment commands in real time at a frequency of 10~20Hz. The probe 2 and the light guide device 3 are connected to the main control module 13. The main control module 13 controls the probe 2 to emit ultrasound or laser. The main control module 13 then receives the ultrasound and photoacoustic signals transmitted back from the probe 2 and performs reconstruction and display. The main control module 13 receives the real-time water temperature data from the temperature control unit 9 and adjusts the reconstructed sound velocity according to the sound velocity and water temperature comparison table to ensure that a high-quality, distortion-free image is reconstructed.

[0095] It should be noted that in ultrasonic and photoacoustic imaging, the propagation speed of sound waves directly determines the geometric accuracy of image reconstruction, and water temperature is a key factor affecting the speed of sound in water; the higher the temperature, the faster the speed of sound, and vice versa. Although the temperature adjustment device 10 in this embodiment of the invention has controlled the water temperature fluctuation within ±0.5°C, this deviation will still cause a change in the speed of sound of approximately ±1.6 m / s. If a fixed speed of sound is used for reconstruction, it may cause a positioning error at the millimeter level. To this end, the main control module 13 receives the current water temperature data fed back by the temperature control unit 9 in real time during the image reconstruction process (the sampling frequency is not less than 20 Hz, and the measurements from the moving sensor at the bottom of the probe and the fixed sensor in the chamber are fused), and dynamically adjusts the speed of sound parameters in the reconstruction algorithm according to a pre-established speed of sound-water temperature comparison table (this comparison table can adopt the internationally accepted empirical formula for the speed of sound in pure water, or be obtained through experimental calibration to compensate for the influence of solutes in the coupling medium). The main control module 13 reads the water temperature value at each imaging trigger moment, calculates the corresponding sound velocity through table lookup or formula, and uses it for delay calculation in ultrasonic beam formation, time-distance conversion in photoacoustic image reconstruction, and spatial registration in 3D image stitching. Through this active sound velocity correction, even with slight fluctuations in water temperature, the actual positioning error of the reconstructed image can be controlled within 0.05 mm, improving the geometric consistency and fusion accuracy of ultrasonic and photoacoustic images, while reducing the stringent requirements on the temperature control system. This is an important and preferred measure to ensure high-quality, distortion-free imaging.

[0096] The imaging system according to embodiments of the present invention can correspond to performing the methods described in the embodiments of the present invention, and the above and other operations and / or functions of each module of the imaging system are respectively for implementing Figure 1 The corresponding process of the method in the illustrated embodiment will not be described in detail here for the sake of brevity.

[0097] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A method for automatic scanning ultrasound-photoacoustic imaging of the foot, characterized in that, The imaging method is applied to an automated foot scanning system, which includes at least: a foot scanning robotic arm and a probe mounted on the robotic arm; the method includes the following steps: In response to the command to start scanning; The foot scanning robotic arm is controlled to move to the scanning starting position inside the immersion imaging chamber and acquire a surface image of the patient's foot to generate a three-dimensional point cloud model of the foot; the immersion imaging chamber contains a coupling medium and is the immersion area of ​​the patient's foot. Based on the three-dimensional point cloud model of the foot, the scanning path information of the probe is planned; the scanning path information includes at least the three-dimensional spatial path point position of the probe at each time point and the probe remains perpendicular to the surface of the patient's foot during the scanning process; During the scanning process, based on the scanning path information, the foot scanning robotic arm is controlled to drive the probe to move according to the scanning path information, and the normal attitude of the probe is adjusted in real time. The predicted water temperature data in the water immersion imaging chamber is acquired in real time, and the water temperature fluctuation in the water immersion imaging chamber is eliminated by a temperature adjustment device arranged in the water immersion imaging chamber. The probe is controlled to emit ultrasound and laser, and to receive the returned ultrasound and photoacoustic signals. Based on the returned ultrasound and photoacoustic signals, an ultrasound image and a photoacoustic image corresponding to the patient's foot are reconstructed. At the same time, the reconstructed photoacoustic image is superimposed on the grayscale background of the ultrasound image in pseudo-color form to display the anatomical structure and functional information of the foot. The scanning path information also includes: the scanning step size of the probe at each time point and the normal orientation that the probe should maintain; The planning of the probe scanning path information specifically includes: The scanning step length is dynamically calculated based on the local curvature of the foot's three-dimensional point cloud model. The effective field of view of the probe is obtained, and the interval between adjacent scanning sub-paths is set according to the effective field of view, so that the overlapping area of ​​adjacent scanning sub-paths satisfies the first preset condition. Using the size of the foot 3D point cloud model as the boundary constraint, and the scanning step size and the interval between adjacent scanning sub-paths as the execution condition constraint, multiple 3D spatial path point positions are generated. Based on the positions of multiple three-dimensional spatial path points, the normal vector of the patient's foot surface at different times is calculated. Based on each surface normal vector, the normal orientation of the probe at each time is set so that the probe remains perpendicular to the patient's foot surface during the scanning process.

2. The automatic scanning ultrasonic photoacoustic imaging method for the foot according to claim 1, characterized in that, The foot scanning robotic arm is equipped with a vision sensor; The generation of the 3D point cloud model of the foot includes: The refractive compensation parameters corresponding to the coupling medium are obtained in advance through underwater calibration, and the imaging model of the vision sensor is refractively corrected based on the refractive compensation parameters. The foot scanning robotic arm is controlled to drive the vision sensor to acquire images from multiple viewpoints around the patient's foot according to the preset scanning angle sequence of the robotic arm, thereby obtaining local point clouds from multiple viewpoints; The iterative nearest point algorithm is used to register the local point clouds corresponding to adjacent viewpoints to generate a complete 3D point cloud model of the foot.

3. The automatic scanning ultrasonic photoacoustic imaging method for the foot according to claim 2, characterized in that, The method also includes: The transformation matrix between the coordinate system of the visual sensor and the end coordinate system of the foot scanning robot arm is determined. At the same time, the fixed transformation relationship between the base coordinate system of the foot scanning robot arm and the coordinate system of the water immersion imaging chamber is calibrated so as to transform the foot three-dimensional point cloud model to the end coordinate system and the base coordinate system in sequence, and unify it to the coordinate system of the water immersion imaging chamber to achieve reference system alignment.

4. The automatic scanning ultrasonic photoacoustic imaging method for the foot according to claim 3, characterized in that, The method also includes: For each target link of the foot scanning robotic arm located in the coupling medium, the equivalent disturbance torque generated by buoyancy at each rotating joint is calculated in real time based on the immersion volume of the target link; The equivalent disturbance torque is used as a feedforward term and superimposed on the joint torque command of the foot scanning robot arm to control and adjust the drive of the foot scanning robot arm.

5. The automatic scanning ultrasonic photoacoustic imaging method for the foot according to claim 1, characterized in that, The method further includes: when controlling the movement of the foot scanning robotic arm, using a seven-segment S-shaped velocity curve to constrain the maximum end velocity and maximum acceleration of the foot scanning robotic arm.

6. The automatic scanning ultrasonic photoacoustic imaging method for the foot according to claim 1, characterized in that, The temperature adjustment device includes multiple heating pads and cooling pads arranged in an array, each element is independently controlled, forming several independent temperature control zones on the immersion imaging chamber; The real-time acquisition of predicted water temperature data within the immersion imaging chamber, and the elimination of water temperature fluctuations within the chamber via a temperature adjustment device located at the immersion imaging chamber, includes: Based on the obtained infrared thermal image distribution of the patient's foot surface and the current temperature of each temperature control zone, a pre-established local temperature field change model inside the immersion imaging chamber is input, and the predicted water temperature corresponding to each temperature control zone is obtained. If any of the predicted water temperatures exceeds the preset water temperature range, the heating pads or cooling pads of the corresponding temperature control zone will be controlled to operate until the water temperature in the corresponding temperature control zone is within the preset water temperature range.

7. The automatic scanning ultrasound photoacoustic imaging method for the foot according to claim 1, characterized in that, The probe is connected to the end of the foot scanning robotic arm via a torque sensor; the method further includes: real-time acquisition of feedback data from the torque sensor, the feedback data being used to reflect the real-time force between the probe and the end of the foot scanning robotic arm; Based on the real-time force, calculate the real-time contact force between the probe and the patient's foot surface; When the real-time contact force is determined to be greater than a preset safety threshold, the foot scanning robotic arm is controlled to pause its movement or retract in the reverse direction along the normal.

8. An automatic foot scanning ultrasonic photoacoustic imaging system, employing the method described in any one of claims 1-7, characterized in that, The imaging system is used in conjunction with a water immersion imaging chamber, which contains a coupling medium and is the immersion area of ​​the patient's foot. The imaging system includes: A foot scanning robotic arm, wherein the end of the foot scanning robotic arm is equipped with a probe and sensor assembly, and the probe integrates ultrasonic imaging and photoacoustic imaging functions; The foot scanning robotic arm is a high-rigidity serial robotic arm, and its rotary joint housing is made of high-strength aluminum alloy or carbon fiber composite material; each rotary joint adopts a dynamic sealing structure combining magnetohydrodynamic seal and O-ring; the sensor assembly includes at least: a vision sensor, which is a binocular structured light camera, encapsulated in a waterproof housing, and the waterproof housing is made of planar optical glass and coated with a hydrophobic coating. A temperature adjustment device, comprising: a plurality of heating pads and cooling pads arranged in an array, wherein the heating pads and cooling pads are arranged in a preset partition on the immersion imaging chamber; A control drive module is connected to the temperature adjustment device and the foot scanning robotic arm via signals, and is used to drive the foot scanning robotic arm to move and acquire water temperature data collected by the temperature adjustment device. The main control module is at least signal-connected to the probe, the sensor assembly, and the control drive module.

9. The automatic foot scanning ultrasonic photoacoustic imaging system according to claim 8, characterized in that, The probe includes an ultrasonic transducer array and a light guide device; The ultrasonic transducer array is used to receive the transmitted ultrasonic and photoacoustic signals. The light guiding device includes an optical fiber bundle that extends along the axial direction of the ultrasonic transducer array housing. The optical fiber bundle is located at the center or side of the acoustic field of the ultrasonic transducer array and is used to transmit laser light into the detection field of view of the ultrasonic transducer array.

Images