A calculus positioning method and device, electronic equipment and storage medium

Abstract

The application provides a stone positioning method and device, electronic equipment and storage medium, which are applied to a stone positioning aiming system. The stone positioning aiming system comprises a collaborative manipulator, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative manipulator. The shock wave source and the ultrasonic probe are respectively fixedly installed with a first positioning tool and a second positioning tool observed by the infrared optical camera. The method comprises the following steps: when receiving a stone positioning instruction, based on real-time pose information of the first positioning tool and the second positioning tool in the infrared optical camera coordinate system and ultrasonic image data containing a target stone, performing coordinate system conversion processing, and determining a target pose to which the end of the collaborative manipulator needs to be moved in order to align the focal point of the shock wave source with the position of the target stone. The present scheme realizes full-automatic and accurate positioning of the stone from the ultrasonic image to the treatment focal point, and improves the stone positioning efficiency and operation precision.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a method, apparatus, electronic device, and storage medium for locating stones. Background Technology

[0002] Extracorporeal shock wave lithotripsy (ESWL) is a non-invasive treatment device that uses high-energy focused shock waves to break up stones inside the body. The key to successful treatment lies in accurately and efficiently locating the stone at the focal point of the shock wave. Therefore, the performance of the stone localization system directly determines the effectiveness and safety of the entire treatment process.

[0003] Currently, the main localization technologies employ X-ray imaging and fixed ultrasound imaging. X-ray localization systems utilize the principle of X-ray penetration imaging, clearly displaying the stone's location in the image based on the difference in X-ray absorption rates between the stone and surrounding tissues, offering the advantage of accurate localization. Ultrasound localization systems, on the other hand, are based on the principle of ultrasound reflection, generating real-time two-dimensional images by receiving echo signals to observe the stone. Both technologies require manual operation by the physician, who must repeatedly search for and confirm the stone's location in the two-dimensional image space, and then manually adjust the shockwave source or the patient's position based on experience to ensure that the stone and the shockwave focus coincide in three-dimensional space.

[0004] However, the above-mentioned technical solutions have the following drawbacks: Although X-ray positioning provides clear imaging, it generates ionizing radiation during use, which may cause potential harm to patients and medical staff; although ultrasound positioning does not involve radiation, it is highly dependent on the experience and spatial imagination of the operating physician, and the manual positioning process is cumbersome and time-consuming, resulting in low overall treatment efficiency and difficulty in achieving rapid and accurate three-dimensional spatial alignment. Summary of the Invention

[0005] This application provides a stone localization method, device, electronic device, and storage medium, which achieves fully automatic and precise localization of stones from ultrasound images to the treatment focus through spatial fusion of infrared optics and ultrasound images and automatic control of a robotic arm, thereby improving stone localization efficiency and operational accuracy.

[0006] In a first aspect, embodiments of this application provide a method for locating kidney stones, applied to a kidney stone location and aiming system. The kidney stone location and aiming system includes a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm. A first positioning tool and a second positioning tool, respectively, are fixedly mounted on the shock wave source and the ultrasonic probe and observed by the infrared optical camera. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. The method includes:

[0007] When a stone location command is received, based on the real-time pose information of the first and second positioning tools in the infrared optical camera coordinate system, and the ultrasound image data containing the target stone, coordinate system transformation processing is performed to determine the target pose that the end effector of the collaborative robotic arm needs to move to so that the focus of the shock wave source is aligned with the target stone position.

[0008] Secondly, this application also provides a stone positioning device for use in a stone positioning and aiming system. The stone positioning and aiming system includes a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm. A first positioning tool and a second positioning tool, respectively, are fixedly mounted on the shock wave source and the ultrasonic probe and observed by the infrared optical camera. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. The device is used for:

[0009] When a stone location command is received, based on the real-time pose information of the first and second positioning tools in the infrared optical camera coordinate system, and the ultrasound image data containing the target stone, coordinate system transformation processing is performed to determine the target pose that the end effector of the collaborative robotic arm needs to move to so that the focus of the shock wave source is aligned with the target stone position.

[0010] Thirdly, embodiments of this application also provide an electronic device, which includes:

[0011] One or more processors;

[0012] Storage device for storing one or more programs.

[0013] When one or more programs are executed by one or more processors, the one or more processors implement a stone location method as described in any of the embodiments of this application.

[0014] Fourthly, embodiments of this application also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform any of the stone localization methods described in the embodiments of this application.

[0015] This application provides a method for locating kidney stones, applied to a kidney stone positioning and aiming system. The kidney stone positioning and aiming system includes a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm. A first positioning tool and a second positioning tool, which are observed by the infrared optical camera, are respectively fixedly installed on the shock wave source and the ultrasonic probe. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. The method includes: when a kidney stone positioning command is received, based on the real-time pose information of the first and second positioning tools in the coordinate system of the infrared optical camera, and the ultrasonic image data containing the target kidney stone, performing coordinate system transformation processing to determine the target pose to which the end of the collaborative robotic arm needs to move in order to align the focus of the shock wave source with the position of the target kidney stone. The technical solution of this application combines real-time spatial positioning with infrared optics, two-dimensional ultrasound image guidance, and automatic control with a collaborative robotic arm to achieve fully automatic and high-precision coordinate transformation and alignment from the two-dimensional position of the stone selected in the ultrasound image to the three-dimensional spatial pose required for the focus of the shock wave source. This solves the problems of conventional ultrasound positioning technology, which relies on manual operation and spatial imagination by doctors, is cumbersome and time-consuming, and is difficult to achieve fast and accurate three-dimensional alignment. It improves the automation level, operational efficiency, and positioning accuracy of stone positioning. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the exemplary embodiments of this application, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the embodiments to be described in this application, and not all of them. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0017] Figure 1 A flowchart illustrating a stone localization method provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of the stone positioning and aiming system involved in the embodiments of this application;

[0019] Figure 3 A flowchart illustrating yet another stone localization method provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram illustrating the data flow relationship between internal modules of a computer program and external devices used to execute the stone location method provided in this embodiment of the present application.

[0021] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0023] Before describing the technical solution proposed in the embodiments of this application, its application scenario is first explained. This embodiment is applicable to various scenarios requiring precise location and aiming of stones using an extracorporeal shock wave lithotripter. Currently, although X-ray imaging positioning or fixed ultrasound imaging positioning is widely used in clinical practice, traditional methods have significant shortcomings. In actual operation, because the stones are located inside the tissue and in a concealed spatial position, and the positioning process needs to achieve rapid and precise alignment from two-dimensional images to the three-dimensional treatment focus while avoiding radiation damage, traditional positioning methods rely on manual operation and experience judgment by doctors, making it difficult to simultaneously meet the requirements of radiation-free, high precision, and high efficiency. This easily leads to problems such as long positioning time, high operational complexity, and unstable treatment accuracy. Therefore, there is an urgent need for a positioning method that can integrate multi-source spatial information and achieve automatic coordinate transformation and precise alignment to improve the intelligence level of stone positioning and treatment efficiency. This embodiment focuses on automatically and accurately aligning the shock wave source with the target stone. Based on real-time acquisition of infrared optical spatial data and ultrasonic image data, the moving target at the end of the robotic arm is determined through coordinate system transformation, thereby ensuring the spatial accuracy and operational automation of the positioning process, effectively improving the reliability, safety and overall efficiency of stone positioning.

[0024] Example 1

[0025] Figure 1 This is a flowchart illustrating a stone location method provided in an embodiment of this application. This embodiment is applicable to various situations where stones need to be accurately located and aimed using an extracorporeal shock wave lithotripter. The method can be executed by a stone location device, which can be implemented in the form of software and / or hardware. The hardware can be a controller, such as a mobile terminal, a PC, or a server.

[0026] The stone localization method provided in this embodiment is applied to a stone localization and aiming system. The stone localization and aiming system includes a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm. A first positioning tool and a second positioning tool, which are observed by the infrared optical camera, are respectively fixedly installed on the shock wave source and the ultrasonic probe. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe.

[0027] In the stone localization and targeting system, the collaborative robotic arm is an automated mechanical device capable of multi-degree-of-freedom movement in physical space and interacting with the environment or operator; the end effector of the collaborative robotic arm is the terminal structure where the robotic arm performs actions. The infrared optical camera is an imaging device that can acquire the position and attitude information of a target object in space by sensing infrared optical signals. The shock wave source is the core component of a physical therapy device used to generate and emit focused shock waves to act on stones in the human body. The ultrasound probe is a medical imaging sensor used to emit ultrasound waves and receive their echoes to generate ultrasound images of the internal structure of the human body. The first positioning tool and the second positioning tool are physical marker components that are fixedly installed on the shock wave source and the ultrasound probe, have specific optical characteristics, and can be uniquely identified and tracked by the infrared optical camera. The working area refers to the entire spatial range in which the physical structure of the collaborative robotic arm, the shock wave source, and the ultrasound probe can move and function effectively during the normal execution of the positioning and treatment tasks of the system.

[0028] This can be understood as applying the stone localization method provided in this embodiment to a stone localization and aiming system composed of a collaborative robotic arm, an infrared optical camera, a shock wave source, and an ultrasonic probe. The shock wave source and ultrasonic probe are rigidly fixed to the end of the collaborative robotic arm, moving with it. To know the exact position and orientation of the shock wave source and ultrasonic probe in space at any time, a first positioning tool is fixed to the shock wave source, and a second positioning tool is fixed to the ultrasonic probe. Both tools can be identified in real time by the infrared optical camera, which provides their respective poses. Simultaneously, the installation angle and field of view of the infrared optical camera are specifically adjusted to monitor the entire range of motion of the collaborative robotic arm, the possible reach area of ​​the shock wave source, and the scanning area of ​​the ultrasonic probe in a single operation. An exemplary schematic diagram of the stone localization and aiming system can be found in [reference needed]. Figure 2 ,like Figure 2 As shown, the device marked by the rectangle above is an infrared optical camera, and the robotic arm-shaped component in the figure is a collaborative robotic arm. The ends of the collaborative robotic arm are respectively fixed with a shock wave source and an ultrasonic probe.

[0029] like Figure 1 As shown, the stone localization method provided in this embodiment of the invention includes the following steps:

[0030] S110. When a stone positioning instruction is received, based on the real-time pose information of the first and second positioning tools in the infrared optical camera coordinate system, and the ultrasound image data containing the target stone, coordinate system transformation processing is performed to determine the target pose that the end effector of the collaborative robotic arm needs to move to so that the focus of the shock wave source is aligned with the position of the target stone.

[0031] Among them, the stone positioning command refers to the start signal and control command issued by the operator or the superior control system to command the stone positioning and aiming system to start the complete process of identifying the target stone from the ultrasound image and determining the position to be reached by the end of the collaborative robotic arm through coordinate transformation calculation.

[0032] Among them, real-time pose information refers to the instantaneous data acquired and calculated in real time by the infrared optical camera, which describes the specific spatial position and three-dimensional spatial attitude of the first or second positioning tool relative to the three-dimensional spatial coordinate system established by the infrared optical camera itself at any specific moment.

[0033] The target stone refers to a specific internal stone that needs to be located using a stone positioning and aiming system and is selected as the focal target of the shock wave source. Ultrasound image data refers to two-dimensional digital images acquired and generated by an ultrasound probe during operation, which reflect the morphology of internal human tissues and contain image information of the target stone.

[0034] The target stone location refers to the specific three-dimensional spatial orientation of the target stone in the imaging space coordinate system corresponding to the ultrasound probe. The target pose refers to the specific spatial position and posture that the end effector of the collaborative robotic arm needs to reach and maintain in its base coordinate system in order to ensure that the treatment focus of the shock wave source fixed at the end effector of the collaborative robotic arm is precisely aligned with the target stone location in three-dimensional space.

[0035] In this embodiment, optionally, the specific implementation of obtaining the real-time pose information of the first positioning tool and the second positioning tool in the infrared optical camera coordinate system may include the following steps:

[0036] (1) The spatial arrangement data of the preset reflective markers on the first and second positioning tools are captured in real time by an infrared optical camera.

[0037] Among them, the preset reflective marker points refer to passive optical marker points that are pre-fixed on the surfaces of the first and second positioning tools according to a specific geometric configuration and can be actively identified and tracked by the infrared optical camera. The spatial arrangement data refers to the raw observation data captured in real time by the infrared optical camera, which describes the three-dimensional spatial distribution and relative positional relationship of the group of preset reflective marker points relative to the coordinate system of the infrared optical camera in the current field of view.

[0038] In this embodiment, the infrared optical camera can continuously emit and receive infrared light, and within its field of view, illuminate and image the preset reflective markers arranged in a specific geometric pattern on the surfaces of the first and second positioning tools. Utilizing the high reflectivity of the reflective material to infrared light, the brightness center of each marker on the camera's imaging plane is extracted. Based on the mutual distance and angular relationship between multiple markers, the relative positions and overall arrangement of these markers in three-dimensional space are reconstructed in real time, thereby obtaining spatial arrangement data. This provides the raw input for subsequent matching with the pre-stored three-dimensional model of the positioning tool and solving the real-time pose information.

[0039] (2) The captured spatial arrangement data is matched with the pre-stored three-dimensional model of the positioning tool to calculate the real-time pose information of the first and second positioning tools in the infrared optical camera coordinate system.

[0040] Among them, the three-dimensional model of the positioning tool refers to the digital reference model that is pre-established and stored in the system to describe the complete three-dimensional geometric shape and spatial configuration of the marker points of the positioning tool.

[0041] In this embodiment, the spatial arrangement data captured in real time by the infrared optical camera can be matched one-to-one with the standard geometric pattern of the marker points described by the three-dimensional model of the positioning tool that has been stored in the memory. By finding the best overlap between the two sets of points, the current spatial orientation and position of the first and second positioning tools in the coordinate system of the infrared optical camera can be deduced, thereby directly calculating the real-time pose information of the two tools for subsequent coordinate transformation and positioning control.

[0042] Specifically, upon receiving a stone localization command, the first real-time pose information of the first positioning tool, measured in real-time under the infrared optical camera coordinate system, can be combined with the real-time pose feedback information from the end effector of the collaborative robotic arm. Using the known fixed installation relationship between the first positioning tool, the shock wave source, and the end effector of the collaborative robotic arm, the global coordinate transformation relationship from the infrared optical camera coordinate system to the base coordinate system of the collaborative robotic arm can be calculated, thereby transforming any point seen by the camera into coordinates that the robotic arm base can understand. Simultaneously, the second real-time pose information of the second positioning tool, measured in real-time under the infrared optical camera coordinate system, the fixed installation relationship between the second positioning tool and the ultrasonic probe, and the ultrasonic image... By combining the preset imaging relationship between the coordinate system and the probe coordinate system, a coordinate mapping relationship is established that can directly map the pixel position in the ultrasound image coordinate system to the infrared optical camera coordinate system. With these two relationships, it is only necessary to read the two-dimensional position of the target stone in the ultrasound image coordinate system. It can first be raised to the infrared optical camera coordinate system through the coordinate mapping relationship, and then lowered to the base coordinate system of the collaborative robotic arm through the global coordinate transformation relationship. At the same time, the focus of the shock wave source is also unified to this coordinate system. After comparing the deviation between the two, the target pose that the end of the collaborative robotic arm must reach is solved in reverse, so that the focus of the shock wave source is finally aligned with the target stone position in physical space, completing the closed-loop positioning.

[0043] This application provides a method for locating kidney stones, applied to a kidney stone positioning and aiming system. The kidney stone positioning and aiming system includes a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm. A first positioning tool and a second positioning tool, which are observed by the infrared optical camera, are respectively fixedly installed on the shock wave source and the ultrasonic probe. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. The method includes: when a kidney stone positioning command is received, based on the real-time pose information of the first and second positioning tools in the coordinate system of the infrared optical camera, and the ultrasonic image data containing the target kidney stone, performing coordinate system transformation processing to determine the target pose to which the end of the collaborative robotic arm needs to move in order to align the focus of the shock wave source with the position of the target kidney stone. The technical solution of this application combines real-time spatial positioning with infrared optics, two-dimensional ultrasound image guidance, and automatic control with a collaborative robotic arm to achieve fully automatic and high-precision coordinate transformation and alignment from the two-dimensional position of the stone selected in the ultrasound image to the three-dimensional spatial pose required for the focus of the shock wave source. This solves the problems of conventional ultrasound positioning technology, which relies on manual operation and spatial imagination by doctors, is cumbersome and time-consuming, and is difficult to achieve fast and accurate three-dimensional alignment. It improves the automation level, operational efficiency, and positioning accuracy of stone positioning.

[0044] Example 2

[0045] Figure 3This is a schematic diagram of a stone localization method provided in an embodiment of this application. Based on the foregoing embodiments, this embodiment will provide a detailed explanation of the specific implementation method of determining the target pose to which the end effector of the collaborative robotic arm needs to move in order to align the focus of the shock wave source with the target stone location. For specific implementation methods, please refer to the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0046] like Figure 3 As shown, the method specifically includes the following steps:

[0047] S210. Based on the first real-time pose information of the first positioning tool in the infrared optical camera coordinate system, the real-time pose feedback information of the end effector of the collaborative robotic arm, and the fixed installation relationship between the first positioning tool, the shock wave source and the end effector of the robotic arm, determine the first global coordinate transformation relationship of the infrared optical camera coordinate system relative to the base coordinate system of the collaborative robotic arm.

[0048] Among them, the first real-time pose information refers to the instantaneous data used to describe the specific spatial position and three-dimensional spatial attitude of the first positioning tool fixed on the shock wave source relative to the three-dimensional spatial coordinate system established by the infrared optical camera itself at any specific moment.

[0049] Among them, the real-time pose feedback information of the collaborative robotic arm end effector refers to the instantaneous data measured and provided in real time by the sensing and control system configured on the collaborative robotic arm itself, which describes the specific spatial position and three-dimensional spatial posture of the collaborative robotic arm end effector relative to the base coordinate system of the collaborative robotic arm at any specific moment.

[0050] Among them, the fixed installation relationship between the first positioning tool, the shock wave source and the end effector of the robotic arm refers to the rigid relative position and relative attitude relationship between the local coordinate systems of the first positioning tool, the shock wave source that fixes the first positioning tool and the end effector of the robotic arm that installs the shock wave source, which is determined in advance through the physical installation and calibration process, and which remains unchanged in spatial position.

[0051] Among them, the first global coordinate transformation relationship can uniformly transform the coordinates of any point in the infrared optical camera coordinate system to the spatial coordinate transformation rules in the base coordinate system of the collaborative robotic arm.

[0052] Specifically, the first real-time pose information of the first positioning tool in its coordinate system provided in real time by the infrared optical camera, the real-time pose feedback information of its end effector in the base coordinate system fed back by the collaborative robotic arm itself, and the known immutable fixed installation relationship between the first positioning tool, the shock wave source and the end effector of the robotic arm can be used to derive and establish a first global coordinate transformation relationship that can unify and transform the coordinate system of the infrared optical camera and the base coordinate system of the collaborative robotic arm through a series of chain-like spatial coordinate transformation calculations.

[0053] In this embodiment, optionally, the specific implementation steps for determining the first global coordinate transformation relationship between the infrared optical camera coordinate system and the base coordinate system of the cooperative robotic arm may include:

[0054] (1) Based on the real-time pose feedback information of the end effector of the collaborative robot arm, determine the first pose to be processed in the coordinate system of the end effector of the robot arm under the coordinate system of the base.

[0055] The first pose to be processed refers to the intermediate calculation data describing the specific spatial position and three-dimensional spatial posture of the end effector coordinate system of the robotic arm relative to the base coordinate system of the collaborative robotic arm at the current moment.

[0056] Specifically, the real-time pose feedback information of the end effector of the collaborative robotic arm can be read from the joint sensors of the collaborative robotic arm itself. This information has integrated the angles of each joint and the parameters of the links. From this, the spatial position and attitude of the end effector coordinate system relative to the base coordinate system can be calculated, which is the first pose to be processed.

[0057] (2) Based on the fixed installation relationship between the first positioning tool, the shock wave source and the end of the robotic arm, determine the first fixed transformation relationship between the coordinate system of the first positioning tool and the coordinate system of the end of the robotic arm.

[0058] The first fixed transformation relationship refers to the constant spatial position and spatial attitude transformation relationship used to describe the coordinate system of the first positioning tool relative to the coordinate system of the end effector of the robotic arm.

[0059] Specifically, since the shock wave source of the first positioning tool and the end effector of the collaborative robotic arm are installed in one go through a rigid structure, and there is no relative movement between them in space, the constant geometric dimensions and angle relationships obtained from factory calibration or on-site hand-eye calibration can be directly used to solidify the offset and rotation between the origin and axis of the first positioning tool coordinate system and the origin and axis of the robotic arm end effector coordinate system, forming a constant first fixed transformation relationship. This allows the corresponding position and orientation of the first positioning tool coordinate system in the same pose to be immediately calculated as long as the pose of the robotic arm end effector coordinate system is known at any time, providing a constant intermediate bridge for subsequent chain coordinate transformations.

[0060] (3) Based on the first real-time pose information, determine the second pose to be processed in the coordinate system of the first positioning tool under the coordinate system of the infrared optical camera.

[0061] The second pose to be processed refers to the intermediate calculation data used to describe the specific spatial position and three-dimensional spatial attitude of the first positioning tool coordinate system relative to the infrared optical camera coordinate system at the current moment.

[0062] Specifically, the first real-time pose information measured by the infrared optical camera can be used as an instantaneous description of the first positioning tool coordinate system relative to the infrared optical camera coordinate system, and is denoted as the second pose to be processed.

[0063] (4) Based on the first pose to be processed, the second pose to be processed and the first fixed transformation relationship, the first global coordinate transformation relationship of the infrared optical camera coordinate system relative to the base coordinate system of the cooperative robotic arm is determined through chain coordinate transformation.

[0064] In this embodiment, the known pose from the end-effector coordinate system to the base coordinate system described by the first pose to be processed, the constant offset and rotation from the first positioning tool coordinate system to the end-effector coordinate system described by the first fixed transformation relationship, and the real-time pose from the first positioning tool coordinate system to the infrared optical camera coordinate system described by the second pose to be processed can be sequentially linked into a coordinate chain. By reverse deducing the overall correspondence of this chain, the complete spatial mapping of the infrared optical camera coordinate system relative to the base coordinate system of the cooperative robot can be directly solved, that is, the first global coordinate transformation relationship. This allows any target point measured under the camera to be transformed into a coordinate environment recognized by the robot base in one go, providing a unified reference for subsequent positioning calculations.

[0065] For example, the first global coordinate transformation relationship between the infrared optical camera coordinate system and the base coordinate system of the collaborative robotic arm can be expressed as:

[0066]

[0067] In the formula, This represents the first global coordinate transformation relationship. This represents the first pose to be processed in the coordinate system of the robotic arm's end effector (midpoint of the end effector) under the coordinate system of the base. This indicates the pose of the shock wave source in the coordinate system of the robotic arm's end effector. This indicates the pose of the first positioning tool in the shock wave source coordinate system. That is, the first fixed transformation relationship between the coordinate system of the first positioning tool and the coordinate system of the end effector of the robotic arm; This represents the second pose to be processed in the coordinate system of the first positioning tool under the coordinate system of the infrared optical camera.

[0068] S220. Based on the second real-time pose information of the second positioning tool in the infrared optical camera coordinate system, the fixed installation relationship between the second positioning tool and the ultrasonic probe, and the preset imaging relationship between the ultrasonic image coordinate system and the probe coordinate system, determine the coordinate mapping relationship for mapping the position in the ultrasonic image coordinate system to the infrared optical camera coordinate system.

[0069] The second real-time pose information refers to the instantaneous data used to describe the specific spatial position and three-dimensional spatial attitude of the second positioning tool fixed on the ultrasonic probe relative to the three-dimensional spatial coordinate system established by the infrared optical camera itself at any specific moment.

[0070] The fixed installation relationship between the second positioning tool and the ultrasonic probe refers to the rigid relative position and relative attitude relationship that is determined in advance through the physical installation and calibration process, describing the corresponding local coordinate system between the second positioning tool and the ultrasonic probe to which it is fixed, and which remains unchanged in space.

[0071] The preset imaging relationship between the ultrasound image coordinate system and the probe coordinate system refers to the fixed spatial geometric relationship that describes the existence between the ultrasound image coordinate system generated by the ultrasound probe and its own probe coordinate system, and characterizes the spatial position and orientation of the ultrasound imaging plane.

[0072] Among them, the coordinate mapping relationship refers to the spatial coordinate transformation rule that can transform and map any two-dimensional position information in the ultrasonic image coordinate system to the three-dimensional spatial coordinate system of the infrared optical camera.

[0073] Specifically, the second real-time pose information obtained by the infrared optical camera from the second positioning tool fixed to the ultrasonic probe can be combined with the fixed installation relationship between the second positioning tool and the ultrasonic probe that has been pre-calibrated and remains unchanged, as well as the inherent preset imaging relationship between the ultrasonic image coordinate system and the probe coordinate system that has been determined by calibration. Through a series of continuous spatial coordinate transformation calculations, a coordinate mapping relationship that can accurately transform the coordinate data of any position in the ultrasonic image coordinate system to the coordinate system of the infrared optical camera can be constructed.

[0074] In this embodiment, optionally, the specific implementation steps for determining the coordinate mapping relationship between the position in the ultrasound image coordinate system and the infrared optical camera coordinate system may include:

[0075] (1) Based on the second real-time pose information, determine the third pose to be processed in the coordinate system of the second positioning tool under the coordinate system of the infrared optical camera.

[0076] The third pose to be processed refers to the intermediate calculation data used to describe the specific spatial position and three-dimensional spatial attitude of the second positioning tool coordinate system relative to the infrared optical camera coordinate system at the current moment.

[0077] Specifically, the second real-time pose information provided by the infrared optical camera can be identified as the instantaneous complete description of the second positioning tool coordinate system relative to the infrared optical camera coordinate system, and denoted as the third pose to be processed.

[0078] (2) Based on the fixed installation relationship between the second positioning tool and the ultrasonic probe, determine the second fixed transformation relationship between the coordinate system of the second positioning tool and the coordinate system of the probe.

[0079] The second fixed transformation relationship refers to the constant spatial position and spatial attitude transformation relationship used to describe the coordinate system of the second positioning tool relative to the coordinate system of the probe.

[0080] Specifically, the rigid connection dimensions and angles between the second positioning tool and the ultrasonic probe, which are measured at the factory or during on-site calibration and never change, can be used to solidify the fixed offsets and rotations between the origin and three axes of the second positioning tool coordinate system and the origin and three axes of the probe coordinate system, forming a constant second fixed transformation relationship. This allows the pose of the probe coordinate system to be uniquely determined as long as the pose of the second positioning tool coordinate system is known.

[0081] (3) Based on the preset imaging relationship between the probe coordinate system and the ultrasound image coordinate system, determine the third fixed transformation relationship of the ultrasound image coordinate system relative to the probe coordinate system.

[0082] The third fixed transformation relationship refers to the constant spatial position and spatial orientation transformation relationship used to describe the ultrasound image coordinate system relative to the probe coordinate system.

[0083] Specifically, the imaging parameters of the probe coordinate system and the ultrasound image coordinate system, which are fixed and no longer change when the probe is calibrated at the factory, can be called. The correspondence between each scan line in the ultrasound image and the physical space direction, the scaling ratio of the pixel size and the actual distance, and the offset of the image origin relative to the reference point on the probe surface are encapsulated into a constant third fixed transformation relationship, so that any pixel position in the ultrasound image coordinate system can be uniquely converted to a three-dimensional spatial point in the probe coordinate system.

[0084] (4) Based on the third pose to be processed, the second fixed transformation relationship and the third fixed transformation relationship, the coordinate mapping relationship used to map the position in the ultrasound image coordinate system to the infrared optical camera coordinate system is determined by chain coordinate transformation.

[0085] In this embodiment, the real-time pose from the second positioning tool coordinate system to the infrared optical camera coordinate system given by the third pose to be processed, the constant offset and rotation from the second positioning tool coordinate system to the probe coordinate system given by the second fixed transformation relationship, and the fixed imaging parameters from the ultrasound image coordinate system to the probe coordinate system given by the third fixed transformation relationship can be sequentially linked into a coordinate chain. By sequentially reversing the overall correspondence of this chain, a complete mapping rule, i.e., a coordinate mapping relationship, can be directly synthesized to transform any position in the ultrasound image coordinate system to the infrared optical camera coordinate system in one go. This allows the coordinates of any subsequent stones in the image to be immediately converted to the three-dimensional space under the camera's viewpoint, providing a unified positioning reference for the shock wave source to align with the target.

[0086] For example, the coordinate mapping relationship between the position in the ultrasound image coordinate system and the infrared optical camera coordinate system can be expressed as:

[0087]

[0088] In the formula, Indicates the coordinate mapping relationship. This represents the third pose to be processed in the coordinate system of the second positioning tool under the coordinate system of the infrared optical camera; This represents the second fixed transformation relationship between the second positioning tool coordinate system and the probe coordinate system. This represents the third fixed transformation relationship between the ultrasound image coordinate system and the probe coordinate system.

[0089] S230. Based on the two-dimensional position of the target stone in the ultrasound image coordinate system, the coordinate mapping relationship, and the first global coordinate transformation relationship, determine the target pose that the end effector of the collaborative robotic arm needs to move to in order to make the focus of the shock wave source align with the position of the target stone.

[0090] The two-dimensional location of the stone refers to coordinate data consisting of two coordinate axes, which characterizes the specific location of the target stone on the two-dimensional image plane in the coordinate system of the ultrasound image generated by the ultrasound probe.

[0091] In this embodiment, optionally, the method for determining the two-dimensional position of the target stone in the ultrasound image coordinate system may include the following steps:

[0092] (1) Receive the user's marking operation on the target stone location in the real-time image interface of the ultrasound probe.

[0093] Among them, the real-time image interface refers to the human-computer interactive graphical user interface in the stone localization and aiming system, which is used to display in real time the ultrasound image data continuously acquired and generated by the ultrasound probe, reflecting the dynamic images of the internal tissues of the human body.

[0094] In practical applications, the ultrasound probe continuously refreshes the ultrasound cross-section image on the real-time image interface. When the operator directly clicks or drags on the dynamic image to indicate the instantaneous image center of the target stone through touch screen, mouse or button, the system immediately captures the screen coordinates corresponding to the interactive action as the input for the marking operation, thereby initiating the subsequent process of converting the pixel coordinates into the two-dimensional position of the stone in the ultrasound image coordinate system.

[0095] (2) Based on the image pixel coordinates corresponding to the marking operation and the image calibration parameters of the ultrasound probe, determine the two-dimensional position of the target stone in the ultrasound image coordinate system.

[0096] Among them, image calibration parameters refer to specific quantitative data or mathematical model parameters obtained by calibrating the ultrasonic probe imaging system in advance, which are used to describe the spatial geometric mapping relationship between the ultrasonic image coordinate system and the probe coordinate system.

[0097] In practical applications, the image pixel coordinates given by the marking operation can be compared with the pre-written image calibration parameters of the ultrasound probe (i.e., the corresponding ratio of pixel rows and columns to actual scanning depth, lateral distance, and image origin offset). This allows the pixel row and column numbers on the screen to be converted into longitudinal depth and lateral distance values ​​in the ultrasound image coordinate system, thereby uniquely determining the two-dimensional position of the target stone in this coordinate system and providing accurate input for subsequent coordinate mapping and robotic arm positioning.

[0098] The target stone location refers to the coordinate data used to describe the specific orientation of the target stone in three-dimensional space, which is determined by a series of spatial coordinate transformations and calculations in the base coordinate system of the collaborative robotic arm or other unified global three-dimensional spatial coordinate system.

[0099] Specifically, the two-dimensional position of the target stone in the ultrasound image coordinate system can be used as the starting point. First, the two-dimensional position is converted into three-dimensional coordinates in the infrared optical camera coordinate system through coordinate mapping. Then, the three-dimensional coordinates under this camera view are further converted into the base coordinate system of the collaborative robotic arm using the first global coordinate transformation relationship to obtain the exact spatial position of the stone in the robotic arm world. Subsequently, combined with the known fixed geometric offset of the shock wave source focus relative to the end of the collaborative robotic arm, the position and orientation that the end of the collaborative robotic arm should have when the shock wave source focus must coincide with this spatial position are calculated in reverse. This outputs the target pose that the end of the collaborative robotic arm needs to move to, ensuring that the focus of the shock wave source is aligned with the position of the target stone.

[0100] In this embodiment, optionally, the specific steps for determining the target pose to which the end effector of the collaborative robotic arm needs to move in order to align the focus of the shock wave source with the target stone location may include:

[0101] (1) Based on the two-dimensional position of the target stone in the ultrasound image coordinate system and the coordinate mapping relationship, determine the three-dimensional coordinates of the target stone in the infrared optical camera coordinate system.

[0102] Among them, the three-dimensional coordinates of the stone refer to spatial coordinate data consisting of three coordinate axis components, which are used to describe the specific location of the target stone in the three-dimensional spatial coordinate system of the infrared optical camera.

[0103] In this embodiment, the two-dimensional position of the target stone in the ultrasound image coordinate system can be used as input. The established coordinate mapping relationship is called, and according to the pixel-physical ratio, scanning direction and depth origin parameters encapsulated in the relationship, the two-dimensional point on the image plane is extended to three-dimensional space in one go to obtain the corresponding longitudinal depth, lateral distance and height values ​​in the infrared optical camera coordinate system. These values ​​are combined to form the three-dimensional coordinates of the stone, thereby completely describing the specific location of the stone in the three-dimensional spatial coordinate system of the infrared optical camera.

[0104] (2) Based on the three-dimensional coordinates of the stone and the first global coordinate transformation relationship, determine the three-dimensional target position of the target stone in the base coordinate system of the collaborative robotic arm.

[0105] Among them, the three-dimensional target position refers to a spatial coordinate data consisting of three coordinate axis components, which describes the specific orientation of the target stone in three-dimensional space under the base coordinate system of the collaborative robotic arm.

[0106] Specifically, the calculated three-dimensional coordinates of the stone can be placed in the coordinate system of the infrared optical camera. By calling the fixed rotation and translation amounts from the camera viewpoint to the robot arm base viewpoint recorded by the first global coordinate transformation relationship, the coordinates along with its spatial orientation can be mapped to the base coordinate system of the collaborative robot arm. This yields the unique three-dimensional target position of the target stone in the robot arm's world coordinates, enabling subsequent back-calculation steps to accurately calculate the pose that the end effector should reach in a unified robot arm reference system.

[0107] (3) Based on the known installation position relationship between the three-dimensional target position and the shock wave source relative to the end of the collaborative robot arm, the target pose to which the end of the collaborative robot arm needs to move is calculated in order to make the focus of the shock wave source coincide with the position of the three-dimensional target.

[0108] In this embodiment, the three-dimensional target position can be regarded as the absolute location that the shock wave source focus must reach. Using the rigid offset and rotation amount fixed by the known installation position relationship between the shock wave source and the end of the collaborative robotic arm, the spatial position and orientation that the end of the collaborative robotic arm should have when the focus and the three-dimensional target position are completely coincident are calculated in reverse within the base coordinate system of the collaborative robotic arm. This directly generates the target pose that the end of the collaborative robotic arm needs to move to, ensuring that the shock wave source focus is aligned with the stone.

[0109] For example, firstly, the position of the target stone in the ultrasound image coordinate system can be determined. Convert the vector to a homogeneous coordinate vector (x, y, 0, 1); based on this, calculate the three-dimensional position of the target stone in the coordinate system of the robotic arm base. Therefore, the calculated three-dimensional target position can be used as a basis. The position that the robotic arm's end effector needs to reach can be determined. This allows you to control the robotic arm to move toward the target point.

[0110] The technical solution of this application embodiment can determine the first global coordinate transformation relationship between the infrared optical camera coordinate system and the base coordinate system of the collaborative robotic arm based on the first real-time pose information of the first positioning tool in the infrared optical camera coordinate system, the real-time pose feedback information of the end effector of the collaborative robotic arm, and the fixed installation relationship between the first positioning tool, the shock wave source and the end effector of the robotic arm. Then, based on the second real-time pose information of the second positioning tool in the infrared optical camera coordinate system, the fixed installation relationship between the second positioning tool and the ultrasonic probe, and the preset imaging relationship between the ultrasonic image coordinate system and the probe coordinate system, a coordinate mapping relationship for mapping the position in the ultrasonic image coordinate system to the infrared optical camera coordinate system is determined. Thus, based on the two-dimensional position of the target stone in the ultrasonic image coordinate system, the coordinate mapping relationship and the first global coordinate transformation relationship, the target pose that the end effector of the collaborative robotic arm needs to move to in order to make the focus of the shock wave source align with the position of the target stone is determined. The technical solution provided by this invention integrates infrared optical positioning and ultrasound image guidance to achieve automated conversion of target stones from two-dimensional image recognition to precise three-dimensional spatial positioning. The two-dimensional position of the stone selected in the ultrasound image is seamlessly converted into the final spatial pose required by the shock wave source carried by the end of the collaborative robotic arm through real-time and precise coordinate system and spatial mapping calculation. This ensures that the focus of shock wave treatment can be automatically and accurately aligned with the target stone, improving the accuracy, efficiency and automation level of positioning.

[0111] Next, a specific implementation method will be used to describe the stone localization method provided in the embodiments of the present invention. Figure 4 This is a schematic diagram illustrating the data flow relationship between internal modules of a computer program and external devices used to execute the stone localization method provided in this embodiment. Figure 4 As shown, the internal modules of the computer program for this stone localization method may include: a fixed coordinate transformation module, a camera data acquisition module, a robotic arm data acquisition module, an ultrasonic data acquisition module, a coordinate transformation module, and a robotic arm control module. In practical applications, the fixed coordinate transformation module reads a fixed transformation relationship: the pose of the shock wave source in the coordinate system of the robotic arm's end effector. The pose of the shock wave source positioning tool in the shock wave source coordinate system The pose of the probe positioning tool in the probe coordinate system and the pose of the ultrasound image coordinate system in the probe coordinate system The camera data acquisition module obtains the conversion relationship between the camera and the positioning tool: the pose of the positioning tool on the shock wave source in the camera coordinate system. And the pose of the positioning tool on the probe in the camera coordinate system. The robot arm data acquisition module obtains the pose of the robot arm's end effector center in the robot arm base coordinate system. Calculate the final position that the robotic arm's end effector needs to reach. The position of the stone in the ultrasound image coordinate system is obtained by the ultrasound data acquisition module. The coordinate transformation module calculates the pose of the camera coordinate system in the robot arm base coordinate system. The pose of the ultrasound image coordinate system in the camera coordinate system The position of the stone in the coordinate system of the robotic arm base. Finally, the robotic arm control module sends commands to the robotic arm to control its movement.

[0112] Example 3

[0113] This application provides a stone positioning device applied to a stone positioning and aiming system. The stone positioning and aiming system includes a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm. A first positioning tool and a second positioning tool, observed by the infrared optical camera, are respectively fixedly mounted on the shock wave source and the ultrasonic probe. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. The device is used to: when receiving a stone positioning command, based on the real-time pose information of the first and second positioning tools in the coordinate system of the infrared optical camera, and the ultrasonic image data containing the target stone, perform coordinate system transformation processing to determine the target pose to which the end of the collaborative robotic arm needs to move so that the focus of the shock wave source is aligned with the position of the target stone.

[0114] This application provides a stone positioning device applied to a stone positioning and aiming system. The stone positioning and aiming system includes a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm. A first positioning tool and a second positioning tool, which are observed by the infrared optical camera, are respectively fixedly installed on the shock wave source and the ultrasonic probe. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. When the device is used, upon receiving a stone positioning command, it performs coordinate system transformation processing based on the real-time pose information of the first and second positioning tools in the coordinate system of the infrared optical camera, and the ultrasonic image data containing the target stone, to determine the target pose that the end of the collaborative robotic arm needs to move to so that the focus of the shock wave source is aligned with the target stone position. The technical solution of this application combines real-time spatial positioning with infrared optics, two-dimensional ultrasound image guidance, and automatic control with a collaborative robotic arm to achieve fully automatic and high-precision coordinate transformation and alignment from the two-dimensional position of the stone selected in the ultrasound image to the three-dimensional spatial pose required for the focus of the shock wave source. This solves the problems of conventional ultrasound positioning technology, which relies on manual operation and spatial imagination by doctors, is cumbersome and time-consuming, and is difficult to achieve fast and accurate three-dimensional alignment. It improves the automation level, operational efficiency, and positioning accuracy of stone positioning.

[0115] Based on the above-mentioned device, an optional stone positioning device includes:

[0116] The global transformation determination module is used to determine the first global coordinate transformation relationship between the infrared optical camera coordinate system and the base coordinate system of the collaborative robotic arm based on the first real-time pose information of the first positioning tool in the infrared optical camera coordinate system, the real-time pose feedback information of the end of the collaborative robotic arm, and the fixed installation relationship between the first positioning tool, the shock wave source and the end of the robotic arm.

[0117] The coordinate mapping determination module is used to determine a coordinate mapping relationship for mapping the position in the ultrasonic image coordinate system to the infrared optical camera coordinate system based on the second real-time pose information of the second positioning tool in the infrared optical camera coordinate system, the fixed installation relationship between the second positioning tool and the ultrasonic probe, and the preset imaging relationship between the ultrasonic image coordinate system and the probe coordinate system.

[0118] The target pose determination module is used to determine the target pose that the end effector of the collaborative robotic arm needs to move to so that the focus of the shock wave source is aligned with the target stone position, based on the two-dimensional position of the target stone in the ultrasound image coordinate system, the coordinate mapping relationship, and the first global coordinate transformation relationship.

[0119] Based on the above-mentioned device, optionally, a global transformation determination module is used to determine, based on the real-time pose feedback information of the end effector of the collaborative robotic arm, a first pose to be processed in the base coordinate system of the robotic arm end effector coordinate system; based on the fixed installation relationship between the first positioning tool, the shock wave source and the end effector of the robotic arm, a first fixed transformation relationship between the coordinate system of the first positioning tool and the coordinate system of the robotic arm end effector coordinate system; based on the first real-time pose information, a second pose to be processed in the coordinate system of the first positioning tool and the infrared optical camera coordinate system; and based on the first pose to be processed, the second pose to be processed and the first fixed transformation relationship, a first global coordinate transformation relationship between the coordinate system of the infrared optical camera and the base coordinate system of the collaborative robotic arm is determined through chained coordinate transformation.

[0120] Based on the above-mentioned device, optionally, a coordinate mapping determination module is used to determine, based on the second real-time pose information, a third pose to be processed in the infrared optical camera coordinate system of the second positioning tool coordinate system; based on the fixed installation relationship between the second positioning tool and the ultrasonic probe, a second fixed transformation relationship between the second positioning tool coordinate system and the probe coordinate system; based on the preset imaging relationship between the probe coordinate system and the ultrasonic image coordinate system, a third fixed transformation relationship between the ultrasonic image coordinate system and the probe coordinate system; and based on the third pose to be processed, the second fixed transformation relationship, and the third fixed transformation relationship, a coordinate mapping relationship for mapping the position in the ultrasonic image coordinate system to the infrared optical camera coordinate system is determined through chained coordinate transformation.

[0121] Based on the above device, optionally, a target pose determination module is used to receive the user's marking operation on the location of the target stone in the real-time image interface of the ultrasound probe; and to determine the two-dimensional position of the target stone in the ultrasound image coordinate system based on the image pixel coordinates corresponding to the marking operation and the image calibration parameters of the ultrasound probe.

[0122] Based on the above-mentioned device, optionally, the target pose determination module is further used to determine the three-dimensional coordinates of the target stone in the infrared optical camera coordinate system based on the two-dimensional position of the target stone in the ultrasound image coordinate system and the coordinate mapping relationship; to determine the three-dimensional target position of the target stone in the base coordinate system of the collaborative robotic arm based on the three-dimensional coordinates of the stone and the first global coordinate transformation relationship; and to calculate the target pose that the end of the collaborative robotic arm needs to move to so that the focus of the shock wave source coincides with the three-dimensional target position based on the three-dimensional target position and the known installation position relationship of the shock wave source relative to the end of the collaborative robotic arm.

[0123] Optionally, based on the above-mentioned device, the device further includes: a real-time pose acquisition module, used to capture in real time the spatial arrangement data of preset reflective markers on the first and second positioning tools through the infrared optical camera; and to match the captured spatial arrangement data with a pre-stored three-dimensional model of the positioning tool to calculate the real-time pose information of the first and second positioning tools in the coordinate system of the infrared optical camera.

[0124] The stone positioning device provided in this application embodiment can execute the stone positioning method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of the method execution.

[0125] It is worth noting that the various units and modules included in the above system are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this application.

[0126] Example 4

[0127] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 A block diagram is shown of an exemplary electronic device 40 suitable for implementing embodiments of the present application. Figure 4 The electronic device 40 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0128] like Figure 4 As shown, electronic device 40 is represented in the form of a general-purpose computing device. The components of electronic device 40 may include, but are not limited to: one or more processors or processing units 401, system memory 402, and bus 403 connecting different system components (including system memory 402 and processing unit 401).

[0129] Bus 403 represents one or more of several bus architectures, including memory buses or memory electronics, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0130] Electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 40, including volatile and non-volatile media, removable and non-removable media.

[0131] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown (usually referred to as a hard drive). Although Figure 4 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a floppy disk) and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0132] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402. Such program modules 407 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 407 typically perform the functions and / or methods described in the embodiments of this application.

[0133] Electronic device 40 can also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), and with one or more devices that enable a user to interact with electronic device 40, and / or with any device that enables electronic device 40 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 411. Furthermore, electronic device 40 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 412. As shown, network adapter 412 communicates with other modules of electronic device 40 via bus 403. It should be understood that, although... Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0134] The processing unit 401 executes various functional applications and page processing by running programs stored in the system memory 402, such as implementing the stone location method provided in the embodiments of this application.

[0135] Example 5

[0136] This application embodiment also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a stone localization method. The stone localization aiming system includes a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm. A first positioning tool and a second positioning tool, observed by the infrared optical camera, are respectively fixedly mounted on the shock wave source and the ultrasonic probe. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. The method includes:

[0137] When a stone location command is received, based on the real-time pose information of the first and second positioning tools in the infrared optical camera coordinate system, and the ultrasound image data containing the target stone, coordinate system transformation processing is performed to determine the target pose that the end effector of the collaborative robotic arm needs to move to so that the focus of the shock wave source is aligned with the target stone position.

[0138] The computer storage medium in this application embodiment can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0139] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0140] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0141] Computer program code for performing the operations of the embodiments of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0142] Note that the above description is merely a preferred embodiment and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, and the scope of this application is determined by the scope of the appended claims.

Claims

1. A method for locating kidney stones, applied to a kidney stone location and aiming system, the kidney stone location and aiming system comprising a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm, characterized in that, A first positioning tool and a second positioning tool, respectively, are fixedly mounted on the shock wave source and the ultrasonic probe, and are observed by the infrared optical camera. The field of view of the infrared optical camera covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. The method includes: When a stone location command is received, based on the first real-time pose information of the first positioning tool in the infrared optical camera coordinate system, the real-time pose feedback information of the end of the collaborative robotic arm, and the fixed installation relationship between the first positioning tool, the shock wave source and the end of the robotic arm, the first global coordinate transformation relationship of the infrared optical camera coordinate system relative to the base coordinate system of the collaborative robotic arm is determined. Based on the second real-time pose information of the second positioning tool in the infrared optical camera coordinate system, the fixed installation relationship between the second positioning tool and the ultrasonic probe, and the preset imaging relationship between the ultrasonic image coordinate system and the probe coordinate system, a coordinate mapping relationship for mapping the position in the ultrasonic image coordinate system to the infrared optical camera coordinate system is determined. Based on the two-dimensional position of the target stone in the ultrasound image coordinate system, the coordinate mapping relationship, and the first global coordinate transformation relationship, the target pose that the end effector of the collaborative robotic arm needs to move to in order to align the focus of the shock wave source with the target stone position is determined.

2. The method according to claim 1, characterized in that, The step of determining the first global coordinate transformation relationship between the infrared optical camera coordinate system and the base coordinate system of the collaborative robotic arm, based on the first real-time pose information of the first positioning tool in the infrared optical camera coordinate system, the real-time pose feedback information of the end effector of the collaborative robotic arm, and the fixed installation relationship between the first positioning tool, the shock wave source, and the end effector of the robotic arm, includes: Based on the real-time pose feedback information of the end effector of the collaborative robotic arm, the first pose to be processed in the coordinate system of the end effector of the robotic arm under the coordinate system of the base is determined; Based on the fixed installation relationship between the first positioning tool, the shock wave source and the end effector of the robotic arm, a first fixed transformation relationship between the coordinate system of the first positioning tool and the coordinate system of the end effector of the robotic arm is determined. Based on the first real-time pose information, determine the second pose to be processed in the coordinate system of the first positioning tool under the coordinate system of the infrared optical camera. Based on the first pose to be processed, the second pose to be processed, and the first fixed transformation relationship, the first global coordinate transformation relationship between the infrared optical camera coordinate system and the base coordinate system of the cooperative robotic arm is determined through chain-like coordinate transformation.

3. The method according to claim 1, characterized in that, The step of determining a coordinate mapping relationship for mapping the position in the ultrasound image coordinate system to the infrared optical camera coordinate system based on the second real-time pose information of the second positioning tool in the infrared optical camera coordinate system, the fixed installation relationship between the second positioning tool and the ultrasound probe, and the preset imaging relationship between the ultrasound image coordinate system and the probe coordinate system includes: Based on the second real-time pose information, determine the third pose to be processed in the coordinate system of the second positioning tool under the coordinate system of the infrared optical camera. Based on the fixed installation relationship between the second positioning tool and the ultrasonic probe, a second fixed transformation relationship between the coordinate system of the second positioning tool and the coordinate system of the probe is determined; Based on the preset imaging relationship between the probe coordinate system and the ultrasound image coordinate system, a third fixed transformation relationship between the ultrasound image coordinate system and the probe coordinate system is determined. Based on the third pose to be processed, the second fixed transformation relationship, and the third fixed transformation relationship, a coordinate mapping relationship for mapping the position in the ultrasound image coordinate system to the infrared optical camera coordinate system is determined through chain-like coordinate transformation.

4. The method according to claim 1, characterized in that, The method further includes: Receives the user's marking operation on the location of the target stone in the real-time image interface of the ultrasound probe; Based on the image pixel coordinates corresponding to the marking operation and the image calibration parameters of the ultrasound probe, the two-dimensional position of the target stone in the ultrasound image coordinate system is determined.

5. The method according to claim 1, characterized in that, The determination of the target pose to which the end effector of the collaborative robotic arm needs to move to align the focus of the shock wave source with the target stone position, based on the two-dimensional position of the target stone in the ultrasound image coordinate system, the coordinate mapping relationship, and the first global coordinate transformation relationship, includes: Based on the two-dimensional position of the target stone in the ultrasound image coordinate system and the coordinate mapping relationship, the three-dimensional coordinates of the target stone in the infrared optical camera coordinate system are determined. Based on the three-dimensional coordinates of the stone and the first global coordinate transformation relationship, the three-dimensional target position of the target stone in the base coordinate system of the collaborative robotic arm is determined; Based on the known installation position relationship between the three-dimensional target position and the shock wave source relative to the end effector of the collaborative robotic arm, the target pose that the end effector of the collaborative robotic arm needs to move to in order to make the focus of the shock wave source coincide with the three-dimensional target position is calculated.

6. The method according to claim 1, characterized in that, Also includes: Acquiring the real-time pose information of the first and second positioning tools in the infrared optical camera coordinate system includes: The infrared optical camera captures in real time the spatial arrangement data of preset reflective markers on the first and second positioning tools. The captured spatial arrangement data is matched with the pre-stored 3D model of the positioning tool to calculate the real-time pose information of the first and second positioning tools in the coordinate system of the infrared optical camera.

7. A stone positioning device, characterized in that, An application in a stone localization and aiming system, the stone localization and aiming system comprising a collaborative robotic arm, an infrared optical camera, and a shock wave source and an ultrasonic probe fixed to the end of the collaborative robotic arm, characterized in that a first positioning tool and a second positioning tool, respectively, are fixedly mounted on the shock wave source and the ultrasonic probe, and the infrared optical camera's field of view covers the working area of ​​the collaborative robotic arm, the shock wave source, and the ultrasonic probe. The device includes: The global transformation determination module is used to determine the first global coordinate transformation relationship between the infrared optical camera coordinate system and the base coordinate system of the collaborative robotic arm when a stone positioning command is received, based on the first real-time pose information of the first positioning tool in the infrared optical camera coordinate system, the real-time pose feedback information of the end of the collaborative robotic arm, and the fixed installation relationship between the first positioning tool, the shock wave source and the end of the robotic arm. The coordinate mapping determination module is used to determine a coordinate mapping relationship for mapping the position in the ultrasonic image coordinate system to the infrared optical camera coordinate system based on the second real-time pose information of the second positioning tool in the infrared optical camera coordinate system, the fixed installation relationship between the second positioning tool and the ultrasonic probe, and the preset imaging relationship between the ultrasonic image coordinate system and the probe coordinate system. The target pose determination module is used to determine the target pose that the end effector of the collaborative robotic arm needs to move to so that the focus of the shock wave source is aligned with the target stone position, based on the two-dimensional position of the target stone in the ultrasound image coordinate system, the coordinate mapping relationship, and the first global coordinate transformation relationship.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the stone localization method as described in any one of claims 1-6.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the stone localization method as described in any one of claims 1-6.

Images