Remote image acquisition method and related products

Abstract

The present application discloses a remote image acquisition method and related products, the remote image acquisition method is applied to a control device, the control device is used to remotely control the image acquisition device to acquire images, the image acquisition device is fixed on a robotic arm, and the control device drives the image acquisition device to move by controlling the movement of the robotic arm, the method comprising: obtaining a first posture of the image acquisition device in a base coordinate system of the robotic arm, the base coordinate system being a coordinate system constructed with the base of the robotic arm; obtaining a first adjustment amount for the posture of the image acquisition device; adjusting the first posture based on the first adjustment amount to obtain a second posture of the image acquisition device in the base coordinate system; obtaining a first conversion relationship, the first conversion relationship being a conversion relationship between the posture of the image acquisition device and the posture of a reference structure, the reference structure including a structure on the robotic arm other than the base; and using the first conversion relationship to convert the second posture into a third posture of the reference structure in the base coordinate system.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a remote image acquisition method and related products. Background Art

[0002] In the medical field, remote surgery is often required, and remote surgery involves remote control of image acquisition. Therefore, how to perform remote image acquisition is of great significance. Summary of the Invention

[0003] The present application provides a remote image acquisition method and related products for remote image acquisition, wherein the related products include a remote image acquisition puncture device, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, a remote image acquisition method is provided. The method is applied to a control device, the control device being used to remotely control an image acquisition device to acquire images. The image acquisition device is fixed to a robotic arm, and the control device drives the image acquisition device to move by controlling the movement of the robotic arm. The method includes:

[0005] Acquire a first pose of the image acquisition device in a base coordinate system of the robotic arm, where the base coordinate system is a coordinate system constructed based on the base of the robotic arm;

[0006] Acquire a first adjustment amount of the posture of the image acquisition device;

[0007] Adjusting the first posture based on the first adjustment amount to obtain a second posture of the image acquisition device in the base coordinate system;

[0008] Acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between a posture of the image acquisition device and a posture of a reference structure, where the reference structure includes a structure on the robotic arm excluding the base;

[0009] The second posture is converted into a third posture of the reference structure in the base coordinate system by using the first conversion relationship.

[0010] In combination with any embodiment of the present application, after converting the second posture into a third posture of the reference structure in the base coordinate system using the first transformation relationship, the method further includes:

[0011] The movement of the robotic arm is controlled by the third posture so that the posture of the reference structure is the third posture.

[0012] In conjunction with any embodiment of the present application, the image captured by the image acquisition device includes an ultrasonic image, and after controlling the movement of the robotic arm through the third posture so that the posture of the reference structure is the third posture, the method includes:

[0013] receiving a target ultrasound image acquired by the image acquisition device in the third posture, wherein the target ultrasound image includes the object to be punctured;

[0014] A puncture path of the object to be punctured is determined based on the target ultrasound image.

[0015] In combination with any embodiment of the present application, the first posture is a posture at a first moment, and obtaining a first adjustment amount of the posture of the image acquisition device includes:

[0016] Acquiring a fourth posture of the control device at the first moment, the control device being used to remotely adjust the posture of the image acquisition device;

[0017] Obtaining a fifth posture of the control device at a second moment, where the second moment is a moment later than the first moment;

[0018] The first adjustment amount is obtained based on a difference between the fifth posture and the fourth posture.

[0019] In combination with any embodiment of the present application, the first adjustment amount includes a first rotation amount and a first translation amount, and using the first transformation relationship to transform the second posture into a third posture of the reference structure in the base coordinate system includes:

[0020] Decomposing the second posture into a second rotation and a second translation;

[0021] adjusting the second rotation amount using the first rotation amount to obtain a third rotation amount;

[0022] adjusting the second translation amount using the first translation amount to obtain a third translation amount;

[0023] The third posture is obtained based on the third rotation amount and the third translation amount.

[0024] In combination with any embodiment of the present application, the base coordinate system includes a horizontal axis, a vertical axis, and a vertical axis, and the first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount, and a first vertical rotation amount, where the first horizontal rotation amount is a component of the first rotation amount on the horizontal axis, the first vertical rotation amount is a component of the first rotation amount on the vertical axis, and the first vertical rotation amount is a component of the first rotation amount on the vertical axis;

[0025] The adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount includes:

[0026] determining a component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount;

[0027] determining a component of the second rotation amount on the longitudinal axis to obtain a second longitudinal rotation amount;

[0028] determining a component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount;

[0029] adjusting the second lateral rotation amount by using the first lateral rotation amount to obtain a third lateral rotation amount;

[0030] adjusting the second longitudinal rotation amount by using the first longitudinal rotation amount to obtain a third longitudinal rotation amount;

[0031] adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount;

[0032] The third rotation amount is obtained based on the third horizontal rotation amount, the third vertical rotation amount, and the third vertical rotation amount.

[0033] In a second aspect, a control device is provided, the control device being used to remotely control an image acquisition device to acquire images, the image acquisition device being fixed to a robotic arm, the control device driving the image acquisition device to move by controlling the movement of the robotic arm, the control device comprising:

[0034] an acquisition unit, configured to acquire a first pose of the image acquisition device in a base coordinate system of the robotic arm, wherein the base coordinate system is a coordinate system constructed based on the base of the robotic arm;

[0035] The acquisition unit is further configured to acquire a first adjustment value of the posture of the image acquisition device;

[0036] an adjusting unit, configured to adjust the first posture to obtain a second posture of the image acquisition device in the base coordinate system based on the first adjustment amount;

[0037] The acquisition unit is further configured to acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between a posture of the image acquisition device and a posture of a reference structure, where the reference structure includes a structure on the robotic arm excluding the base;

[0038] A conversion unit is used to convert the second posture into a third posture of the reference structure in the base coordinate system by using the first conversion relationship.

[0039] In combination with any embodiment of the present application, the control device further includes:

[0040] A control unit is used to control the movement of the robotic arm through the third posture so that the posture of the reference structure is the third posture.

[0041] In combination with any embodiment of the present application, the image captured by the image acquisition device includes an ultrasound image, and the control device further includes: a receiving unit for receiving a target ultrasound image captured by the image acquisition device in the third posture, wherein the target ultrasound image includes the object to be punctured;

[0042] A determining unit is configured to determine a puncture path of the object to be punctured based on the target ultrasound image.

[0043] In combination with any embodiment of the present application, the acquiring unit is configured to:

[0044] Acquiring a fourth posture of the control device at the first moment, the control device being used to remotely adjust the posture of the image acquisition device;

[0045] Obtaining a fifth posture of the control device at a second moment, where the second moment is a moment later than the first moment;

[0046] The first adjustment amount is obtained based on a difference between the fifth posture and the fourth posture.

[0047] In combination with any embodiment of the present application, the first adjustment amount includes a first rotation amount and a first translation amount, and the conversion unit is configured to:

[0048] Decomposing the second posture into a second rotation and a second translation;

[0049] adjusting the second rotation amount using the first rotation amount to obtain a third rotation amount;

[0050] adjusting the second translation amount using the first translation amount to obtain a third translation amount;

[0051] The third posture is obtained based on the third rotation amount and the third translation amount.

[0052] In combination with any embodiment of the present application, the base coordinate system includes a horizontal axis, a vertical axis, and a vertical axis, and the first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount, and a first vertical rotation amount, where the first horizontal rotation amount is a component of the first rotation amount on the horizontal axis, the first vertical rotation amount is a component of the first rotation amount on the vertical axis, and the first vertical rotation amount is a component of the first rotation amount on the vertical axis;

[0053] The conversion unit is used to:

[0054] determining a component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount;

[0055] determining a component of the second rotation amount on the longitudinal axis to obtain a second longitudinal rotation amount;

[0056] determining a component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount;

[0057] adjusting the second lateral rotation amount by using the first lateral rotation amount to obtain a third lateral rotation amount;

[0058] adjusting the second longitudinal rotation amount by using the first longitudinal rotation amount to obtain a third longitudinal rotation amount;

[0059] adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount;

[0060] The third rotation amount is obtained based on the third horizontal rotation amount, the third vertical rotation amount, and the third vertical rotation amount.

[0061] In a third aspect, an electronic device is provided, comprising: a processor and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation method thereof.

[0062] In a fourth aspect, another electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation method thereof.

[0063] In a fifth aspect, a computer-readable storage medium is provided, in which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method as described in the first aspect above and any possible implementation method thereof.

[0064] In a sixth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is enabled to execute the method of the above-mentioned first aspect and any possible implementation thereof.

[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0066] In an embodiment of the present application, an image acquisition device is fixed on a robotic arm, wherein a control device drives the image acquisition device to move by controlling the movement of the robotic arm. After the control device obtains a first posture of the image acquisition device in the base coordinate system of the robotic arm, it obtains a first adjustment amount of the posture of the image acquisition device. Then, based on the first adjustment amount, the first posture is adjusted to obtain a second posture of the image acquisition device in the base coordinate system. Then, a first conversion relationship is obtained, wherein the first conversion relationship is a conversion relationship between the posture of the image acquisition device and the posture of a reference structure, and the reference structure is the structure of the robotic arm. Finally, the second posture is converted into a third posture of the reference structure in the base coordinate system using the first conversion relationship, so that the first adjustment amount of the posture of the image acquisition device can be converted into the posture of the reference structure of the robotic arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0068] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0069] Figure 1 A flowchart of a remote image acquisition method provided in an embodiment of the present application;

[0070] Figure 2 A schematic diagram of a scenario in which an image acquisition device is controlled by a joystick to capture an image, provided in an embodiment of the present application;

[0071] Figure 3a A schematic diagram of the horizontal and vertical axes of a flange coordinate system provided in an embodiment of the present application;

[0072] Figure 3b A schematic diagram of the longitudinal axis and vertical axis of a flange coordinate system provided in an embodiment of the present application;

[0073] Figure 4 A schematic diagram of a puncture scenario based on an ultrasound probe provided in an embodiment of the present application;

[0074] Figure 5 A schematic structural diagram of a control device provided in an embodiment of the present application;

[0075] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0077] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0078] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, that the embodiments described herein may be combined with other embodiments. It should be understood that, in this application, "at least one (item)" means one or more, "a plurality" means two or more, and "at least two (items)" means two or three or more.

[0079] An embodiment of the present application provides a remote image acquisition method, which is applied to a control device. The control device is configured to remotely control an image acquisition device to acquire images. The image acquisition device is fixed to a robotic arm, and the control device drives the image acquisition device by controlling the movement of the robotic arm. By controlling the movement of the robotic arm, the image acquisition device can be driven to move, thereby adjusting the shooting angle of the image acquisition device and, therefore, the content of the images captured by the image acquisition device.

[0080] In one possible implementation scenario, the image acquisition device includes an ultrasound probe, wherein the ultrasound probe is used to acquire ultrasound images. The ultrasound probe is fixed to a robotic arm, and the robotic arm can be deployed in an operating room. The control device can be any electronic device. Optionally, the control device includes one of the following: a mobile phone, a computer, or a tablet computer. The control device can be deployed outside the operating room. In this implementation scenario, a user can control the image acquisition device in the operating room to acquire images through the control device outside the operating room, thereby realizing remote image acquisition. For example, a doctor can control the image acquisition device in the operating room to acquire images through the control device outside the operating room. In the embodiment of the present application, the execution entity of the remote image acquisition method is the above-mentioned control device.

[0081] It should be understood that the method embodiment of the present application can also be implemented by a processor executing computer program code. The following describes the embodiment of the present application in conjunction with the drawings in the embodiment of the present application. Figure 1 , Figure 1 A flowchart of a remote image acquisition method provided in an embodiment of the present application.

[0082] 101. Obtain a first position of the image acquisition device in the base coordinate system of the robotic arm.

[0083] In the embodiments of the present application, the base coordinate system is a coordinate system constructed using the base of the robotic arm. The first pose is the pose of the image acquisition device in the base coordinate system. Optionally, the pose in the embodiments of the present application includes position and pose, with the first pose including the position and pose of the image acquisition device in the base coordinate system.

[0084] In an implementation of obtaining the first posture, a control device receives the first posture input by the user through an input component to obtain the first posture, wherein the input component includes: a mouse, a keyboard, a touch screen, a touchpad, and an audio input device.

[0085] In another implementation method of obtaining the first posture, the control device receives the first posture sent by the user through the terminal to obtain the first posture, wherein the terminal includes: a mobile phone, a computer, a tablet computer, and a smart wearable device.

[0086] In another implementation method of obtaining the first pose, the control device includes a positioning imaging device, wherein the positioning imaging device includes an imaging device, and the positioning imaging device is used to determine the first pose. Optionally, the positioning imaging device is a laser radar. By scanning the image acquisition device through the laser radar, a reference pose of the image acquisition device in the world coordinate system can be obtained. Then, based on the conversion relationship between the world coordinate system and the base coordinate system, the reference pose is converted to obtain the first pose, wherein the conversion relationship includes a rotation relationship between the world coordinate system and the base coordinate system, and a translation relationship between the world coordinate system and the base coordinate system. After rotating the world coordinate system based on the rotation relationship and translating the world coordinate system based on the translation relationship, the world coordinate system can be aligned with the base coordinate system.

[0087] In one implementation scenario based on this implementation, the control device is deployed outside the operating room, and the image acquisition device is deployed inside the operating room. Through this implementation, the position and posture of the image acquisition device in the operating room can be obtained outside the operating room.

[0088] Optionally, the positioning imaging device is a two-dimensional imaging device. The two-dimensional imaging device captures a two-dimensional scene image including the image acquisition device. The two-dimensional scene image is then input into a pose estimation model, so that the pose estimation model estimates the pose of the image acquisition device in the two-dimensional scene image in the base coordinate system to obtain a first pose. The pose estimation model is configured to estimate the pose of an object in the two-dimensional image in the base coordinate system based on the two-dimensional image.

[0089] 102. Obtain a first adjustment amount of the posture of the image acquisition device.

[0090] In an embodiment of the present application, the first adjustment amount is the adjustment amount of the posture of the image acquisition device. Adjusting the posture of the image acquisition device based on the first adjustment amount can change the posture of the image acquisition device, and then change the shooting angle and shooting range of the image acquisition device, thereby changing the content of the image captured by the image acquisition device.

[0091] In one possible implementation, the control device includes a joystick, wherein the joystick is used to adjust the posture of the image acquisition device. Specifically, the posture of the image acquisition device can be changed by adjusting the posture of the joystick. A user can input a first adjustment amount of the posture of the image acquisition device to the control device by moving the joystick. Optionally, the adjustment amount of the posture of the image acquisition device is based on the adjustment amount of the posture of the joystick. Figure 2 , Figure 2 A schematic diagram of a scene in which an image acquisition device is controlled by a joystick to capture an image is provided in an embodiment of the present application. Figure 2As shown, the image acquisition device is fixed on the robotic arm, and the operating rod can drive the image acquisition device to move by controlling the movement of the robotic arm, thereby controlling the image acquisition device to capture images of the object to be captured, wherein the image acquisition device can be an ultrasound probe and the object to be captured can be a patient. It should be understood that Figure 2 The joystick and the robotic arm are not fixedly connected. In actual applications, the joystick and the robotic arm can be deployed in different spaces respectively. For example, the joystick can be deployed outside the operating room and the robotic arm can be deployed inside the operating room.

[0092] 103. Based on the first adjustment amount, adjust the first posture to obtain a second posture of the image acquisition device in the base coordinate system.

[0093] As described in step 102, the first adjustment amount is an adjustment amount for the posture of the image acquisition device. Therefore, the control device can adjust the first posture of the image acquisition device in the base coordinate system based on the first adjustment amount. After adjustment, the posture of the image acquisition device in the base coordinate system is a second posture. In one possible implementation, the control device obtains the second posture by determining the sum of the first posture and the first adjustment amount. In another possible implementation, after adjusting the first posture based on the first adjustment amount to obtain the second posture, the control device further plans a reference path for adjusting the posture of the image acquisition device from the first posture to the second posture based on a three-dimensional scene image. The three-dimensional scene image is a three-dimensional image of the scene in which the robotic arm and the image acquisition device are located, and the three-dimensional scene image is captured by the control device. The reference path is the motion path of the robotic arm. Movement of the robotic arm according to the reference path can adjust the posture of the image acquisition device on the robotic arm from the first posture to the second posture. In this implementation, since the scene in which the robotic arm and the image acquisition device are located is an operating room, where people or objects may be present, the motion path of the robotic arm should avoid people or objects. The control device is located outside the operating room. Therefore, by capturing a 3D image of the scene within the operating room (i.e., the scene where the robotic arm and image acquisition device are located), the control device can obtain more comprehensive and accurate information about the scene within the operating room. Based on this 3D scene image, a reference path is then planned. When the robotic arm moves along the reference path, the image acquisition device's position changes from a first position to a second position. This reduces the chance of collisions between the robotic arm and image acquisition device and people or objects, thereby improving safety.

[0094] 104. Obtain a first conversion relationship.

[0095] In an embodiment of the present application, the first conversion relationship is a conversion relationship between the posture of the image acquisition device and the posture of the reference structure, that is, the posture of the image acquisition device can be converted into the posture of the reference structure using the first conversion relationship. For example, when it is known that the posture of the image acquisition device is posture 1, the posture 1 is converted using the first conversion relationship to obtain the posture of the reference structure. The reference structure is the structure of the robotic arm. Optionally, the reference structure is the structure of the robotic arm excluding the base. Optionally, the reference structure is the structure on the robotic arm connected to the image acquisition device. For example, if the image acquisition device is fixed on the flange of the robotic arm, then the reference structure is the flange.

[0096] Optionally, when the reference structure is a flange of a robotic arm, the first transformation relationship is a transformation relationship between a base coordinate system and a flange coordinate system, wherein the flange coordinate system is a coordinate system constructed based on the flange. Figure 3a A schematic diagram of the horizontal and vertical axes of a flange coordinate system provided in an embodiment of the present application, Figure 3b This is a schematic diagram of the longitudinal axis and vertical axis of a flange coordinate system provided in an embodiment of the present application. Figure 3a and Figure 3b As shown, the flange includes a first end face and a second end face, wherein the first end face is the end face in contact with the image acquisition device. The origin of the flange coordinate system (i.e. Figure 3a O1 and Figure 3b O1 in the figure is the center of the first end face. Figure 3a The X1 axis in the figure) and the vertical axis (i.e. Figure 3a The Y1 axis and Figure 3b The Y1 axis in the figure is parallel to the radial direction of the first end face, and the horizontal axis is perpendicular to the vertical axis. Figure 3b The Z1 axis in FIG is perpendicular to the first end face.

[0097] When the image acquisition device is fixed on the flange, the first conversion relationship can be determined based on the structural relationship between the image acquisition device and the flange. At this time, the first conversion relationship can be used to convert the posture of the image acquisition device in the base coordinate system into the posture of the flange in the flange coordinate system. Based on the first conversion relationship, the robotic arm can then convert the posture of the flange in the flange coordinate system into the posture of the flange in the base coordinate system. For example, Flange is used to represent the first conversion relationship, where Flange is a matrix, and based on the first conversion relationship, the posture of the image acquisition device in the base coordinate system can be converted into the posture of the flange in the flange coordinate system. Then based on Flange -1 The position of the flange in the flange coordinate system can be converted to the position of the flange in the base coordinate system, where Flange -1 is the inverse matrix of Flange.

[0098] Since the structure that the robotic arm can control is the structure of the robotic arm, and it cannot control objects other than the structure of the robotic arm, in other words, the robotic arm can control the position and posture of the robotic arm structure, but cannot control the position and posture of objects fixed to the robotic arm. The reference structure is the structure of the robotic arm, and the image acquisition device is not the structure of the robotic arm. Therefore, the robotic arm can control the position and posture of the reference structure, but cannot control the position and posture of the image acquisition device. It should be understood that the robotic arm controlling the position and posture of the robotic arm structure can be understood as the robotic arm including a central processing unit (CPU), which can control the structure of the robotic arm. Specifically, the CPU can adjust the position and posture of the robotic arm structure in the base coordinate system of the robotic arm by controlling the movement of the robotic arm. For example, the CPU of the robotic arm can adjust the position and posture of the reference structure of the robotic arm in the base coordinate system by controlling the movement of each joint of the robotic arm.

[0099] 105. Use the first transformation relationship to transform the second posture into a third posture of the reference structure in the base coordinate system.

[0100] Because the first transformation relationship is the transformation relationship between the image acquisition device's pose and the reference structure's pose, the control device can use the first transformation relationship to transform the image acquisition device's pose in the base coordinate system into the reference structure's pose in the base coordinate system. This means transforming the second pose into the reference structure's pose in the base coordinate system, which is the third pose. As described in step 104, the robotic arm can only control the structure of the robotic arm and cannot control objects other than the robotic arm's structure. Therefore, the control device obtains the third pose by executing step 105. The robotic arm can adjust the pose of the image acquisition device to the second pose by adjusting the reference structure's pose to the third pose.

[0101] In an embodiment of the present application, an image acquisition device is fixed on a robotic arm, wherein a control device drives the image acquisition device to move by controlling the movement of the robotic arm. After the control device obtains a first posture of the image acquisition device in the base coordinate system of the robotic arm, it obtains a first adjustment amount of the posture of the image acquisition device. Then, based on the first adjustment amount, the first posture is adjusted to obtain a second posture of the image acquisition device in the base coordinate system. Then, a first conversion relationship is obtained, wherein the first conversion relationship is a conversion relationship between the posture of the image acquisition device and the posture of a reference structure, and the reference structure is the structure of the robotic arm. Finally, the second posture is converted into a third posture of the reference structure in the base coordinate system using the first conversion relationship, so that the first adjustment amount of the posture of the image acquisition device can be converted into the posture of the reference structure of the robotic arm.

[0102] As an optional embodiment, after the control device converts the second posture into the third posture of the reference structure in the base coordinate system using the first conversion relationship, the control device controls the movement of the manipulator through the third posture so that the posture of the reference structure is the third posture. Optionally, after obtaining the third posture, the control device sends a control instruction carrying the third posture to the manipulator, wherein the control instruction is used to instruct the manipulator to adjust the posture of the reference structure to the third posture. Because when the posture of the reference structure in the base coordinate system is the third posture, the posture of the image acquisition device in the base coordinate system is the second posture, the control device can adjust the posture of the image acquisition device to the second posture based on the first adjustment amount by sending a control instruction carrying the third posture to the manipulator.

[0103] In one possible implementation scenario, the image acquisition device includes an ultrasound probe, which is used to acquire ultrasound images. The ultrasound probe is fixed to a robotic arm, which can be deployed within an operating room, and a control device is deployed outside the operating room. A user, through the control device, can send a control command containing a third posture to the robotic arm, thereby adjusting the posture of the ultrasound probe in the operating room to a second posture from outside the operating room.

[0104] As an optional embodiment, the image captured by the above-mentioned image acquisition device includes an ultrasonic image. After the control device controls the movement of the above-mentioned robotic arm through the above-mentioned third posture so that the posture of the above-mentioned reference structure is the above-mentioned third posture, the control device further performs the following steps: receiving the target ultrasonic image captured by the above-mentioned image acquisition device in the above-mentioned third posture, the above-mentioned target ultrasonic image includes the object to be punctured; and determining the puncture path of the above-mentioned object to be punctured based on the above-mentioned target ultrasonic image.

[0105] In one possible implementation scenario, the image acquisition device includes an ultrasound probe, which is used to acquire ultrasound images. The ultrasound probe is fixed to a robotic arm, which can be deployed within an operating room, and a control device is deployed outside the operating room. A user, through the control device, can send a control command containing a third posture to the robotic arm, thereby adjusting the posture of the ultrasound probe inside the operating room to a second posture outside the operating room. After the ultrasound probe acquires an ultrasound image of the patient, a puncture path for the patient can be determined based on the ultrasound image, and then a needle insertion device can be controlled based on the puncture path to perform puncture on the patient.

[0106] See also Figure 4 , Figure 4 A schematic diagram of a puncture scenario based on an ultrasound probe provided in an embodiment of the present application. Figure 4As shown, the scene includes an ultrasound probe, a robotic arm, and a needle insertion device, wherein both the ultrasound probe and the needle insertion device are fixed to the robotic arm. The ultrasound probe is used to scan and acquire ultrasound images of the patient, and the needle insertion device is used to control the puncture needle for puncture.

[0107] like Figure 4 As shown, the scanning area of ​​the ultrasound probe is a sector-shaped area. When the ultrasound probe scans the patient, it can image the information in the scanning area to obtain an ultrasound image. After obtaining the ultrasound image, the needle insertion point and the patient's lesion can be determined from the ultrasound image. The position of the needle insertion point in the ultrasound image is then converted to a position in the world coordinate system, and the position of the lesion in the ultrasound image is converted to a position in the world coordinate system, where Figure 4 The O2X2Y2Z2 diagram is the world coordinate system. Specifically, O2 is the origin of the world coordinate system, X2 is the horizontal axis, Y2 is the vertical axis, and Z2 is the vertical axis. Based on the position of the needle insertion point and the lesion in the world coordinate system, a puncture path for the patient's lesion can be determined. Based on this puncture path, the robotic arm is controlled to drive the needle insertion device, allowing the device to control the puncture needle along the puncture path.

[0108] As an optional embodiment, the above-mentioned first posture is the posture at the first moment, and the control device obtains the first adjustment amount of the posture of the image acquisition device by executing the following steps: obtaining the fourth posture of the control device at the above-mentioned first moment, and the above-mentioned control device is used to remotely adjust the posture of the above-mentioned image acquisition device; obtaining the fifth posture of the above-mentioned control device at the second moment, and the above-mentioned second moment is a moment later than the above-mentioned first moment; based on the difference between the above-mentioned fifth posture and the above-mentioned fourth posture, the above-mentioned first adjustment amount is obtained.

[0109] In this embodiment, the control device is used to remotely adjust the posture of the image acquisition device, that is, the control device can control the posture of the image acquisition device. For example, the control device is Figure 2The operating rod in the image acquisition device. Optionally, the control device includes a manipulation device. The first moment is the initial moment of adjusting the posture of the image acquisition device by the manipulation device, and the second moment is the termination moment of adjusting the posture of the image acquisition device by the manipulation device, wherein the first posture is the posture of the image acquisition device at the first moment, that is, at the first moment, the posture of the image acquisition device in the base coordinate system is the first posture. In a possible implementation scenario, at the first moment, the posture of the image acquisition device in the base coordinate system of the manipulator is the first posture, and the posture of the manipulation device is the fourth posture. It is used to adjust the posture of the manipulation device (for example, move the manipulation stick) starting from the first moment so that the posture of the manipulation device at the second moment is the second posture. The control device converts the change in the posture of the manipulation device from the first moment to the second moment into a first adjustment amount, and adjusts the posture of the image acquisition device based on the first adjustment amount.

[0110] Optionally, the fourth posture and the fifth posture are both postures of the manipulation device in a manipulation coordinate system, where the manipulation coordinate system is a coordinate system constructed based on the manipulation device. The control device uses a second transformation relationship to convert the fourth posture into a sixth posture in the base coordinate system, and uses a second transformation relationship to convert the fifth posture into a seventh posture in the base coordinate system. The first adjustment amount is then obtained based on the difference between the seventh posture and the sixth posture.

[0111] As an optional embodiment, the above-mentioned first adjustment amount includes a first rotation amount and a first translation amount. The control device performs the following steps during the execution of the step of "using the above-mentioned first transformation relationship to convert the above-mentioned second posture into the third posture of the above-mentioned reference structure in the above-mentioned base coordinate system": decomposing the above-mentioned second posture into a second rotation amount and a second translation amount; using the above-mentioned first rotation amount to adjust the above-mentioned second rotation amount to obtain a third rotation amount; using the above-mentioned first translation amount to adjust the above-mentioned second translation amount to obtain a third translation amount; based on the above-mentioned third rotation amount and the above-mentioned third translation amount, obtaining the above-mentioned third posture.

[0112] In this embodiment, since the first adjustment amount includes a first rotation amount and a first translation amount, the control device can adjust the posture of the image acquisition device based on the first adjustment amount by rotating the image acquisition device and translating the image acquisition device. Therefore, the control device first decomposes the second posture into a second rotation amount and a second translation amount, and then uses the first rotation amount and the first translation amount to adjust the rotational component and the translational component of the second posture, respectively, to obtain a third posture.

[0113] As an optional embodiment, the above-mentioned base coordinate system includes a horizontal axis, a vertical axis and a vertical axis, and the above-mentioned first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount and a first vertical rotation amount. The above-mentioned first horizontal rotation amount is the component of the above-mentioned first rotation amount on the above-mentioned horizontal axis, the above-mentioned first vertical rotation amount is the component of the above-mentioned first rotation amount on the above-mentioned vertical axis, and the above-mentioned first vertical rotation amount is the component of the above-mentioned first rotation amount on the above-mentioned vertical axis.

[0114] The control device performs the following steps during the process of executing the step "adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount": determining the component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount. Determining the component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount. Determining the component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount. Adjusting the second horizontal rotation amount by using the first horizontal rotation amount to obtain a third horizontal rotation amount. Adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount. Adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount. Obtaining the third rotation amount based on the third horizontal rotation amount, the third vertical rotation amount, and the third vertical rotation amount.

[0115] In this embodiment, the first rotation amount includes components on each coordinate axis of the base coordinate system (i.e., a first transverse rotation amount, a first longitudinal rotation amount, and a first vertical rotation amount). Through the above steps, the control device decomposes the second rotation amount into components on each coordinate axis of the base coordinate system, obtaining a second transverse rotation amount, a second longitudinal rotation amount, and a second vertical rotation amount. The components of the first rotation amount on each coordinate axis are then used to adjust the components on the corresponding coordinate axis, thereby obtaining adjusted components on each coordinate axis (i.e., a third transverse rotation amount, a third longitudinal rotation amount, and a third vertical rotation amount). Finally, based on the adjusted components on each coordinate axis, an adjusted rotation amount (i.e., a third rotation amount) is obtained.

[0116] Optionally, the horizontal axis of the base coordinate system is expressed as , the vertical axis of the base coordinate system is expressed as , the vertical axis of the base coordinate system is expressed as ,in, Represented as (1, 0, 0), Represented as (0, 1, 0), Represented as (0, 0, 1). The control device will By multiplying the rotation amount, the component of the rotation amount on the horizontal axis can be obtained. For example, by multiplying the second rotation amount with (1, 0, 0), the second horizontal rotation amount can be obtained. Similarly, the control device can By multiplying the rotation amount, the component of the rotation amount on the vertical axis can be obtained. Multiplying it by the amount of rotation gives the component of the rotation on the vertical axis.

[0117] Based on the remote image acquisition method provided above, the embodiment of the present application also provides a possible application scenario. Specifically, the image acquisition device can be an ultrasound probe, which is used to scan the patient to obtain an ultrasound image of the patient, so as to further determine the puncture path for puncturing the lesion in the patient's body based on the ultrasound image. In order to avoid the aorta and the great vein in the patient's body when planning the puncture path. When the patient is scanned by the ultrasound probe, a contrast agent is injected into the patient. In this way, if the ultrasound probe scans and obtains the ultrasound image of the patient within the development time of the contrast agent, then in the ultrasound image, the aorta and the great vein are in a development state, that is, in the ultrasound image, the aorta and the great vein can be highlighted. If the ultrasound probe does not scan and obtain the ultrasound image of the patient within the development time of the contrast agent, then in the ultrasound image, the aorta and the great vein are not in a development state, that is, in the ultrasound image, the aorta and the great vein may not be highlighted.

[0118] Based on the remote image acquisition method provided above, the control device controls the robotic arm to move the ultrasound probe, causing the ultrasound probe to scan the patient multiple times, thereby obtaining multiple puncture ultrasound images of the patient. When these multiple puncture ultrasound images are displayed, the physician can determine the lesion and needle insertion point based on these images, and further determine the puncture path based on the lesion and needle insertion point.

[0119] Because different ultrasound images are obtained by scanning patients at different positions and angles, they contain different lesion information. Lesion information includes information used to identify the lesion. For example, if the lesion is located in the lungs, the lesion information includes lung tissue. For another example, if the lesion is located in the renal pelvis, the lesion information includes the renal pelvis. Obviously, the richer and clearer the lesion information contained in the ultrasound image, the higher the accuracy of the lesion identification based on the ultrasound image, and thus the higher the accuracy of the puncture path determined based on the lesion. Therefore, before scanning a patient with an ultrasound probe, the ultrasound probe's puncture position can be determined. When the ultrasound probe is in the puncture position, the ultrasound image obtained by the ultrasound probe contains lesion information with information content and clarity that meet preset requirements. For example, the information content of the lesion information is greater than or equal to an information content threshold, and the clarity of the lesion information is greater than or equal to a clarity threshold. The doctor can then determine the lesion and needle insertion point based on the puncture ultrasound image. Optionally, before scanning the patient, a remote image acquisition method can be used to control the ultrasound probe to scan the test subject in different postures, thereby obtaining multiple test ultrasound images. Based on the information content and clarity of the lesion information contained in the ultrasound image, a reference ultrasound image can then be determined from the multiple ultrasound images. The reference ultrasound image is an ultrasound image whose information content and clarity of the lesion information meet preset requirements, for example, the information content of the lesion information is greater than or equal to an information content threshold, and the clarity of the lesion information is greater than or equal to a clarity threshold. Finally, the posture of the ultrasound probe when acquiring the reference ultrasound image is determined to be the puncture posture.

[0120] After determining the puncture position, the ultrasound probe can be controlled using the remote image acquisition method described above to achieve the puncture position. The ultrasound probe can then be controlled to scan the patient in the puncture position, obtaining multiple puncture ultrasound images. The physician can then determine the lesion and needle insertion point based on these multiple puncture ultrasound images.

[0121] Given the limited development time of contrast agents, many puncture ultrasound images include those acquired outside the contrast agent development time. When acquired outside this time, the aorta and veins are not highlighted within the puncture ultrasound images. This can cause doctors to overlook these aortas when identifying the lesion and needle insertion point, potentially leading to the puncture path passing through the aorta or vein, potentially posing a fatal risk to the patient.

[0122] Because multiple puncture ultrasound images are acquired by the ultrasound probe in the puncture position, and the patient's position typically remains unchanged during the acquisition of these images by the ultrasound probe, the positions of the aorta and great vein are identical in each puncture ultrasound image. Based on this, the control device first identifies, from the multiple ultrasound images, a developed ultrasound image whose acquisition time falls within the contrast agent's development time. It then determines the positions of the aorta and great vein within the developed ultrasound image as the developed position. Finally, for all non-developed ultrasound images in the multiple ultrasound images, excluding the developed ultrasound image, the area designated as the developed position is highlighted. In this way, the physician can determine the lesion and needle insertion point based on the developed and non-developed ultrasound images from the multiple puncture ultrasound images, reducing the probability that the puncture path determined based on the lesion and needle insertion point will pass through the aorta or great vein.

[0123] After the lesion and needle entry point are determined from the puncture ultrasound image, the position of the lesion in the puncture ultrasound image is converted to a position in the world coordinate system based on the third transformation relationship, and the position of the needle entry point in the puncture ultrasound image is converted to a position in the world coordinate system. Finally, the puncture path in the world coordinate system can be determined based on the position of the lesion in the world coordinate system and the position of the needle entry point in the world coordinate system.

[0124] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0125] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0126] The above describes in detail the method of the embodiment of the present application, and the following provides an apparatus of the embodiment of the present application.

[0127] See also Figure 5 , Figure 5 This is a structural diagram of a control device provided in an embodiment of the present application. The control device 1 is used to remotely control an image acquisition device to acquire images. The image acquisition device is fixed to a robotic arm. The control device 1 drives the image acquisition device to move by controlling the movement of the robotic arm. The control device 1 includes: an acquisition unit 11, an adjustment unit 12, and a conversion unit 13. Optionally, the control device 1 also includes: a control unit 14, a receiving unit 15, and a determination unit 16. Specifically:

[0128] An acquisition unit 11 is configured to acquire a first pose of the image acquisition device in a base coordinate system of the robotic arm, where the base coordinate system is a coordinate system constructed based on the base of the robotic arm;

[0129] The acquisition unit 11 is further configured to acquire a first adjustment value of the posture of the image acquisition device;

[0130] An adjusting unit 12, configured to adjust the first posture to obtain a second posture of the image acquisition device in the base coordinate system based on the first adjustment amount;

[0131] The acquisition unit 11 is further configured to acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between a posture of the image acquisition device and a posture of a reference structure, where the reference structure includes a structure on the robotic arm excluding the base;

[0132] The conversion unit 13 is configured to convert the second posture into a third posture of the reference structure in the base coordinate system by using the first conversion relationship.

[0133] In combination with any embodiment of the present application, the control device 1 further includes:

[0134] The control unit 14 is configured to control the movement of the robotic arm through the third posture so that the posture of the reference structure is the third posture.

[0135] In combination with any embodiment of the present application, the image captured by the image acquisition device includes an ultrasonic image, and the control device 1 further includes: a receiving unit 15, configured to receive a target ultrasonic image captured by the image acquisition device in the third posture, wherein the target ultrasonic image includes the object to be punctured;

[0136] The determining unit 16 is configured to determine a puncture path of the object to be punctured based on the target ultrasound image.

[0137] In combination with any embodiment of the present application, the acquiring unit 11 is configured to:

[0138] Acquiring a fourth posture of the control device at the first moment, the control device being used to remotely adjust the posture of the image acquisition device;

[0139] Obtaining a fifth posture of the control device at a second moment, where the second moment is a moment later than the first moment;

[0140] The first adjustment amount is obtained based on a difference between the fifth posture and the fourth posture.

[0141] In combination with any embodiment of the present application, the first adjustment amount includes a first rotation amount and a first translation amount, and the conversion unit 13 is configured to:

[0142] Decomposing the second posture into a second rotation and a second translation;

[0143] adjusting the second rotation amount using the first rotation amount to obtain a third rotation amount;

[0144] adjusting the second translation amount using the first translation amount to obtain a third translation amount;

[0145] The third posture is obtained based on the third rotation amount and the third translation amount.

[0146] In combination with any embodiment of the present application, the base coordinate system includes a horizontal axis, a vertical axis, and a vertical axis, and the first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount, and a first vertical rotation amount, where the first horizontal rotation amount is a component of the first rotation amount on the horizontal axis, the first vertical rotation amount is a component of the first rotation amount on the vertical axis, and the first vertical rotation amount is a component of the first rotation amount on the vertical axis;

[0147] The conversion unit is used to:

[0148] determining a component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount;

[0149] determining a component of the second rotation amount on the longitudinal axis to obtain a second longitudinal rotation amount;

[0150] determining a component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount;

[0151] adjusting the second lateral rotation amount by using the first lateral rotation amount to obtain a third lateral rotation amount;

[0152] adjusting the second longitudinal rotation amount by using the first longitudinal rotation amount to obtain a third longitudinal rotation amount;

[0153] adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount;

[0154] The third rotation amount is obtained based on the third horizontal rotation amount, the third vertical rotation amount, and the third vertical rotation amount.

[0155] In an embodiment of the present application, an image acquisition device is fixed on a robotic arm, wherein a control device drives the image acquisition device to move by controlling the movement of the robotic arm. After the control device obtains a first posture of the image acquisition device in the base coordinate system of the robotic arm, it obtains a first adjustment amount of the posture of the image acquisition device. Then, based on the first adjustment amount, the first posture is adjusted to obtain a second posture of the image acquisition device in the base coordinate system. Then, a first conversion relationship is obtained, wherein the first conversion relationship is a conversion relationship between the posture of the image acquisition device and the posture of a reference structure, and the reference structure is the structure of the robotic arm. Finally, the second posture is converted into a third posture of the reference structure in the base coordinate system using the first conversion relationship, so that the first adjustment amount of the posture of the image acquisition device can be converted into the posture of the reference structure of the robotic arm.

[0156] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0157] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, the memory 22, the input device 23 and the output device 24 are coupled via a connector, and the connector includes various interfaces, transmission lines or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, connection through various interfaces, transmission lines, buses, etc.

[0158] The processor 21 may be one or more graphics processing units (GPUs). If the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Alternatively, the processor 21 may be a processor group consisting of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Alternatively, the processor may be another type of processor, and the present embodiment is not limiting.

[0159] The memory 22 can be used to store computer program instructions and various computer program codes, including program codes for executing the solution of the present application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), and is used for related instructions and data.

[0160] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.

[0161] It can be understood that in the embodiment of the present application, the memory 22 can be used not only to store relevant instructions, but also to store relevant data. The embodiment of the present application does not limit the specific data stored in the memory.

[0162] It is understandable that Figure 6 Only a simplified design of an electronic device is shown. In actual applications, the electronic device may further include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the scope of protection of the present application.

[0163] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0164] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here. Those skilled in the art will also clearly understand that the descriptions of the various embodiments of this application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, reference can be made to the descriptions of other embodiments.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0166] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0167] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0168] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0169] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium. When executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A remote image acquisition method, characterized in that: The method is applied to a control device, the control device including a manipulation device, the control device being used to remotely control an image acquisition device to acquire images, the image acquisition device being fixed to a robotic arm, the control device driving the image acquisition device to move by controlling the movement of the robotic arm, the control device being deployed outside the operating room, the robotic arm being deployed inside the operating room, the image acquisition device including an ultrasound probe, and the method comprising: Acquire a first pose of the image acquisition device in a base coordinate system of the robotic arm, where the base coordinate system is a coordinate system constructed based on the base of the robotic arm; Obtaining a first adjustment amount for the posture of the image acquisition device, where the first posture is the posture at a first moment; obtaining the first adjustment amount for the posture of the image acquisition device includes: obtaining a fourth posture of the control device at the first moment, where the control device is used to remotely adjust the posture of the image acquisition device; obtaining a fifth posture of the control device at a second moment, where the second moment is later than the first moment; and obtaining the first adjustment amount based on a difference between the fifth posture and the fourth posture; The obtaining of the first adjustment amount based on the difference between the fifth posture and the fourth posture comprises: the control device converting the fourth posture into a sixth posture in the base coordinate system by using a second transformation relationship, and converting the fifth posture into a seventh posture in the base coordinate system by using the second transformation; obtaining the first adjustment amount based on the difference between the seventh posture and the sixth posture; the fourth posture and the fifth posture are both postures of the control device in a control coordinate system, wherein the control coordinate system is a coordinate system constructed based on the control device; Based on the first adjustment amount, adjusting the first posture to obtain a second posture of the image acquisition device in the base coordinate system, where the second posture is a puncture posture. When the posture of the ultrasound probe is the puncture posture, the information content and clarity of the lung lesion information contained in the ultrasound image scanned by the ultrasound probe meet preset requirements; Acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between a posture of the image acquisition device and a posture of a reference structure, where the reference structure is a flange of the robotic arm; Converting the second posture into a third posture of the reference structure in the base coordinate system by using the first conversion relationship; According to the principle of avoiding people or objects in the operating room, based on the three-dimensional scene image and the third posture, planning a reference path for the image acquisition device to adjust its posture from the first posture to the second posture, wherein the three-dimensional scene image is a three-dimensional image of the scene in which the robotic arm and the image acquisition device are located, the three-dimensional scene image is acquired by the control device, the reference path is the movement path of the robotic arm, and the robotic arm moves according to the reference path so that the posture of the image acquisition device on the robotic arm is adjusted from the first posture to the second posture, and the posture of the reference structure is adjusted to the third posture; Based on the reference path and the third posture, controlling the movement of the robotic arm so that the posture of the reference structure is the third posture; receiving a plurality of puncture ultrasound images acquired by the ultrasound probe in the puncture posture, wherein the plurality of puncture ultrasound images all include the object to be punctured; Determining, from the plurality of puncture ultrasound images, a developed ultrasound image whose acquisition time falls within the development time of the contrast agent; determining positions of the pulmonary aorta and the pulmonary venae in the development ultrasound image as development positions; For the non-developed ultrasound images other than the developed ultrasound image in the plurality of puncture ultrasound images, the region at the developed position is highlighted to obtain a highlighted non-developed ultrasound image; displaying the developed ultrasound image and the highlighted non-developed ultrasound image; A puncture path is determined based on the lesion and the needle insertion point determined from the developed ultrasound image and the highlighted non-developed ultrasound image. The puncture path is used to puncture the lesion in the lung.

2. The method according to claim 1, characterized in that The first adjustment amount includes a first rotation amount and a first translation amount, and the converting the second posture into a third posture of the reference structure in the base coordinate system by using the first conversion relationship includes: Decomposing the second posture into a second rotation and a second translation; adjusting the second rotation amount using the first rotation amount to obtain a third rotation amount; adjusting the second translation amount using the first translation amount to obtain a third translation amount; The third posture is obtained based on the third rotation amount and the third translation amount.

3. The method according to claim 2, characterized in that The base coordinate system includes a horizontal axis, a vertical axis, and a vertical axis, and the first rotation amount includes a first horizontal rotation amount, a first vertical rotation amount, and a first vertical rotation amount, wherein the first horizontal rotation amount is a component of the first rotation amount on the horizontal axis, the first vertical rotation amount is a component of the first rotation amount on the vertical axis, and the first vertical rotation amount is a component of the first rotation amount on the vertical axis; The adjusting the second rotation amount by using the first rotation amount to obtain a third rotation amount includes: determining a component of the second rotation amount on the horizontal axis to obtain a second horizontal rotation amount; determining a component of the second rotation amount on the longitudinal axis to obtain a second longitudinal rotation amount; determining a component of the second rotation amount on the vertical axis to obtain a second vertical rotation amount; adjusting the second lateral rotation amount by using the first lateral rotation amount to obtain a third lateral rotation amount; adjusting the second longitudinal rotation amount by using the first longitudinal rotation amount to obtain a third longitudinal rotation amount; adjusting the second vertical rotation amount by using the first vertical rotation amount to obtain a third vertical rotation amount; The third rotation amount is obtained based on the third horizontal rotation amount, the third vertical rotation amount, and the third vertical rotation amount.

4. A control device, characterized in that: The control device includes a manipulation device, the control device is used to remotely control an image acquisition device to acquire images, the image acquisition device is fixed to a robotic arm, the control device drives the image acquisition device to move by controlling the movement of the robotic arm, the control device is deployed outside the operating room, the robotic arm is deployed inside the operating room, the image acquisition device includes an ultrasound probe, and the control device includes: an acquisition unit, configured to acquire a first pose of the image acquisition device in a base coordinate system of the robotic arm, wherein the base coordinate system is a coordinate system constructed based on the base of the robotic arm; The acquisition unit is further configured to acquire a first adjustment amount for the posture of the image acquisition device, the first posture being the posture at a first moment; acquiring the first adjustment amount for the posture of the image acquisition device comprises: acquiring a fourth posture of the control device at the first moment, the control device being configured to remotely adjust the posture of the image acquisition device; acquiring a fifth posture of the control device at a second moment, the second moment being a moment later than the first moment; and obtaining the first adjustment amount based on a difference between the fifth posture and the fourth posture; The obtaining of the first adjustment amount based on the difference between the fifth posture and the fourth posture comprises: the control device converting the fourth posture into a sixth posture in the base coordinate system by using a second transformation relationship, and converting the fifth posture into a seventh posture in the base coordinate system by using the second transformation; obtaining the first adjustment amount based on the difference between the seventh posture and the sixth posture; the fourth posture and the fifth posture are both postures of the control device in a control coordinate system, wherein the control coordinate system is a coordinate system constructed based on the control device; an adjustment unit, configured to adjust the first posture based on the first adjustment amount to obtain a second posture of the image acquisition device in the base coordinate system, wherein the second posture is a puncture posture, and when the posture of the ultrasound probe is the puncture posture, the information content and clarity of the lung lesion information contained in the ultrasound image scanned by the ultrasound probe meet preset requirements; The acquisition unit is further configured to acquire a first conversion relationship, where the first conversion relationship is a conversion relationship between a posture of the image acquisition device and a posture of a reference structure, where the reference structure is a flange of the robotic arm; A conversion unit, configured to convert the second posture into a third posture of the reference structure in the base coordinate system by using the first conversion relationship; a planning unit, configured to plan, based on a three-dimensional scene image and the third posture, a reference path for adjusting the posture of the image acquisition device from the first posture to the second posture in accordance with a principle of avoiding people or objects in the operating room, wherein the three-dimensional scene image is a three-dimensional image of a scene in which the robotic arm and the image acquisition device are located, the three-dimensional scene image being acquired by the control device, the reference path being a movement path of the robotic arm, and the robotic arm moving according to the reference path so that the posture of the image acquisition device on the robotic arm is adjusted from the first posture to the second posture, and the posture of the reference structure is adjusted to the third posture; a control unit, configured to control the movement of the robotic arm based on the reference path and the third posture, so that the posture of the reference structure is the third posture; a receiving unit, configured to receive a plurality of puncture ultrasound images acquired by the ultrasound probe in the puncture posture, wherein the plurality of puncture ultrasound images all include the object to be punctured; a determining unit, configured to determine, from the plurality of puncture ultrasound images, a developed ultrasound image whose acquisition time falls within the development time of the contrast agent; The determining unit is further configured to determine the positions of the pulmonary aorta and the pulmonary venae in the developed ultrasound image as developed positions; a display unit configured to highlight an area of ​​the non-developed ultrasound image other than the developed ultrasound image in the plurality of puncture ultrasound images, where the area is the developed location, to obtain a highlighted non-developed ultrasound image; The display unit is further configured to display the developed ultrasonic image and the highlighted non-developed ultrasonic image; The planning unit is further configured to determine a puncture path based on the lesion and the needle insertion point determined from the developed ultrasound image and the highlighted non-developed ultrasound image, wherein the puncture path is used to puncture the lesion in the lung.

5. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the electronic device executes the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 3.

7. A computer program product, characterized in that The computer program product comprises a computer program; when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 3.

Images