Simulation-based navigation method and device, surgical robot, and electronic device

Abstract

This application discloses a simulation-based navigation method and device, a surgical robot, and an electronic device. The method includes: acquiring a target 3D image and a target path; the target 3D image includes a first object; acquiring a first 3D model of the first object and a second 3D model of a guide, the second 3D model including a hole placement model; obtaining a first pose of the first 3D model in a first coordinate system based on the target 3D image; obtaining a second pose of the second 3D model in the first coordinate system based on the first 3D model, the second 3D model, and the first pose; and, given the second pose of the second 3D model, the axis of the hole placement model is aligned with the target path, and the first 3D model and the second 3D model do not intersect. Simulation based on this method yields a guide pose that satisfies the following conditions: the axis of the hole placement model in the guide is aligned with the target path, and the guide does not collide with the first object.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to a simulation-based navigation method and device, a surgical robot, and electronic equipment. Background Technology

[0002] In the medical field, after planning the path from the skin region of a first object to its body, a guide can be navigated based on this path to align the axis of the guide's placement hole with the path. This allows a second object to be moved from the skin region of the first object towards its body by moving it along the axis of the placement hole. Therefore, determining the guide's pose based on this path is of paramount importance when navigating the guide. Summary of the Invention

[0003] This application provides a simulation-based navigation method and device, surgical robot, and electronic device to obtain the pose of a guide that satisfies the following conditions through simulation: the axis of the placement hole in the guide is aligned with the target path, and the guide does not collide with the first object.

[0004] Firstly, a simulation-based navigation method is provided, the method comprising:

[0005] Acquire a target 3D image and a target path, wherein the target 3D image includes a first object and the target path is a path that moves from the skin region of the first object toward a target point within the first object;

[0006] Obtain a first three-dimensional model of the first object and a second three-dimensional model of the guide. The second three-dimensional model includes a placement hole model, which is a model corresponding to the placement hole in the guide. When the second object is placed in the placement hole, the second object can move along the axis of the placement hole. The second object is an object that moves from the skin area of ​​the first object toward a target point inside the first object.

[0007] Based on the target 3D image, the first pose of the first 3D model in the first coordinate system is obtained, where the first coordinate system is the image coordinate system of the target 3D image;

[0008] Based on the first 3D model, the second 3D model, and the first pose, the second pose of the second 3D model in the first coordinate system is obtained. When the pose of the second 3D model is the second pose, the axis of the model where the hole is placed is aligned with the target path, and the first 3D model and the second 3D model do not intersect.

[0009] In any embodiment of this application, the first structure of the guide is connected to the robotic arm;

[0010] Before obtaining the second pose of the second three-dimensional model in the first coordinate system based on the first three-dimensional model, the second three-dimensional model, and the first pose, the method further includes:

[0011] The third pose of the third 3D model of the robotic arm in the first coordinate system and the fourth pose of the second 3D model in the first coordinate system are obtained. When the third 3D model has not started to move, the pose of the third 3D model is the third pose and the pose of the second 3D model is the fourth pose.

[0012] Based on the third pose and the fourth pose, a first orientation relationship is obtained. The first orientation relationship indicates whether the first sub-model in the second three-dimensional model is facing the second sub-model in the third three-dimensional model. The first sub-model is the model corresponding to the first structure, and the second sub-model is the model corresponding to the base of the robotic arm.

[0013] The step of obtaining the second pose of the second 3D model in the first coordinate system based on the first 3D model, the second 3D model, and the first pose includes:

[0014] Based on the first 3D model, the second 3D model, and the first pose, n first candidate poses are obtained, where n is a positive integer. When the pose of the second 3D model is the first candidate pose, the axis of the placement of the Confucius model is aligned with the target path, and the first 3D model and the second 3D model do not intersect.

[0015] Based on the first orientation relationship, the second pose is determined from the n first candidate poses. The second orientation relationship indicates whether the first sub-model is facing the second sub-model when the pose of the second 3D model is the second pose. The first orientation relationship is the same as the second orientation relationship.

[0016] In conjunction with any embodiment of this application, before determining the second pose from the n first candidate poses based on the first orientation relationship, the method further includes:

[0017] Obtain the first range of motion in the first coordinate system, where the first range of motion is the range of motion of the third three-dimensional model;

[0018] Based on the first motion range, m second candidate poses are obtained, where m is a positive integer. When the third 3D model moves within the first motion range, the pose of the second 3D model is the second candidate pose.

[0019] The step of determining the second pose from the n first candidate poses based on the first orientation relationship includes:

[0020] Based on the first orientation relationship, x third candidate poses are determined from the m second candidate poses, where x is a positive integer less than or equal to m;

[0021] The second pose is obtained based on the x third candidate poses and the n first candidate poses, wherein the x third candidate poses and the n first candidate poses all include the second pose.

[0022] In any embodiment of this application, determining the second pose from the n first candidate poses based on the first orientation relationship includes:

[0023] Based on the first orientation relationship, y fourth candidate poses are determined from the n first candidate poses, where y is a positive integer less than or equal to n. The third orientation relationship indicates whether the first sub-model is oriented towards the second sub-model when the pose of the second 3D model is the fourth candidate pose. The first orientation relationship is the same as the third orientation relationship.

[0024] Based on the y fourth candidate poses, r first motion paths are obtained. When the third 3D model moves based on the first motion paths, the second 3D model moves from the fourth pose to the fourth candidate pose, where r is a positive integer less than or equal to y.

[0025] The shortest path among the r first motion paths is determined as the second motion path;

[0026] The second pose is obtained based on the pose corresponding to the second motion path among the y fourth candidate poses.

[0027] In any embodiment of this application, obtaining r first motion paths based on the y fourth candidate poses includes:

[0028] Based on the y fourth candidate poses, y third motion paths are obtained. The third motion paths correspond one-to-one with the fourth candidate poses. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose.

[0029] Based on the y third motion paths, y first fault tolerance ranges are obtained. The first fault tolerance range corresponds one-to-one with the third motion path. The first fault tolerance range is the range where the distance to the third motion path is less than or equal to a first threshold.

[0030] The number of the third motion paths within each of the y first fault tolerance ranges is determined to obtain the number of y paths;

[0031] The ranges corresponding to the r largest quantities among the y first fault tolerance ranges are determined as r second fault tolerance ranges;

[0032] The path that corresponds to the r second fault tolerance range among the y third motion paths is determined as the r first motion paths.

[0033] In any embodiment of this application, obtaining r first motion paths based on the y fourth candidate poses includes:

[0034] Based on the y fourth candidate poses, y third motion paths are obtained. The third motion paths correspond one-to-one with the fourth candidate poses. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose.

[0035] Determine the maximum curvature of the y third motion paths to obtain the y first curvatures;

[0036] Based on the r curvatures that are the smallest among the y first curvatures, r second curvatures are obtained;

[0037] Based on the paths corresponding to the r second curvatures among the y third motion paths, the r first motion paths are obtained.

[0038] In any embodiment of this application, the guide is connected to the robotic arm, and the method further includes:

[0039] Based on the second pose and the first transformation relationship, the third pose of the guide in the second coordinate system is obtained, where the second coordinate system is a coordinate system constructed based on the base of the robotic arm;

[0040] Based on the third pose, the movement of the robotic arm is controlled so that the pose of the guide in the second coordinate system is the third pose.

[0041] Secondly, a simulation-based navigation device is provided, the simulation-based navigation device comprising:

[0042] An acquisition unit is used to acquire a target 3D image and a target path. The target 3D image includes a first object, and the target path is a path that moves from the skin region of the first object toward a target point within the first object.

[0043] The acquisition unit is further configured to acquire a first three-dimensional model of the first object and a second three-dimensional model of the guide. The second three-dimensional model includes a placement hole model, which is a model corresponding to the placement hole in the guide. When the second object is placed in the placement hole, the second object can move along the axis of the placement hole. The second object is an object that moves from the skin area of ​​the first object toward a target point inside the first object.

[0044] The processing unit is configured to obtain the first pose of the first three-dimensional model in a first coordinate system based on the target three-dimensional image, wherein the first coordinate system is the image coordinate system of the target three-dimensional image;

[0045] The processing unit is further configured to obtain a second pose of the second three-dimensional model in the first coordinate system based on the first three-dimensional model, the second three-dimensional model, and the first pose. When the pose of the second three-dimensional model is the second pose, the axis of the model for placing the hole is aligned with the target path, and the first three-dimensional model and the second three-dimensional model do not intersect.

[0046] In any embodiment of this application, the first structure of the guide is connected to the robotic arm;

[0047] The acquisition unit is further configured to acquire the third pose of the third three-dimensional model of the robotic arm in the first coordinate system and the fourth pose of the second three-dimensional model in the first coordinate system. When the third three-dimensional model has not started to move, the pose of the third three-dimensional model is the third pose and the pose of the second three-dimensional model is the fourth pose.

[0048] The processing unit is further configured to obtain a first orientation relationship based on the third pose and the fourth pose. The first orientation relationship indicates whether the first sub-model in the second three-dimensional model is facing the second sub-model in the third three-dimensional model. The first sub-model is a model corresponding to the first structure, and the second sub-model is a model corresponding to the base of the robotic arm.

[0049] The processing unit is further configured to obtain n first candidate poses based on the first three-dimensional model, the second three-dimensional model, and the first pose, where n is a positive integer. When the pose of the second three-dimensional model is the first candidate pose, the axis of the placement of the Confucius model is aligned with the target path, and the first three-dimensional model and the second three-dimensional model do not intersect.

[0050] The processing unit is further configured to determine the second pose from the n first candidate poses based on the first orientation relationship. The second orientation relationship indicates whether the first sub-model is facing the second sub-model when the pose of the second 3D model is the second pose. The first orientation relationship is the same as the second orientation relationship.

[0051] In any embodiment of this application, the acquisition unit is further configured to acquire a first motion range in the first coordinate system, wherein the first motion range is the motion range of the third three-dimensional model;

[0052] The processing unit is further configured to obtain m second candidate poses based on the first motion range, where m is a positive integer, and when the third three-dimensional model moves within the first motion range, the pose of the second three-dimensional model is the second candidate pose;

[0053] The processing unit is further configured to determine x third candidate poses from the m second candidate poses based on the first orientation relationship, where x is a positive integer less than or equal to m;

[0054] The processing unit is further configured to obtain the second pose based on the x third candidate poses and the n first candidate poses, wherein the x third candidate poses and the n first candidate poses all include the second pose.

[0055] In any embodiment of this application, the processing unit is further configured to determine y fourth candidate poses from the n first candidate poses based on the first orientation relationship, where y is a positive integer less than or equal to n, and the third orientation relationship indicates whether the first sub-model is oriented towards the second sub-model when the pose of the second three-dimensional model is the fourth candidate pose, and the first orientation relationship is the same as the third orientation relationship.

[0056] The processing unit is further configured to obtain r first motion paths based on the y fourth candidate poses, and when the third 3D model moves based on the first motion paths, the second 3D model moves from the fourth pose to the fourth candidate pose, where r is a positive integer less than or equal to y.

[0057] The processing unit is further configured to determine the shortest path among the r first motion paths as the second motion path;

[0058] The processing unit is further configured to obtain the second pose based on the pose corresponding to the second motion path among the y fourth candidate poses.

[0059] In conjunction with any embodiment of this application, the processing unit is further configured to:

[0060] Based on the y fourth candidate poses, y third motion paths are obtained. The third motion paths correspond one-to-one with the fourth candidate poses. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose.

[0061] Based on the y third motion paths, y first fault tolerance ranges are obtained. The first fault tolerance range corresponds one-to-one with the third motion path. The first fault tolerance range is the range where the distance to the third motion path is less than or equal to a first threshold.

[0062] The number of the third motion paths within each of the y first fault tolerance ranges is determined to obtain the number of y paths;

[0063] The ranges corresponding to the r largest quantities among the y first fault tolerance ranges are determined as r second fault tolerance ranges;

[0064] The path that corresponds to the r second fault tolerance range among the y third motion paths is determined as the r first motion paths.

[0065] In conjunction with any embodiment of this application, the processing unit is further configured to:

[0066] Based on the y fourth candidate poses, y third motion paths are obtained. The third motion paths correspond one-to-one with the fourth candidate poses. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose.

[0067] Determine the maximum curvature of the y third motion paths to obtain the y first curvatures;

[0068] Based on the r curvatures that are the smallest among the y first curvatures, r second curvatures are obtained;

[0069] Based on the paths corresponding to the r second curvatures among the y third motion paths, the r first motion paths are obtained.

[0070] In any embodiment of this application, the guide is connected to the robotic arm, and the processing unit is further configured to:

[0071] Based on the second pose and the first transformation relationship, the third pose of the guide in the second coordinate system is obtained, where the second coordinate system is a coordinate system constructed based on the base of the robotic arm;

[0072] Based on the third pose, the movement of the robotic arm is controlled so that the pose of the guide in the second coordinate system is the third pose.

[0073] Thirdly, a surgical robot is provided, including a simulation-based navigation device as described in the second aspect. In the third aspect, the surgical robot can execute a simulation-based navigation method via the simulation-based navigation device, and the pose of the guide can be obtained through simulation that satisfies the following conditions: the axis of the placement hole in the guide is aligned with the target path, and the guide does not collide with the first object.

[0074] Fourthly, an electronic device is provided, comprising: a processor and a memory, the memory for storing computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs a method as described in the first aspect above and any possible implementation thereof.

[0075] Fifthly, another electronic device is provided, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs the method as described in the first aspect above and any possible implementation thereof.

[0076] In a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored therein, the computer program including program instructions that, when executed by a processor, cause the processor to perform a method as described in the first aspect above and any possible implementation thereof.

[0077] In a seventh aspect, a computer program product is provided, the computer program product comprising a computer program or instructions, wherein, when the computer program or instructions are executed on a computer, the computer performs the method described in the first aspect and any possible implementation thereof.

[0078] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application.

[0079] In this embodiment, after acquiring a target 3D image and a first 3D model, the navigation device can obtain a first pose of the first 3D model in a first coordinate system based on the target 3D image. Then, based on the first 3D model, a second 3D model, and the first pose, a second pose of the second 3D model in the first coordinate system can be obtained. When the pose of the second 3D model is the second pose, the axis of the hole placement model is aligned with the target path, and the first 3D model and the second 3D model do not intersect. Thus, through simulation, the pose of the guide that satisfies the following conditions can be obtained: the axis of the hole placement in the guide is aligned with the target path, and the guide does not collide with the first object. Attached Figure Description

[0080] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments of this application will be described below.

[0081] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0082] Figure 1 A flowchart illustrating a simulation-based navigation method provided in this application embodiment;

[0083] Figure 2 This is a schematic diagram of the structure of a guide provided in an embodiment of this application;

[0084] Figure 3 A schematic diagram illustrating the connection between a guide and a robotic arm, provided as an embodiment of this application;

[0085] Figure 4 A schematic diagram of a first three-dimensional model and a second three-dimensional model provided for an embodiment of this application;

[0086] Figure 5 A schematic diagram of another first three-dimensional model and a second three-dimensional model provided for embodiments of this application;

[0087] Figure 6 A schematic diagram of yet another first three-dimensional model and a second three-dimensional model provided in the embodiments of this application;

[0088] Figure 7 A schematic diagram of a simulation navigation interface provided for an embodiment of this application;

[0089] Figure 8 A schematic diagram illustrating a collision between a first three-dimensional model and a second three-dimensional model, provided as an embodiment of this application;

[0090] Figure 9 A schematic diagram illustrating a first three-dimensional model and a second three-dimensional model that do not collide, provided for an embodiment of this application;

[0091] Figure 10 A schematic diagram illustrating another instance of a first three-dimensional model and a second three-dimensional model not colliding, provided in an embodiment of this application;

[0092] Figure 11 A schematic diagram of a simulation-based navigation device provided for an embodiment of this application;

[0093] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0094] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0095] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0096] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one" means one or more, "more" means two or more, and "at least two" means two or three or more.

[0097] The execution subject of this application embodiment is a simulation-based navigation device (hereinafter referred to as a navigation device), wherein the navigation device can be any electronic device capable of executing the technical solutions disclosed in the method embodiments of this application. Optionally, the navigation device can be one of the following: a computer, a platform server.

[0098] It should be understood that the method embodiments of this application can also be implemented by a processor executing computer program code. The embodiments of this application are described below with reference to the accompanying drawings. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating a simulation-based navigation method provided in an embodiment of this application.

[0099] 101. Obtain the target's 3D image and target path.

[0100] In this embodiment, the target three-dimensional image can be a medical image. Optionally, the target three-dimensional image is one of the following: a three-dimensional ultrasound image or a three-dimensional computed tomography (CT) image.

[0101] The target 3D image includes a first object. Optionally, the first object is a person. The target path is a path that moves from the skin region of the first object toward a target point within the first object. In other words, the starting point of the target path is a point within the skin region of the first object, and the ending point of the target path is a target point, exemplarily a target point within the first object.

[0102] Optionally, the target path is a path in the first coordinate system, where the first coordinate system is the image coordinate system of the target 3D image.

[0103] In one implementation of acquiring a target 3D image, the navigation device controls a CT scanning device to scan a first object to obtain a target 3D image. In another implementation of acquiring a target 3D image, the navigation device receives a target 3D image input by a user through an input component, wherein the input component includes at least one of the following: a mouse, a keyboard, and a touch screen.

[0104] 102. Obtain the first three-dimensional model of the first object and the second three-dimensional model of the guide.

[0105] In this embodiment of the application, the three-dimensional model (including the first three-dimensional model, the second three-dimensional model, and the third and fourth three-dimensional models mentioned below) can be one of the following: triangular mesh model, computer-aided design (CAD) three-dimensional model, or point cloud three-dimensional model.

[0106] The second 3D model includes a placement hole model, which corresponds to a placement hole in the guide. When a second object is placed within the placement hole, the second object can move along the axis of the placement hole, wherein the second object is an object that moves from the skin region of the first object toward a target point within the first object. For example, the second object is a needle. With the axis of the placement hole aligned with the target path, placing the needle within the placement hole allows the needle to move from the skin region of the first object toward the target point within the first object by moving the needle.

[0107] Optionally, the axis of the placement hole is the center line of the placement hole, and when the second object is placed in the placement hole, the second object can move along the center line of the placement hole.

[0108] For example, Figure 2 This is a schematic diagram of the structure of a guide provided in an embodiment of this application, as shown below. Figure 2 As shown, the guide includes four optical markers and placement holes. The optical markers are capable of reflecting light, and the optical tracking device positions the optical markers based on the light reflected from them. Figure 2 The axis of the hole (i.e.) is also shown. Figure 2 (The dotted line in the image) indicates that when the second object is placed inside the placement hole, the second object coincides with the axis of the placement hole.

[0109] 103. Based on the target 3D image, obtain the first pose of the first 3D model in the first coordinate system.

[0110] Since the target 3D model includes the first object, the pose of the first object in the first coordinate system can be obtained based on the target 3D image, and then the pose can be used as the pose of the first 3D model in the first coordinate system (i.e., the first pose mentioned above).

[0111] In one possible implementation, the navigation device performs semantic segmentation on the target 3D image to determine a first object in the target 3D image, and then determines the pose of the first object in the target 3D image, thereby obtaining a first pose based on the pose.

[0112] 104. Based on the first three-dimensional model, the second three-dimensional model, and the first pose, the second pose of the second three-dimensional model in the first coordinate system is obtained.

[0113] In this embodiment, when the pose of the second 3D model is the second pose, the axis of the placement hole model is aligned with the target path. That is, when the pose of the second 3D model in the first coordinate system is the second pose, the axis of the placement hole model in the second 3D model is aligned with the target path. At this time, if the 3D model corresponding to the second object is placed within the placement hole model, the 3D model corresponding to the second object can be aligned with the target path.

[0114] Optionally, after obtaining the second pose, the navigation device can adjust the pose of the second three-dimensional model to the second pose so that the axis of the placement hole model of the second three-dimensional model is aligned with the target path.

[0115] In some solutions, the guide is connected to a robotic arm. By controlling the movement of the robotic arm, the guide can be moved, thereby adjusting its pose. If the information used to control the robotic arm's movement is in a second coordinate system—for example, a coordinate system constructed based on the robotic arm's base—then after determining the second pose, a third pose of the guide in the second coordinate system can be obtained based on the second pose. The robotic arm can then be controlled based on this third pose to adjust the guide's pose to the third pose. With the guide in its third pose, placing a second object within a placement hole aligns the second object with the hole's axis. Moving the second object along the axis of the placement hole allows it to move along a target path, thus moving it from the skin region of the first object towards a target point within the first object's body.

[0116] When the pose of the second 3D model is the second pose, the first 3D model and the second 3D model do not intersect; that is, the first 3D model and the second 3D model do not collide. In other words, if the pose of the guide is adjusted based on the second pose so that the axis of the placement hole in the guide is aligned with the target path, then when the axis of the placement hole in the guide is aligned with the target path, the guide and the first object will not collide.

[0117] Optionally, when the pose of the second 3D model is the second pose, the distance between the second 3D model and the starting point of the target path is greater than or equal to a distance threshold. This reduces the probability of the guide colliding with the first object when adjusting the pose of the guide based on the second pose.

[0118] exist Figure 1 In the simulation-based navigation method shown, after acquiring a target 3D image and a first 3D model, the navigation device can obtain the first pose of the first 3D model in a first coordinate system based on the target 3D image. Then, based on the first 3D model, the second 3D model, and the first pose, the second pose of the second 3D model in the first coordinate system can be obtained. When the pose of the second 3D model is the second pose, the axis of the hole placement model is aligned with the target path, and the first 3D model and the second 3D model do not intersect. Thus, through simulation, the pose of the guide that satisfies the following conditions can be obtained: the axis of the hole placement in the guide is aligned with the target path, and the guide does not collide with the first object.

[0119] In some scenarios, the guide's pose is uncertain when the axis of the placement hole in the guide is aligned with the target path. Specifically, the guide can rotate around the axis of the placement hole when aligned with the target path. Therefore, if the guide's pose is adjusted based on the target path after planning to align the axis of the placement hole with the target path, a collision between the guide and the first object may occur, potentially causing damage to the first object. However, after obtaining the target path, the pose is first adjusted based on... Figure 1 The method of simulation to obtain the second pose, and then adjusting the pose of the guide based on the second pose, can reduce the probability of the guide colliding with the first object, thereby improving safety.

[0120] As an optional implementation, the first structure of the guide is connected to the robotic arm. Optionally, the first structure is a flange. Before executing the step "obtaining the second pose of the second three-dimensional model in the first coordinate system based on the first three-dimensional model, the second three-dimensional model, and the first pose", the navigation device further performs the following steps: obtaining the third pose of the third three-dimensional model of the robotic arm in the first coordinate system and the fourth pose of the second three-dimensional model in the first coordinate system, wherein, when the third three-dimensional model has not started moving, the pose of the second three-dimensional model is the fourth pose. Based on the third pose and the fourth pose, a first orientation relationship is obtained, wherein the first orientation relationship indicates whether the first sub-model in the second three-dimensional model is oriented towards the second sub-model in the third three-dimensional model, the first sub-model being the model corresponding to the first structure, and the second sub-model being the model corresponding to the base of the robotic arm. At this point, the step "Based on the first 3D model, the second 3D model, and the first pose, obtain the second pose of the second 3D model in the first coordinate system" includes the following steps: Based on the first 3D model, the second 3D model, and the first pose, obtain n first candidate poses, where n is a positive integer. When the pose of the second 3D model is a first candidate pose, the axis of the placement of the hole model is aligned with the target path, and the first 3D model and the second 3D model do not intersect. Based on the first orientation relationship, determine the second pose from the n first candidate poses, where the second orientation relationship indicates whether the first sub-model faces the second sub-model when the pose of the second 3D model is the second pose. The first orientation relationship is the same as the second orientation relationship.

[0121] For example, Figure 3 This is a schematic diagram illustrating the connection between a guide and a robotic arm, as provided in an embodiment of this application. Figure 3 As shown, the guide is connected to the robotic arm via the first structure. Figure 3 The structure of the guide in Figure 2 The guides shown have the same structure, based on Figure 2It can be seen that one end of the guide is a structure including a placement hole, and the other end of the guide is the end face of a cylinder. Figure 3 In the middle, the first structure is the end face of the cylinder. And in... Figure 3 In the middle, the first structure faces the base of the robotic arm.

[0122] The third pose mentioned above is the pose when the robotic arm returns to zero; that is, each movement of the robotic arm begins with the third pose as the initial pose. In other words, when the third 3D model has not started moving, the pose of the third 3D model is the third pose. Furthermore, when the third 3D model has not started moving, the pose of the second 3D model is the fourth pose; in other words, when the robotic arm returns to zero, the pose of the second 3D model is the fourth pose.

[0123] In one possible implementation, after obtaining a first candidate pose, the navigation device can adjust the pose of the second 3D model based on the first candidate pose and display the adjusted second 3D model and the first 3D model. This allows the user to determine whether the orientations of the first and second sub-models are the same as the first orientation. For example, based on the actual positional relationship between the robotic arm and the guide, the user can determine whether the orientations of the first and second sub-models are the same as the first orientation when the guide is facing the left side of the first object. Alternatively, the user can determine whether the orientations of the first and second sub-models are the same as the first orientation when the guide is facing the right side of the first object. This allows for adjustment of the pose of the second 3D model to ensure that the orientations of the first and second sub-models are the same as the first orientation. For example... Figure 4 This is a schematic diagram of a first three-dimensional model and a second three-dimensional model provided in an embodiment of this application, wherein, Figure 4 The pose of the second 3D model in the first coordinate system is the first candidate pose. Figure 5 This is a schematic diagram of another first three-dimensional model and a second three-dimensional model provided in an embodiment of this application. Figure 6 This is a schematic diagram illustrating yet another first three-dimensional model and a second three-dimensional model provided in an embodiment of this application. Wherein, Figure 5 The second three-dimensional model and Figure 6 The second three-dimensional models in the text are all obtained through the analysis of... Figure 4 The pose of the second 3D model in the model is adjusted.

[0124] As mentioned earlier, by controlling the movement of the robotic arm, the guide can be moved, thereby adjusting the guide's posture. If we define the range of motion when the orientation relationship between the first structure and the base changes as the first motion difficulty, and the range of motion when the orientation relationship between the first structure and the base remains unchanged as the second motion difficulty, then based on... Figure 3It is evident that the first motion is more difficult than the second motion. Specifically, because it changes the orientation relationship between the first structure and the base, the robotic arm's motion needs to satisfy at least one of the following: a larger range of motion, a larger pose change, or a longer motion path. Therefore, the first motion is more difficult than the second motion.

[0125] Therefore, during the simulation process, the navigation device can determine the second pose of the second 3D model based on whether the orientation relationship between the model corresponding to the first structure and the model corresponding to the base changes. This reduces the difficulty of the robotic arm's movement and thus improves the difficulty and accuracy of controlling the robotic arm's movement based on the second pose.

[0126] In this implementation, the first orientation relationship is the orientation relationship between the model corresponding to the first structure (i.e., the aforementioned first sub-model) and the model corresponding to the base (i.e., the second sub-model) when the robotic arm is zeroed. The second orientation relationship is the orientation relationship between the first sub-model and the second sub-model when the pose of the second 3D model is the second pose. If the first orientation relationship and the second orientation relationship are the same, the movement difficulty of the third 3D model from the third pose to the final pose can be reduced. Specifically, when the pose of the third 3D model is the final pose, the pose of the second 3D model is the second pose. Therefore, the navigation device first obtains n first candidate poses based on the first 3D model, the second 3D model, and the first pose, which align the axis of the hole model with the target path. Then, it selects from the n first candidate poses the pose that has the same first and second orientation relationships as the second pose.

[0127] Optionally, "the first orientation relationship and the second orientation relationship are the same" means that both the first orientation relationship and the second orientation relationship indicate that the first sub-model is oriented toward the second sub-model, or that both the first orientation relationship and the second orientation relationship indicate that the first sub-model is not oriented toward the second sub-model.

[0128] Optionally, the first structure is the end face of the guide, in which case the first sub-model is the model corresponding to this end face. When the angle between the normal vector and the reference vector of the first sub-model is less than or equal to a second threshold, the first sub-model faces the second sub-model; when the angle between the normal vector and the reference vector of the first sub-model is greater than the second threshold, the first sub-model does not face the second sub-model. The reference vector is perpendicular to the bottom surface of the robot arm's base, its starting point is the center of the bottom surface of the base, and it points towards the base. The robot arm is fixed to the ground, and the bottom surface of the base is in contact with the ground.

[0129] As an optional implementation, before the navigation device performs the step "determine the second pose from n first candidate poses based on the first orientation relationship", the following steps are also performed: Obtaining a first motion range in a first coordinate system, wherein the first motion range is the motion range of the third 3D model. Based on the first motion range, obtaining m second candidate poses, where m is a positive integer, and the pose of the second 3D model is a second candidate pose when the third 3D model moves within the first motion range. Given m second candidate poses, determining the second pose from n first candidate poses based on the first orientation relationship includes the following steps: Based on the first orientation relationship, determining x third candidate poses from the m second candidate poses, where x is a positive integer less than or equal to m. Based on the x third candidate poses and n first candidate poses, obtaining the second pose, wherein both the x third candidate poses and the n first candidate poses include the second pose.

[0130] In this implementation, the first range of motion is determined based on the movable range of the robotic arm, which is determined based on the movable range of each joint of the robotic arm. Since the robotic arm cannot exceed its movable range during actual movement, after acquiring the first range of motion, the navigation device first obtains m second candidate poses based on the first range of motion. When the pose of the third 3D model is any one of the m second candidate poses, the third 3D model is within the first range of motion. After obtaining the m second candidate poses, based on the first orientation relationship, x third candidate poses that satisfy the first orientation relationship are selected from the m second candidate poses. At this point, when the pose of the third 3D model is any one of the x third candidate poses, the third 3D model is within the first range of motion, and the orientation relationship between the first sub-model and the second sub-model is the same as the first orientation relationship. Finally, based on the intersection of x third candidate poses and n first candidate poses, the second pose can be obtained. Thus, the pose of the third 3D model can satisfy the following conditions when it is the second pose: the third 3D model is within the first motion range, the orientation relationship between the first sub-model and the second sub-model is the same as the first orientation relationship, the axis of the placed hole model is aligned with the target path, and the first 3D model and the second 3D model do not intersect.

[0131] As an optional implementation, determining the second pose from n first candidate poses based on a first orientation relationship includes the following steps: Based on the first orientation relationship, determining y fourth candidate poses from the n first candidate poses, where y is a positive integer less than or equal to n; a third orientation relationship indicates whether the first sub-model faces the second sub-model when the pose of the second 3D model is a fourth candidate pose, and the first orientation relationship is the same as the third orientation relationship. Based on the y fourth candidate poses, obtaining r first motion paths, where, when the third 3D model moves based on the first motion path, the second 3D model moves from the fourth pose to the fourth candidate pose, and r is a positive integer less than or equal to y. Determining the shortest path among the r first motion paths as the second motion path. Based on the poses corresponding to the second motion path among the y fourth candidate poses, obtaining the second pose.

[0132] In this implementation, the navigation device first selects y fourth candidate poses from n first candidate poses that satisfy a first orientation relationship. Then, based on the y fourth candidate poses, r first motion paths are obtained. The third 3D model moves based on the first motion paths, which can adjust the pose of the third 3D model from the third pose to the fourth candidate pose. Optionally, the first motion path is a structure in the robotic arm connected to the first structure; for example, the first motion path is the motion path of a flange. Then, the shortest path among the r first motion paths is determined as the second motion path, and the second pose is obtained based on the pose corresponding to the second motion path among the y fourth candidate poses. By controlling the robotic arm movement based on the second pose, the movement distance of the robotic arm can be reduced, thereby reducing the energy consumption of the robotic arm and lowering the difficulty of controlling the robotic arm.

[0133] Optionally, optical markers are affixed to the surface of the first object. Correspondingly, the first 3D model includes an optical marker sub-model. During simulation, the navigation device also detects whether the second and third 3D models occlude the optical marker sub-model during movement. Specifically, if the optical marker sub-model is occluded, the light reflected from it cannot be detected by the optical tracking 3D model, which is the 3D model of the optical tracking device. Based on the detection results of whether the optical marker sub-model is occluded, a second motion path can be determined so that the third 3D model, when moving along the second motion path, will not cause the optical marker sub-model to be occluded.

[0134] For example, Figure 7 This is a schematic diagram of a simulated navigation interface provided in an embodiment of this application. Figure 7As shown, the simulation navigation interface includes a first 3D model, a second 3D model, a third 3D model, and an optical tracking 3D model. The first 3D model includes an optical marker sub-model. During the simulation, controlling the movement of the third 3D model can move the second 3D model, thus bringing the second 3D model closer to the first 3D model. During this process, the navigation device can detect whether the optical marker sub-model is obstructed.

[0135] In one optional implementation, the navigation device can combine the first two implementations to obtain the second pose. Specifically, before determining the second pose from n first candidate poses based on the first orientation relationship, the navigation device performs the following steps: obtaining a first motion range in a first coordinate system. Based on the first motion range, obtaining m second candidate poses. Then, during the process of performing the step "determining the second pose from n first candidate poses based on the first orientation relationship," the following steps are performed: determining x third candidate poses from the m second candidate poses based on the first orientation relationship. Based on the x third candidate poses, obtaining s first motion paths, where s is a positive integer less than or equal to x. Determining the shortest path among the s first motion paths as the fourth motion path. Based on the pose corresponding to the fourth motion path among the x third candidate poses, obtaining the second pose.

[0136] As an optional implementation, based on y fourth candidate poses, r first motion paths are obtained, including the following steps: Based on y fourth candidate poses, y third motion paths are obtained, wherein each third motion path corresponds one-to-one with a fourth candidate pose. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose. Based on y third motion paths, y first tolerance ranges are obtained, wherein each first tolerance range corresponds one-to-one with a third motion path, and the first tolerance range is the range where the distance to the third motion path is less than or equal to a first threshold. The number of third motion paths within each y first tolerance range is determined, resulting in y path counts. The ranges corresponding to the r largest counts among the y first tolerance ranges are determined as r second tolerance ranges. The paths corresponding to the r second tolerance ranges among the y third motion paths are determined as r first motion paths.

[0137] On the one hand, since the y third motion paths are obtained based on the fourth candidate pose, the third 3D model can satisfy the following conditions when moving based on any of the third motion paths: the third 3D model is within the first motion range, the orientation relationship between the first and second sub-models is the same as the first orientation relationship, the axis of the hole model is aligned with the target path, and the first and second 3D models do not intersect. In other words, any third motion path is a valid motion path. On the other hand, when the third 3D model moves based on the third motion path, there may be a deviation between the actual motion path of the third 3D model and the third motion path. However, if any of the third motion paths is valid, for any one of the y third motion paths, the more other third motion paths surrounding it, the greater the probability that the actual motion path of the third 3D model, even if it deviates from the third motion path, is still a valid motion path. In other words, the more other third motion paths surrounding the third motion path, the greater the tolerance space for the third 3D model to move based on that third motion path.

[0138] Based on the above two aspects, in this embodiment of the navigation device, y first tolerance ranges are obtained based on y third motion paths. For example, the y third motion paths include third motion path p1 and third motion path p2, and the y first tolerance ranges include first tolerance range f1 and first tolerance range f2. The first tolerance range f1 is the range where the distance to third motion path p1 is less than or equal to a first threshold, and the first tolerance range f2 is the range where the distance to third motion path p2 is less than or equal to the first threshold. That is, the first tolerance range f1 corresponds to third motion path p1, and the first tolerance range f2 corresponds to third motion path p2. Optionally, the first threshold is 1 centimeter, meaning the first tolerance range is the range within 1 centimeter of the third motion path.

[0139] After obtaining y first tolerance ranges, the number of third motion paths within each first tolerance range is determined, resulting in y path counts. A larger path count indicates a stronger tolerance capability of the first tolerance range, meaning a larger tolerance space for the third 3D model based on the third motion paths corresponding to the first tolerance ranges. For example, the y third motion paths include third motion path p1, third motion path p2, and third motion path p3; the y first tolerance ranges include first tolerance range f1, first tolerance range f2, and first tolerance range f3. First tolerance range f1 corresponds to third motion path p1, first tolerance range f2 corresponds to third motion path p2, and first tolerance range f3 corresponds to third motion path p3. If first tolerance ranges f1 and f2 both include third motion paths p1 and p2, and first tolerance range f3 includes third motion path p3, then the path counts for first tolerance range f1 and f2 are both 2, and the path count for first tolerance range f3 is 1.

[0140] After obtaining the number of y paths, the navigation device determines the ranges corresponding to the r largest numbers among the y first tolerance ranges as r second tolerance ranges. In other words, the r second tolerance ranges are the r ranges with the strongest fault tolerance among the y first tolerance ranges. Then, it determines the paths corresponding to the r second tolerance ranges among the y third motion paths as r first motion paths. This improves the fault tolerance of the r first motion paths, thus increasing the probability that the actual motion path of the third 3D model is a reasonable motion path when it moves based on any of the r first motion paths.

[0141] As an optional implementation, based on y fourth candidate poses, r first motion paths are obtained, including the following steps: Based on y fourth candidate poses, y third motion paths are obtained, wherein each third motion path corresponds one-to-one with a fourth candidate pose. While the third 3D model moves based on the third motion paths, the second 3D model moves from the fourth pose to the fourth candidate pose. The maximum curvature of the y third motion paths is determined to obtain y first curvatures. Based on the r smallest curvatures among the y first curvatures, r second curvatures are obtained. Based on the paths among the y first motion paths that correspond to the r second curvatures, r first motion paths are obtained.

[0142] Because the greater the maximum curvature of the motion path of the third 3D model, the greater the difficulty of the third 3D model's motion, which is also the greater the difficulty of the robotic arm's motion, in this embodiment of the navigation device, based on the smallest r curvatures among y first curvatures, obtains r second curvatures, and then obtains r first motion paths based on the r second curvatures, which can reduce the difficulty of the third 3D model's motion based on the first motion paths.

[0143] As an optional implementation, the navigation device obtains r first motion paths based on the first two implementations. Specifically, during the execution of step "obtaining r first motion paths based on y fourth candidate poses", the navigation device performs the following steps: Based on y fourth candidate poses, obtain y third motion paths. Based on y third motion paths, obtain y first tolerance ranges. Determine the number of third motion paths within each of the y first tolerance ranges, obtaining y path counts. Determine the ranges corresponding to the largest t counts among the y path counts within the y first tolerance ranges as t second tolerance ranges, where t is a positive integer, t is less than or equal to y, and t is greater than or equal to r. Determine the paths corresponding to the t second tolerance ranges among the y third motion paths as t fifth motion paths. Determine the maximum curvature of the t fifth motion paths, obtaining t third curvatures. Based on the smallest r curvatures among the t third curvatures, obtain r second curvatures. Based on the paths corresponding to the r second curvatures among the t fifth motion paths, obtain r first motion paths.

[0144] As an optional implementation, the guide is connected to the robotic arm, and the navigation device further performs the following steps: based on the second pose and the first transformation relationship, a third pose of the guide in a second coordinate system is obtained, wherein the second coordinate system is a coordinate system constructed based on the base of the robotic arm. Based on the third pose, the movement of the robotic arm is controlled so that the pose of the guide in the second coordinate system is the third pose.

[0145] Since the robotic arm moves based on pose information in the second coordinate system, in this embodiment, after obtaining the second pose, the navigation device first converts the second pose into a third pose in the second coordinate system based on a first transformation relationship. Then, based on the third pose, the robotic arm's movement can be controlled so that the guide's pose in the second coordinate system is the third pose. This allows for control of the robotic arm's movement based on the second pose.

[0146] In some schemes, the navigation device will convert the second pose into a sixth pose in the optical coordinate system based on the second transformation relation. Optionally, the second pose is represented by TTargetTracer2CT, the sixth pose by TTargetTracer2OTS, and the second transformation relation by registrationMatrix. Then, TTargetTracer2CT, TTargetTracer2OTS, and registrationMatrix satisfy the following equation: TTargetTracer2OTS = registrationMatrix × TTargetTracer2CT.

[0147] The navigation device acquires the seventh pose of the guide in the optical coordinate system, where the seventh pose is obtained by locating optical markers in the guide using an optical tracking device. Optionally, all poses obtained by the optical tracking device are poses in the optical coordinate system. Then, based on the sixth and seventh poses, a first transformation matrix is ​​obtained, where the first transformation matrix is ​​used to transform the seventh pose into the sixth pose or vice versa. Optionally, let TTargetTracer2CurrentTracer represent the first transformation matrix, TTargetTracer2OTS represent the sixth pose, and TCurrentTracer2OTS represent the seventh pose. Then, TTargetTracer2CurrentTracer, TTargetTracer2OTS, and TCurrentTracer2OTS satisfy the following equation: TTargetTracer2CurrentTracer = TCurrentTracer2OTS - ¹ × TTargetTracer2OTS. After obtaining the first transformation matrix, based on the relative positional relationship between the guide and the base of the robotic arm, the first transformation matrix is ​​converted into the target pose in the second coordinate system.

[0148] Optionally, before executing the step "obtaining the second pose of the second 3D model in the first coordinate system based on the first 3D model, the second 3D model, and the first pose," the navigation device also obtains the eighth pose of the fourth 3D model in the first coordinate system, where the fourth 3D model is a 3D model of a bed, and the first object lies on the bed. At this time, obtaining the second pose of the second 3D model in the first coordinate system based on the first 3D model, the second 3D model, and the first pose includes: obtaining the second pose of the second 3D model in the first coordinate system based on the first 3D model, the second 3D model, the fourth 3D model, the first pose, and the eighth pose. In this case, when the pose of the second 3D model is the second pose, the axis of the hole model is aligned with the target path, and the first 3D model does not intersect with either the second or fourth 3D model. Thus, if the pose of the guide is adjusted based on the second pose so that the axis of the hole in the guide is aligned with the target path, then when the axis of the hole in the guide is aligned with the target path, the guide will not collide with the first object or the bed.

[0149] Optionally, when the pose of the second 3D model is the second pose, the angle between the axis of the Confucius model and the starting point normal vector is less than or equal to a third threshold, wherein the starting point normal vector is the normal vector at the starting point of the target path.

[0150] Optionally, the navigation device can also receive a reference pose input by the user, where the reference pose is the pose of the second 3D model in the first coordinate system. Based on the first 3D model, the second 3D model, the third 3D model, the first pose, the reference pose, and the third pose, the navigation device simulates the motion process of the third 3D model starting from the third pose, using the pose of the second 3D model as the reference pose. It can also detect whether collisions occur during this motion and whether the optical marker sub-model is occluded. The detection results can then be displayed so that the user can obtain the detection results.

[0151] Optionally, if the detection results indicate that a collision has occurred and / or occlusion exists, the collision and / or occlusion situation can be highlighted, such as by highlighting or displaying text describing the collision and / or occlusion situation, which can help the user adjust the reference pose.

[0152] For example, Figure 8 This is a schematic diagram illustrating a collision between a first three-dimensional model and a second three-dimensional model, provided as an embodiment of this application. Figure 8 As shown, the first 3D model intersects with the second 3D model, meaning a collision occurs. In this case, the user can adjust the reference pose to prevent the first 3D model from colliding with the second 3D model. For example, Figure 9 This is a schematic diagram illustrating a first three-dimensional model and a second three-dimensional model that do not collide, as provided in an embodiment of this application. Figure 10 This is a schematic diagram illustrating another instance of a first three-dimensional model and a second three-dimensional model not colliding, provided as an embodiment of this application. Figure 9 and Figure 10 All results are obtained from simulations based on the user's adjusted reference pose. For example... Figure 9 and Figure 10 As shown, the first 3D model and the second 3D model do not intersect, meaning that the first 3D model and the second 3D model do not collide.

[0153] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0154] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, while using clear signs / information to inform users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0155] The methods of the embodiments of this application have been described in detail above, and the apparatus of the embodiments of this application is provided below.

[0156] Please see Figure 11 , Figure 11 This is a schematic diagram of a simulation-based navigation device provided in an embodiment of this application. The simulation-based navigation device 1 includes: an acquisition unit 11 and a processing unit 12, wherein:

[0157] The acquisition unit 11 is used to acquire a target three-dimensional image and a target path. The target three-dimensional image includes a first object, and the target path is a path that moves from the skin area of ​​the first object toward a target point inside the first object.

[0158] The acquisition unit 11 is also used to acquire a first three-dimensional model of the first object and a second three-dimensional model of the guide. The second three-dimensional model includes a placement hole model, which is a model corresponding to the placement hole in the guide. When the second object is placed in the placement hole, the second object can move along the axis of the placement hole. The second object is an object that moves from the skin area of ​​the first object toward a target point inside the first object.

[0159] Processing unit 12 is used to obtain the first pose of the first three-dimensional model in a first coordinate system based on the target three-dimensional image, wherein the first coordinate system is the image coordinate system of the target three-dimensional image;

[0160] The processing unit 12 is further configured to obtain a second pose of the second three-dimensional model in the first coordinate system based on the first three-dimensional model, the second three-dimensional model, and the first pose. When the pose of the second three-dimensional model is the second pose, the axis of the placement hole model is aligned with the target path, and the first three-dimensional model and the second three-dimensional model do not intersect.

[0161] In any embodiment of this application, the first structure of the guide is connected to the robotic arm;

[0162] The acquisition unit 11 is further configured to acquire the third pose of the third three-dimensional model of the robotic arm in the first coordinate system and the fourth pose of the second three-dimensional model in the first coordinate system. When the third three-dimensional model has not started to move, the pose of the third three-dimensional model is the third pose and the pose of the second three-dimensional model is the fourth pose.

[0163] The processing unit 12 is further configured to obtain a first orientation relationship based on the third pose and the fourth pose. The first orientation relationship indicates whether the first sub-model in the second three-dimensional model is facing the second sub-model in the third three-dimensional model. The first sub-model is a model corresponding to the first structure, and the second sub-model is a model corresponding to the base of the robotic arm.

[0164] The processing unit 12 is further configured to obtain n first candidate poses based on the first three-dimensional model, the second three-dimensional model, and the first pose, where n is a positive integer. When the pose of the second three-dimensional model is the first candidate pose, the axis of the placement of the hole model is aligned with the target path, and the first three-dimensional model and the second three-dimensional model do not intersect.

[0165] The processing unit 12 is further configured to determine the second pose from the n first candidate poses based on the first orientation relationship. The second orientation relationship indicates whether the first sub-model is facing the second sub-model when the pose of the second three-dimensional model is the second pose. The first orientation relationship is the same as the second orientation relationship.

[0166] In any embodiment of this application, the acquisition unit 11 is further configured to acquire a first motion range in the first coordinate system, wherein the first motion range is the motion range of the third three-dimensional model;

[0167] The processing unit 12 is further configured to obtain m second candidate poses based on the first motion range, where m is a positive integer. When the third three-dimensional model moves within the first motion range, the pose of the second three-dimensional model is the second candidate pose.

[0168] The processing unit 12 is further configured to determine x third candidate poses from the m second candidate poses based on the first orientation relationship, wherein x is a positive integer less than or equal to m;

[0169] The processing unit 12 is further configured to obtain the second pose based on the x third candidate poses and the n first candidate poses, wherein the x third candidate poses and the n first candidate poses all include the second pose.

[0170] In any embodiment of this application, the processing unit 12 is further configured to determine y fourth candidate poses from the n first candidate poses based on the first orientation relationship, where y is a positive integer less than or equal to n, and the third orientation relationship indicates whether the first sub-model is oriented towards the second sub-model when the pose of the second three-dimensional model is the fourth candidate pose, and the first orientation relationship is the same as the third orientation relationship.

[0171] The processing unit 12 is further configured to obtain r first motion paths based on the y fourth candidate poses, and when the third three-dimensional model moves based on the first motion paths, the second three-dimensional model moves from the fourth pose to the fourth candidate pose, where r is a positive integer less than or equal to y.

[0172] The processing unit 12 is further configured to determine the shortest path among the r first motion paths as the second motion path;

[0173] The processing unit 12 is further configured to obtain the second pose based on the pose corresponding to the second motion path among the y fourth candidate poses.

[0174] In conjunction with any embodiment of this application, the processing unit 12 is further configured to:

[0175] Based on the y fourth candidate poses, y third motion paths are obtained. The third motion paths correspond one-to-one with the fourth candidate poses. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose.

[0176] Based on the y third motion paths, y first fault tolerance ranges are obtained. The first fault tolerance range corresponds one-to-one with the third motion path. The first fault tolerance range is the range where the distance to the third motion path is less than or equal to a first threshold.

[0177] The number of the third motion paths within each of the y first fault tolerance ranges is determined to obtain the number of y paths;

[0178] The ranges corresponding to the r largest quantities among the y first fault tolerance ranges are determined as r second fault tolerance ranges;

[0179] The path that corresponds to the r second fault tolerance range among the y third motion paths is determined as the r first motion paths.

[0180] In conjunction with any embodiment of this application, the processing unit 12 is further configured to:

[0181] Based on the y fourth candidate poses, y third motion paths are obtained. The third motion paths correspond one-to-one with the fourth candidate poses. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose.

[0182] Determine the maximum curvature of the y third motion paths to obtain the y first curvatures;

[0183] Based on the r curvatures that are the smallest among the y first curvatures, r second curvatures are obtained;

[0184] Based on the paths corresponding to the r second curvatures among the y third motion paths, the r first motion paths are obtained.

[0185] In any embodiment of this application, the guide is connected to the robotic arm, and the processing unit 12 is further configured to:

[0186] Based on the second pose and the first transformation relationship, the third pose of the guide in the second coordinate system is obtained, where the second coordinate system is a coordinate system constructed based on the base of the robotic arm;

[0187] Based on the third pose, the movement of the robotic arm is controlled so that the pose of the guide in the second coordinate system is the third pose.

[0188] In this embodiment, after acquiring a target 3D image and a first 3D model, the navigation device can obtain a first pose of the first 3D model in a first coordinate system based on the target 3D image. Then, based on the first 3D model, a second 3D model, and the first pose, a second pose of the second 3D model in the first coordinate system can be obtained. When the pose of the second 3D model is the second pose, the axis of the hole placement model is aligned with the target path, and the first 3D model and the second 3D model do not intersect. Thus, through simulation, the pose of the guide that satisfies the following conditions can be obtained: the axis of the hole placement in the guide is aligned with the target path, and the guide does not collide with the first object.

[0189] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0190] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, memory 22, input device 23, and output device 24 are coupled together via connectors, which include various interfaces, transmission lines, or buses, etc., and are not limited in this embodiment. It should be understood that in the various embodiments of this application, coupling refers to mutual connection in a specific way, including direct connection or indirect connection through other devices, such as through various interfaces, transmission lines, buses, etc.

[0191] The processor 21 can be one or more graphics processing units (GPUs). If the processor 21 is a GPU, the GPU can be a single-core GPU or a multi-core GPU. Optionally, the processor 21 can be a processor group composed of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Optionally, the processor can also be other types of processors, etc., which are not limited in this embodiment.

[0192] The memory 22 can be used to store computer program instructions, as well as various types of computer program code, including program code for executing the scheme of this application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0193] Input device 23 is used to input data and / or signals, and output device 24 is used to output data and / or signals. Input device 23 and output device 24 can be independent devices or an integrated device.

[0194] It is understood that in this embodiment of the application, the memory 22 can be used not only to store related instructions, but also to store related data. This embodiment of the application does not limit the specific data stored in the memory.

[0195] Understandable Figure 12 This is merely a simplified design of an electronic device. In practical applications, the electronic device may also include other necessary components, including, but not limited to, any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of this application are within the protection scope of this application.

[0196] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0197] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will also readily understand that the various embodiments of this application have different focuses, and for the sake of convenience and brevity, the same or similar parts may not be repeated in different embodiments. Therefore, parts not described or not described in detail in one embodiment can be referred to the descriptions in other embodiments.

[0198] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0199] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0200] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0201] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0202] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A simulation-based navigation method, characterized by, The method includes: Acquire a target 3D image and a target path, wherein the target 3D image includes a first object, and the target path is a path that moves from the skin region of the first object toward a target point within the first object; A first three-dimensional model of the first object and a second three-dimensional model of the guide are obtained. The second three-dimensional model includes a placement hole model, which is a model corresponding to the placement hole in the guide. When the second object is placed in the placement hole, the second object can move along the axis of the placement hole. The second object is an object that moves from the skin area of ​​the first object toward a target point in the body of the first object. The first structure of the guide is connected to the robotic arm. Based on the target 3D image, the first pose of the first 3D model in the first coordinate system is obtained, where the first coordinate system is the image coordinate system of the target 3D image; The third pose of the third 3D model of the robotic arm in the first coordinate system and the fourth pose of the second 3D model in the first coordinate system are obtained. When the third 3D model has not started to move, the pose of the third 3D model is the third pose and the pose of the second 3D model is the fourth pose. Based on the third pose and the fourth pose, a first orientation relationship is obtained. The first orientation relationship indicates whether the first sub-model in the second three-dimensional model is facing the second sub-model in the third three-dimensional model. The first sub-model is the model corresponding to the first structure, and the second sub-model is the model corresponding to the base of the robotic arm. Based on the first 3D model, the second 3D model, and the first pose, the second pose of the second 3D model in the first coordinate system is obtained. The process of obtaining the second pose of the second 3D model in the first coordinate system based on the first 3D model, the second 3D model, and the first pose includes: obtaining n first candidate poses based on the first 3D model, the second 3D model, and the first pose, where n is a positive integer. When the pose of the second 3D model is the first candidate pose, the axis of the placement of the Confucius model is aligned with the target path, and the first 3D model and the second 3D model do not intersect. Based on the first orientation relationship, the second pose is determined from the n first candidate poses. The second orientation relationship indicates whether the first sub-model is facing the second sub-model when the pose of the second 3D model is the second pose. The first orientation relationship is the same as the second orientation relationship. When the pose of the second 3D model is the second pose, the axis of the hole placement model is aligned with the target path, and the first 3D model and the second 3D model do not intersect.

2. The method of claim 1, wherein, Before determining the second pose from the n first candidate poses based on the first orientation relationship, the method further includes: Obtain the first range of motion in the first coordinate system, where the first range of motion is the range of motion of the third three-dimensional model; Based on the first motion range, m second candidate poses are obtained, where m is a positive integer. When the third 3D model moves within the first motion range, the pose of the second 3D model is the second candidate pose. The step of determining the second pose from the n first candidate poses based on the first orientation relationship includes: Based on the first orientation relationship, x third candidate poses are determined from the m second candidate poses, where x is a positive integer less than or equal to m; The second pose is obtained based on the x third candidate poses and the n first candidate poses, wherein the x third candidate poses and the n first candidate poses all include the second pose.

3. The method of claim 1, wherein, The step of determining the second pose from the n first candidate poses based on the first orientation relationship includes: Based on the first orientation relationship, y fourth candidate poses are determined from the n first candidate poses, where y is a positive integer less than or equal to n. The third orientation relationship indicates whether the first sub-model is oriented towards the second sub-model when the pose of the second 3D model is the fourth candidate pose. The first orientation relationship is the same as the third orientation relationship. Based on the y fourth candidate poses, r first motion paths are obtained. When the third 3D model moves based on the first motion paths, the second 3D model moves from the fourth pose to the fourth candidate pose, where r is a positive integer less than or equal to y. The shortest path among the r first motion paths is determined as the second motion path; The second pose is obtained based on the pose corresponding to the second motion path among the y fourth candidate poses.

4. The method of claim 3, wherein, The process of obtaining r first motion paths based on the y fourth candidate poses includes: Based on the y fourth candidate poses, y third motion paths are obtained. The third motion paths correspond one-to-one with the fourth candidate poses. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose. Based on the y third motion paths, y first fault tolerance ranges are obtained. The first fault tolerance range corresponds one-to-one with the third motion path. The first fault tolerance range is the range where the distance to the third motion path is less than or equal to a first threshold. The number of the third motion paths within each of the y first fault tolerance ranges is determined to obtain the number of y paths; The ranges corresponding to the r largest quantities among the y first fault tolerance ranges are determined as r second fault tolerance ranges; The path that corresponds to the r second fault tolerance range among the y third motion paths is determined as the r first motion paths.

5. The method of claim 3, wherein, The process of obtaining r first motion paths based on the y fourth candidate poses includes: Based on the y fourth candidate poses, y third motion paths are obtained. The third motion paths correspond one-to-one with the fourth candidate poses. When the third 3D model moves based on the third motion path, the second 3D model moves from the fourth pose to the fourth candidate pose. Determine the maximum curvature of the y third motion paths to obtain the y first curvatures; Based on the r curvatures that are the smallest among the y first curvatures, r second curvatures are obtained; Based on the paths corresponding to the r second curvatures among the y third motion paths, the r first motion paths are obtained.

6. The method according to any one of claims 1 to 5, characterized in that, The guide is connected to the robotic arm, and the method further includes: Based on the second pose and the first transformation relationship, the third pose of the guide in the second coordinate system is obtained, where the second coordinate system is a coordinate system constructed based on the base of the robotic arm; Based on the third pose, the movement of the robotic arm is controlled so that the pose of the guide in the second coordinate system is the third pose.

7. A simulation-based navigation device, characterized by The simulation-based navigation device includes: An acquisition unit is used to acquire a target 3D image and a target path. The target 3D image includes a first object, and the target path is a path that moves from the skin region of the first object toward a target point within the first object. The acquisition unit is further configured to acquire a first three-dimensional model of the first object and a second three-dimensional model of the guide. The second three-dimensional model includes a placement hole model, which is a model corresponding to the placement hole in the guide. When the second object is placed in the placement hole, the second object can move along the axis of the placement hole. The second object is an object that moves from the skin area of ​​the first object toward a target point inside the first object. The first structure of the guide is connected to the robotic arm. The processing unit is configured to obtain the first pose of the first three-dimensional model in a first coordinate system based on the target three-dimensional image, wherein the first coordinate system is the image coordinate system of the target three-dimensional image; The acquisition unit is further configured to acquire the third pose of the third three-dimensional model of the robotic arm in the first coordinate system and the fourth pose of the second three-dimensional model in the first coordinate system. When the third three-dimensional model has not started to move, the pose of the third three-dimensional model is the third pose and the pose of the second three-dimensional model is the fourth pose. The processing unit is further configured to obtain a first orientation relationship based on the third pose and the fourth pose. The first orientation relationship indicates whether the first sub-model in the second three-dimensional model is facing the second sub-model in the third three-dimensional model. The first sub-model is a model corresponding to the first structure, and the second sub-model is a model corresponding to the base of the robotic arm. The processing unit is further configured to obtain a second pose of the second three-dimensional model in the first coordinate system based on the first three-dimensional model, the second three-dimensional model, and the first pose. Obtaining the second pose of the second three-dimensional model in the first coordinate system based on the first three-dimensional model, the second three-dimensional model, and the first pose includes: obtaining n first candidate poses based on the first three-dimensional model, the second three-dimensional model, and the first pose, where n is a positive integer; when the pose of the second three-dimensional model is one of the first candidate poses, the axis of the placement of the hole model is aligned with the target path, and the first three-dimensional model and the second three-dimensional model do not intersect; determining the second pose from the n first candidate poses based on the first orientation relationship; the second orientation relationship indicates whether the first sub-model faces the second sub-model when the pose of the second three-dimensional model is the second pose; the first orientation relationship is the same as the second orientation relationship; when the pose of the second three-dimensional model is the second pose, the axis of the placement of the hole model is aligned with the target path, and the first three-dimensional model and the second three-dimensional model do not intersect.

8. A surgical robot, characterized by Including the simulation-based navigation device as described in claim 7.

9. An electronic device, comprising: include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1 to 6.

Images