Osteotomy control apparatus, computer device, and computer program product

Abstract

The embodiment of the invention is suitable for the technical field of computer-aided medical treatment and data processing, and provides an osteotomy control device, computer equipment and a computer program product, the device comprises a visual sensor used for capturing the spatial geometric features of an operation area and the six-dimensional pose of a mechanical arm end effector; the data processing unit is used for reconstructing to obtain a safe osteotomy operation boundary and projecting the safe osteotomy operation boundary and the contour of the end effector to a two-dimensional plane; a real-time feedback signal is generated based on the position relation between the safe osteotomy operation boundary in the two-dimensional plane and the outline of the end effector; and the mechanical arm control unit is used for controlling the mechanical arm to drive the end effector to move towards the interior of the safe osteotomy operation boundary according to the real-time feedback signal when the end effector exceeds the safe osteotomy operation boundary. By the adoption of the method, it can be guaranteed that osteotomy operation is conducted in a safe area all the time, and operation safety is guaranteed.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202610183657.9 (22) Application Date 2026.02.09 (71) Applicant: Gushengyuan Robotics (Shenzhen) Co., Ltd. Address: Room 2101, Building D1, Nanshan Zhiyuan, Changyuan Community, Taoyuan Street, Nanshan District, Shenzhen, Guangdong Province, 518000 (72) Inventors: Wang Weiwei, Liu Tingting (74) Patent Agency: Shenzhen Zhongyi United Intellectual Property Agency Co., Ltd. 44414 Patent Attorney: Ren Min (51) Int.Cl. A61B 17 / 16 (2006.01) A61B 34 / 10 (2016.01) A61B 34 / 30 (2016.01) A61B 34 / 00 (2016.01) G06T 7 / 70 (2017.01) B25J 9 / 16 (2006.01) (54) Invention Title: Osteotomy Control Device, Computer Equipment, and Computer Program Product (57) Abstract: This application applies to the fields of computer-aided medical technology and data processing technology, and provides an osteotomy control device, computer equipment, and computer program product. The device includes: a vision sensor for capturing the spatial geometric features of the surgical area and the six-dimensional pose of the end effector of a robotic arm; a data processing unit for reconstructing a safe osteotomy operation boundary and projecting the safe osteotomy operation boundary and the contour of the end effector onto a two-dimensional plane; generating a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary and the contour of the end effector in the two-dimensional plane; and a robotic arm control unit for controlling the robotic arm to move the end effector into the safe osteotomy operation boundary when the end effector crosses the safe osteotomy operation boundary according to the real-time feedback signal. By using the above method, it can be ensured that the osteotomy operation is always performed within a safe area, thus ensuring surgical safety. Claims (3 pages), Description (14 pages), Drawings (7 pages), CN 121667803 A, 2026.03.17, CN 1 21 66 78 03 A. 1. An osteotomy control device, characterized in that it comprises: a vision sensor for capturing the spatial geometric features of the surgical area and the six-dimensional pose of a robotic arm end effector; a data processing unit for reconstructing a safe osteotomy operation boundary based on the spatial geometric features, and projecting the safe osteotomy operation boundary and the contour of the end effector characterized by the six-dimensional pose onto a two-dimensional plane; generating a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary and the contour of the end effector in the two-dimensional plane; and a robotic arm control unit for controlling the end effector to exceed the safe osteotomy according to the real-time feedback signal.When operating at the boundary, the robotic arm is controlled to move the end effector inward into the safe osteotomy operation boundary. 2. The device according to claim 1, wherein the data processing unit is specifically configured to: determine the projection point of the center point of the surgical area in the two-dimensional plane, and connect the projection point with the first endpoint and the second endpoint of the end effector to form a first line segment and a second line segment; calculate whether the first line segment and the second line segment intersect the safe osteotomy operation boundary; and generate a real-time feedback signal when the first line segment and / or the second line segment intersects the safe osteotomy operation boundary. 3. The apparatus according to claim 2, wherein the safe osteotomy operation boundary is composed of multiple boundary line segments, and the data processing unit is further configured to: calculate, during the process of the robotic arm control unit controlling the robotic arm to drive the end effector to perform osteotomy along any boundary line segment, calculate the cross product between the first vector and the second vector corresponding to the first line segment and the second line segment and the boundary vector representing the current boundary line segment; determine whether the first line segment and the second line segment intersect the current boundary line segment based on the cross product; determine the direction and magnitude of the force when the first line segment and / or the second line segment intersects the current boundary line segment, and generate a real-time feedback signal based on the direction and magnitude of the force; wherein the direction of the force is perpendicular to the current boundary line segment and points towards the boundary line segment, the magnitude of the force is proportional to the distance between the target endpoint and the current boundary line segment, and the target endpoint is an endpoint located outside the safe osteotomy operation boundary. 4. The apparatus according to claim 2 or 3, wherein the data processing unit is further configured to: determine the angle type when the first line segment or the second line segment intersects the safe osteotomy operation boundary, wherein the angle is formed by connecting the target endpoint of the end effector with the two endpoints of two adjacent boundary line segments, wherein the two adjacent boundary line segments include the boundary line segment currently intersecting the first line segment or the second line segment in the safe osteotomy operation boundary and another boundary line segment adjacent to the intersecting boundary line segment; determine the direction and magnitude of the force when the angle is obtuse, and generate a real-time feedback signal based on the direction and magnitude of the force; wherein the direction of the force is from a point located outside the safe osteotomy operation boundary on the angle bisector of the two adjacent boundary line segments to the target endpoint; determine the line segment angle formed by the two adjacent boundary line segments when the angle is acute, and generate a real-time feedback signal based on the line segment angle. 5. The apparatus according to claim 4, wherein generating the real-time feedback signal based on the line segment angle includes:When the angle between the line segments formed by the two adjacent boundary segments is a right angle, the direction and magnitude of the force are determined, and a real-time feedback signal is generated based on the direction and magnitude of the force; wherein, the direction of the force is determined by the combination of the directions perpendicularly pointing from the target endpoint to the two adjacent boundary segments; when the angle between the line segments formed by the two adjacent boundary segments is an obtuse angle, the direction and magnitude of the force are determined, and a real-time feedback signal is generated based on the direction and magnitude of the force; wherein, the direction of the force is from the target endpoint to a point located inside the safe osteotomy operation boundary on the angle bisector of the two adjacent boundary segments. 6. The apparatus according to any one of claims 2, 3, or 5, wherein the data processing unit is further specifically configured to: determine, when both the first line segment and the second line segment intersect the safe osteotomy operation boundary, a graphic type formed based on the intersecting safe osteotomy operation boundary, the graphic type including a concave graphic or a convex graphic; determine the direction and magnitude of the force according to the graphic type, and generate a real-time feedback signal according to the direction and magnitude of the force. 7. The apparatus according to claim 6, wherein the data processing unit is further configured to: sort and sequentially assign edge numbers to a plurality of boundary segments constituting the safe osteotomy operation boundary; if the absolute value of the difference between the edge numbers of two boundary segments intersecting the first segment and the second segment is greater than 1, determine a first midpoint between the two closest points of the two boundary segments, and a second midpoint between the first boundary segment and the last boundary segment of the safe osteotomy operation boundary; calculate a first distance between the second midpoint and the end effector and a second distance between the first midpoint and the second midpoint; if the first distance is greater than the second distance, determine that the shape formed based on the two boundary segments is a concave shape; if the absolute value of the difference between the edge numbers of two boundary segments intersecting the first segment and the second segment is equal to 1, or the first distance is less than or equal to the second distance, determine that the shape formed based on the two boundary segments is a convex shape. 8. The apparatus according to claim 7, wherein the data processing unit is further configured to: when the shape formed by the two boundary line segments is a concave shape, determine that the direction of the force is perpendicular to the line connecting the two closest points of the two boundary line segments and points from the first midpoint into the interior of the safe osteotomy operation boundary; when the shape formed by the two boundary line segments is a convex shape, determine that the direction of the force is along the angle bisector of the two boundary line segments and points into the interior of the safe osteotomy operation boundary.9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, the computer device performs the following method: capturing spatial geometric features of a surgical area and a six-dimensional pose of a robotic arm end effector; reconstructing a safe osteotomy operation boundary based on the spatial geometric features, and projecting the safe osteotomy operation boundary and the contour of the end effector characterized by the six-dimensional pose onto a two-dimensional plane; generating a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary and the end effector contour in the two-dimensional plane; and, based on the real-time feedback signal, controlling the robotic arm to move the end effector inward into the safe osteotomy operation boundary when the end effector crosses the safe osteotomy operation boundary. 10. A computer program product, comprising a computer program, characterized in that, when the computer program is executed, the following method is performed: capturing the spatial geometric features of the surgical area and the six-dimensional pose of the end effector of the robotic arm; reconstructing a safe osteotomy operation boundary based on the spatial geometric features, and projecting the safe osteotomy operation boundary and the contour of the end effector characterized by the six-dimensional pose onto a two-dimensional plane; generating a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary and the contour of the end effector in the two-dimensional plane; and controlling the robotic arm to move the end effector inward into the safe osteotomy operation boundary when the end effector crosses the safe osteotomy operation boundary according to the real-time feedback signal. Claims 3 / 3 page 4 CN 121667803 A Osteotomy Control Device, Computer Equipment and Computer Program Product Technical Field

[0001] The embodiments of this application belong to the fields of computer-aided medical technology and data processing technology, and particularly relate to an osteotomy control device, computer equipment and computer program product. Background Art

[0002] Osteotomy is one of the common orthopedic surgical procedures. With the widespread application of computer-aided medical technology, osteotomy performed using computer equipment can reduce the operational difficulty for surgeons and improve surgical efficiency.

[0003] Computer-aided osteotomy usually involves installing an actuator at the end of a robotic arm to perform the osteotomy operation. During this process, it is necessary to collect relevant information about the osteotomy area to generate a safe boundary for the osteotomy operation, and ensure that the end effector does not exceed the safe boundary during the osteotomy process. Therefore, how to accurately determine the positional relationship between the end effector and the safe boundary to ensure that the osteotomy operation is always performed within a safe area is one of the problems that urgently needs to be solved. Summary of the Invention

[0004] In view of this, embodiments of this application provide an osteotomy control device, a computer device, and a computer program product, used to accurately determine the positional relationship between the end effector and the safety boundary, prevent the end effector from exceeding the safety boundary, ensure that the osteotomy operation is always performed within a safe area, and ensure the safety of the osteotomy surgery.

[0005] A first aspect of embodiments of this application provides an osteotomy control device, including:

[0006] a vision sensor, used to capture the spatial geometric features of the surgical area and the six-dimensional pose of the robotic arm end effector; a data processing unit, used to reconstruct a safe osteotomy operation boundary based on the spatial geometric features, and project the safe osteotomy operation boundary and the contour of the end effector represented by the six-dimensional pose onto a two-dimensional plane; and generate a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary and the contour of the end effector in the two-dimensional plane; and a robotic arm control unit, used to control the robotic arm to move the end effector into the safe osteotomy operation boundary when the end effector exceeds the safe osteotomy operation boundary according to the real-time feedback signal.

[0007] Optionally, the data processing unit is specifically used for: determining the projection point of the center point of the surgical area in the two-dimensional plane, and connecting the projection point with the first endpoint and the second endpoint of the end effector to form a first line segment and a second line segment; calculating whether the first line segment and the second line segment intersect the safe osteotomy operation boundary respectively; and generating a real-time feedback signal when the first line segment and / or the second line segment intersects the safe osteotomy operation boundary.

[0008] Optionally, the safe osteotomy operation boundary is composed of multiple boundary line segments. The data processing unit is further configured to: calculate the cross product between the first vector and the second vector corresponding to the first line segment and the boundary vector representing the current boundary line segment, respectively, during the process of the robotic arm control unit controlling the robotic arm to drive the end effector to perform osteotomy along any boundary line segment; determine whether the first line segment and the second line segment intersect with the current boundary line segment based on the cross product; determine the direction and magnitude of the force when the first line segment and / or the second line segment intersects with the current boundary line segment, and generate a real-time feedback signal based on the direction and magnitude of the force; wherein the direction of the force is perpendicular to the current boundary line segment and points to the boundary line segment, the magnitude of the force is proportional to the distance between the target endpoint and the current boundary line segment, and the target endpoint is the endpoint located outside the safe osteotomy operation boundary.

[0009] Optionally, the data processing unit is further configured to:When the first or second line segment intersects the safe osteotomy operation boundary, the corner type is determined. The corner is formed by connecting the target endpoint of the end effector with the two endpoints of two adjacent boundary lines. The two adjacent boundary lines include the boundary line segment currently intersecting the first or second line segment in the safe osteotomy operation boundary and another boundary line segment adjacent to the intersecting boundary line segment. When the corner is obtuse, the direction and magnitude of the force are determined, and a real-time feedback signal is generated based on the direction and magnitude of the force. The direction of the force is from a point on the angle bisector of the two adjacent boundary lines located outside the safe osteotomy operation boundary to the target endpoint. When the corner is acute, the angle of the line segment formed by the two adjacent boundary lines is determined, and a real-time feedback signal is generated based on the line segment angle.

[0010] Optionally, generating a real-time feedback signal based on the line segment angle includes: determining the direction and magnitude of the force when the angle between the two adjacent boundary line segments is a right angle, and generating a real-time feedback signal based on the direction and magnitude of the force; wherein the direction of the force is determined by combining the directions of the target endpoint perpendicularly pointing to the two adjacent boundary line segments; determining the direction and magnitude of the force when the angle between the two adjacent boundary line segments is an obtuse angle, and generating a real-time feedback signal based on the direction and magnitude of the force; wherein the direction of the force is from the target endpoint to a point located inside the safe osteotomy operation boundary on the angle bisector of the two adjacent boundary line segments.

[0011] Optionally, the data processing unit is further specifically used for: determining the graphic type formed based on the intersecting safe osteotomy operation boundary when both the first line segment and the second line segment intersect the safe osteotomy operation boundary, the graphic type including a concave graphic or a convex graphic; determining the direction and magnitude of the force based on the graphic type, and generating a real-time feedback signal based on the direction and magnitude of the force.

[0012] Optionally, the data processing unit is further configured to: sort the multiple boundary segments constituting the safe osteotomy operation boundary and assign edge numbers sequentially; if the absolute value of the difference between the edge numbers of two boundary segments intersecting the first and second line segments is greater than 1, determine the first midpoint between the two closest points of the two boundary segments, and the second midpoint between the first and last boundary segments of the safe osteotomy operation boundary; calculate the first distance between the second midpoint and the end effector, and the second distance between the first midpoint and the second midpoint; if the first distance is greater than the second distance, determine that the shape formed based on the two boundary segments is a concave shape; Specification 2 / 14 pages 6 CN121667803 A If the absolute value of the difference between the edge indices of the two boundary line segments intersecting the first line segment and the second line segment is equal to 1, or if the first distance is less than or equal to the second distance, the figure formed by the two boundary line segments is determined to be a convex figure.

[0013] Optionally, the data processing unit is further configured to: if the figure formed by the two boundary line segments is a concave figure, determine that the direction of the force is perpendicular to the line connecting the two closest points of the two boundary line segments and points from the first midpoint to the interior of the safe osteotomy operation boundary; if the figure formed by the two boundary line segments is a convex figure, determine that the direction of the force is along the angle bisector of the two boundary line segments and points to the interior of the safe osteotomy operation boundary.

[0014] A second aspect of this application provides an osteotomy control method, comprising: capturing the spatial geometric features of a surgical area and the six-dimensional pose of a robotic arm end effector; reconstructing a safe osteotomy operation boundary based on the spatial geometric features, and projecting the safe osteotomy operation boundary and the contour of the end effector characterized by the six-dimensional pose onto a two-dimensional plane; generating a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary and the end effector contour in the two-dimensional plane; and controlling the robotic arm to move the end effector inward into the safe osteotomy operation boundary when the end effector exceeds the safe osteotomy operation boundary according to the real-time feedback signal.

[0015] A third aspect of this application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device performs the function of the apparatus as described in any of the first aspects above; or performs the method as described in the second aspect above.

[0016] A fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the function of the apparatus as described in any of the first aspects above; or implements the method as described in the second aspect above.

[0017] A fifth aspect of the present application provides a computer program product including a computer program that, when executed, causes the function of the apparatus as described in any of the first aspects above to be implemented, or causes the method as described in the second aspect above to be executed.

[0018] Compared with the prior art, the embodiments of the present application have the following beneficial effects: In the embodiments of the present application, by utilizing a visual sensor to capture the spatial geometric features of the surgical area and the six-dimensional pose of the robotic arm end effector, a safe osteotomy operation boundary can be reconstructed. Based on this, the safe osteotomy operation boundary...The outline of the boundary and the end effector are projected onto a two-dimensional plane. Based on the positional relationship between the boundary and the outline in the two-dimensional plane, a real-time feedback signal can be generated. This signal is used to control the operation of the robotic arm by the robotic arm control unit when the end effector exceeds the safety osteotomy operation boundary, thereby driving the end effector to move inward towards the boundary, ensuring the safety and accuracy of the osteotomy process. In this embodiment, by projecting the midpoint of the three-dimensional space onto a two-dimensional plane and then using methods such as cross-legal methods to determine whether the corresponding line segments in the two-dimensional plane intersect, it is possible to quickly determine whether the end effector on the robotic arm has exceeded the safety boundary and to control the robotic arm in a timely manner when the end effector exceeds the boundary. By applying the device provided in this embodiment, the accuracy and safety of the operation can be ensured during the osteotomy process, and the timeliness of the relevant control process can be guaranteed. Specification 3 / 14 pages 7 CN 121667803 A Brief Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0020] Figure 1 is a schematic diagram of calculating whether line segments intersect using cross-legal calculations according to an embodiment of this application; Figure 2 is a schematic diagram of an osteotomy control device according to an embodiment of this application; Figure 3 is a schematic diagram of the positional relationship between the safe osteotomy operation boundary and the end effector contour in a two-dimensional plane according to an embodiment of this application; Figure 4 is a schematic diagram of the positional relationship between a portion of the boundary line segments of the safe osteotomy operation boundary and the end effector contour in a two-dimensional plane according to an embodiment of this application; Figure 5 is a schematic diagram of another positional relationship between the safe osteotomy operation boundary and the end effector contour in a two-dimensional plane according to an embodiment of this application; Figure 6 is a schematic diagram of an obtuse angle formed by the endpoints of two adjacent boundary line segments of the safe osteotomy operation boundary and the target endpoint of the end effector in a two-dimensional plane according to an embodiment of this application; Figure 7 is a schematic diagram of an acute angle formed by the endpoints of two adjacent boundary line segments of the safe osteotomy operation boundary and the target endpoint of the end effector in a two-dimensional plane according to an embodiment of this application; Figure 8 is a schematic diagram of an acute angle formed by the endpoints of two adjacent boundary line segments of the safe osteotomy operation boundary and the target endpoint of the end effector in a two-dimensional plane according to an embodiment of this application. Figure 9 is a schematic diagram of an end effector intersecting two boundary line segments simultaneously, provided in an embodiment of this application; Figure 10 is a schematic diagram of another end effector intersecting two boundary line segments simultaneously, provided in an embodiment of this application; Figure 11 is a schematic diagram of an osteotomy control method provided in an embodiment of this application; Figure 12 is a schematic diagram of another osteotomy control device provided in an embodiment of this application.Figure 13 is a schematic diagram of a computer device provided in an embodiment of this application. Detailed Description

[0021] In the following description, specific details such as particular system structures and technologies are set forth for illustration rather than limitation in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art should understand that this application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary details.

[0022] As described in the background art, computer-aided osteotomy surgery requires ensuring that the end effector does not exceed the safety boundary of the osteotomy area during the osteotomy process. In some solutions, osteotomy boundary points can be planned and then transformed to the coordinate system corresponding to the robotic arm device, and the control of the end effector can be achieved based on the relevant processing in the robotic arm coordinate system. However, this processing method involves multiple coordinate transformations, and the data cannot be synchronized in real time between various devices, which seriously affects the accuracy of the osteotomy operation.

[0023] To address the above problems, this application provides an osteotomy control device that can quickly determine whether the end effector is at risk of exceeding the safe osteotomy operation boundary by projecting relevant data in three-dimensional space onto a two-dimensional plane and then judging the positional relationship between line segments in the two-dimensional plane using cross-legal calculations. In this way, when the end effector exceeds or is about to exceed the safe osteotomy operation boundary, the robotic arm can be controlled in a timely manner to move the end effector towards the safe osteotomy operation boundary.

[0024] Before introducing the specific technical solutions of this application's embodiments, the method of calculating the positional relationship between line segments using cross-legal calculations is first described.

[0025] As shown in Figure 1, this application provides a schematic diagram of calculating whether line segments intersect using cross-legal calculations. Taking Figure 1 as an example, line segments and vectors are defined. For example, given two line segments AB and CD as shown in Figure 1(b), and defined as vectors AB = B-A and CD = D-C, then combining with Figure 1(a), we have: AC = C-A, AD = D-A, BC = C-B, BD = D-B, CA = A-C, CB = B-C, etc.

[0026] Then, the cross product between each vector can be calculated, for example: Calculate the cross product of vector AB and vector AC: cross1 = AB × AC; Calculate the cross product of vector AB and vector AD: cross2 = AB × AD; Calculate the cross product of vector CD and vector CA: cross3 = CD × CA; Calculate the cross product of vector CD and vector CB: cross4 = CD × CB.

[0027] For vectors AB and AC, if the calculated cross product between them is greater than 0, it indicates that vector AC...The counterclockwise direction of vector AB means that rotating vector AB to vector AC requires rotating vector AB counterclockwise. This also means that point C is to the left of line segment AB. Conversely, if the cross product between the two is less than 0, it means that vector AC is in the clockwise direction of vector AB. Rotating vector AB to vector AC requires rotating vector AB clockwise. This also means that point C is to the right of line segment AB.

[0028] Therefore, in conjunction with the above introduction, when using the cross-legal method to determine the intersection relationship between line segments, the following judgment conditions are available: (1) If cross1 × cross2 ≤ 0, it means that point C and point D are on opposite sides of line segment AB.

[0029] (2) If cross3 × cross4 ≤ 0, it means that point A and point B are on opposite sides of line segment CD.

[0030] (3) Only when both of the above conditions (1) and (2) are satisfied, line segments AB and CD intersect.

[0031] The technical solution of this application will be described below through specific embodiments.

[0032] Referring to FIG2, a schematic diagram of an osteotomy control device provided in an embodiment of this application is shown. The device 200 includes a vision sensor 201, a data processing unit 202, and a robotic arm control unit 203. The robotic arm control unit 203 is used to control the robotic arm 2031, and an actuator, namely the end effector 2032 shown in FIG2, is installed at the end of the robotic arm 2031 to realize the control of the osteotomy process during surgery. Specifically: The vision sensor 201 is used to capture the spatial geometric features of the surgical area and the six-dimensional pose of the end effector 2032 installed at the end of the robotic arm 2031.

[0033] The data processing unit 202 is used to reconstruct the safe osteotomy operation boundary according to the captured spatial geometric features, and project the safe osteotomy operation boundary and the outline of the end effector 2032 represented by the above-mentioned six-dimensional pose onto a two-dimensional plane; Based on the positional relationship between the safe osteotomy operation boundary and the outline of the end effector 2032 in the two-dimensional plane, a real-time feedback signal is generated.

[0034] The robotic arm control unit 203 is used to control the robotic arm 2031 to move the end effector 2032 into the aforementioned safe osteotomy operation boundary when the end effector 2032 exceeds the safe osteotomy operation boundary, based on the generated real-time feedback signal.

[0035] In this embodiment, the surgical area can be determined and the safe osteotomy operation boundary can be planned during the preoperative planning stage. During the osteotomy process controlled by the robotic arm 2031, the end effector 2032 should be kept within the aforementioned safe osteotomy operation boundary. Unless otherwise specified, the terms "safe osteotomy operation boundary," "safe boundary," or "boundary," "osteotomy boundary," etc., used in this embodiment all refer to the aforementioned "safe osteotomy operation boundary."

[0036] In this embodiment, the vision sensor 201 can be a high-resolution three-dimensional vision sensor, capable of capturing the spatial geometric features of the surgical area in real time, such as the osteotomy boundary and preoperative planning path. Furthermore, while capturing the spatial geometric features of the surgical area, the vision sensor 201 can also capture the six-dimensional pose of the actuator at the end of the robotic arm 2031. It should be noted that, unless otherwise specified, the "end-effector" or "actuator" mentioned in this embodiment refers to the end-effector 2032 shown in Figure 2. The end-effector 2032 can be a surgical tool such as a oscillating saw installed at the end of the robotic arm 2031, capable of performing osteotomy operations.

[0037] The spatial geometric features of the surgical area captured by the vision sensor 201 and the six-dimensional pose of the end-effector 2032 will be transmitted to the data processing unit 202. Through processing by the data processing unit 202, it is possible to determine in real time whether the end-effector 2032 has exceeded the safe osteotomy operation boundary.

[0038] In this embodiment, the data processing unit 202 can reconstruct the safe osteotomy operation boundary based on the spatial geometric features captured by the visual sensor 201. This process may include registration of the aforementioned spatial geometric features and preoperative planning information. Based on the registration, a three-dimensional anatomical model is reconstructed, and then a safe osteotomy operation boundary with sub-millimeter precision is generated in the three-dimensional anatomical model. This embodiment does not elaborate on the registration, three-dimensional reconstruction, and other related processes.

[0039] Based on the reconstructed three-dimensional anatomical model, the data processing unit 202 can simultaneously superimpose the end effector 2032 onto the aforementioned model. Then, in order to quickly determine the positional relationship between the end effector 2032 and the safe osteotomy operation boundary, the data processing unit 202 can project the safe osteotomy operation boundary and the contour of the end effector 2032 characterized by the aforementioned six-dimensional pose onto a two-dimensional plane. The safe osteotomy operation boundary and the contour of the end effector 2032 can be regarded as being composed of a series of point sets, and the process of projecting them onto a two-dimensional plane is also the process of projecting a point in three-dimensional space onto a two-dimensional plane.

[0040] Thus, based on the positional relationship between the safe osteotomy operation boundary in the two-dimensional plane and the contour of the end effector 2032, it can be determined whether the end effector 2032 has exceeded the boundary. The data processing unit 202 can generate a real-time feedback signal when the end effector 2032 exceeds the boundary and transmit it to the robotic arm control unit 203.

[0041] The determination of whether line segments intersect in the two-dimensional plane can be achieved by using a cross-law method, that is, by calculating the cross product between the vectors corresponding to the two line segments, and determining whether the two intersect based on the cross product calculation result. Therefore, after projecting the safe osteotomy operation boundary and the contour of the end effector 2032 onto the two-dimensional plane, the safe osteotomy operation boundary can be regarded as a lineLine segment: The contour of the end effector 2032 is considered as another line segment. The cross product of the corresponding vectors of the two line segments is calculated to determine whether they intersect.

[0042] In another possible implementation of the embodiments of this application, after projecting the contour of the end effector 2032 onto a two-dimensional plane, the endpoints of the end effector 2032 can be used to determine whether it intersects with the safety boundary. For example, two points representing the left and right endpoints of the end effector 2032 in the two-dimensional plane can be connected to a point in the safety boundary to form two line segments, and then it can be determined whether these two line segments intersect with the safety boundary. If any line segment intersects with the safety boundary, it can be considered that the end effector 2032 has exceeded the safety boundary.

[0043] In the embodiments of this application, the above-mentioned real-time feedback signal can be generated when the end effector 2032 exceeds the boundary. Therefore, the real-time feedback signal can include information such as the magnitude and direction of the force indicating the movement of the end effector 2032 into the safety osteotomy operation boundary.

[0044] Thus, after receiving the real-time feedback signal, the robotic arm control unit 203 can control the movement of the robotic arm 2031 according to the signal, and drive the end effector 2032 to move inward toward the safety boundary. Specification 6 / 14 pages 10 CN 121667803 A

[0045] In this embodiment, by using a visual sensor to capture the spatial geometric features of the surgical area and the six-dimensional pose of the robotic arm end effector, the safe osteotomy operation boundary can be reconstructed. Based on this, the safe osteotomy operation boundary and the contour of the end effector are projected onto a two-dimensional plane. Based on the positional relationship between the boundary and the contour in the two-dimensional plane, a real-time feedback signal can be generated. This signal is used to control the operation of the robotic arm by the robotic arm control unit when the end effector crosses the safe osteotomy operation boundary, thereby driving the end effector to move inward toward the boundary, ensuring the safety and accuracy of the osteotomy process. This application embodiment projects a point in three-dimensional space onto a two-dimensional plane, and then uses methods such as cross-cutting to determine whether corresponding line segments in the two-dimensional plane intersect. This allows for rapid determination of whether the end effector on the robotic arm has exceeded the safety boundary, and timely control of the robotic arm when the end effector exceeds the boundary. Using the device provided in this application embodiment, the accuracy and safety of the operation during osteotomy can be guaranteed, as well as the timeliness of related control processes.

[0046] In one possible implementation of this application embodiment, when the data processing unit 202 generates a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary in the two-dimensional plane and the contour of the end effector, it can first determine the projection point of the center point of the surgical area in the two-dimensional plane, and connect this projection point with the first and second endpoints of the end effector to form a first line segment and a second line segment. Then, the first and second line segments are respectively calculated relative to the safe osteotomy boundary.Whether the bone manipulation boundary intersects. A real-time feedback signal is generated when the first line segment and / or the second line segment intersects the safe osteotomy boundary.

[0047] Figure 3 shows a schematic diagram of the positional relationship between the safe osteotomy boundary and the end effector contour in a two-dimensional plane according to an embodiment of this application. In Figure 3, the safe osteotomy boundary is composed of multiple boundary line segments, namely line segments l1 to l5 shown in Figure 3. Each line segment is connected end-to-end, together forming the safe boundary of the osteotomy surgery. In Figure 3, S300 represents the contour of the end effector 2032 in the two-dimensional plane. Since the end effector 2032 includes two endpoints, namely the left endpoint and the right endpoint, projecting the end effector 2032 onto the two-dimensional plane will also form two endpoints, namely the first endpoint L (corresponding to the left endpoint of the end effector 2032) and the second endpoint R (corresponding to the right endpoint of the end effector 2032) in Figure 3. The center point of the surgical area is point O in Figure 3. Connecting this center point O to the first endpoint L and the second endpoint R of the end effector 2032 respectively, two line segments can be obtained, namely the first line segment OL and the second line segment OR in Figure 3.

[0048] When determining the positional relationship between the end effector 2032 and the safe osteotomy operation boundary, the intersection of the first line segment OL and the second line segment OR with the safe boundary can be calculated. For example, the cross product between the vectors corresponding to the two line segments and the safe boundary can be calculated using the cross-legal method described above, and the intersection of the line segments can be determined based on the cross product.

[0049] In this embodiment of the application, since the safe osteotomy operation boundary can be composed of multiple boundary line segments, when calculating the cross product between the first line segment OL and the second line segment OR and the safe boundary using the cross-legal method, the calculation can be performed separately for each boundary line segment. For example, the cross product between the vectors corresponding to the first line segment OL and the second line segment OR and the vectors corresponding to the line segments l1 to l5 shown in Figure 3 can be calculated respectively.

[0050] In one possible implementation of this application embodiment, the above-mentioned processing of the data processing unit 202 may be performed during the process of the robotic arm control unit 203 controlling the robotic arm 2031 to drive the end effector 2032 to perform osteotomy along any boundary line segment. Referring to Figures 3 and 4, Figures 3 and 4 show an example of the end effector 2032 performing osteotomy along the boundary line segment l2. Among them, Figure 4 is an example of a part of the area in Figure 3, that is, Figure 4 only shows the area of ​​the boundary line segment l2 in Figure 3. During this osteotomy process, the data processing unit 202 can calculate the cross product between the first vector and the second vector corresponding to the first line segment OL and the second line segment OR and the boundary vector representing the current boundary line segment l2, and determine whether the first line segment OL and the second line segment OR intersect with the current boundary line segment l2 based on the cross product.

[0051] In the case where the first line segment OL and / or the second line segment OR intersect with the current boundary line segment l2, for example as shown in the figureAs shown, the first line segment OL intersects the current boundary line segment l2 at point X1. At this time, the data processing unit 202 can determine the direction and magnitude of the force, and generate a real-time feedback signal based on the direction and magnitude of the force. The real-time feedback signal will be used to control the robotic arm control unit 203 to generate a force of the corresponding magnitude and act on the end effector 2032 in the corresponding direction, so that the end effector 2032 that has crossed the boundary moves to the inside of the boundary.

[0052] In the embodiment of this application, during the process of the robotic arm control unit 203 controlling the robotic arm 2031 to drive the end effector 2032 to perform osteotomy along any boundary line segment, if the end effector 2032 crosses the safety boundary, the direction of the force can be perpendicular to the current boundary line segment and point to the boundary line segment. The magnitude of the force can be proportional to the distance between the target endpoint and the current boundary line segment. The target endpoint can be an endpoint located outside the safety osteotomy operation boundary. For example, as shown in Figure 4, the target endpoint is point L. The direction of the force F1 is perpendicular to the current boundary line segment l2 from point L outside the boundary and points towards the boundary line segment l2. The magnitude of the force F1 is proportional to the distance d1 between the target endpoint L and the current boundary line segment l2.

[0053] In another possible implementation of the embodiments of this application, the relationship between related line segments in the two-dimensional plane also includes the case of polygon intersection. The polygon intersection can refer to the case where the first line segment and the second line segment formed by connecting the projection points of the left and right endpoints of the end effector 2032 in the two-dimensional plane with the center point of the surgical area intersect with two different boundary line segments respectively. In this case, the magnitude of the force can be proportional to the minimum value of the distances from the corresponding endpoint to all boundary line segments. This is used to determine the magnitude and direction of the force and generate a real-time feedback signal.

[0054] In another possible implementation of this application embodiment, if either the first line segment and the second line segment formed by the center point of the surgical area and the two endpoints of the end effector 2032 intersect the safe osteotomy operation boundary, the data processing unit can determine the magnitude and direction of the force that subsequently acts on the end effector 2032 and can move the end effector 2032 into the boundary based on the corner type. The aforementioned corner can be formed by connecting the target endpoint of the end effector 2032 with the two endpoints of two adjacent boundary line segments. The two adjacent boundary line segments include the boundary line segment in the safe osteotomy operation boundary that currently intersects with the first line segment or the second line segment and another boundary line segment adjacent to the intersecting boundary line segment. The target endpoint is the endpoint outside the safe boundary in the two-dimensional plane.

[0055] For example, as shown in FIG5, another safe osteotomy operation edge in the two-dimensional plane provided by this application embodiment is shown.A schematic diagram of the positional relationship between the boundary and the contour of the end effector. Figure 5 shows two different cases: the left endpoint of the end effector 2032 is located outside the safety boundary, and the first line segment obtained by connecting the left endpoint to the center point of the surgical area intersects with a certain boundary line segment; and the right endpoint of the end effector 2032 is located outside the safety boundary, and the second line segment obtained by connecting the right endpoint to the center point of the surgical area intersects with another boundary line segment.

[0056] Specifically, as shown in Figure 5, the projection point of the left endpoint of the end effector 2032 in the two-dimensional plane is point L in Figure 5, and the first line segment obtained by connecting point L with the center point of the surgical area intersects with the boundary line segment l1 (the above intersection relationship is not shown in Figure 5). The target endpoint at this time is point L in Figure 5, and the two adjacent boundary line segments are boundary line segments l1 and l2 in Figure 5. The angle formed by connecting point L with the two endpoints of boundary line segments l1 and l2 is also the angle ALC formed by connecting point L with one endpoint A of boundary line segment l1 and one endpoint C of boundary line segment l2 in Figure 5.

[0057] In another case, as shown in Figure 5, the projection point of the → endpoint of the end effector 2032 in the two-dimensional plane is point R in Figure 5. The second line segment obtained by connecting point R with the center point of the surgical area intersects with boundary line segment l3 or l4 (the above intersection relationship is not shown in Figure 5). The target endpoint at this time is point R in Figure 5, and the two adjacent boundary line segments are boundary line segments l3 and l4 in Figure 5. The angle formed by connecting point R with the two endpoints of boundary line segments l3 and l4 is also the angle CRE formed by connecting point R with one endpoint C of boundary line segment l3 and one endpoint E of boundary line segment l4 in Figure 5.

[0058] It should be noted that the two situations described above may exist simultaneously or only one of them may exist. For example, both endpoints of the end effector 2032 are located outside the safety boundary at the same time, forming a situation where points L and R are both outside the safety boundary, as shown in Figure 5 (page 8 / 14 of specification, 12 CN 121667803 A). Alternatively, only one of the two endpoints of the end effector 2032 is located outside the safety boundary. For example, only point L shown in Figure 5 is outside the safety boundary; or only point R shown in Figure 5 is outside the safety boundary. This application embodiment does not limit this.

[0059] In this application embodiment, for the situation shown in Figure 5, the data processing unit 202 can determine the corner type, that is, the type of corner ALC or corner CRE shown in Figure 5, and perform targeted processing according to different boundary types.

[0060] In this embodiment of the application, if the corner is an obtuse angle, for example, angle ALC as shown in Figure 5 is an obtuse angle, in the case of such an obtuse angle, the data processing unit 202 can determine the direction and magnitude of the force to be generated at this time, and according to the action...The direction and magnitude of the force generate a real-time feedback signal.

[0061] Specifically, when the corner is obtuse, the direction of the force to be generated can be from a point on the angle bisector of the two adjacent boundary segments outside the safe osteotomy operation boundary to the target endpoint.

[0062] For example, as shown in Figure 6, when the corner ALC shown in Figure 5 is obtuse, the force to be generated is force F2, and the direction of force F2 is from point W1 in Figure 6 to the target endpoint L (i.e., the endpoint outside the safe boundary). The point W1 is a point on the angle bisector lh1 of the two adjacent boundary segments l1 and l2 (corresponding to segments AB and BC in Figure 6) and located outside the safe osteotomy operation boundary. The position of the point W1 can be determined according to the actual situation. For example, the point W1 can usually be set on the angle bisector lh1 about 5 cm away from point B.

[0063] In another possible implementation of this application embodiment, if the corner is an acute angle, for example, the angle CRE shown in FIG5 is an acute angle, in the case of the acute angle, the data processing unit 202 can determine the angle of the line segment formed by two adjacent boundary line segments, and generate a real-time feedback signal based on different line segment angles. Among them, in the case of the acute angle, the angle of the line segment formed by two adjacent boundary line segments includes two cases: right angle and obtuse angle.

[0064] For example, as shown in FIG5 and FIG7, it is a schematic diagram in the case of the acute angle CRE, where the angle of the line segment formed by two adjacent boundary line segments l3 and l4 is a right angle. As shown in FIG8, it is a schematic diagram in the case of the acute angle CRE, where the angle of the line segment formed by two adjacent boundary line segments l3 and l4 (corresponding to line segments CD and DE in FIG8) is an obtuse angle.

[0065] The data processing unit 202 can determine the direction and magnitude of the force to be generated for the above two cases respectively.

[0066] Specifically, as shown in Figures 5 and 7, when the angle between the line segments formed by two adjacent boundary line segments l3 and l4 (i.e., angle CDE in Figures 5 and 7) is a right angle, the force to be generated by the data processing unit 202 can be force F3. The direction of force F3 can be determined by combining the directions perpendicularly pointed from the target endpoint (i.e., point R in Figures 5 and 7) to the two adjacent boundary line segments. For example, referring to Figure 7, the directions perpendicularly pointed from the target endpoint R to the two adjacent boundary line segments CD and DE are directions F3x and F3y shown in Figure 7. The combination of these two directions yields the direction of force F3 as shown in Figure 7. The magnitude of force F3 can be determined based on the distance between the target endpoint R and the corresponding second line segment boundary. After determining the direction and magnitude of force F3, the data processing unit 202 can generate a real-time feedback signal based on the direction and magnitude of force F3 to move the end effector 2032 inside the safety boundary.

[0067] In another possible implementation, as shown in Figure 8, when the angle between the two adjacent boundary segments l3 and l4 (i.e., angle CDE in Figure 8) is obtuse, the force to be generated determined by the data processing unit 202 can be force F4. The direction of force F4 can be from the target endpoint R to a point on the angle bisector of the two adjacent boundary segments and located inside the safe osteotomy operation boundary. For example, referring to Figure 8, the direction of force F4 can be from the target endpoint R to point W2, which is a point on the angle bisector lh2 of the two adjacent boundary segments l3 and l4 (corresponding to segments CD and DE in Figure 8) and located inside the safe osteotomy operation boundary. The data processing unit 202 can generate a real-time feedback signal based on the direction and magnitude of force F4, which is used to move the end effector 2032 to the inside of the safe boundary.

[0068] In another possible implementation of the embodiments of this application, during the process of the robotic arm 2031 controlling the end effector 2032 to perform osteotomy or movement, there may be a situation where the end effector 2032 intersects with two boundary line segments simultaneously, that is, the first line segment and the second line segment formed by connecting the center point of the surgical area with the two endpoints of the end effector 2032 both intersect with the safe osteotomy operation boundary. At this time, the data processing unit 202 can determine the graphic type formed based on the intersecting safe osteotomy operation boundary, and determine the direction and magnitude of the force according to the graphic type, thereby generating a real-time feedback signal according to the determined direction and magnitude of the force. The above graphic type may include a concave graphic or a convex graphic. Among them, a concave graphic can refer to a graphic formed by the adjacent boundary recessing into the osteotomy area; a convex graphic can refer to a graphic formed by the adjacent boundary protruding outward from the osteotomy area.

[0069] As shown in Figures 9 and 10, these are schematic diagrams of the end effector intersecting with two boundary line segments simultaneously provided in the embodiments of this application. Figure 9 shows an example where the end effector S300 intersects with boundary segments l03 and l05 simultaneously, and Figure 10 shows an example where the end effector S300 intersects with boundary segments l11 and l12 simultaneously. The aforementioned intersections can refer to the intersections of the first and second line segments with the boundary segments. The first and second line segments are two line segments connecting the center point of the surgical area to the two endpoints of the end effector S300.

[0070] Combining Figures 9 and 10, when the end effector S300 intersects with two boundary segments simultaneously, the data processing unit 202 can sort and assign edge numbers to multiple boundary segments constituting the boundary of the safe osteotomy operation. For example, after assigning edge numbers to the multiple boundary segments in Figure 9, they are respectively edges l01 to l07, and the multiple boundary segments in Figure 10 are...After assigning edge numbers, they are identified as edges l11 to l12.

[0071] Then, the data processing unit 202 can calculate the absolute value of the difference between the edge numbers of the two boundary segments intersecting the first and second line segments. If the absolute value of the difference between the edge numbers of the two boundary segments intersecting the first and second line segments is greater than 1, the data processing unit 202 can further determine the first midpoint between the two closest points of the two boundary segments, and the second midpoint between the first and last boundary segments of the safe osteotomy operation boundary. For example, the first midpoint can be point O01 in Figure 9, the first boundary segment of the safe osteotomy operation boundary in Figure 9 is l01, the last boundary segment is l07, and the second midpoint between them can be point O02 in Figure 9.

[0072] The data processing unit 202 can calculate the first distance between the second midpoint O02 and the end effector S300, and the second distance between the first midpoint O01 and the second midpoint O02. In the intersection diagram shown in Figure 9, the first distance is greater than the second distance.

[0073] In this embodiment of the application, when the first distance determined in the above manner is greater than the second distance, the shape formed based on the two boundary line segments can be determined to be a concave shape. When the absolute value of the difference between the edge numbers of the two boundary line segments intersecting with the first line segment and the second line segment is equal to 1, or when the first distance determined in the above manner is less than or equal to the second distance, the shape formed based on the two boundary line segments can be determined to be a convex shape. That is, in the intersection diagram shown in Figure 9, the shape formed based on the two boundary line segments is a concave shape. In the intersection diagram shown in Figure 10, the shape formed based on the two boundary line segments is a convex shape. In the intersection example shown in Figure 10, the first midpoint is point O11, and the second midpoint is point O12.

[0074] The data processing unit 202 can perform targeted processing according to the different types of the formed shapes to determine the magnitude and direction of the corresponding forces to be generated.

[0075] Specifically, when the shape formed by the two boundary line segments is a concave shape, such as the concave shape formed by the intersection as shown in Figure 9, the data processing unit 202 can determine that the direction of the force to be generated is perpendicular to the line connecting the two closest points of the two boundary line segments and points from the first midpoint to the inside of the safe osteotomy operation boundary. Referring to Figure 9, the force to be generated is the force F5 shown in Figure 9. The direction of this force F5 is perpendicular to the line connecting the two closest points of the two boundary line segments (page 10 / 14 of specification, CN 121667803 A), which is the boundary line segment l04 shown in Figure 9, and points to the inside of the safe osteotomy operation boundary.

[0076] When the shape formed by the two boundary line segments is a convex shape, such as the convex shape formed by the intersection as shown in Figure 10, the data processing unit 202 can determine that the direction of the force to be generated is along the angle bisector of the two boundary line segments.The line points inside the safe osteotomy operation boundary. Referring to Figure 10, the force to be generated is force F6 shown in Figure 10. The direction of force F6 is along the angle bisector of the two boundary line segments, i.e., angle bisector lh3 in Figure 10, pointing inside the safe osteotomy operation boundary.

[0077] The foregoing embodiments, in conjunction with the various intersection situations shown in Figures 3 to 10, respectively describe the direction and magnitude of the force required to control the end effector, which is about to cross or has already crossed the safe boundary, to move inside the safe boundary under different conditions. The data processing unit can generate a real-time feedback signal based on the direction and magnitude of the corresponding force and send it to the robotic arm control unit 203, so that the robotic arm control unit 203 can generate the corresponding force and act on the end effector 2032 in a determined direction, ensuring that the end effector 2032 can perform osteotomy operations within the set safe osteotomy area.

[0078] In one possible implementation of this application embodiment, the force to be generated for different intersection situations can ultimately act on the end effector 2032. Taking the 2031 robotic arm as an example, which is a seven-axis robotic arm, it is necessary to convert the force into the force on each axis of the seven-axis robotic arm, that is, to convert the force in the Cartesian coordinate system into the force in the joint space. The above conversion relationship can be expressed as: τ = J(q)F. Wherein, τ is the joint torque 7×1 vector; J(q) is the Jacobian matrix (6×7 matrix) of the robot at the current position q, which describes the relationship between the end effector Cartesian space velocity and the joint space velocity; F is the direction of the superimposed end effector force, which can be expressed as a 6×1 vector [Fx, Fy, Fz, Mx, My, Mz].

[0079] By applying the osteotomy control device provided in the embodiments of this application, the boundary constraint control of the orthopedic surgical robotic arm can be realized by integrating three-dimensional vision sensing technology, and the autonomous perception and active intervention of the intraoperative safety boundary can be realized through three-dimensional data fusion. Among them, by using high-resolution three-dimensional vision sensors, the spatial geometric features of the surgical area can be captured in real time, such as the osteotomy boundary, the preoperative planned path, and the six-dimensional pose of the robotic arm end effector. Based on this, using an embedded data processing unit, a three-dimensional anatomical model of the surgical area can be reconstructed based on point cloud registration and statistical modeling algorithms, and a safe operating boundary with sub-millimeter precision can be dynamically generated. The robotic arm control unit, by receiving real-time feedback signals from the data processing unit, can trigger an impedance control strategy when the end effector approaches a preset safety threshold, achieving non-contact boundary constraint by applying a reverse operating force. The osteotomy control device provided in this application integrates spatiotemporal data of the patient's anatomical structure, surgical instruments, and the robotic arm system through a unified spatial coordinate system, which can significantly reduce human error and improve osteotomy accuracy.

[0080] Based on the foregoing embodiments, this application also provides an osteotomy control method. Figure 11 shows a schematic diagram of an osteotomy control method provided by this application. This method specifically includes the following steps: S1101, capturing the spatial geometric features of the surgical area and the six-dimensional pose of the robotic arm end effector.

[0081] S1102, reconstructing a safe osteotomy operation boundary based on the spatial geometric features, and projecting the safe osteotomy operation boundary and the contour of the end effector represented by the six-dimensional pose onto a two-dimensional plane.

[0082] S1103, generating a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary and the end effector contour in the two-dimensional plane.

[0083] S1104, according to the real-time feedback signal, when the end effector exceeds the safe osteotomy operation boundary, controlling the robotic arm to drive the end effector to move inwards towards the safe osteotomy operation boundary. Instruction manual, pages 11 / 14, 15 CN 121667803 A

[0084] The above method can be applied to computer equipment, that is, the execution subject of the method can be computer equipment. The computer equipment can reconstruct the safe osteotomy operation boundary based on the spatial geometric features by capturing the spatial geometric features of the surgical area and the six-dimensional pose of the end effector of the robotic arm, and project the safe osteotomy operation boundary and the outline of the end effector represented by the six-dimensional pose onto a two-dimensional plane. On this basis, the computer equipment can control the robotic arm to drive the end effector to move into the safe osteotomy operation boundary when the end effector exceeds the safe osteotomy operation boundary according to the real-time feedback signal.

[0085] The above method implemented by the computer equipment is similar to the functions that can be realized by each unit or module of the osteotomy control device in the above embodiments. For example, the steps of S1101 implemented by the computer equipment can refer to the description of the vision sensor 201 in the above embodiments; the steps of S1102 can refer to the description of the data processing unit 202 in the above embodiments; the steps of S1103 can refer to the description of the robotic arm control unit 203 in the above embodiments. This application embodiment will not elaborate further.

[0086] On the other hand, referring to FIG12, a schematic diagram of another osteotomy control device provided in this application embodiment is shown, which may specifically include a capture module 1201, a reconstruction module 1202, a projection module 1203, a generation module 1204 and a control module 1205, wherein: the capture module 1201 is used to capture the spatial geometric features of the surgical area and the six-dimensional pose of the robotic arm end effector.

[0087] The reconstruction module 1202 is used to reconstruct the safe osteotomy operation boundary according to the spatial geometric features.

[0088] The projection module 1203 is used to project the safe osteotomy operation boundary and the end effector represented by the six-dimensional pose.The contour of the actuator is projected onto a two-dimensional plane.

[0089] The generation module 1204 is used to generate a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary in the two-dimensional plane and the contour of the end effector.

[0090] The control module 1205 is used to control the robotic arm to move the end effector into the safe osteotomy operation boundary when the end effector exceeds the safe osteotomy operation boundary according to the real-time feedback signal.

[0091] Wherein, the function implemented by the capture module 1201 is similar to the function implemented by the vision sensor 201 in the foregoing embodiments; the functions implemented by the reconstruction module 1202, projection module 1203 and generation module 1204 are similar to the functions implemented by the data processing unit 202 in the foregoing embodiments; the function implemented by the control module 1205 is similar to the function implemented by the robotic arm control unit 203 in the foregoing embodiments. For relevant details, please refer to the description of the foregoing embodiments.

[0092] Referring to FIG13, a schematic diagram of a computer device provided in an embodiment of this application is shown. As shown in FIG13, the computer device 1300 in this embodiment includes: a processor 1310, a memory 1320, and a computer program 1321 stored in the memory 1320 and executable on the processor 1310. When the processor 1310 executes the computer program 1321, it implements the steps in the various embodiments of the osteotomy control method described above, such as steps S1101 to S1104 shown in FIG11. Alternatively, when the processor 1310 executes the computer program 1321, it implements the functions of each module / unit in the various device embodiments described above, such as the functions of devices or units 201 to 203 shown in FIG2; or the functions of modules 1201 to 1205 shown in FIG12.

[0093] For example, the computer program 1321 can be divided into one or more modules / units, and the one or more modules / units are stored in the memory 1320 and executed by the processor 1310 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions. These instruction segments may be used to describe the execution process of the computer program 1321 in the computer device 1300. For example, the computer program 1321 may be divided into a capture module, a reconstruction module, a projection module, a generation module, and a control module. The specific functions of each module are as follows: The capture module is used to capture the spatial geometric features of the surgical area and the six-dimensional pose of the robotic arm end effector.

[0094] The reconstruction module is used to reconstruct the safe osteotomy operation boundary based on the spatial geometric features.

[0095] The projection module is used to project the safe osteotomy operation boundary and the end effector represented by the six-dimensional pose.The outline of the device is projected onto a two-dimensional plane.

[0096] A generation module is used to generate a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary in the two-dimensional plane and the outline of the end effector.

[0097] A control module is used to control the robotic arm to move the end effector into the safe osteotomy operation boundary when the end effector crosses the safe osteotomy operation boundary, according to the real-time feedback signal.

[0098] The computer device 1300 may be a device capable of implementing the functions of each step in the aforementioned method embodiments, or a device capable of implementing the relevant functions that can be implemented by each unit and module in the aforementioned device embodiments. The computer device 1300 may be a desktop computer, a cloud server, or other devices. The computer device 1300 may include, but is not limited to, a processor 1310 and a memory 1320. Those skilled in the art will understand that Figure 13 is merely an example of computer device 1300 and does not constitute a limitation on computer device 1300. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device 1300 may also include input / output devices, network access devices, buses, etc.

[0099] The processor 1310 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0100] The memory 1320 may be an internal storage unit of computer device 1300, such as the hard disk or memory of computer device 1300. The memory 1320 can also be an external storage device of the computer device 1300, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the computer device 1300. Furthermore, the memory 1320 can include both internal storage units of the computer device 1300 and external storage devices. The memory 1320 is used to store the computer program 1321 and other programs and data required by the computer device 1300.It can also be used to temporarily store data that has been output or will be output.

[0101] This application embodiment also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the methods described in the foregoing embodiments.

[0102] This application embodiment also discloses a computer-readable storage medium storing a computer program. When the computer program is executed by a computer, it implements the methods described in the foregoing embodiments.

[0103] This application embodiment also discloses a computer program product, including a computer program. When the computer program is run on a computer, it causes the computer to execute the methods described in the foregoing embodiments. Specification 13 / 14 pages 17 CN 121667803 A

[0104] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. Instruction Manual 14 / 14 Page 18 CN 121667803 A Figure 1 Figure 2 Instruction Manual Figure 1 / 7 Page 19 CN 121667803 A Figure 3 Figure 4 Instruction Manual Figure 2 / 7 Page 20 CN 121667803 A Figure 5 Figure 6 Instruction Manual Figure 3 / 7 Page 21 CN 121667803 A Figure 7 Figure 8 Instruction Manual Figure 4 / 7 Page 22 CN 121667803 A Figure 9 Figure 10 Instruction Manual Figure 5 / 7 Page 23 CN 121667803 A Figure 11 Figure 12 Instruction Manual Figure 6 / 7 Page 24 CN 121667803 A Figure 13 Instruction Manual Figure 7 / 7 Page 25 CN 121667803 A OSTEOTOMY CONTROL APPARATUS, COMPUTER DEVICE, AND COMPUTER PROGRAM PRODUCT ABSTRACT The embodiments of the present application are applicable to the technical field of computer-assistedmedical technology and data processing, and provide an osteotomy control apparatus, a computer device, and a computer program product. The device includes: a vision sensor, used for capturing spatial geometric features of a surgical area and a six-dimensional pose of an end effector of a mechanical arm; a data processing unit, used for reconstructing a safe osteotomy operation boundary, and projecting the safe osteotomy operation boundary and a contour of the end effector to a two-dimensional plane; generating a real-time feedback signal based on a positional relationship between the safe osteotomy operation boundary and the end effector contour in the two-dimensional plane; and a mechanical arm control unit, used for controlling the mechanical arm to drive the end effector to move to the inside of the safe osteotomy operation boundary when the end effector exceeds the safe osteotomy operation boundary according to the real-time feedback signal. By employing the above method, it can beensured that the osteotomy operation is always performed within a safe region, thereby ensuring surgical safety.

Claims

1. An osteotomy control device, characterized by, The method comprises the following steps: a visual sensor is used to capture the spatial geometric features of the surgical area and the six-dimensional pose of the end effector of the mechanical arm; a data processing unit is used to reconstruct a safe osteotomy operation boundary according to the spatial geometric features, project the safe osteotomy operation boundary and the contour of the end effector represented by the six-dimensional pose to a two-dimensional plane, and generate a real-time feedback signal based on the positional relationship between the safe osteotomy operation boundary and the end effector contour in the two-dimensional plane; a mechanical arm control unit is used to control the mechanical arm to move the end effector inside the safe osteotomy operation boundary when the end effector exceeds the safe osteotomy operation boundary according to the real-time feedback signal.

2. The apparatus of claim 1, wherein, The data processing unit is specifically configured to: determine the projection point of the center point of the surgical area in the two-dimensional plane, and connect the projection point with the first end point and the second end point of the end effector to form a first line segment and a second line segment; calculate whether the first line segment and the second line segment intersect with the safe osteotomy operation boundary, respectively; generate a real-time feedback signal in the case that the first line segment and / or the second line segment intersects with the safe osteotomy operation boundary.

3. The apparatus of claim 2, wherein, The safe osteotomy operation boundary is composed of a plurality of boundary line segments, and the data processing unit is further configured to: calculate the cross product between the first vector and the second vector corresponding to the first line segment and the second line segment and the boundary vector representing the current boundary line segment during the process in which the mechanical arm control unit controls the mechanical arm to perform osteotomy along any boundary line segment; determine whether the first line segment and the second line segment intersect with the current boundary line segment according to the cross product; determine the direction and size of the force and generate a real-time feedback signal according to the direction and size of the force in the case that the first line segment and / or the second line segment intersects with the current boundary line segment; wherein the direction of the force is perpendicular to the current boundary line segment and points to the boundary line segment, and the size of the force is proportional to the distance between the target end point and the current boundary line segment, and the target end point is an end point located outside the safe osteotomy operation boundary.

4. The apparatus of claim 2 or 3, wherein, The data processing unit is further configured to: determine the type of the edge angle in the case that the first line segment or the second line segment intersects with the safe osteotomy operation boundary, the edge angle being formed by connecting the target end point of the end effector and the two end points of the two adjacent boundary line segments, the two adjacent boundary line segments including the current boundary line segment intersecting with the first line segment or the second line segment in the safe osteotomy operation boundary and another boundary line segment adjacent to the intersecting boundary line segment; determine the direction and size of the force and generate a real-time feedback signal according to the direction and size of the force in the case that the edge angle is an obtuse angle, wherein the direction of the force is from a point on the angle bisector of the two adjacent boundary line segments located outside the safe osteotomy operation boundary to the target end point. In the case that the corner is an acute angle, a line segment angle formed by the two adjacent boundary line segments is determined, and a real-time feedback signal is generated based on the line segment angle.

5. The apparatus of claim 4, wherein, The generating of the real-time feedback signal based on the line segment angle comprises: In the case that the line segment angle formed by the two adjacent boundary line segments is a right angle, a direction and a size of an acting force are determined, and a real-time feedback signal is generated according to the direction and the size of the acting force; wherein the direction of the acting force is determined by a direction of the target endpoint perpendicular to a direction of each of the two adjacent boundary line segments. In the case that the line segment angle formed by the two adjacent boundary line segments is an obtuse angle, a direction and a size of an acting force are determined, and a real-time feedback signal is generated according to the direction and the size of the acting force; wherein the direction of the acting force is determined by a point on an angle bisector of the two adjacent boundary line segments, which is located inside the safe osteotomy operation boundary.

6. The apparatus of any one of claims 2 or 3 or 5, wherein, The data processing unit is further configured to: In the case that the first line segment and the second line segment both intersect with the safe osteotomy operation boundary, a type of a figure formed based on the intersection of the safe osteotomy operation boundary is determined, and the type of the figure comprises a concave figure or a convex figure; a direction and a size of an acting force are determined according to the type of the figure, and a real-time feedback signal is generated according to the direction and the size of the acting force.

7. The apparatus of claim 6, wherein, The data processing unit is further configured to: sort the plurality of boundary line segments forming the safe osteotomy operation boundary and assign a boundary sequence number in sequence; in the case that an absolute value of a difference between the boundary sequence numbers of the two boundary line segments intersecting with the first line segment and the second line segment is greater than 1, a first midpoint between the closest two points in the two boundary line segments and a second midpoint between a first boundary line segment and a last boundary line segment of the safe osteotomy operation boundary are determined; a first distance between the second midpoint and the end effector and a second distance between the first midpoint and the second midpoint are calculated; if the first distance is greater than the second distance, it is determined that a figure formed based on the two boundary line segments is a concave figure; in the case that the absolute value of the difference between the boundary sequence numbers of the two boundary line segments intersecting with the first line segment and the second line segment is equal to 1, or the first distance is less than or equal to the second distance, it is determined that a figure formed based on the two boundary line segments is a convex figure.

8. The apparatus of claim 7, wherein, The data processing unit is further configured to: in the case that the figure formed by the two boundary line segments is a concave figure, the direction of the acting force is determined to be perpendicular to a line connecting the closest two points in the two boundary line segments and pointing to the inside of the safe osteotomy operation boundary from the first midpoint; in the case that the figure formed by the two boundary line segments is a convex figure, the direction of the acting force is determined to be along an angle bisector of the two boundary line segments and pointing to the inside of the safe osteotomy operation boundary.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the computer device implements the following method: capture spatial geometric features of a surgical region and a six-dimensional pose of an end effector of a mechanical arm; According to the spatial geometric features, a safe osteotomy operation boundary is reconstructed, and the safe osteotomy operation boundary and a contour of the end effector represented by the six-dimensional pose are projected to a two-dimensional plane; Based on a positional relationship between the safe osteotomy operation boundary and the end effector contour in the two-dimensional plane, a real-time feedback signal is generated; According to the real-time feedback signal, when the end effector exceeds the safe osteotomy operation boundary, the mechanical arm is controlled to drive the end effector to move inside the safe osteotomy operation boundary.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed, the following method is performed: spatial geometric features of a surgical region and a six-dimensional pose of an end effector of a mechanical arm are captured; According to the spatial geometric features, a safe osteotomy operation boundary is reconstructed, and the safe osteotomy operation boundary and a contour of the end effector represented by the six-dimensional pose are projected to a two-dimensional plane; Based on a positional relationship between the safe osteotomy operation boundary and the end effector contour in the two-dimensional plane, a real-time feedback signal is generated; According to the real-time feedback signal, when the end effector exceeds the safe osteotomy operation boundary, the mechanical arm is controlled to drive the end effector to move inside the safe osteotomy operation boundary.

Images