Tarsometatarsal joint internal fracture minimally invasive fixing method and system based on reinforcement learning and terminal

Abstract

The invention provides a minimally invasive fixation method, system and terminal for tarsometatarsal joint internal fracture based on reinforcement learning, and the method comprises the steps: carrying out the preprocessing of collected tarsometatarsal joint three-dimensional CT data and real operation screw scanning point cloud data, and importing the obtained tarsometatarsal joint three-dimensional model data and operation screw three-dimensional model data into Unity; and acquiring a three-dimensional tarsometatarsal joint and a three-dimensional screw in the operation simulation scene, calculating an initial implantation point position of the three-dimensional screw and an implantation target area, and obtaining an optimal orientation for guiding the three-dimensional screw to be implanted so as to determine an optimal implantation path of the three-dimensional screw. The problem that in the prior art, the continuous three-dimensional structure characteristics of the fracture in the tarsometatarsal joint are ignored, and consequently the screw implantation path planning result is not accurate is solved. According to the method, the optimal implantation path of the screw-fixed tarsometatarsal joint internal fracture is found in the three-dimensional model on the basis of reinforcement learning in combination with a clinical experience design optimization algorithm, and the operation precision and safety are greatly improved in actual clinical application.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent medical treatment, in particular to a metatarsal tarsal joint intraosseous fracture minimally invasive fixation method, system and terminal based on reinforcement learning. BACKGROUND

[0002] In the field of medical image processing and surgical planning, the clinical treatment of metatarsal tarsal joint intraosseous fracture (also known as Lisfranc fracture) has always been a serious challenge. The fracture occurs in the key load-bearing area of the foot, involving the complex joint structure between the metatarsal bones and the tarsal bones. The individual differences in anatomical morphology are large, and the surrounding important nerves, blood vessels and tendon tissues are also involved, which puts high requirements on the accuracy and safety of the operation. The postoperative recovery effect directly affects the foot function and walking ability of the patient, so the accuracy of surgical planning is crucial.

[0003] Traditional metatarsal tarsal joint intraosseous fracture minimally invasive fixation surgery scheme highly depends on the clinical experience and spatial judgment ability of the lead surgeon. The surgeon needs to conceive a three-dimensional surgical path based on two-dimensional image data. This experience-driven planning method has significant limitations. On the one hand, individual anatomical differences may cause the experience-based path design to be mismatched with the actual bone structure, increasing the risk of screw misplacement into the joint cavity or damage to adjacent tissues. On the other hand, path optimization under complex fracture types lacks quantitative evaluation criteria, making it difficult to ensure the consistency and optimality of surgical results, and the incidence of postoperative complications such as malunion and traumatic arthritis is high.

[0004] With the penetration of computer-aided technology in the medical field, some algorithms have begun to be applied to surgical path planning assistance. Among them, traditional path search algorithms such as Dijkstra algorithm have certain foundation in path optimization, but most of them are limited to two-dimensional plane scenarios or simplified three-dimensional models, and cannot completely restore the three-dimensional anatomical features of the bone. This kind of method usually ignores key spatial information such as bone surface curvature and internal cancellous bone distribution, resulting in deviation between the planned path and the actual surgical requirements.

[0005] In recent years, intelligent planning algorithms based on reinforcement learning have gradually attracted attention, but their application in the field of orthopedic minimally invasive surgery still has technical bottlenecks. Existing reinforcement learning schemes mostly use voxelization to process three-dimensional image data to simplify the computational complexity. This discrete processing method loses the continuous structural features of the bone, making it difficult to accurately capture key anatomical details such as joint space and bone cortex thickness. In addition, existing algorithms do not fully integrate clinical surgical standards and physician experience knowledge, and the generated path may not meet the mechanical requirements and safety criteria of minimally invasive operations, limiting its actual clinical applicability. SUMMARY

[0006] In view of the above-mentioned disadvantages of the prior art, the purpose of the present application is to provide a tarsometatarsal joint intraosseous fracture minimally invasive fixation method, system and terminal based on reinforcement learning, which is used to solve the problem that the continuous three-dimensional structural characteristics of the tarsometatarsal joint intraosseous fracture are ignored in the prior art, resulting in inaccurate screw implantation path planning results.

[0007] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a tarsometatarsal joint intraosseous fracture minimally invasive fixation method based on reinforcement learning, comprising: preprocessing the collected tarsometatarsal joint three-dimensional CT data and real surgery screw scanning point cloud data to obtain tarsometatarsal joint three-dimensional model data and surgery screw three-dimensional model data respectively; importing the tarsometatarsal joint three-dimensional model data and the surgery screw three-dimensional model data into Unity to obtain a three-dimensional tarsometatarsal joint and a three-dimensional screw in a surgery simulation scene, and calculate the initial implantation point position of the three-dimensional screw and the target region of implantation to obtain the optimal orientation for guiding the implantation of the three-dimensional screw, so as to determine the optimal implantation path of the three-dimensional screw.

[0008] In some embodiments of the first aspect of the present application, the preprocessing of the collected tarsometatarsal joint three-dimensional CT data and real screw scanning point cloud data comprises: wherein the tarsometatarsal joint three-dimensional CT data is segmented according to the bone structure, and the segmented bone data is then reconstructed to obtain the tarsometatarsal joint three-dimensional model data; and the real surgery screw scanning point cloud data is processed to obtain the surgery screw three-dimensional model data.

[0009] In some embodiments of the first aspect of the present application, a Raycast ray is arranged on the three-dimensional screw, which is used to detect the distance and orientation angle deviation of the three-dimensional screw relative to the target region through the Raycast ray.

[0010] In some embodiments of the first aspect of the present application, the calculation of the initial implantation point position of the three-dimensional screw comprises: calculating the maximum cross-section nine-square grid on the pre-surgery bone based on the pre-surgery bone in the three-dimensional tarsometatarsal joint, and taking the center coordinate positions of each grid point of the maximum cross-section nine-square grid as the initial implantation point positions of the three-dimensional screw implanted into the pre-surgery bone of the three-dimensional tarsometatarsal joint.

[0011] In some embodiments of the first aspect of the present application, the calculation of the target region of implantation comprises: determining the center column line of the corresponding one or more pre-surgery bones in the three-dimensional tarsometatarsal joint based on the one or more pre-surgery bones in the three-dimensional tarsometatarsal joint, and selecting a specific region on the center column line as the target region of implantation according to the actual surgery demand.

[0012] In some embodiments of the first aspect of the present application, the obtaining the optimal orientation for guiding the three-dimensional screw implantation comprises: implanting a three-dimensional screw according to each initial implantation point position, detecting the distance and orientation angle deviation of the three-dimensional screw relative to the target region through Raycast on the three-dimensional screw, and adjusting the tip orientation angle of the three-dimensional screw based thereon to determine the optimal orientation for guiding the three-dimensional screw implantation.

[0013] In some embodiments of the first aspect of the present application, after each adjustment of the tip orientation angle of the three-dimensional screw, it is detected whether a plurality of detection points on the tip fall within the target region; when falling within, the tip orientation angle of the three-dimensional screw is taken as the optimal orientation for guiding the three-dimensional screw implantation; when not falling within, the path of the implantation action is optimized according to the feedback of the reward function to adjust the tip orientation angle of the three-dimensional screw based on the path until it is detected that the plurality of detection points on the tip fall within the target region.

[0014] In some embodiments of the first aspect of the present application, the reward function is related to the distance of the three-dimensional screw relative to the target region, the distance of the three-dimensional screw relative to the non-target bone and other three-dimensional screws, and the degree of completion of the three-dimensional screw task.

[0015] To achieve the above object and other related objects, the second aspect of the present application provides a metatarsal tarsal joint intra-fracture minimally invasive fixation system based on reinforcement learning, comprising: a preprocessing module for preprocessing collected metatarsal tarsal joint three-dimensional CT data and real surgery screw scanning point cloud data to obtain metatarsal tarsal joint three-dimensional model data and surgery screw three-dimensional model data; a simulation optimization module for importing the metatarsal tarsal joint three-dimensional model data and the surgery screw three-dimensional model data into Unity, obtaining a three-dimensional metatarsal tarsal joint and a three-dimensional screw in a surgery simulation scene, and calculating an initial implantation point position and a target region of implantation of the three-dimensional screw to obtain an optimal orientation for guiding the three-dimensional screw implantation to determine an optimal implantation path of the three-dimensional screw.

[0016] To achieve the above object and other related objects, the third aspect of the present application provides an electronic terminal comprising a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the metatarsal tarsal joint intra-fracture minimally invasive fixation method based on reinforcement learning.

[0017] As described above, the reinforcement learning-based minimally invasive fixation method, system, and terminal for intra-articular tarsometatarsal fractures of the present invention have the following beneficial effects: The present invention preprocesses the acquired 3D CT data of the tarsometatarsal joint and the real surgical screw scan point cloud data, imports the obtained 3D model data of the tarsometatarsal joint and the 3D model data of the surgical screw into Unity, obtains the 3D tarsometatarsal joint and 3D screw in the surgical simulation scenario, calculates the initial implantation point position and target area of ​​the 3D screw, obtains the optimal orientation to guide the 3D screw implantation, and determines the optimal implantation path of the 3D screw. The present invention solves the problem in the prior art of ignoring the continuous 3D structural features of intra-articular tarsometatarsal fractures and resulting in inaccurate screw implantation path planning. The present invention uses reinforcement learning combined with clinical experience to design an optimization algorithm in the 3D model to find the optimal implantation path for screw fixation of intra-articular tarsometatarsal fractures, significantly improving surgical accuracy and safety in actual clinical applications. Attached Figure Description

[0018] Figure 1 The diagram shows a flowchart of an embodiment of the minimally invasive fixation method for intra-articular tarsometatarsal fractures based on reinforcement learning according to the present invention.

[0019] Figure 2 The image shows the reconstruction effect of a tarsometatarsal joint fracture model in one embodiment of the minimally invasive fixation method for intra-articular tarsometatarsal fractures based on reinforcement learning according to the present invention.

[0020] Figure 3 The diagram shows a Unity platform environment simulation of one embodiment of the minimally invasive fixation method for intra-articular tarsometatarsal fractures based on reinforcement learning according to the present invention.

[0021] Figure 4 The diagram shows a schematic of the initial implantation point calculation in one embodiment of the minimally invasive fixation method for intra-articular tarsometatarsal fractures based on reinforcement learning according to the present invention.

[0022] Figure 5 The diagram shows a schematic of the central column in one embodiment of the minimally invasive fixation method for intra-articular tarsometatarsal fractures based on reinforcement learning according to the present invention.

[0023] Figure 6 The diagram shows a structural schematic of an embodiment of the minimally invasive fixation system for intra-articular tarsometatarsal fractures based on reinforcement learning according to the present invention.

[0024] Figure 7 The diagram shows a structural schematic of an embodiment of the minimally invasive fixation terminal for intra-articular metatarsophalangeal fractures based on reinforcement learning according to the present invention. Detailed Implementation

[0025] The present application can be implemented or applied in other different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following examples and features in the examples can be combined with each other without conflict.

[0026] Before further detailing the present application, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations:

[0027] <1> Unity platform: Unity platform is a cross-platform game engine that supports the development and interactive experience on multiple platforms. Unity platform can be used to create medical simulators, surgical training tools, patient education applications and virtual reality treatment programs, etc.

[0028] <2> Agent: Agent refers to a software or hardware system that can perceive, decide and act in an environment, usually used in reinforcement learning or multi-agent systems.

[0029] <3> Reward function: Reward function is used to quantify the reward or punishment of the agent after performing a specific action in the environment, guiding the agent to learn the optimal strategy.

[0030] <4> ML-Agents toolkit: ML-Agents toolkit is a machine learning toolkit provided by Unity for training agents in Unity environment, supporting various reinforcement learning algorithms such as PPO.

[0031] <5> PPO algorithm: PPO algorithm (Proximal Policy Optimization) is a commonly used reinforcement learning algorithm, which introduces constraints in the process of policy update to ensure that the new policy is close to the old policy, improving the stability and efficiency of training.

[0032] <6> Mimics Research: Mimics Research is a medical image processing software used for normalizing, multi-skeletal instance segmentation and high-precision surface reconstruction of metatarsal joint three-dimensional CT data to retain continuous three-dimensional structural features and generate personalized skeletal three-dimensional models.

[0033] <7>Convex: Convex is a setting in three-dimensional modeling that enables convex hull collision bodies. Convex hull refers to the smallest convex polyhedron that encloses the model. The combination of the two can optimize the collision detection efficiency of complex models such as bones in Unity, while preserving key geometric features to avoid collision errors.

[0034] To facilitate understanding of the embodiments of the present application, first, the Figure 1 detailed description. Figure 1 A flowchart of a minimally invasive fixation method for intra-tarsal joint fracture based on reinforcement learning in an embodiment of the present application is shown. The minimally invasive fixation method for intra-tarsal joint fracture based on reinforcement learning in the present embodiment mainly includes the following steps:

[0035] Step S11: Preprocess the collected intra-tarsal joint three-dimensional CT data and real surgical screw scanning point cloud data to obtain intra-tarsal joint three-dimensional model data and surgical screw three-dimensional model data, respectively.

[0036] In an embodiment of the present application, the traditional method based on 2D or 3D voxel data has low accuracy and is difficult to meet the needs of personalized surgery for patients, so the original intra-tarsal joint three-dimensional CT data needs to be segmented accordingly. The purpose is to accurately extract the bony structure of the intra-tarsal joint including the metatarsal bone, tarsal bone and joint space between them from the mixed image containing skin, muscle, fat and other soft tissues. And reconstruct a new surgical screw three-dimensional model for the segmented image.

[0037] The preprocessing of the collected intra-tarsal joint three-dimensional CT data and real screw scanning point cloud data includes: segmenting the intra-tarsal joint three-dimensional CT data according to the bone structure, and then reconstructing the segmented bone data to obtain intra-tarsal joint three-dimensional model data. The collected intra-tarsal joint three-dimensional CT data comes from a large number of real clinical cases.

[0038] Specifically, the first step is data normalization. Using the software's built-in grayscale calibration function, CT data acquired from different devices and with different scanning parameters are uniformly mapped to a standard grayscale range. For example, the high attenuation coefficient value corresponding to the bone cortex is standardized to a fixed grayscale range, eliminating grayscale fluctuations caused by device errors and ensuring stable and discernible grayscale differences between bones and soft tissues during subsequent segmentation. Simultaneously, spatial coordinate normalization is performed, establishing a local coordinate system with the center point of the metatarsophalangeal joint as the origin. This unifies the anatomical position reference benchmark for different patients, laying the foundation for subsequent cross-individual path planning algorithm adaptation. Secondly, multi-bone instance segmentation: Based on the morphological characteristics of the metatarsals and tarsals, preliminary region division is first performed using the natural boundary between bone tissue with high grayscale values ​​and soft tissue with low grayscale values. Then, for abnormal structures such as bone fragments and joint dislocations that may exist at fracture sites, the boundaries are refined by manually outlining the contours or using specialized tools, ensuring that each bone is segmented as an independent instance. Crucial anatomical landmarks such as fracture lines and joint spaces are clearly preserved, avoiding feature loss due to structural adhesions in traditional voxelization processing. Finally, high-precision surface reconstruction is performed: surface reconstruction is carried out on each segmented bone. The software generates a continuous 3D surface model by calculating the isosurface of the bone contour. During the reconstruction process, the mesh cells are densified in areas sensitive to path accuracy, such as the bone cortex edges and articular surfaces, while the density is appropriately reduced in non-critical areas such as the interior of cancellous bone, balancing computational efficiency while ensuring the complete preservation of continuous 3D structural features. The initial 3D mesh model generated by surface reconstruction may contain noise, redundant topology, or geometric defects, which need to be cleaned up to meet the accuracy requirements of reinforcement learning algorithms for continuous 3D structural features. The final generated 3D model contains the 3D coordinates, normal vectors, and bone type labels of each vertex, providing a 3D model that can be used for surgical planning in subsequent reinforcement learning algorithms.

[0039] For example, such as Figure 2 The image shown is a reconstruction result of a metatarsophalangeal joint fracture model in one embodiment. The clinician first read the original three-dimensional CT scan data of the patient's foot and imported it into Mimics Research to perform multi-step refinement processing to solve the problem of loss of bone structure features in the prior art.

[0040] It is worth pointing out that the real surgical screw scan point cloud data also needs to be processed to obtain the surgical screw three-dimensional model data. In order to ensure the adaptability of the planned path and the actual screw implantation, the processing flow of the real screw used in the operation is as follows: first, screw point cloud data acquisition and import: full surface scanning of the real screw is performed to obtain high-density point cloud data, and key physical characteristics such as screw thread form, tip angle, and rod diameter are captured. Second, import the point cloud data into Mimics Research software, and align the screw coordinate system with the bone model coordinate system through the coordinate alignment tool. Defects such as scanning blind area at the thread groove that may occur during the scanning process are processed to ensure that the point cloud data completely reflects the three-dimensional shape of the screw. Then, the repaired point cloud data is converted into a three-dimensional model that can be used for surgical planning.

[0041] Step S12: Import the metatarsal joint three-dimensional model data and the surgical screw three-dimensional model data into Unity to obtain the three-dimensional metatarsal joint and the three-dimensional screw in the surgical simulation scene, and calculate the initial implantation point position and the target area of the three-dimensional screw implantation, and obtain the optimal orientation of the guided three-dimensional screw implantation, to determine the optimal implantation path of the three-dimensional screw.

[0042] In an embodiment of the present application, the metatarsal joint three-dimensional model data and the surgical screw three-dimensional model data are imported into Unity to obtain the three-dimensional metatarsal joint and the three-dimensional screw in the surgical simulation scene, and appropriate variable attributes are set for the three-dimensional metatarsal joint and the three-dimensional screw, including but not limited to the scale in each direction, rigid body, and Convex setting, to ensure that the simulated environment is consistent with the real data of the patient to the greatest extent in modeling.

[0043] Specifically, the Unity surgery simulation scene takes the geometric center of the three-dimensional tarsometatarsal joint as the scene origin, so that the initial position of the three-dimensional screw maintains a preset distance from the three-dimensional tarsometatarsal joint. After importing the preprocessed three-dimensional model of the tarsometatarsal joint and the three-dimensional model of the surgical screw into Unity, accurate scaling parameters are set in Unity according to the physical resolution of the CT scan, so that the size of the imported three-dimensional tarsometatarsal joint in the Unity scene is the same as the size of the real bone of the patient. For the three-dimensional screw, the scaling ratio is calibrated according to its clinical specifications to ensure that the key physical characteristics such as thread spacing and tip angle are consistent with the real screw, avoiding errors in screw and bone adaptability caused by proportion deviation in path planning. Regarding the Convex setting, a mesh collision component is added to each bone model in the three-dimensional tarsometatarsal joint for Convex setting. For continuous and smooth areas such as bone cortex surface, the high-precision features of the original mesh collision body are retained; for irregular bone fragments in the fracture area, the collision body topology is optimized through Convex to ensure collision detection efficiency while avoiding collision misjudgment caused by mesh deformation. In addition, a rigid body component is added to each bone model, and the parameters are configured according to the physical properties of the bone tissue: the mass parameter is set to the actual mass of the bone to simulate the fixed state of the bone in the human body.

[0044] In an embodiment of the present application, a Raycast ray is set on the three-dimensional screw to detect the distance and orientation angle deviation of the three-dimensional screw relative to the target area through the Raycast ray, achieving high-precision measurement of distance and orientation angle; the three-dimensional screw is set as an agent (Agent) in reinforcement learning; to realize precise regulation and control of the reinforcement learning algorithm on the screw implantation path, a dedicated Raycast ray perception system is configured for each three-dimensional screw in the Unity environment, and the screw action is limited according to the actual needs of clinical surgery, the specific design is as follows: in terms of ray deployment position and direction calibration, the emission point of the Raycast ray is fixed at the tip of the screw, and the ray direction is consistent with the actual implantation path of the screw; the length and angle parameters can be customized: the ray length (Length) can be dynamically adjusted according to the depth of the fracture site to ensure that the ray can stably hit the target area. The perception range (Angle) adopts a conical angle setting, which simultaneously detects the distance and angle deviation of each bone of the three-dimensional tarsometatarsal joint within the fan-shaped range; if the target area within the range exceeds the range, a target loss warning is triggered, prompting the algorithm to adjust the orientation of the three-dimensional screw.

[0045] It should be noted that in order to simulate the operation characteristics of screw implantation in real surgery and simplify the state space and computational complexity of reinforcement learning, the degrees of freedom of the three-dimensional screw are limited in the following aspects: in terms of the range of rotation angles, the rotation of the three-dimensional screw in the three-dimensional space is limited within ±20°; this design is based on clinical practice: in minimally invasive surgery of the tarsal joint, the adjustment range of the surgical screw is usually small to avoid excessive traction of soft tissue by the instrument, and excessive rotation is easy to cause the path to deviate from the safe area of the bone cortex. In addition, the Y-axis of the three-dimensional screw Local coordinate system is set to a rotation lock state, so that the three-dimensional screw cannot perform a self-rotation action around its own axis; this limitation is consistent with clinical practice: in minimally invasive fixation surgery, the surgical screw is implanted by the instrument, and the self-rotation is only used for the surgical screw to be screwed into the bone tissue, and the posture adjustment in the path planning stage does not require self-rotation action; if self-rotation is allowed, it may cause the Raycast ray direction to be misaligned with the actual advancement direction of the surgical screw, increasing the complexity of the algorithm for judging the posture, and after locking, it can ensure that the ray direction is always consistent with the implantation direction of the surgical screw, reducing the amount of invalid calculation.

[0046] As shown in Figure 3 is a Unity platform environment simulation diagram in an embodiment, the bones of the three-dimensional tarsal joint are simulated in reinforcement learning by setting various variable attributes in the Unity platform to simulate the characteristics of actual clinical bones, and the plurality of three-dimensional screws are each provided with a Raycast ray, which can be used to detect a target region on the preoperative bone in the three-dimensional tarsal joint to determine the optimal orientation of the three-dimensional screw by using a reinforcement learning algorithm, so as to train an optimal implantation path.

[0047] In an embodiment of the present application, before building a reinforcement learning task framework, an experimental scheme needs to be designed according to clinical needs, and the initial implantation point of the surgical screw needs to be specified in advance. According to the minimally invasive fixation method of the tarsal joint fracture in clinical experience, the physician usually takes the dorsal surface of the medial cuneiform bone as a reference, finds a rectangular section with the largest area, divides the rectangular section into a nine-square structure, and places one or more surgical screws at different positions of the nine-square structure as initial implantation points to obtain maximum stability and fully expose the tarsal joint. Therefore, as shown in Figure 4 is a calculation diagram of the initial implantation point in an embodiment. The calculation of the initial implantation point position of the three-dimensional screw includes: calculating a maximum section nine-square structure on the preoperative bone based on the preoperative bone in the three-dimensional tarsal joint, and taking the center coordinate positions of each grid point of the maximum section nine-square structure as the initial implantation point positions of the three-dimensional screw implanted in the preoperative bone of the three-dimensional tarsal joint.

[0048] Specifically, based on the Unity environment settings, a Convexhull structure of the pre-operative bone for the 3D tarsometatarsal joint is obtained. This Convexhull eliminates internal cavities and complex concave surfaces while retaining the outer contour features to reduce computational complexity. All faces of the Convexhull are traversed, and the area of ​​each face is calculated to find the largest face as the largest cross-section of the medial wedge. Planar normal vectors are calculated based on at least three vertices of the largest cross-section, and the smallest bounding rectangle of this largest cross-section is further calculated based on the normal vectors as the final rectangular cross-section. After obtaining the final rectangular cross-section, a 3×3 grid is evenly divided based on the coordinates of the four vertices of the rectangle, and the center coordinates of each grid point are calculated. According to clinical needs, the center coordinates of any grid point are input into the position variable of the 3D screw to be implanted, achieving the effect of placing the screw at the center of a certain grid point. Preferably, implanting two surgical screws between the pre-operative bones generally achieves the best stability.

[0049] In one embodiment of the present invention, to calculate the optimal orientation angle of the three-dimensional screw, the orientation can be adjusted by calculating the distance and orientation angle deviation between the three-dimensional screw and the target area on the pre-operated bone. The calculation of the target area for implantation includes: determining the central column of one or more pre-operated bones in the three-dimensional tarsometatarsal joint based on one or more pre-operated bones, and selecting a specific area on the central column as the target area for implantation according to actual surgical needs.

[0050] It should be noted that the optimal orientation in clinical practice generally follows the principle of "facing the instep and facing outwards." This is because facing the instep, passing through the metatarsals to reach the fracture site, can stabilize the fracture while avoiding damage to the medial plantar neurovascular bundle and interference with blood supply. Facing outwards avoids collisions between surgical screws at other metatarsal sites and transversely fixed surgical screws, especially those that traverse inwards from the medial cuneiform bone, in some cases. Therefore, this invention calculates the optimal orientation of the surgical screw during the determination of the optimal implantation path. To this end, this invention designs a central column line in the pre-operative bone, and selects a target area on the central column line according to clinical surgical needs for reference in calculating the optimal orientation of the screw.

[0051] Specifically, such as Figure 5The center column line is shown as a schematic diagram in an embodiment, which is based on the Convex of the preoperative bone, and a cylinder is generated at the center of the Convex, which is consistent with the orientation of the preoperative bone. By using the principal component analysis (PCA) algorithm, the covariance matrix of the line segment is determined by calculating the coordinate array of multiple vertices according to the correlation of the distribution of the vertices of the Convex in each direction, and the eigenvector corresponding to the maximum eigenvalue is obtained, so as to determine the longest vector which is consistent with the orientation of the preoperative bone. According to the coordinates of the start and end points of the longest vector, a line segment is generated at the center of the Convex, which is parallel to the longest vector. The cylinder with the line segment as the axis and r as the radius, and r can be adjusted according to the surgical requirements. In addition, in order to prevent the deviation of the obtained center column line from the actual orientation of the preoperative bone due to the irregularity of the Convex when calculating the longest vector, the three-dimensional coordinate values of the center column line can be manually defined and modified by the doctor, so that the orientation of the center column line is consistent with that of the preoperative bone. At this time, the optimal orientation of the three-dimensional screw is only to adjust the orientation angle of the screw to the target area on the center column line based on the reinforcement learning algorithm.

[0052] In an embodiment of the present application, obtaining the optimal orientation for guiding the implantation of the three-dimensional screw includes: implanting the three-dimensional screw according to the position of each initial implantation point, and detecting the distance and orientation angle deviation of the three-dimensional screw relative to the target area by Raycast ray on the three-dimensional screw, and adjusting the orientation angle of the tip of the three-dimensional screw based on this to determine the optimal orientation for guiding the implantation of the three-dimensional screw. In the reinforcement learning of the optimal orientation for guiding the implantation of the three-dimensional screw, when the screw moves along the candidate path in the scene, the Raycast ray is triggered to detect the ray: the tip axis ray is emitted from the emission point along the screw axis to the target area marker point, if the ray hits the target area, the straight line distance value from the ray starting point, i.e. the screw tip, to the hitting point, i.e. the surface of the target area, is returned; if the ray does not hit the target area but hits the non-target area of the bone, the shortest collision distance is returned and marked.

[0053] Specifically, the detection logic of the orientation angle deviation and the distance: the distance detection is realized by the hitting feedback of the Raycast ray, when the Raycast ray hits the surface of the target area, the Unity engine returns the straight line distance value from the ray starting point to the hitting point, and simultaneously records the X, Y, Z axis components of the distance in the three-dimensional space, providing the position deviation data in the three-dimensional direction for the algorithm; the orientation angle deviation is calculated by the included angle between the ray direction vector and the normal vector of the target area, the surface normal vector of the hitting point of the target area is obtained by the three-dimensional tarsal joint model mesh, the included angle between the screw ray direction and the normal vector is calculated, and then the action strategy is dynamically adjusted by the reinforcement learning algorithm according to the size of the included angle deviation.

[0054] In an embodiment of the present application, after adjusting the tip portion of the three-dimensional screw each time, it is detected whether the plurality of detection points on the tip portion fall within the target region; wherein, when falling within, the three-dimensional screw tip portion orientation angle is taken as the optimal orientation for guiding the three-dimensional screw implantation; when not falling within, the path of the implantation action is optimized according to the feedback of the reward function to adjust the tip portion of the three-dimensional screw orientation angle based on the path until the plurality of detection points on the tip portion are detected to fall within the target region. Specifically, a plurality of feature detection points are taken on the tip portion of the three-dimensional screw to form a spatial distribution network: a tip vertex detection point, a plurality of edge detection points uniformly distributed around it along the cross-sectional circumference of the screw. The coordinates of the detection points are dynamically calculated through the screw Local coordinate system, and the spatial positions are updated in real time with the adjustment of the screw orientation to ensure real-time detection of whether the tip portion falls within the target region.

[0055] In an embodiment of the present application, the three-dimensional screw finds the optimal implantation path guided by the reward function, and the reward function is related to the distance of the three-dimensional screw relative to the target region, the distance of the three-dimensional screw relative to the non-target bone and other three-dimensional screws, and the task completion degree of the three-dimensional screw. The reward function is combined by the distance of the three-dimensional screw relative to the target region, the distance relative to the non-target bone and other three-dimensional screws, and the task completion degree of the three-dimensional screw, and is given different weights to finally obtain a comprehensive reward value. The higher the reward value represents the more the implantation path of the three-dimensional screw conforms to the task target.

[0056] Specifically, the reward function r navigation is related to the distance of the three-dimensional screw relative to the target region, and if it is closer to the target region, a bonus reward will be given; in addition, if the Raycast ray detects that the three-dimensional screw is consistent with the target region when it is closer to the target region, an additional bonus reward will be given. The reward function r scenario is related to the distance of the three-dimensional screw relative to the non-target bone and other three-dimensional screws, and if the action of the three-dimensional screw collides with the non-target bone or other three-dimensional screws in the simulation environment, a deduction penalty will be given; the reward function r penalty is related to the task completion degree of the three-dimensional screw, and the average time length is set for the time used for the action of the three-dimensional screw, and a deduction penalty is given for the action that exceeds the average time length, so that the task of the optimal implantation path can be completed in a shorter time.

[0057] In an embodiment of the present application, the present application simulates different surgical plans, evaluates the pros and cons of each plan according to the surgical target and constraint conditions, finally selects the optimal implantation path, and displays the specific operation steps of the plan to the doctor, helping the doctor to formulate the actual surgical plan. The reinforcement learning training toolkit that can be used in reinforcement learning includes but is not limited to: ML-Agents toolkit, TensorFlow Agents toolkit, PyTorch RL toolkit, etc. The optimization algorithm of reinforcement learning that can be used includes but is not limited to: Q-Learning algorithm, SARS A algorithm, Deep Q-Network (DQN) algorithm, Policy Gradient algorithm, Actor-Critic algorithm, Proximal Policy Optimization (PPO) algorithm, Trust Region Policy Optimization (TRPO) algorithm, etc.

[0058] Preferably, the PPO algorithm supported by the ML-Agents toolkit optimizes the strategy in real-time model training in the simulation environment. The advantage of the PPO algorithm in the ML-Agents toolkit in the application of the present application is that the PPO algorithm supports continuous action space optimization, without the need for discretization processing of continuous parameters such as screw orientation angle and position, perfectly matching the complex continuous three-dimensional anatomical structure characteristics of the tarsal joint. During the screw orientation adjustment process, the algorithm can directly output the continuous rotation angle around the coordinate axis. This continuous regulation ensures that the screw tip part orientation can smoothly fit the natural curvature of the bone surface, avoids the path deviation caused by discretization truncation, and solves the problem of discontinuous path caused by structure feature loss in voxelization processing.

[0059] It should be noted that in the embodiments of the present application, the words such as "exemplary" or "for example" represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words such as "exemplary" or "for example" are intended to present the relevant concept in a specific manner.

[0060] Figure 6 is a structural schematic diagram of a tarsal joint intraosseous fracture minimally invasive fixation system based on reinforcement learning provided by an embodiment of the present application. As shown in Figure 6 , the device includes a preprocessing module, a simulation optimization module.

[0061] The preprocessing module 601 is used for preprocessing the collected tarsal joint three-dimensional CT data and real surgical screw scanning point cloud data to obtain tarsal joint three-dimensional model data and surgical screw three-dimensional model data;

[0062] The simulation optimization module 602 is configured to import the three-dimensional model data of the tarsometatarsal joint and the three-dimensional model data of the surgical screw into Unity, obtain the three-dimensional tarsometatarsal joint and the three-dimensional screw in the surgical simulation scene, and calculate the initial implantation point position of the three-dimensional screw and the target region of implantation, obtain the optimal orientation for guiding the implantation of the three-dimensional screw, and determine the optimal implantation path of the three-dimensional screw.

[0063] It should be understood that the specific process of each module performing the corresponding steps described above has been described in detail in the method embodiments described above, and for the sake of brevity, will not be repeated here.

[0064] It should also be understood that the division of the modules in the embodiments of the present application is illustrative, and is only a logical functional division. In actual implementation, there can be another division manner. In addition, each functional module in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0065] Figure 7 is a structural schematic diagram of an electronic terminal provided by an embodiment of the present application. As shown in Figure 7 , the electronic terminal includes at least one processor 701, a memory 702, at least one network interface 703, and a user interface 705. Each component in the apparatus is coupled together through a bus system 704. It can be understood that the bus system 704 is used to realize the connection and communication between the components. In addition to including a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all kinds of buses are marked as the bus system in Figure 7 .

[0066] Among them, the user interface 705 can include a display, a keyboard, a mouse, a trackball, a click gun, a key, a button, a touchpad, or a touch screen, etc.

[0067] It is to be understood that the memory 702 can be volatile or nonvolatile memory, or both. Nonvolatile memory can be read only memory (ROM), programmable ROM (PROM), flash memory, or the like, used as external cache. By way of example, and not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous dynamic random access memory (SDRAM), and the like. The memory described in the embodiments of the present application is intended to include, without being limited to, these and any other suitable type of memory.

[0068] The memory 702 in the embodiments of the present application is configured to store various types of data to support the operation of the electronic terminal 700. Examples of the data include any executable programs for operating on the electronic terminal 700, such as an operating system 7021 and an application program 7022. The operating system 7021 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application program 7022 can include various application programs, such as a media player, a browser, and the like, for implementing various application services. The method for fixing a metatarsal fracture provided by the embodiments of the present application can be included in the application program 7022.

[0069] The method disclosed in the embodiments of the present application can be applied to the processor 701 or implemented by the processor 701. The processor 701 can be an integrated circuit chip having a processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 701. The processor 701 described above can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like. The processor 701 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor 701 can be a microprocessor or any conventional processor, and the like. The steps of the method for optimizing the accessories provided by the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium, which is located in the memory. The processor reads the information in the memory and combines the hardware to complete the steps of the above method.

[0070] In exemplary embodiments, the electronic terminal 700 can be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), or the like for performing the aforementioned methods.

[0071] As used in this description, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. By way of illustration, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and / or thread of execution and a component can be localized, partially localized, and / or distributed across two or more computers. Also, these components can execute from various computer readable media having various data structures stored thereon. The components can communicate by way of local and / or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and / or across a network such as the Internet with other systems via the signal), software applications, other information, and / or the like.

[0072] Those of ordinary skill in the art will realize and understand, the various illustrative logical blocks and steps described in connection with the embodiments disclosed herein can be implemented, either in electronic hardware, or in a combination of computer software and electronic hardware. The choice of hardware or software implementation depends on the specific application and design constraints imposed on the technical solution. Those of ordinary skill in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0073] Those of ordinary skill in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here again.

[0074] In several embodiments provided by the present application, it should be understood that the disclosed system, apparatus and method can be implemented in other manners. For example, the apparatus embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0075] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0076] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be a physically separate unit, or two or more units can be integrated into a unit.

[0077] In the above embodiments, the functions of the functional units can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, the whole or part of the flow or function according to the embodiments of the present application is generated. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as high-density digital video disc (Digital Video Disc, DVD)) or semiconductor media (such as solid state disk (Solid State Disk, SSD) and the like.

[0078] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0079] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0080] To sum up, the present application provides a kind of based on the minimally invasive fixation method, system and terminal of tarsal joint fracture of reinforcement learning, pre-processing is carried out to the three-dimensional CT data and real operation screw point cloud data of tarsal joint fracture collected by the present application, three-dimensional model data of tarsal joint and three-dimensional model data of surgical screw are obtained and imported into Unity, three-dimensional tarsal joint and three-dimensional screw under the operation simulation scene are obtained, and the initial implantation point position of three-dimensional screw and the target area of implantation are calculated, the optimal orientation of guiding three-dimensional screw implantation is obtained, to determine the optimal implantation path of three-dimensional screw. The present application solves the problem that the continuous three-dimensional structure characteristics of tarsal joint fracture are ignored in the prior art, and the screw implantation path planning result is not accurate. The present application finds the optimal implantation path of screw fixation tarsal joint fracture in three-dimensional model based on reinforcement learning combined with clinical experience to design optimization algorithm, which greatly improves the operation precision and safety in actual clinical application.

[0081] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical idea disclosed by the present application should be covered by the claims of the present application.

Claims

1. A method of minimally invasive fixation of intra-tarsal joint fracture based on reinforcement learning, characterized in that, The method comprises the following steps: The collected metatarsal tarsal joint three-dimensional CT data and the real operation screw scanning point cloud data are preprocessed to obtain metatarsal tarsal joint three-dimensional model data and operation screw three-dimensional model data; The metatarsal tarsal joint three-dimensional model data and the operation screw three-dimensional model data are imported into Unity to obtain a three-dimensional metatarsal tarsal joint and a three-dimensional screw in a surgical simulation scene, and the initial implantation point position and the target region of the three-dimensional screw are calculated to obtain the optimal orientation of the guided three-dimensional screw implantation, so as to determine the optimal implantation path of the three-dimensional screw.

2. The method of claim 1, wherein the method further comprises: The preprocessing of the collected metatarsal tarsal joint three-dimensional CT data and the real screw scanning point cloud data comprises the following steps: the three-dimensional CT data of the metatarsal tarsal joint is segmented according to the bone structure, and the segmented bone data is reconstructed to obtain the three-dimensional model data of the metatarsal tarsal joint; The real operation screw scanning point cloud data is processed to obtain the three-dimensional model data of the operation screw.

3. The reinforcement learning based minimally invasive method for fixing intra- osseous fracture of the tarsal joint as claimed in claim 2, wherein, A Raycast ray is arranged on the three-dimensional screw, which is used to detect the distance and orientation angle deviation of the three-dimensional screw relative to the target region through the Raycast ray.

4. The reinforcement learning based minimally invasive fixation method for intra- tarsal joint fracture according to claim 3, characterized in that, The initial implantation point position of the three-dimensional screw is calculated, which comprises the following steps: the maximum cross-section nine-square grid on the preoperative bone of the three-dimensional metatarsal tarsal joint is calculated based on the preoperative bone of the three-dimensional metatarsal tarsal joint, and the center coordinates of each grid point of the maximum cross-section nine-square grid are taken as the initial implantation point positions of the three-dimensional screw implanted in the preoperative bone of the three-dimensional metatarsal tarsal joint.

5. The reinforcement learning based minimally invasive method for fixing intra- osseous fracture of the tarsal joint as claimed in claim 4, wherein, The target region of the implantation is calculated, which comprises the following steps: the center column line of one or more preoperative bones in the three-dimensional metatarsal tarsal joint is determined based on the one or more preoperative bones in the three-dimensional metatarsal tarsal joint, and a specific region on the center column line is selected as the target region of the implantation according to the actual operation requirement.

6. The reinforcement learning based minimally invasive method for fixing intra- osseous fracture of the tarsal joint as claimed in claim 5, wherein, The optimal orientation of the guided three-dimensional screw implantation is obtained, which comprises the following steps: The three-dimensional screw is implanted according to each initial implantation point position, the distance and orientation angle deviation of the three-dimensional screw relative to the target region are detected through the Raycast ray on the three-dimensional screw, and the orientation angle of the tip of the three-dimensional screw is adjusted based on this to determine the optimal orientation of the guided three-dimensional screw implantation.

7. The reinforcement learning based minimally invasive method for fixing intra- osseous fracture of the tarsal joint as claimed in claim 6, wherein, After adjusting the orientation angle of the tip of the three-dimensional screw each time, it is detected whether the plurality of detection points on the tip fall into the target region; when falling into the target region, the orientation angle of the tip of the three-dimensional screw is taken as the optimal orientation of the guided three-dimensional screw implantation; when not falling into the target region, the path of the implantation action is optimized according to the feedback of the reward function, so that the orientation angle of the tip of the three-dimensional screw is adjusted based on the path, until it is detected that the plurality of detection points on the tip fall into the target region.

8. The reinforcement learning based minimally invasive method for fixing intra- osseous fracture of the tarsal joint as claimed in claim 7, wherein, The reward function is related to the distance of the three-dimensional screw relative to the target region, the distance of the three-dimensional screw relative to the non-target bone and other three-dimensional screws, and the task completion degree of the three-dimensional screw.

9. A minimally invasive system for intra-tarsal fracture fixation based on reinforcement learning, characterized in that, The method comprises the following steps: A preprocessing module is arranged to preprocess the collected metatarsal tarsal joint three-dimensional CT data and real operation screw scanning point cloud data to obtain metatarsal tarsal joint three-dimensional model data and operation screw three-dimensional model data; The simulation optimization module is used for importing the tarsometatarsal joint three-dimensional model data and the surgical screw three-dimensional model data into Unity, obtaining the three-dimensional tarsometatarsal joint and the three-dimensional screw under the surgical simulation scene, and calculating the initial implantation point position of the three-dimensional screw and the target area of implantation, obtaining the optimal orientation for guiding the three-dimensional screw implantation, and determining the optimal implantation path of the three-dimensional screw.

10. An electronic terminal comprising a memory, a processor and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the tarsometatarsal joint intraosseous fracture minimally invasive fixation method based on the reinforcement learning in any one of claims 1 to 8.

Images