Intelligentized auxiliary generation method for optimal channel in operation based on augmented reality technology

Abstract

The invention discloses an intra-operative optimal channel intelligent auxiliary generation method based on an augmented reality technology, an orthopedic operation channel is generated based on the AR technology, a doctor is assisted in selecting an optimal intra-operative channel to implement an operation, the time for searching for the optimal channel in the early stage is saved, and the operation progress is prevented from being delayed. In addition, an intra-operative channel of an orthopedic surgery can be simulated in an augmented reality environment, doctors can be conveniently guided to accurately implement the surgery, and large deviation in the surgery is avoided, so that surgical risks are avoided. By means of AR intra-operative channel construction, deep operation channel guidance can be provided for doctors, operation guidance can be conveniently provided for the doctors from the channel implementation depth, and large operation wounds are avoided. In addition, during intraoperative implementation, the intraoperative channel can be synchronously displayed based on AR, whether the intraoperative channel deviates from the preset AR optimal channel or not is supervised by a doctor and other administrators, an alarm is given if deviation occurs, the accuracy of operation implementation can be supervised, and the influence of expansion caused by deviation on an operative wound of a patient is avoided.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical technology, and in particular to an intelligent auxiliary generation method of an optimal intraoperative channel based on augmented reality technology, an intelligent auxiliary generation device of an optimal intraoperative channel based on augmented reality technology, and electronic equipment. Background Art

[0002] The pre-generation of surgical channels (also known as surgical access channels) is an important technology during surgery, which helps reduce risks, improve efficiency and achieve more precise surgical operations. With the continuous development and innovation of medical technology, the pre-generation of surgical channels can provide patients with safer, more efficient and comfortable options for surgical treatment.

[0003] A surgical corridor is a path or passage established during surgery to reach the surgical area. Figure 1 The femoral intramedullary nailing transpyriform sinus surgical approach (the dotted line) is shown. Figure 2 The surgical approach for fixation of intercondylar fracture of humerus shown in the figure (cross lines around the fixation pin). The pre-generation of surgical corridor refers to the pre-planning, construction or preparation of surgical corridor through various techniques and means before the operation begins, so that the operation can proceed smoothly.

[0004] Pre-generating surgical corridors has many advantages. First, it helps reduce risks and uncertainties during surgery. By planning and building corridors in advance, doctors can better understand the structures and anatomical relationships in the surgical area, thereby avoiding possible injuries or complications. Second, pre-generating corridors can improve the efficiency and precision of surgery. Doctors can position and operate surgical instruments more accurately according to the pre-set corridor path, thereby speeding up the operation and reducing errors.

[0005] However, in practical applications, the pre-generation of surgical corridors mainly adopts the following two methods:

[0006] A common method is to use medical imaging technology, such as CT and MRI, to perform detailed scans and images of the surgical area. Through these imaging data, doctors can accurately plan the path and size of the surgical corridor and use it as a reference and guide during the operation.

[0007] Another approach is to use navigation systems and robotics technology to achieve automatic construction and adjustment of surgical channels through computer-assisted positioning and navigation.

[0008] Although the above method can construct the corresponding surgical channel, it has the following clinical use defects when constructing the surgical channel for clinical orthopedic surgery:

[0009] First, the image data used to construct the surgical corridor is mainly two-dimensional image data, and doctors can only Figure 1 and attached Figure 2 The corresponding surgical path is constructed on the plane image shown, so there is no path design in depth; and bones have different shapes, and the spatial channel for performing surgery needs to present a sense of spatial hierarchy. Therefore, the surgical channel designed in the traditional way cannot allow doctors to intuitively learn the three-dimensional spatial channel to reach the bone lesion, and cannot provide doctors with channel depth guidance during surgery, lacking spatial hierarchical surgical guidance;

[0010] Secondly, each time a surgical channel is constructed, multiple surgical paths can actually be provided to the patient, but this requires the doctor to exercise clinical control and determine which surgical path to choose as the optimal intraoperative channel for clinical surgery. This requires the doctor to have rich clinical surgical experience and higher professional capabilities. For some early surgeons, it is impossible to quickly determine the optimal intraoperative channel, and it will take a lot of time to verify and find the optimal intraoperative channel, delaying the progress of the operation. Summary of the invention

[0011] In order to solve the technical problems existing in the prior art, the embodiments of the present invention provide the following technical solutions:

[0012] On the one hand, a method for intelligently assisting generation of an optimal intraoperative channel based on augmented reality technology is provided, the method being implemented by an electronic device, and the method comprising:

[0013] Construct three-dimensional skeletal medical models of skeletal areas;

[0014] On the three-dimensional bone medical model, a plurality of preset surgical pre-incision position points p1 and bone lesion position points p2 are marked;

[0015] Constructing a plurality of bone surgery intraoperative paths between the surgical pre-incision position point p1 and the bone lesion position point p2 on the three-dimensional bone medical model;

[0016] Marking the path-related elements surrounding each of the intraoperative paths of the bone surgery that affect the implementation of the intraoperative surgery, and finding the optimal intraoperative path for implementing the intraoperative surgery based on the path-related elements;

[0017] Taking the intraoperative optimal path as a rotation axis and stretching according to a preset contour line, a corresponding intraoperative optimal path channel is generated in the three-dimensional bone medical model;

[0018] Based on AR integration technology, an AR three-dimensional model containing the optimal path channel during the operation is constructed and sent to the AR device for display.

[0019] As a preferred embodiment of the present invention, the path association element includes a path priority element x and a path failure element p, wherein:

[0020] The path priority element x is used to characterize the location points that should be included in the intraoperative operation, including the natural gaps in the bone area and the location points where the anatomical structures are located;

[0021] The path fault element p is used to characterize the location points that should not be included in the intraoperative surgery, including the location points where the nerves, blood vessels and soft tissues in the bone area are located.

[0022] As a preferred embodiment of the present invention, the marking of path-related elements surrounding each of the intraoperative paths of the bone surgery that affect the intraoperative surgical implementation includes:

[0023] Let any point on the path of the bone surgery be v(x, y, z), and the three-dimensional coordinate system be V:

[0024] V=F(x,y,z), x,y,z represent the corresponding coordinate variables respectively,

[0025] in:

[0026] x=y and its value range is [-a,+a] (a is a custom natural number), z≥0;

[0027] Traversing along the bone surgery intraoperative path through the three-dimensional coordinate system V, identifying the path priority element x and the path fault element p located in the space where the three-dimensional coordinate system V is located, and marking them;

[0028] After the traversal is completed, path-related elements that affect the implementation of the intraoperative surgery and surround the intraoperative path of the bone surgery are obtained;

[0029] According to the above steps, the path-related elements that affect the intraoperative surgical implementation on the remaining bone surgery intraoperative paths are identified and marked.

[0030] As a preferred embodiment of the present invention, the step of finding the optimal intraoperative path for performing the intraoperative surgery based on the path association element comprises:

[0031] Traversing the path-related elements marked on each of the bone surgery intraoperative paths, and reading the characteristic value of each of the path-related elements;

[0032] Respectively counting the number of the path priority element x and the number of the path failure element p in the path association element: Ix and Ip;

[0033] Select the bone surgery intraoperative pathway that meets the following eligibility criteria:

[0034] αIx+βIp≥k,

[0035] in:

[0036] α and β are weight coefficients,

[0037] α is the mean of the eigenvalues ​​of each of the path priority elements x in the path association element,

[0038] β is the mean of the characteristic values ​​of each of the path fault elements p in the path association element,

[0039] α is greater than β and α+β=1;

[0040] If there is only one intraoperative path for the bone surgery that satisfies: αIx+βIp≥k, then it is taken as the optimal intraoperative path;

[0041] If there are at least two intraoperative bone surgery paths that satisfy: αIx+βIp≥k, then selection is performed according to the following first search condition:

[0042] Ix-Ip≥(k / 4-k / 2)>0,

[0043] If there is a bone surgery intraoperative path that satisfies: Ix-Ip≥(k / 4-k / 2)>0, it is used as the intraoperative optimal path;

[0044] If there is no intraoperative path for the bone surgery that satisfies: Ix-Ip≥(k / 4-k / 2)>0, the selection is performed according to the following second search condition:

[0045] Ix-Ip>(k / 4-0),

[0046] If there is a bone surgery intraoperative path that satisfies: Ix-Ip>(k / 4-0), it is used as the intraoperative optimal path;

[0047] If it does not exist, the k value is gradually reduced according to the preset arithmetic progression: kd, k-2d, k-3d, etc., until the optimal path during the operation is found.

[0048] As a preferred embodiment of the present invention, the construction of a three-dimensional bone medical model of a bone region comprises:

[0049] Determine the area of ​​bone where the bone lesion is located;

[0050] Performing a three-dimensional X-ray scan on the bone region to obtain a corresponding three-dimensional bone X-ray image;

[0051] Performing MSCT scanning on the bone region to obtain MSCT three-dimensional images of the path-related elements around the bone;

[0052] Based on the three-dimensional X-ray image, three-dimensional reconstruction is performed to generate a three-dimensional bone medical model corresponding to the bone region, and the three-dimensional bone medical model is preprocessed;

[0053] Based on the MSCT three-dimensional image, three-dimensional reconstruction is performed, and the corresponding path-related elements are generated around the three-dimensional skeletal medical model simultaneously, and each of the path-related elements is rendered respectively according to element attributes;

[0054] The model file of the three-dimensional skeletal medical model is saved in a background database.

[0055] As a preferred embodiment of the present invention, the AR three-dimensional model including the optimal path channel during the operation is constructed based on AR integration technology, including:

[0056] Importing the model file of the three-dimensional bone medical model into the AR background;

[0057] On the AR background, based on AR integration technology, the three-dimensional skeletal medical model is subjected to structural visualization processing to generate an AR three-dimensional model of the three-dimensional skeletal medical model;

[0058] Adding an AR display path matching the intraoperative optimal path to the intraoperative optimal path channel in the AR three-dimensional model, wherein the AR display path is used to display the intraoperative optimal path channel to the surgeon according to the intraoperative optimal path on an AR device;

[0059] The AR 3D model is saved to the AR background.

[0060] As a preferred embodiment of the present invention, the method further comprises:

[0061] During the intraoperative surgery, surgical navigation technology is used to collect X-ray images of the surgical corridor in real time;

[0062] Sending the X-ray image to the AR background for image enhancement processing, generating a corresponding intraoperative tracking AR image in the intraoperative optimal path channel, and synchronizing it to the AR device;

[0063] According to the intraoperative tracking AR image, determine whether the real-time surgical channel deviates from the pre-built intraoperative optimal path channel:

[0064] If it deviates, an alarm signal will be issued;

[0065] Otherwise give up.

[0066] On the other hand, a device for intelligently assisting generation of an optimal intraoperative channel based on augmented reality technology is provided, and the device for intelligently assisting generation of an optimal intraoperative channel based on augmented reality technology is used to implement the above-mentioned method for intelligently assisting generation of an optimal intraoperative channel based on augmented reality technology, and the device includes:

[0067] A three-dimensional modeling module is used to construct a three-dimensional bone medical model of the bone area;

[0068] A position point marking module, used to mark a plurality of preset surgical pre-incision position points p1 and bone lesion position points p2 on the three-dimensional bone medical model;

[0069] A path construction module, used to construct a plurality of bone surgery intraoperative paths between the surgical pre-incision position point p1 and the bone lesion position point p2 on the three-dimensional bone medical model;

[0070] An optimal matching module, used for marking path-related elements surrounding each of the intraoperative paths of the bone surgery that affect the implementation of the intraoperative surgery, and finding the optimal intraoperative path for implementing the intraoperative surgery based on the path-related elements;

[0071] A channel stretching module, used to use the intraoperative optimal path as a rotation axis and to stretch according to a preset contour line to generate a corresponding intraoperative optimal path channel in the three-dimensional bone medical model;

[0072] The AR background module is used to build an AR three-dimensional model containing the optimal path channel during the operation based on AR integration technology, and send it to the AR device for display.

[0073] On the other hand, an electronic device is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned methods for intelligently assisting in generating optimal intraoperative channels based on augmented reality technology is implemented.

[0074] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned methods for intelligently assisting generation of optimal intraoperative channels based on augmented reality technology.

[0075] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0076] Based on the implementation of the present invention, by marking several preset surgical pre-incision positions p1 and bone lesion positions p2 on a three-dimensional bone medical model, and constructing several bone surgery intraoperative paths between the surgical pre-incision positions p1 and the bone lesion positions p2; marking the path-related elements that affect the implementation of the intraoperative surgery around each of the bone surgery intraoperative paths, and finding the intraoperative optimal path for implementing the intraoperative surgery based on the path-related elements; taking the intraoperative optimal path as the rotation axis and stretching according to the preset contour line, a corresponding intraoperative optimal path channel is generated in the three-dimensional bone medical model; based on AR integration technology, an AR three-dimensional model containing the intraoperative optimal path channel is constructed, and sent to an AR device for display. It is possible to generate an orthopedic surgery channel based on AR technology, assist doctors in selecting the optimal intraoperative channel to perform surgery, save time in the early stage of finding the optimal channel, and avoid delaying the progress of the surgery. The present invention can also simulate the intraoperative channel of orthopedic surgery in an augmented reality environment, which is convenient for guiding doctors to accurately perform surgery, avoid large intraoperative deviations, and cause surgical risks.

[0077] Through the construction of AR intraoperative channels, doctors can be provided with in-depth surgical channel guidance, which is convenient for providing surgical guidance to doctors based on the depth of channel implementation and avoiding larger surgical incisions.

[0078] The present invention can also synchronously display the intraoperative channel based on AR during implementation during surgery, and doctors and other administrators can monitor whether the intraoperative channel deviates from the preset AR optimal channel. If there is a deviation, an alarm will be triggered. This can monitor the accuracy of the operation and avoid deviations that may increase the impact on the patient's surgical incision. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0080] Figure 1 This is a schematic diagram of the surgical approach for intramedullary femoral nailing via the pyriform sinus;

[0081] Figure 2 This is a schematic diagram of another surgical approach for fixation of intercondylar humeral fractures;

[0082] Figure 3 This is a flow chart of a method for intelligently assisting in generating an optimal intraoperative channel based on augmented reality technology provided by an embodiment of the present invention;

[0083] Figure 4This is a block diagram of an intelligent auxiliary generation device for optimal intraoperative channel based on augmented reality technology provided by an embodiment of the present invention;

[0084] Figure 5 is an exemplary schematic diagram of a three-dimensional intraoperative path provided by an embodiment of the present invention;

[0085] Figure 6 is a three-dimensional space schematic diagram of a three-dimensional coordinate system provided by an embodiment of the present invention;

[0086] Figure 7 This is a schematic diagram of an example of three intraoperative paths a, b, and c provided in an embodiment of the present invention;

[0087] Figure 8 is a structural schematic diagram of a stretching channel provided by an embodiment of the present invention;

[0088] Fig. 9 is a schematic diagram of a three-dimensional skeleton example of a MSCT three-dimensional reconstruction technology provided by an embodiment of the present invention;

[0089] Fig.10 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0090] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0091] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0092] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0093] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0094] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0095] An embodiment of the present invention provides an intelligent auxiliary generation method for an optimal intraoperative channel based on augmented reality technology. The method can be implemented by an electronic device, which can be a terminal or a server.

[0096] like Figure 3 The flowchart of the method for intelligently assisting in generating the optimal intraoperative channel based on augmented reality technology is shown in the figure. The processing flow of the method may include the following steps:

[0097] S1. Construct a three-dimensional bone medical model of the bone area;

[0098] S2, marking a plurality of preset surgical pre-incision position points p1 and bone lesion position points p2 on the three-dimensional bone medical model;

[0099] S3, constructing a plurality of bone surgery intraoperative paths between the surgical pre-incision position point p1 and the bone lesion position point p2 on the three-dimensional bone medical model;

[0100] S4, marking the path-related elements surrounding each of the intraoperative paths of the bone surgery that affect the implementation of the intraoperative surgery, and finding the optimal intraoperative path for implementing the intraoperative surgery based on the path-related elements;

[0101] S5, taking the intraoperative optimal path as a rotation axis and stretching according to a preset contour line, to generate a corresponding intraoperative optimal path channel in the three-dimensional bone medical model;

[0102] S6. Based on AR integration technology, an AR three-dimensional model including the optimal path channel during the operation is constructed and sent to an AR device for display.

[0103] In orthopedic surgery, constructing the optimal intraoperative channel is a key step to ensure surgical success and reduce patient injury. The following are some methods to construct the optimal intraoperative channel:

[0104] 1. Detailed preoperative evaluation:

[0105] Through imaging examinations (such as X-ray, CT, MRI, etc.), we can understand the fracture site, displacement, surrounding tissue structure and possible pathological changes in detail.

[0106] Understand the patient's general condition, including age, nutritional status, comorbidities, etc., in order to assess surgical risks and develop a personalized surgical plan.

[0107] 2. Choose the appropriate surgical approach:

[0108] The most appropriate surgical approach is selected based on the fracture type, location, and surrounding anatomical structures.

[0109] Consider the advantages and disadvantages of the surgical approach, such as the range of exposure, damage to surrounding tissues, and postoperative recovery.

[0110] 3. Utilize navigation technology:

[0111] Use computer navigation technology to assist surgery and improve surgical accuracy and efficiency.

[0112] Through navigation technology, the position and angle of surgical instruments can be displayed in real time to ensure that operations are performed in the optimal channel.

[0113] 4. Real-time monitoring during surgery:

[0114] Use fluoroscopic equipment (such as C-arm) to monitor fracture reduction and fixation in real time during surgery.

[0115] According to the real-time monitoring results, the surgical operation is adjusted in time to ensure the best surgical effect.

[0116] 5. Protect surrounding tissues:

[0117] When constructing the optimal intraoperative channel, minimize damage to surrounding blood vessels, nerves and other tissues.

[0118] Use appropriate surgical instruments and techniques to reduce traction and compression on surrounding tissues.

[0119] 6. Postoperative evaluation and adjustment:

[0120] The patient's recovery was evaluated after surgery, including pain, functional recovery, etc.

[0121] According to the postoperative evaluation results, the rehabilitation plan is adjusted in time to ensure that the patient recovers as soon as possible.

[0122] Constructing the optimal intraoperative channel requires comprehensive consideration of the patient's specific conditions, fracture type, and surrounding anatomical structures. Through detailed preoperative evaluation, selection of appropriate surgical approaches, use of navigation technology, real-time intraoperative monitoring, and protection of surrounding tissues, the success of the operation and the patient's rapid recovery can be ensured. At the same time, postoperative evaluation and adjustment are also important links to ensure the effectiveness of the operation.

[0123] The present invention uses AR technology to construct an optimal intraoperative path for performing intraoperative surgery based on path association elements, and based on AR integration technology, constructs an AR three-dimensional model containing the intraoperative optimal path channel, and sends it to an AR device for display, showing the doctor the priority nodes for avoiding intraoperative obstacles and passing through tissue gaps, thereby guiding the doctor to perform surgery according to the three-dimensionally constructed intraoperative path and carry out the surgical route of the optimal path.

[0124] like Figure 4 The system functions and processes shown. The present invention generates an orthopedic surgical channel based on AR technology to assist doctors in selecting the optimal intraoperative channel to perform surgery, saving time in the early stage of finding the optimal channel and avoiding delays in the progress of the surgery. The present invention can also simulate the intraoperative channel of orthopedic surgery in an augmented reality environment, which is convenient for guiding doctors to perform surgery accurately and avoid large deviations during surgery, resulting in surgical risks. Through the construction of AR intraoperative channels, doctors can be provided with in-depth surgical channel guidance, which is convenient for providing doctors with surgical guidance from the depth of channel implementation and avoiding large surgical wounds.

[0125] Therefore, the present invention can synchronously display the intraoperative channel based on AR during the operation, and doctors and other administrators can monitor whether the intraoperative channel deviates from the preset AR optimal channel. If it deviates, an alarm will be triggered, which can monitor the accuracy of the operation and avoid deviations that may increase the impact on the patient's surgical wound.

[0126] The AR 3D model of the optimal intraoperative path channel allows doctors to learn the optimal intraoperative path channel route intuitively and in 3D through AR devices such as AR glasses, allowing doctors to understand and learn the next intraoperative surgical route more vividly. The specific 3D display of the intraoperative 3D path can be understood in combination with the existing AR 3D display effect. Figure 5 An example of a three-dimensional intraoperative path is shown. Transforaminal endoscopic lumbar discectomy (TELD) differs from microscopic and endoscopic translaminar discectomy in that it accesses the intervertebral disc space through Kambin's triangle.

[0127] Therefore, through AR three-dimensional virtual display of the optimal intraoperative channel, doctors can be provided with more vivid and intuitive intraoperative surgical path guidance, guiding doctors to learn the intraoperative surgical path in advance.

[0128] The implementation principle of the present invention will be further described below.

[0129] As a preferred embodiment of the present invention, the path association element includes a path priority element x and a path failure element p, wherein:

[0130] The path priority element x is used to characterize the location points that should be included in the intraoperative operation, including the natural gaps in the bone area and the location points where the anatomical structures are located;

[0131] The path fault element p is used to characterize the location points that should not be included in the intraoperative surgery, including the location points where the nerves, blood vessels and soft tissues in the bone area are located.

[0132] During the operation, maintain a clear field of vision and stable operation.

[0133] Use appropriate surgical instruments and implants to ensure stability and durability of the surgery.

[0134] Pay attention to protect the surrounding nerves, blood vessels and soft tissues to avoid unnecessary damage.

[0135] After constructing the three-dimensional skeletal medical model, it is necessary to carry out intraoperative path construction work for the three-dimensional skeletal medical model.

[0136] The surgical path needs to go from the incision to the lesion, and the surrounding nerves, blood vessels and soft tissues need to be avoided along the way. At the same time, if there are natural gaps and anatomical structures, priority will be given to passing through these natural gaps and anatomical structures to avoid causing greater damage to the patient's bones and other parts.

[0137] When constructing the system, doctors can choose several incision locations, so that several surgical pathways can be formed between the incisions and the lesions. Which pathway to choose requires calculating the optimal path.

[0138] Specifically, according to a number of surgical pre-incision position points p1 and bone lesion position points p2 constructed on the three-dimensional bone medical model, a number of bone surgery intraoperative paths between the incisions and the lesions are constructed, and the path nodes x and path fault points p around each bone surgery intraoperative path are marked;

[0139] Pathway nodes (pathway priority elements) x: natural gaps and anatomical structures; the intraoperative path preferably passes through these nodes;

[0140] Path failure points (path failure elements) p: nerves, blood vessels and soft tissues; the intraoperative path should preferentially avoid these nodes.

[0141] Select the bone surgery intraoperative path with the most path nodes x and the least path failure points p as the optimal intraoperative path;

[0142] Perform circular scanning with the optimal intraoperative path as the center line, and generate the corresponding optimal intraoperative path channel in the three-dimensional bone medical model;

[0143] Based on AR integration technology, an AR three-dimensional model of the optimal path channel is constructed and sent to the AR device for display.

[0144] Optimal path selection principle:

[0145] On the path from the incision to the bone lesion, check whether there is a nearest path node x:

[0146] If it exists, mark the path L1 between the cut and the nearest path node x1;

[0147] If not, check whether there is a path failure point p:

[0148] If it exists, bypass the path fault point p around the incision;

[0149] If it does not exist, the intraoperative path is set in combination with the path node x.

[0150] This will be described in detail below.

[0151] As a preferred embodiment of the present invention, the marking of path-related elements surrounding each of the intraoperative paths of the bone surgery that affect the intraoperative surgical implementation includes:

[0152] Let any point on the path of the bone surgery be v(x, y, z), and the three-dimensional coordinate system be V:

[0153] V=F(x,y,z), x,y,z represent the corresponding coordinate variables respectively,

[0154] in:

[0155] x=y and its value range is [-a,+a] (a is a custom natural number), z≥0;

[0156] Traversing along the bone surgery intraoperative path through the three-dimensional coordinate system V, identifying the path priority element x and the path fault element p located in the space where the three-dimensional coordinate system V is located, and marking them;

[0157] After the traversal is completed, path-related elements that affect the implementation of the intraoperative surgery and surround the intraoperative path of the bone surgery are obtained;

[0158] According to the above steps, the path-related elements that affect the intraoperative surgical implementation on the remaining bone surgery intraoperative paths are identified and marked.

[0159] x=y and its value range is [-a,+a] (a is a custom natural number), z≥0, which represents the upper semi-axis space of a three-dimensional coordinate system (such as Figure 6 As shown, it is converted to the upper semi-axis space where +y is located). A simple analogy is: place the bone part on a plane A, and construct a plane B parallel to A on the horizontal plane at the center of the part. The upper space of plane B is used as the three-dimensional space of the three-dimensional coordinate axis. The surgical path is set in this space to avoid the surgical path passing through the lower part, which will increase the difficulty of the operation and the wound area, and increase the incision.

[0160] If it is necessary to pass through the three-dimensional space below, adjust the position of the part, or rotate the three-dimensional coordinate system xyz.

[0161] The intraoperative paths of the bone surgeries mentioned earlier can be constructed by the user according to the position between the incision and the lesion. Figure 7 Schematic diagram of the three intraoperative pathways a, b, and c shown in the figure.

[0162] Since the corresponding three-dimensional bone medical model has been constructed based on three-dimensional scanning in the early stage, the coordinate positions of each spatial point on the three-dimensional bone medical model or the surrounding associated elements that affect the surgical path can also be marked in the three-dimensional space where the model is located. Therefore, the administrator can traverse the constructed intraoperative paths of various bone surgeries in the three-dimensional space (the three-dimensional space of the three-dimensional modeling software), mark the path nodes and fault nodes around the path, and mark the relevant path-associated elements around each path by manual marking. Different nodes are assigned different characteristic values, or marked by different marking methods, so as to facilitate the subsequent classification, identification and statistics of different nodes marked on each path.

[0163] According to the direction from the incision to the lesion, the path priority nodes and fault nodes around each path are marked in turn.

[0164] Marking within a certain range around the surgical path:

[0165] Take each path as the edge and set a reference plane (that is, the channel aperture that limits the stretching), the channel constructed by stretching (such as Figure 8 As shown in the figure, mark each node in the channel. Because in the construction of the three-dimensional model, a certain edge (the intraoperative path of bone surgery) can be stretched around the axis, and a rotation stretching channel is obtained by rotation stretching, so as to construct a stretching range, and mark each priority node and fault node in the channel that affects the path.

[0166] As a preferred embodiment of the present invention, the step of finding the optimal intraoperative path for performing the intraoperative surgery based on the path association element comprises:

[0167] Traversing the path-related elements marked on each of the bone surgery intraoperative paths, and reading the characteristic value of each of the path-related elements;

[0168] Respectively counting the number of the path priority element x and the number of the path failure element p in the path association element: Ix and Ip;

[0169] Select the bone surgery intraoperative pathway that meets the following eligibility criteria:

[0170] αIx+βIp≥k,

[0171] in:

[0172] α and β are weight coefficients,

[0173] α is the mean of the eigenvalues ​​of each of the path priority elements x in the path association element,

[0174] β is the mean of the characteristic values ​​of each of the path fault elements p in the path association element,

[0175] α is greater than β and α+β=1;

[0176] If there is only one intraoperative path for the bone surgery that satisfies: αIx+βIp≥k, then it is taken as the optimal intraoperative path;

[0177] If there are at least two intraoperative bone surgery paths that satisfy: αIx+βIp≥k, then selection is performed according to the following first search condition:

[0178] Ix-Ip≥(k / 4-k / 2)>0,

[0179] If there is a bone surgery intraoperative path that satisfies: Ix-Ip≥(k / 4-k / 2)>0, it is used as the intraoperative optimal path;

[0180] If there is no intraoperative path for the bone surgery that satisfies: Ix-Ip≥(k / 4-k / 2)>0, the selection is performed according to the following second search condition:

[0181] Ix-Ip>(k / 4-0),

[0182] If there is a bone surgery intraoperative path that satisfies: Ix-Ip>(k / 4-0), it is used as the intraoperative optimal path;

[0183] If it does not exist, the k value is gradually reduced according to the preset arithmetic progression: kd, k-2d, k-3d, etc., until the optimal path during the operation is found.

[0184] Each path may have path priority nodes and path failure nodes around it that affect the implementation of the surgery. Ideally, the optimal digital path has as many path priority nodes as possible and as few path failure nodes as possible.

[0185] Therefore, in order to select the optimal intraoperative path from various surgical paths, this scheme determines the optimal path by judging the number of priority points and failure points around each surgical path.

[0186] This solution sets a method for calculating the number of influencing elements of a surgical path: αIx+βIp≥k (k is the threshold set by the system). After counting the number of priority nodes and fault nodes around each path, the qualified path is found according to the above judgment method, where the weight coefficient is preferentially configured as α greater than β. The specific weight value can be debugged and finally confirmed by the user.

[0187] In the calculation, it is assumed that at least one path meets the above requirements. If there is only one path, the qualified path is taken as the optimal path in the number; if there are two or more paths, the path with the largest number of path priority elements is taken as the optimal path in the number. If there are two or more paths with the same number of path priority elements, the path with the least path failure elements is selected as the surgical path. In extreme cases, if the path priority elements and the roadblock path failure elements are the same (this situation is rare), but if it occurs, the user can choose one by himself to determine the optimal path during the operation according to surgical habits.

[0188] Therefore, the above scheme is adopted. This scheme can comprehensively consider the influencing nodes around the surgical path to select the best surgical path. The above specific formula is used to select the optimal path during the operation. The background server automatically completes the calculation and recommends the calculated optimal path during the operation to the doctor's front-end terminal for the doctor to choose.

[0189] Here, because the number of path priority elements around each path may exceed 30-50, or even more, in order to accurately find the best intraoperative path for bone surgery as the "intraoperative optimal path", the present invention adopts the following method for retrieval.

[0190] First, select the bone surgery intraoperative path that meets the following eligibility conditions:

[0191] αIx+βIp≥k,

[0192] in:

[0193] α and β are weight coefficients,

[0194] α is the mean of the eigenvalues ​​of each of the path priority elements x in the path association element,

[0195] β is the mean of the characteristic values ​​of each of the path fault elements p in the path association element,

[0196] α is greater than β and α+β=1;

[0197] Secondly, because there may be multiple paths, it is necessary to select a path with more path priority elements x. Therefore, if there is only one intraoperative path of the bone surgery that satisfies: αIx+βIp≥k, it is taken as the intraoperative optimal path;

[0198] If there are at least two intraoperative bone surgery paths that satisfy: αIx+βIp≥k, then selection is performed according to the following first search condition:

[0199] Ix-Ip≥(k / 4-k / 2)>0,

[0200] If there is a bone surgery intraoperative path that satisfies: Ix-Ip≥(k / 4-k / 2)>0, it is used as the intraoperative optimal path;

[0201] When there are multiple qualified paths, each qualified path has more path priority elements x. At this time, the path is selected according to the difference value here. For example, the number of path priority elements x is k / 4-k / 2 more than the path failure element p (if k is 100, the threshold is 25-50), that is, the path with 25-50 more path priority elements x than path failure elements p is used as the optimal path in the operation.

[0202] The same below:

[0203] If there is no intraoperative path for the bone surgery that satisfies: Ix-Ip≥(k / 4-k / 2)>0, the selection is performed according to the following second search condition:

[0204] Ix-Ip>(k / 4-0),

[0205] If there is a bone surgery intraoperative path that satisfies: Ix-Ip>(k / 4-0), it is used as the intraoperative optimal path;

[0206] If it does not exist, the k value is gradually reduced according to the preset arithmetic progression: kd, k-2d, k-3d, etc., until the optimal path during the operation is found.

[0207] If it is finally found that the threshold range is insufficient or the k value is too large, the k value will be reduced to reduce the threshold range, making it easier to explore the best qualified path from large to small.

[0208] When adjusting the threshold k, it is necessary to adjust and test it according to the regularity. Here, the threshold k is adjusted according to the arithmetic progression. For example, if k is 100 and the arithmetic progression d is 5, the first threshold k is 100, the next threshold k is 95, the next is 90, and so on, until the optimal path in the operation is obtained.

[0209] After the doctor has selected the optimal intraoperative path, he can also learn the path nodes x and path failure points p around the optimal intraoperative path (which will also be displayed in the AR model channel);

[0210] Pathway nodes (pathway priority elements) x: natural gaps and anatomical structures; the intraoperative path preferably passes through these nodes;

[0211] Path failure points (path failure elements) p: nerves, blood vessels and soft tissues; the intraoperative path should preferentially avoid these nodes.

[0212] It is convenient to implement a better surgical route during surgery and reduce the impact of surgical trauma on patients.

[0213] As a preferred embodiment of the present invention, the construction of a three-dimensional bone medical model of a bone region comprises:

[0214] Determine the area of ​​bone where the bone lesion is located;

[0215] Performing a three-dimensional X-ray scan on the bone region to obtain a corresponding three-dimensional bone X-ray image;

[0216] Performing MSCT scanning on the bone region to obtain MSCT three-dimensional images of the path-related elements around the bone;

[0217] Based on the three-dimensional X-ray image, three-dimensional reconstruction is performed to generate a three-dimensional bone medical model corresponding to the bone region, and the three-dimensional bone medical model is preprocessed;

[0218] Based on the MSCT three-dimensional image, three-dimensional reconstruction is performed, and the corresponding path-related elements are generated around the three-dimensional skeletal medical model simultaneously, and each of the path-related elements is rendered respectively according to element attributes;

[0219] The model file of the three-dimensional skeletal medical model is saved in a background database.

[0220] Scanning bones using MSCT (multi-slice spiral CT) and obtaining a 3D image of the bones is a precise process involving multiple steps. Here is an overview of the steps based on information from reference articles:

[0221] 1.MSCT scanning principle:

[0222] MSCT uses multiple rows of detectors to acquire and record images from multiple layers at the same time.

[0223] Compared with traditional CT scans, MSCT can greatly shorten scanning time, improve image quality, and reduce the patient's exposure to radiation.

[0224] 2. Scan preparation:

[0225] Before having an MSCT scan, you may need to prepare yourself by changing into hospital clothing and removing metal jewelry and other items that may interfere with the scan.

[0226] Depending on the area being scanned, the patient may need to assume a specific position, such as supine or prone.

[0227] 3. Scanning process:

[0228] The patient enters the scanning room and lies on the scanning table.

[0229] The MSCT equipment starts working, the X-ray tube detector rotates around the patient, and the examination bed moves at a constant speed.

[0230] The detector receives the X-rays that pass through the human body and converts them into electrical signals, which are then reconstructed and processed by the data acquisition system (DAS) to generate a two-dimensional image.

[0231] 4. Image post-processing:

[0232] Using specialized software, doctors can post-process two-dimensional images to generate three-dimensional images of the bones.

[0233] Three-dimensional images can display the structure and morphology of bones more intuitively, helping doctors diagnose diseases more accurately.

[0234] 5. Notes:

[0235] During the MSCT scan, the patient needs to remain still and avoid moving to ensure image clarity.

[0236] Since MSCT uses X-rays, patients may be exposed to a certain amount of radiation. However, the radiation dose of modern MSCT equipment has been greatly reduced, and doctors will adjust the scanning parameters according to the patient's specific situation to minimize radiation exposure.

[0237] like Fig. 9 Shown is an example of a three-dimensional bone using MSCT three-dimensional reconstruction technology.

[0238] However, the present invention uses AR to achieve three-dimensional reconstruction and virtual enhanced display of the three-dimensional skeletal medical model.

[0239] As a preferred embodiment of the present invention, the AR three-dimensional model including the optimal path channel during the operation is constructed based on AR integration technology, including:

[0240] Importing the model file of the three-dimensional bone medical model into the AR background;

[0241] On the AR background, based on AR integration technology, the three-dimensional skeletal medical model is subjected to structural visualization processing to generate an AR three-dimensional model of the three-dimensional skeletal medical model;

[0242] Adding an AR display path matching the intraoperative optimal path to the intraoperative optimal path channel in the AR three-dimensional model, wherein the AR display path is used to display the intraoperative optimal path channel to the surgeon according to the intraoperative optimal path on an AR device;

[0243] The AR 3D model is saved to the AR background.

[0244] AR (augmented reality) integration technology plays a vital role in the structural visualization of 3D bone models, allowing users to intuitively see and operate 3D bone models in a real environment. The following are the general steps to generate an AR 3D model of bones:

[0245] 1. Data acquisition: First, detailed data of the bones needs to be obtained. This can usually be achieved through medical scans (such as CT or MRI scans) or 3D scanning equipment. For medical applications, this data may come from scanned images of patients.

[0246] 2. 3D reconstruction: Using the acquired data, professional 3D modeling software can be used to perform 3D reconstruction of the bones. This process includes data processing (such as denoising, smoothing, etc.) and converting the data into a 3D model through modeling software.

[0247] 3. Textures and materials: In order to make the 3D model more realistic, you can add textures and materials to the bones. This can include properties such as the color, glossiness, and reflection of the bones.

[0248] 4. AR Integration: Next, the 3D model needs to be integrated into the AR environment. This usually involves using AR development tools (such as ARKit, ARCore, etc.) to create AR applications or experiences. In the AR environment, the skeletal model can be placed anywhere the user chooses and manipulated using a variety of interactive methods.

[0249] 5. Interaction design: In order to make the AR experience more interesting and practical, good interaction methods need to be designed. For example, users may be able to rotate, scale, or move a skeleton model through gestures or voice commands. In addition, other functions can be added, such as highlighting specific bones or displaying detailed information about bones.

[0250] Through the above steps, AR integration technology is used to generate a realistic bone AR three-dimensional model to provide doctors with visual guidance for clinical bone surgery.

[0251] AR backend, i.e., the backend application service for implementing AR technology, can be determined according to the selected AR device model, and its backend management software can be installed and deployed on the backend server. Specifically, the administrator completes the corresponding AR integration and sharing on the backend server.

[0252] For the interactive link between the AR background and AR devices such as AR glasses, please refer to the working principle of existing AR devices.

[0253] As a preferred embodiment of the present invention, the method further comprises:

[0254] During the intraoperative surgery, surgical navigation technology is used to collect X-ray images of the surgical corridor in real time;

[0255] Sending the X-ray image to the AR background for image enhancement processing, generating a corresponding intraoperative tracking AR image in the intraoperative optimal path channel, and synchronizing it to the AR device;

[0256] According to the intraoperative tracking AR image, determine whether the real-time surgical channel deviates from the pre-built intraoperative optimal path channel:

[0257] If it deviates, an alarm signal will be issued;

[0258] Otherwise give up.

[0259] The use of X-ray images for intraoperative navigation is an important application of the combination of medical imaging and surgical technology, especially showing significant advantages in improving surgical accuracy and safety. The following is an introduction to the main aspects of using X-ray images for intraoperative navigation:

[0260] 1. Basic principles

[0261] Intraoperative navigation technology combines the imaging results of medical imaging with actual surgery, and uses computer software to match the patient's X-ray image data with surgical instruments to achieve precise positioning and operation. Doctors can view the patient's anatomical structure in real time during surgery and guide surgical operations based on X-ray image data, thereby avoiding accidental injury to normal tissues and improving the safety and success rate of surgery.

[0262] 2. Application fields

[0263] Intraoperative navigation using X-ray images has been widely used in various surgical procedures, including but not limited to the following areas:

[0264] 1. Neurosurgery: During brain surgery, X-ray image navigation can help doctors accurately locate brain tumors, vascular abnormalities and other lesions, and precisely remove them without damaging surrounding important tissues.

[0265] 2. Orthopedics: In orthopedic surgery, X-ray image navigation can help doctors accurately locate the fracture site, guide orthopedic surgical operations, and improve surgical efficacy.

[0266] 3. Cardiac Surgery: During cardiac surgery, X-ray image navigation can assist doctors in viewing the internal structure of the heart, guide interventional surgery, and improve the success rate of surgery.

[0267] 3. Technical advantages

[0268] Compared with traditional surgery, using X-ray images for intraoperative navigation has the following advantages:

[0269] 1. High precision: Through X-ray image navigation, doctors can view the patient's anatomical structure in real time, ensure the precise positioning and operation of surgical instruments, and reduce errors.

[0270] 2. High safety: Intraoperative navigation technology can reduce damage to normal tissues, reduce surgical risks, and ensure patient safety.

[0271] 3. High efficiency: X-ray image navigation technology can shorten the operation time, improve the efficiency of the operation, and enable patients to recover faster.

[0272] 4. Technical Implementation

[0273] In practical applications, the technical implementation of intraoperative navigation using X-ray images mainly includes the following aspects:

[0274] 1. Image processing: Preprocess and enhance X-ray images to improve image quality and clarity, making it easier for doctors to observe and judge.

[0275] 2. Matching and registration: Match and register the patient’s X-ray image data with the surgical instruments to ensure accurate positioning and operation during the operation.

[0276] 3. Real-time tracking: Through optical positioning tracking technology or other sensor technology, the position and posture of surgical instruments are monitored in real time to ensure real-time navigation during the operation.

[0277] In this embodiment, X-ray images can be used for intraoperative navigation. The images of the doctor's surgical instruments and the parts to be operated on are processed, and combined with AR integration technology, the real-time intraoperative surgical status is synchronously displayed in the AR three-dimensional model of the intraoperative optimal path channel (the doctor wears AR glasses to perform the operation), so as to determine whether the doctor's surgical route is in the preset intraoperative channel. If it is found that it deviates from the preset optimal intraoperative channel, an alarm can be issued. The specific judgment of whether there is deviation can be monitored by the background nurse.

[0278] On the other hand, a device for intelligently assisting generation of an optimal intraoperative channel based on augmented reality technology is provided, and the device for intelligently assisting generation of an optimal intraoperative channel based on augmented reality technology is used to implement the above-mentioned method for intelligently assisting generation of an optimal intraoperative channel based on augmented reality technology, and the device includes:

[0279] A three-dimensional modeling module is used to construct a three-dimensional bone medical model of the bone area;

[0280] A position point marking module, used to mark a plurality of preset surgical pre-incision position points p1 and bone lesion position points p2 on the three-dimensional bone medical model;

[0281] A path construction module, used to construct a plurality of bone surgery intraoperative paths between the surgical pre-incision position point p1 and the bone lesion position point p2 on the three-dimensional bone medical model;

[0282] An optimal matching module, used for marking path-related elements surrounding each of the intraoperative paths of the bone surgery that affect the implementation of the intraoperative surgery, and finding the optimal intraoperative path for implementing the intraoperative surgery based on the path-related elements;

[0283] A channel stretching module, used to use the intraoperative optimal path as a rotation axis and to stretch according to a preset contour line to generate a corresponding intraoperative optimal path channel in the three-dimensional bone medical model;

[0284] The AR background module is used to build an AR three-dimensional model containing the optimal path channel during the operation based on AR integration technology, and send it to the AR device for display.

[0285] Please understand the various functions of the above-mentioned device and the interaction between the functional modules in conjunction with the specific implementation principles in the method.

[0286] Fig.10 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Fig.10 As shown, the electronic device may include the above Figure 4 The device for intelligently assisting in generating the optimal intraoperative channel based on augmented reality technology is shown. Optionally, the electronic device 410 may include a first processor 2001 .

[0287] Optionally, the electronic device 410 may further include a memory 2002 and a transceiver 2003 .

[0288] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0289] Combine the following Fig.10 The components of the electronic device 410 are described in detail:

[0290] The first processor 2001 is the control center of the electronic device 410, and may be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be application specific integrated circuits (ASICs), or may be one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0291] Optionally, the first processor 2001 can execute various functions of the electronic device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0292] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Fig.10 CPU0 and CPU1 are shown in FIG.

[0293] In a specific implementation, as an embodiment, the electronic device 410 may also include multiple processors, such as Fig.10 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0294] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0295] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently and access the first processor 2001 through the interface circuit ( Fig.10 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0296] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0297] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Fig.10 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0298] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Fig.10 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0299] It should be noted that Fig.10 The structure of the electronic device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or fewer components than those shown in the figure, or combine certain components, or arrange the components differently.

[0300] In addition, the technical effects of the electronic device 410 can refer to the technical effects of the intelligent auxiliary generation method of the optimal intraoperative channel based on augmented reality technology described in the above method embodiment, and will not be repeated here.

[0301] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0302] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0303] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0304] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0305] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0306] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0307] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0308] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0309] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0310] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0311] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0312] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0313] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for intelligently assisting the generation of optimal intraoperative channels based on augmented reality technology, characterized in that: The method comprises: Construct three-dimensional skeletal medical models of skeletal areas; On the three-dimensional bone medical model, a plurality of preset surgical pre-incision position points p1 and bone lesion position points p2 are marked; Constructing a plurality of bone surgery intraoperative paths between the surgical pre-incision position point p1 and the bone lesion position point p2 on the three-dimensional bone medical model; Marking the path-related elements surrounding each of the intraoperative paths of the bone surgery that affect the implementation of the intraoperative surgery, and finding the optimal intraoperative path for implementing the intraoperative surgery based on the path-related elements; Taking the intraoperative optimal path as a rotation axis and stretching according to a preset contour line, a corresponding intraoperative optimal path channel is generated in the three-dimensional bone medical model; Based on AR integration technology, an AR three-dimensional model containing the optimal path channel during the operation is constructed and sent to the AR device for display.

2. The method for intelligently assisting generation of optimal intraoperative channels based on augmented reality technology according to claim 1, characterized in that: The path association element includes a path priority element x and a path failure element p, wherein: The path priority element x is used to characterize the location points that should be included in the intraoperative operation, including the natural gaps in the bone area and the location points where the anatomical structures are located; The path fault element p is used to characterize the location points that should not be included in the intraoperative surgery, including the location points where the nerves, blood vessels and soft tissues in the bone area are located.

3. The method for intelligently assisting generation of optimal intraoperative channels based on augmented reality technology according to claim 1, characterized in that: The path-related elements that are marked around each of the intraoperative paths of the bone surgery and that affect the implementation of the intraoperative surgery include: Let any point on the path of the bone surgery be v (x, y, z), and its three-dimensional coordinate system be V: V=F(x, y, z), x, y, z represent the corresponding coordinate variables respectively, in: x=y and its value range is [-a,+a] (a is a custom natural number), z≥0; Traversing along the bone surgery intraoperative path through the three-dimensional coordinate system V, identifying the path priority element x and the path fault element p located in the space where the three-dimensional coordinate system V is located, and marking them; After the traversal is completed, path-related elements that affect the implementation of the intraoperative surgery and surround the intraoperative path of the bone surgery are obtained; According to the above steps, the path-related elements that affect the intraoperative surgical implementation on the remaining bone surgery intraoperative paths are identified and marked.

4. The method for intelligently assisting generation of optimal intraoperative channels based on augmented reality technology according to claim 3 is characterized in that: The step of finding the optimal intraoperative path for performing the intraoperative surgery based on the path association element comprises: Traversing the path-related elements marked on each of the bone surgery intraoperative paths, and reading the characteristic value of each of the path-related elements; Respectively counting the number of the path priority element x and the number of the path failure element p in the path association element: Ix and Ip; Select the bone surgery intraoperative pathway that meets the following eligibility criteria: αIx+βIp≥k, in: α and β are weight coefficients, α is the mean of the eigenvalues ​​of each of the path priority elements x in the path association element, β is the mean of the characteristic values ​​of each of the path fault elements p in the path association element, α is greater than β and α+β=1; If there is only one intraoperative path for the bone surgery that satisfies: αIx+βIp≥k, then it is taken as the optimal intraoperative path; If there are at least two intraoperative bone surgery paths that satisfy: αIx+βIp≥k, then selection is performed according to the following first search condition: Ix-Ip≥(k / 4-k / 2)>0, If there is a bone surgery intraoperative path that satisfies: Ix-Ip≥(k / 4-k / 2)>0, it is used as the intraoperative optimal path; If there is no intraoperative path for the bone surgery that satisfies: Ix-Ip≥(k / 4-k / 2)>0, the selection is performed according to the following second search condition: Ix-Ip>(k / 4-0), If there is a bone surgery intraoperative path that satisfies: Ix-Ip>(k / 4-0), it is used as the intraoperative optimal path; If it does not exist, the k value is gradually reduced according to the preset arithmetic progression: kd, k-2d, k-3d, etc., until the optimal path during the operation is found.

5. The method for intelligently assisting generation of optimal intraoperative channels based on augmented reality technology according to claim 1, characterized in that: The constructing of a three-dimensional bone medical model of the bone region comprises: Determine the area of ​​bone where the bone lesion is located; Performing a three-dimensional X-ray scan on the bone region to obtain a corresponding three-dimensional bone X-ray image; Performing MSCT scanning on the bone region to obtain MSCT three-dimensional images of the path-related elements around the bone; Based on the three-dimensional X-ray image, three-dimensional reconstruction is performed to generate a three-dimensional bone medical model corresponding to the bone region, and the three-dimensional bone medical model is preprocessed; Based on the MSCT three-dimensional image, three-dimensional reconstruction is performed, and the corresponding path-related elements are generated around the three-dimensional skeletal medical model simultaneously, and each of the path-related elements is rendered respectively according to element attributes; The model file of the three-dimensional skeletal medical model is saved in a background database.

6. The method for intelligently assisting generation of optimal intraoperative channels based on augmented reality technology according to claim 1, characterized in that: The AR three-dimensional model including the optimal path channel during the operation is constructed based on the AR integration technology, including: Importing the model file of the three-dimensional bone medical model into the AR background; On the AR background, based on AR integration technology, the three-dimensional skeletal medical model is subjected to structural visualization processing to generate an AR three-dimensional model of the three-dimensional skeletal medical model; Adding an AR display path matching the intraoperative optimal path to the intraoperative optimal path channel in the AR three-dimensional model, wherein the AR display path is used to display the intraoperative optimal path channel to the surgeon according to the intraoperative optimal path on an AR device; The AR 3D model is saved to the AR background.

7. The method for intelligently assisting generation of optimal intraoperative channels based on augmented reality technology according to claim 6, characterized in that: The method further comprises: During the intraoperative surgery, surgical navigation technology is used to collect X-ray images of the surgical corridor in real time; Sending the X-ray image to the AR background for image enhancement processing, generating a corresponding intraoperative tracking AR image in the intraoperative optimal path channel, and synchronizing it to the AR device; According to the intraoperative tracking AR image, determine whether the real-time surgical channel deviates from the pre-built intraoperative optimal path channel: If it deviates, an alarm signal will be issued; Otherwise give up.

8. An intelligent auxiliary generation device for the optimal intraoperative channel based on augmented reality technology, wherein the intelligent auxiliary generation device for the optimal intraoperative channel based on augmented reality technology is used to implement the intelligent auxiliary generation method for the optimal intraoperative channel based on augmented reality technology as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A three-dimensional modeling module is used to construct a three-dimensional bone medical model of the bone area; A position point marking module, used to mark a plurality of preset surgical pre-incision position points p1 and bone lesion position points p2 on the three-dimensional bone medical model; A path construction module, used to construct a plurality of bone surgery intraoperative paths between the surgical pre-incision position point p1 and the bone lesion position point p2 on the three-dimensional bone medical model; An optimal matching module, used for marking path-related elements surrounding each of the intraoperative paths of the bone surgery that affect the implementation of the intraoperative surgery, and finding the optimal intraoperative path for implementing the intraoperative surgery based on the path-related elements; A channel stretching module, used to use the intraoperative optimal path as a rotation axis and to stretch according to a preset contour line to generate a corresponding intraoperative optimal path channel in the three-dimensional bone medical model; The AR background module is used to build an AR three-dimensional model containing the optimal path channel during the operation based on AR integration technology, and send it to the AR device for display.

9. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images