Fracture block space positioning method based on augmented reality technology fracture two-dimensional image

Abstract

The invention discloses a fracture block spatial positioning method based on an augmented reality technology fracture two-dimensional image, and the method comprises the steps: carrying out the three-dimensional scanning along the body surface position of a fracture part, obtaining a corresponding fracture medical X image data set K, carrying out the distributed storage, facilitating the centralized calling of image pictures, carrying out the distributed processing of the image pictures, and achieving the positioning of a fracture block. And the operation pressure of the server is reduced. On the basis of an FDK algorithm and an MIP fracture reconstruction technology, corresponding fracture models are constructed and fused, the fused fracture models are integrated by using an AR augmented reality technology, a three-dimensional fracture AR model for displaying a fracture part through AR positioning is generated, a doctor can intuitively observe a three-dimensional structure of a fracture on AR equipment, and the three-dimensional structure of the fracture is displayed on the AR equipment. Therefore, deep fracture parameters are obtained, understanding of fracture is deepened, doctors are helped to locate and diagnose parameters such as fracture positions, experience and professional requirements on the doctors can be reduced, the method is suitable for clinical use of the doctors, and clinical treatment time of fracture patients is shortened.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical technology, and in particular to a method for spatially locating a fracture fragment based on a two-dimensional fracture image using augmented reality technology, a device for spatially locating a fracture fragment based on a two-dimensional fracture image using augmented reality technology, and equipment for spatially locating a fracture fragment. Background Art

[0002] Fracture localization is an important step in fracture treatment. Its main purpose is to determine the specific location and type of fracture in order to choose appropriate treatment methods.

[0003] Fracture location usually relies on medical imaging tests, such as X-rays, CT scans or MRIs, etc. These tests can provide detailed images of the fracture site, helping doctors determine the exact location of the fracture, the direction of the fracture line, the size and displacement of the fracture fragments, etc.

[0004] Once the fracture location is determined, the doctor can develop a targeted treatment plan. This may include conservative treatment measures such as manual reduction, fixation, and traction, or more complex treatments such as surgical reduction and internal fixation.

[0005] Therefore, the accuracy of fracture positioning is crucial to the treatment effect. If the positioning is inaccurate, it may lead to improper treatment, affecting fracture healing and patient recovery. Therefore, after a fracture occurs, you should seek medical attention as soon as possible and receive professional medical imaging examinations to ensure the accuracy of fracture positioning and the effectiveness of treatment.

[0006] In the existing clinical fracture localization, the traditional method is mainly to take X-ray images (X-ray images) of the spatial position of the bone, and the doctor determines the fracture location based on experience. However, the traditional fracture localization method has the following technical defects:

[0007] First, doctors need to have rich clinical experience and be able to quickly find the fracture site and its crease from X-rays, but not all orthopedic surgeons have rich clinical experience, especially doctors in the early stages. They lack such experience and need to spend a long time to make a determination or consult older doctors or look up materials, etc., which will undoubtedly delay the clinical treatment time for fracture patients.

[0008] Second, as attached Figure 1 As shown, in the two-dimensional images displayed by X-ray images, only the front fracture creases and locations can be displayed. The deep creases, depths, fracture surfaces, etc. cannot be displayed to doctors, especially for some clinical symptoms of adhesion after fractures. It is also impossible to provide accurate three-dimensional fracture data, and it is impossible to better assist doctors in fully understanding the overall situation of the fracture site.

[0009] Third, there are many X images, which makes it inconvenient to call and process each X image later. Centralized storage and processing will increase the computing pressure of the server. Summary of the invention

[0010] In order to solve the technical problems existing in the prior art, the present invention provides the following technical solutions:

[0011] On the one hand, a method for spatial positioning of fracture fragments based on a two-dimensional fracture image using augmented reality technology is provided. The method is implemented by a device for spatial positioning of fracture fragments. The method comprises:

[0012] The method comprises:

[0013] Perform a three-dimensional scan along the body surface where the fracture site is located, obtain the corresponding fracture medical X-ray image dataset K and perform distributed storage;

[0014] Parse the fracture medical X-ray image dataset to obtain the fracture medical film k in each scanning sequence:

[0015] K = ∏ n i=1 SCAN(k i ) I ,

[0016] I represents the projection intensity of X-rays,

[0017] SCAN represents the scanning function.

[0018] SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting the i-th scanning sequence under the projection intensity I i ,

[0019] K represents the fracture medical film k under several scanning sequences i The product is obtained;

[0020] Based on the FDK algorithm, the back-projected images of each of the fracture medical films k are obtained, and a fracture medical three-dimensional image dataset K' composed of each of the back-projected images is obtained, and the fracture site is three-dimensionally reconstructed based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model;

[0021] Based on the above three-dimensional scanning path, the MIP technology is synchronously used to obtain the MIP two-dimensional projection images under each scanning sequence, and the MIP fracture reconstruction image data set K'' is obtained from each of the MIP two-dimensional projection images, and the fracture site is three-dimensionally reconstructed based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model;

[0022] fusing the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrating the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model;

[0023] The three-dimensional fracture AR model is superimposed on a preset AR device, and three-dimensionally displayed in the AR device to locate and display the three-dimensional model structure of the fracture site.

[0024] As a preferred embodiment of the present invention, the three-dimensional scanning is performed along the body surface position where the fracture site is located, and the corresponding fracture medical X-ray image data set K is obtained and distributedly stored, including:

[0025] Presetting the three-dimensional scanning path of the fracture site and its corresponding scanning parameters;

[0026] Start X-ray scanning, perform three-dimensional scanning along the body surface where the fracture site is located, sequentially obtain fracture medical films k under different scanning sequences, and bind each fracture medical film k to the corresponding scanning sequence ID;

[0027] Performing denoising and contrast enhancement processing on the fracture medical film k;

[0028] After processing, traverse and identify the idle data nodes in the Hadoop distributed file system - datanode (x);

[0029] Each of the fracture medical films k is stored in the corresponding data node - datanode (x) in sequence according to the scanning sequence, and the scanning sequence ID is written into the name node of the Hadoop distributed file system;

[0030] Through the name node name node, the fracture medical film k in each data node -data node (x) is traversed and extracted according to the scan sequence ID to obtain the fracture medical X-image dataset K.

[0031] As a preferred embodiment of the present invention, the back-projection images of each of the fracture medical films k are obtained based on the FDK algorithm, and a fracture medical three-dimensional image dataset K' composed of each of the back-projection images is obtained, and a fracture site is three-dimensionally reconstructed based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model, including:

[0032] Through the name node, read the corresponding fracture medical films k in order according to the scan sequence ID;

[0033] Based on the FDK algorithm, the fracture medical film k is back-projected according to its corresponding scanning sequence to generate the corresponding back-projected image;

[0034] Performing three-dimensional voxel assignment on the back-projected images of all the fracture medical films k respectively, to obtain the fracture medical three-dimensional image data set K' composed of the back-projected images of all the fracture medical films k;

[0035] Importing the fracture medical three-dimensional image data set K' into a preset three-dimensional coordinate system, performing three-dimensional reconstruction of the fracture site, and generating the first fracture three-dimensional model;

[0036] After post-processing the first three-dimensional fracture model, the model file of the first three-dimensional fracture model is stored in a background database.

[0037] As a preferred embodiment of the present invention, based on the above three-dimensional scanning path, the MIP technology is synchronously used to obtain the MIP two-dimensional projection images under each scanning sequence, and the MIP fracture reconstruction image data set K'' of each of the MIP two-dimensional projection images is obtained, including:

[0038] Starting the MIP scan according to the pre-set three-dimensional scanning path and its corresponding scanning parameters;

[0039] Perform perspective scanning along the body surface where the fracture site is located, sequentially obtain MIP two-dimensional projection images under different scanning sequences, and bind each of the MIP two-dimensional projection images to the corresponding scanning sequence ID;

[0040] Performing denoising and contrast enhancement processing on the MIP two-dimensional projection image;

[0041] After processing, traverse and identify the idle data nodes in the Hadoop distributed file system - datanode (x);

[0042] According to the scanning sequence ID, each of the MIP two-dimensional projection images is stored in the corresponding data node -data node (x), and coexists with the fracture medical film k bound to the scanning sequence ID;

[0043] Through the name node name node, the MIP two-dimensional projection image in each data node -data node (x) is traversed and extracted according to the scan sequence ID to obtain the MIP fracture reconstruction image data set K''.

[0044] As a preferred embodiment of the present invention, the three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model includes:

[0045] Obtaining the fracture three-dimensional MIP data to be reconstructed: through the name node, sequentially reading the corresponding MIP two-dimensional projection images according to the scanning sequence ID, to obtain the MIP fracture reconstruction image data set K'';

[0046] Sequentially traverse and calculate the MIP values ​​of each of the MIP two-dimensional projection images in the MIP fracture reconstruction image data set K'', and generate corresponding MIP fracture reconstruction images through projection;

[0047] According to the above steps, the MIP two-dimensional projection images under all scanning sequences are projected in a preset three-dimensional space to generate the corresponding second fracture three-dimensional model;

[0048] After post-processing the second fracture three-dimensional model, the model file of the second fracture three-dimensional model is stored in a background database.

[0049] As a preferred embodiment of the present invention, the fusing of the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrating the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model, comprises:

[0050] Reading the model file, projecting the first fracture three-dimensional model and the second fracture three-dimensional model into the same three-dimensional space and performing rendering processing;

[0051] Taking a scanning vector of a scanning sequence in the three-dimensional scanning path as a reference, and positioning the first three-dimensional fracture model and the second three-dimensional fracture model on the reference, so as to achieve model overlap and fusion processing under the same reference;

[0052] Performing surface structure analysis on the overlapped and fused models, and trimming the first three-dimensional fracture model based on the surface of the second three-dimensional fracture model, to obtain the three-dimensional fracture model after trimming;

[0053] The fracture three-dimensional model is exported and input into an AR background, and the fracture three-dimensional model is AR instantiated and integrated on the AR background to generate and save the corresponding three-dimensional fracture AR model.

[0054] On the other hand, a device for spatially locating fracture fragments based on a two-dimensional fracture image of augmented reality technology is provided, and the device for spatially locating fracture fragments based on a two-dimensional fracture image of augmented reality technology is used to implement the above-mentioned method for spatially locating fracture fragments based on a two-dimensional fracture image of augmented reality technology, and the device comprises:

[0055] An X-ray image acquisition module is used to perform a three-dimensional scan along the body surface where the fracture site is located, obtain the corresponding fracture medical X-ray image data set K and perform distributed storage;

[0056] The parsing module is used to parse the fracture medical X-ray image data set to obtain the fracture medical film k in each scanning sequence:

[0057] K = ∏ n i=1 SCAN(k i ) I ,

[0058] I represents the projection intensity of X-rays,

[0059] SCAN represents the scanning function.

[0060] SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting the i-th scanning sequence under the projection intensity I i ,

[0061] K represents the fracture medical film k under several scanning sequences i The product is obtained;

[0062] An X-film reconstruction module, for obtaining the back-projected images of each of the fracture medical films k based on the FDK algorithm, and obtaining a fracture medical three-dimensional image dataset K' composed of each of the back-projected images, and performing three-dimensional reconstruction of the fracture site based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model;

[0063] A MIP image acquisition module is used to synchronously use the MIP technology to acquire MIP two-dimensional projection images under each scanning sequence based on the above three-dimensional scanning path, and obtain a MIP fracture reconstruction image data set K'' from each of the MIP two-dimensional projection images;

[0064] An MIP reconstruction module is used to perform three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model;

[0065] The AR background is used to fuse the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, integrate the three-dimensional fracture model with AR technology, generate a corresponding three-dimensional fracture AR model, and superimpose the three-dimensional fracture AR model on a preset AR device;

[0066] AR equipment is used for three-dimensional display and positioning and display of the three-dimensional model structure of the fracture site.

[0067] On the other hand, a device for spatial positioning of fracture fragments is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned methods for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology is implemented.

[0068] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned methods for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology.

[0069] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0070] Based on the implementation of the present invention, the present invention performs a three-dimensional scan along the body surface position where the fracture site is located, obtains the corresponding fracture medical X-ray image data set K and performs distributed storage, which is convenient for centralized retrieval of image pictures and image picture distribution processing, thereby reducing the computing pressure of the server. Based on the FDK algorithm and MIP fracture reconstruction technology, the corresponding fracture models are respectively constructed and the models are fused, and then the AR augmented reality technology is used to integrate the fused fracture models to generate a three-dimensional fracture AR model that displays the fracture site through AR positioning, so that doctors can intuitively see the three-dimensional structure of the fracture on the AR device, thereby obtaining deep fracture parameters, deepening the understanding of the fracture, and helping doctors locate and diagnose parameters such as the fracture location, which can reduce the experience and professional requirements for doctors, is suitable for clinical use by a large number of doctors, and speeds up the clinical treatment time for fracture patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0072] Figure 1 It is a two-dimensional fracture diagram shown by the X-ray image;

[0073] Figure 2 is a flow chart of a method for spatial positioning of fracture fragments based on a two-dimensional fracture image using augmented reality technology provided by an embodiment of the present invention (the formula for k is not shown in step S2);

[0074] Figure 3 It is a block diagram of a device for spatially locating fracture fragments based on a two-dimensional fracture image using augmented reality technology provided by an embodiment of the present invention;

[0075] Figure 4 It is a functional architecture diagram of a Hadoop distributed file system provided by an embodiment of the present invention;

[0076] Figure 5 It is a schematic diagram of model positioning fusion under a benchmark provided by an embodiment of the present invention;

[0077] Figure 6 It is a schematic structural diagram of a bone fragment spatial positioning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0078] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0079] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0080] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0081] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0082] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0083] An embodiment of the present invention provides a method for spatially locating bone fragments of a two-dimensional fracture image based on augmented reality technology. The method can be implemented by a bone fragment spatial positioning device, which can be a terminal or a server.

[0084] like Figure 2 and Figure 3 The flowchart of the method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology is shown. The processing flow of the method may include the following steps:

[0085] On the one hand, a method for spatial positioning of fracture fragments based on a two-dimensional fracture image using augmented reality technology is provided. The method is implemented by a device for spatial positioning of fracture fragments. The method comprises:

[0086] S1, perform a three-dimensional scan along the body surface where the fracture site is located, obtain the corresponding fracture medical X-ray image dataset K and perform distributed storage;

[0087] S2. Analyze the fracture medical X-ray image dataset to obtain the fracture medical film k in each scanning sequence:

[0088] K = ∏ n i=1 SCAN(k i ) I ,

[0089] I represents the projection intensity of X-rays,

[0090] SCAN represents the scanning function.

[0091] SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting the i-th scanning sequence under the projection intensity I i ,

[0092] K represents the fracture medical film k under several scanning sequences i The product is obtained;

[0093] S3, based on the FDK algorithm, obtaining the back-projected images of each of the fracture medical films k, and obtaining a fracture medical three-dimensional image dataset K' composed of each of the back-projected images, and performing three-dimensional reconstruction of the fracture site based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model;

[0094] S4, based on the above three-dimensional scanning path, synchronously using MIP technology to obtain MIP two-dimensional projection images under each scanning sequence, and obtain a MIP fracture reconstruction image dataset K'' from each of the MIP two-dimensional projection images, and based on the MIP fracture reconstruction image dataset K'', perform three-dimensional reconstruction of the fracture site to generate a corresponding second fracture three-dimensional model;

[0095] S5, fusing the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrating the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model;

[0096] S6. Overlaying the three-dimensional fracture AR model onto a preset AR device, performing three-dimensional display in the AR device, and positioning and displaying the three-dimensional model structure of the fracture site.

[0097] In this case, AR technology was used to convert X-ray images into three-dimensional space to help doctors locate themselves.

[0098] Converting X-ray images into three-dimensional space through AR technology is an innovative and practical application. This technology combines the advantages of augmented reality (AR) and medical imaging, providing doctors, researchers and patients with a more intuitive and comprehensive way to observe and understand X-ray images.

[0099] First, we need to understand the basic principles of AR technology and X-ray images. AR technology is a technology that can integrate virtual information with the real world. It simulates virtual information such as computer-generated text, images, and three-dimensional models, and then applies them to the real world to achieve "enhancement" of the real world. X-ray images are images obtained after X-rays penetrate human tissues and are processed. They can show the internal structure and pathological conditions of the human body.

[0100] Combining the two, the three-dimensional space transformation of X-ray images can be achieved through the following steps:

[0101] 1. X-ray image acquisition and processing: First, obtain the patient's X-ray image and perform necessary preprocessing, such as denoising and contrast enhancement, to improve image quality.

[0102] 2. 3D reconstruction: Using computer vision and image processing technology, X-ray images are reconstructed in 3D. This usually involves steps such as image segmentation and surface reconstruction to extract 3D structural information from 2D images.

[0103] 3. AR integration: Integrate the reconstructed 3D model with AR technology. This includes aligning the 3D model with the spatial position of the real world so that it can be displayed correctly in the AR device.

[0104] The integration of 3D models and AR technology is a forward-looking combination of technologies in the contemporary digital field. This integration not only greatly enriches the way users interact with digital content, but also brings innovative applications to multiple industries.

[0105] A 3D model is a digital object created by a specific software tool. It can be displayed in three dimensions on a computer and can be viewed from any angle. This model can accurately reflect the shape, structure, texture and other characteristics of an object, so it has a wide range of applications in design, manufacturing, visualization and other fields.

[0106] AR technology, or augmented reality technology, is a technology that cleverly integrates virtual information with the real world. It uses computer-generated text, images, three-dimensional models and other virtual information to present it to users through specific display devices (such as AR glasses, smartphones, etc.), allowing users to perceive this virtual information in the real world.

[0107] When 3D models are combined with AR technology, the effect is amazing. This integration makes 3D models no longer limited to display on computer screens or specific devices, but can be directly integrated into the user's real environment. Users can interact with these 3D models in the real world through AR devices to gain a more intuitive and realistic experience.

[0108] This integration has broad application prospects in many fields. For example, in the field of architectural design, designers can use the integration of 3D models and AR technology to present design plans to customers in a more realistic way, helping them better understand and feel the design effects. In the field of education and training, teachers can combine 3D models with AR technology to create a more vivid and intuitive learning environment for students and improve teaching effectiveness. In the field of entertainment and games, this integration can bring users a more immersive experience and enhance the fun and attractiveness of games.

[0109] Of course, to achieve the effective integration of 3D models and AR technology, some technical challenges need to be overcome. For example, with the continuous expansion of progress and application scenarios, it will bring more convenience and innovation to our lives in the future.

[0110] 4. AR display and interaction: Through AR devices, such as AR glasses or smartphones, the 3D model is superimposed on the real world and allows users to interact with it. Doctors or researchers can rotate, scale, move, and perform other operations on the 3D model through gestures, voice, etc., so as to observe and understand the lesion more comprehensively.

[0111] Through this technology, doctors can more intuitively understand the patient's lesions and improve the accuracy and efficiency of diagnosis. At the same time, patients can also better understand their own conditions and communicate more effectively with doctors. In addition, this technology can also be applied to medical education and training to help students and doctors better learn and master medical imaging knowledge.

[0112] Therefore, this solution adopts a three-dimensional scanning method, uses X-rays to obtain the three-dimensional scanning data of the fracture site, and uses the FDK algorithm to perform three-dimensional reconstruction of the three-dimensional scanning data to generate the first fracture three-dimensional model of the fracture site based on the X-ray scan; using the same three-dimensional scanning path, and then using the mip scanning technology, use the MIP two-dimensional projection to reconstruct the second fracture three-dimensional model of the same site. Subsequently, the fracture three-dimensional model of the fracture site is obtained by fusing the two fracture three-dimensional models, so that the model constructed by the MIP two-dimensional projection can correct the three-dimensional model generated by scanning the fracture site under X-rays. The three-dimensional model of the fracture site is deeply constructed by model fusion to improve the reconstruction accuracy of the fracture site.

[0113] Using augmented reality technology, a virtual simulation model of the corresponding model is generated in the AR background, and the fracture site is displayed in three dimensions in the AR device. Therefore, doctors can view the three-dimensional shape structure and parameters of the fracture site more intuitively. Through AR display, doctors can understand the three-dimensional morphology of the current patient's fracture site more intuitively and with spatial hierarchy, which is convenient for better customization of diagnostic medical plans.

[0114] The 3D scanning of X-ray medical images plays a vital role in the medical field. It can provide doctors with detailed 3D information about the structures and lesions in the patient's body. The following is a detailed explanation of the path, parameters and methods of 3D scanning of X-ray medical images:

[0115] 1. Scanning path

[0116] In medical imaging, a 3D scanning path usually refers to how a scanning device (such as CT or MRI) traverses the patient's body to obtain image data at multiple levels. These paths can be customized according to the scanning area and purpose, but generally follow the following basic principles:

[0117] 1. Continuous slice scanning: The device continuously scans multiple slices along the patient's body axis (such as sagittal, coronal or transverse plane) to obtain complete three-dimensional data.

[0118] 2. Multi-angle scanning: In some cases, in order to obtain more comprehensive information, multi-angle scanning may be required, such as oblique scanning or rotation scanning.

[0119] (II) Scanning parameters

[0120] Scan parameters are key settings that control how the scanning device works. They directly affect the quality and resolution of the image. Here are some common scanning parameters:

[0121] 1. Slice thickness: The thickness of each scan layer, usually measured in millimeters. Smaller slice thickness can achieve higher spatial resolution, but will increase scan time and radiation dose (for CT).

[0122] 2. Slice spacing: The distance between two adjacent layers. In some cases, in order to save time or reduce radiation dose, a slice spacing greater than the slice thickness may be selected.

[0123] 3. Scan Range: The area of ​​the body covered by the scanning device. This can be adjusted as needed to include or exclude specific anatomical structures.

[0124] 4. Reconstruction algorithm: The mathematical method used to convert the raw scan data into a 3D image. Different reconstruction algorithms may have different effects on the resolution, contrast, and noise characteristics of the image.

[0125] 5. Tube voltage and tube current (projection intensity): For CT scans, tube voltage and tube current determine the energy and dose of X-rays. Higher voltage and current can improve image quality, but also increase radiation dose.

[0126] In X-scan, which set of data is used for scanning? Specifically, the corresponding scanning function (three-dimensional scanning along the body surface position (such as the sagittal plane) where the fracture site is located) can be selected for scanning. The user can scan according to the various types of scanning functions (including the corresponding scanning paths, parameters and scanning methods) provided by the system.

[0127] 3. Scanning method

[0128] In medical imaging, there are several methods to obtain three-dimensional image data, the most commonly used of which are computed tomography (CT) and magnetic resonance imaging (MRI):

[0129] 1. Computed Tomography (CT):

[0130] The human body is scanned using an X-ray beam, and the X-rays that pass through the human body are received by the detector and converted into electrical signals.

[0131] The electrical signal is converted into digital data by an analog / digital converter and input into a computer for processing.

[0132] A computer uses a reconstruction algorithm to convert the digital data into a three-dimensional image.

[0133] 2. Magnetic Resonance Imaging (MRI):

[0134] Use magnetic fields and radio waves to excite hydrogen atoms in the human body and detect the signals they produce.

[0135] These signals are converted into digital data and fed into a computer for processing.

[0136] Computers use specific algorithms to convert digital data into three-dimensional images.

[0137] The 3D scanning path, parameters and methods of X-ray medical imaging are the key to ensuring high-quality 3D images. Doctors and technicians need to select appropriate scanning paths, parameters and methods based on the patient's specific conditions and scanning purposes. With the continuous advancement of technology, more advanced 3D scanning technologies may appear in the future to further improve image quality and diagnostic accuracy.

[0138] The principle of the present invention will be further described below.

[0139] As a preferred embodiment of the present invention, the three-dimensional scanning is performed along the body surface position where the fracture site is located, and the corresponding fracture medical X-ray image data set K is obtained and distributedly stored, including:

[0140] Presetting the three-dimensional scanning path of the fracture site and its corresponding scanning parameters;

[0141] Start X-ray scanning, perform three-dimensional scanning along the body surface where the fracture site is located, sequentially obtain fracture medical films k under different scanning sequences, and bind each fracture medical film k to the corresponding scanning sequence ID;

[0142] Performing denoising and contrast enhancement processing on the fracture medical film k;

[0143] After processing, traverse and identify the idle data nodes in the Hadoop distributed file system - datanode (x);

[0144] Each of the fracture medical films k is stored in the corresponding data node - datanode (x) in sequence according to the scanning sequence, and the scanning sequence ID is written into the name node of the Hadoop distributed file system;

[0145] Through the name node name node, the fracture medical film k in each data node -data node (x) is traversed and extracted according to the scan sequence ID to obtain the fracture medical X-image dataset K.

[0146] like Figure 3As shown, the 3D scanning path and scanning parameters of the fracture site are set by the user according to the normal X-ray 3D scanning operation method, so that the fracture site can be 3D scanned and the corresponding 3D scanning data can be obtained. The specific setting and operation of parameters and paths on the X-ray imaging scanning system are not elaborated in this solution and are handled by the user.

[0147] The 3D scanning path of the fractured part can be saved and used to guide subsequent VIP scanning.

[0148] In a three-dimensional scanning sequence, a corresponding fracture medical film will be obtained in each sequence. Therefore, in order to process each fracture medical film, this solution binds the scanning sequence ID with the corresponding fracture medical film, so that it is convenient to traverse and retrieve each fracture medical film by conveniently scanning the virtual ID, and use it for other film processing steps, thereby improving the film processing efficiency.

[0149] For the pre-processing methods of noise reduction and contrast enhancement of the film, you can refer to the noise reduction and contrast enhancement processing methods on conventional films, and it can be completed by the medical imaging system.

[0150] In order to avoid the problem of difficult management of the storage and scheduling of the background three-dimensional data by the system, this solution sets up a Hadoop distributed file system on the background system to store the fracture medical films under each sequence, wherein the name node can manage the storage status of each data node and identify the idle state of the data node. If it is idle, it is used for data storage. The specific use of the Hadoop distributed file system and the identification of the data nodes in the idle state for storing the fracture medical films under different scanning sequences can be implemented by referring to the working principle of the distributed file system, and this solution will not be elaborated.

[0151] Subsequently, a request for the corresponding sequence of films can be sent to the name node, allowing the name node to retrieve the corresponding sequence of fracture medical films from the corresponding data node and feed them back to the subsequent film processing module for the next step of processing.

[0152] like Figure 4 The schematic diagram of the architecture of the Hadoop distributed file system is shown in FIG.

[0153] The nodes in Hadoop Distributed File System (HDFS) are mainly divided into two categories: Master Node or Name Node, and Slave Node or Data Node. The following is a detailed description of these two types of nodes:

[0154] 1. Master Node or Name Node (Administrator)

[0155] Function: Manage the file system namespace, maintain the file system tree and all files and directories on the entire tree.

[0156] Data structure:

[0157] FsImage: A snapshot of the file system tree, along with metadata about all files and folders in the tree.

[0158] EditLog: records all operations such as creation, deletion, and renaming of files.

[0159] Record information: The location information of the data node where each block in each file is located (including the number of copies and user operations on HDFS). However, the location information of the block is not permanently saved. This information will be rebuilt based on the data node information when the system starts.

[0160] Name Node in HA (High Availability) Cluster: Hadoop supports High Availability (HA) configuration, which involves Active Name Node and Standby Name Node. When the Active Name Node fails, the Standby NameNode can take over the service. Zookeeper and ZKFC (Failure Detection Control) are used to ensure that this transition is smooth.

[0161] 2. Slave Node or Data Node

[0162] Function: Stores split data blocks and provides read and write requests from file system clients.

[0163] Communication mechanism: When the DN thread is started, it will report block information to the NN and keep in touch with it by sending heartbeats through the NN (3 seconds by default). If the NN does not receive the heartbeat of the DN for a long time, it will be considered invalid and may copy the blocks on it to other DNs.

[0164] Data block copy strategy:

[0165] The first copy is placed in the DN of the uploaded file. If the file is submitted outside the cluster, a node with a not too full disk and a not too busy CPU is randomly selected.

[0166] The second replica is placed on a different rack node from the first replica.

[0167] The third replica is on the same rack node as the second replica.

[0168] More replicas: Randomly select nodes.

[0169] Data nodes in HA clusters: In HA clusters, all Data Nodes must be configured with the addresses of two Name Nodes and send data block location information and heartbeats to both of them.

[0170] In summary, the nodes in the Hadoop distributed file system work together to ensure high availability, fault tolerance, and scalability of data.

[0171] Therefore, the present invention uses the Hadoop distributed file system to store the fracture medical film k scanned and obtained under each scanning sequence. Each fracture medical film k is stored in the data node in the Hadoop distributed file system in an idle state according to the scanning sequence ID, and the scanning sequence ID is stored in the name node, so that the administrator can read the fracture medical film k stored in the corresponding Data Node according to the scanning sequence ID, which is convenient for the subsequent call processing of each X-ray image, and avoids the centralized storage and processing that will increase the computing pressure of the server (otherwise it will be stored in the background database together, and a search library or SQL is required to search and call the data, and the calculation is relatively complicated and troublesome).

[0172] As a preferred embodiment of the present invention, the back-projection images of each of the fracture medical films k are obtained based on the FDK algorithm, and a fracture medical three-dimensional image dataset K' composed of each of the back-projection images is obtained, and a fracture site is three-dimensionally reconstructed based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model, including:

[0173] Through the name node, read the corresponding fracture medical films k in order according to the scan sequence ID;

[0174] Based on the FDK algorithm, the fracture medical film k is back-projected according to its corresponding scanning sequence to generate the corresponding back-projected image;

[0175] Performing three-dimensional voxel assignment on the back-projected images of all the fracture medical films k respectively, to obtain the fracture medical three-dimensional image data set K' composed of the back-projected images of all the fracture medical films k;

[0176] Importing the fracture medical three-dimensional image data set K' into a preset three-dimensional coordinate system, performing three-dimensional reconstruction of the fracture site, and generating the first fracture three-dimensional model;

[0177] After post-processing the first three-dimensional fracture model, the model file of the first three-dimensional fracture model is stored in a background database.

[0178] This scheme uses the FDK algorithm to perform three-dimensional reconstruction of the fracture site.

[0179] The FDK algorithm, also known as the Feldkamp-Davis-Kress algorithm, is an algorithm widely used in computed tomography (CT) image reconstruction. Its core principle is an image reconstruction method based on filtered back projection. During the CT scanning process, the FDK algorithm first projects X-rays at multiple angles on the sample to be tested to obtain a series of two-dimensional projection data. Subsequently, these projection data are Fourier transformed and filtered to remove low-frequency signals and other noise. Finally, these processed data are converted into internal structure images of three-dimensional objects through back projection technology.

[0180] An important feature of the FDK algorithm is that it can approximate all cone beam projection data that do not pass through the geometric center plane as projection data obtained by tilting the fan beam of the geometric center plane at an angle. This approximate processing enables the FDK algorithm to more accurately restore the internal structure of the object during image reconstruction.

[0181] The FDK algorithm is a commonly used method in computed tomography (CT) image reconstruction. It can restore the structure of a three-dimensional object from a series of two-dimensional projection images. The following are the basic steps to reconstruct a three-dimensional model using the FDK algorithm:

[0182] 1. Projection data acquisition: First, multiple two-dimensional projection images of the object to be reconstructed need to be acquired. These projection images are usually obtained by rotating the object and scanning it using X-rays or other radiation sources.

[0183] 2. Preprocessing: Preprocess the acquired projection data, including removing noise, correcting distortion, etc., to improve the reconstruction quality.

[0184] 3. Fourier Transform: Perform a Fourier transform on each projection image to convert it from the spatial domain to the frequency domain. This step helps in applying filtering operations in subsequent steps.

[0185] 4. Filtering: Filter the projection data in the frequency domain. The purpose of filtering is to remove high-frequency noise introduced by the blurring effect during the projection process while retaining low-frequency information to improve the quality of the reconstructed image.

[0186] 5. Inverse Fourier transform: The filtered frequency domain data is converted back to the spatial domain through inverse Fourier transform to obtain the filtered projection image.

[0187] 6. Back-projection: The filtered projection images are back-projected, that is, they are re-projected back into the three-dimensional space. This step is achieved by rotating each projection image in the opposite direction along its corresponding rotation angle and superimposing them.

[0188] 7. 3D voxel assignment: During the back-projection process, voxels in 3D space are assigned values ​​based on the projection data. The value of each voxel can be determined by comprehensively considering the contribution of all projection images.

[0189] 8. Post-processing: Perform post-processing on the reconstructed 3D model, such as smoothing and segmentation, to further optimize the reconstruction results.

[0190] The steps of 3D reconstruction using the FDK algorithm are as follows:

[0191] 1. Data acquisition: First, a series of two-dimensional image data needs to be acquired through X-ray technology. These images are usually continuous tomographic images that can show the internal structure of the object.

[0192] 2. Data cleaning and processing: After obtaining the original image data, a series of processing tasks are required, including denoising and filtering, to improve the quality and clarity of the image and lay the foundation for subsequent reconstruction work.

[0193] 3. Data reconstruction and merging: This is the core step of 3D reconstruction. By using specific reconstruction algorithms, such as the FDK algorithm (Feldkamp-Davis-Kress algorithm), the 2D image data is converted into a 3D model. At the same time, the 2D images at different angles and levels need to be accurately registered and merged to form a complete 3D structure.

[0194] 4. Structural analysis and visualization: Finally, the reconstructed 3D model is subjected to structural analysis and visualization, which includes rendering, measuring, and cutting the model to more intuitively display the internal structure of the object and provide doctors or researchers with detailed diagnosis and analysis basis.

[0195] Specifically, please combine the FDK steps and the corresponding fracture medical 3D image data set K' to complete the reconstruction and obtain the first fracture 3D model.

[0196] The implementation of the FDK algorithm requires certain conditions to be met, such as the detector and the source must be fixed, the scanned object must spin around the vertical axis, and the scanned object must be completely within the cone beam of the X-ray source. At the same time, the X-ray source must be a point light source, and the reconstruction result is usually represented by a voxel model.

[0197] Specifically, data collection can be completed in combination with the above-mentioned scanning path setting.

[0198] Next, the MIP technology will be combined to reconstruct the fracture 3D model and used for model fusion to improve the accuracy of the fracture 3D model.

[0199] As a preferred embodiment of the present invention, based on the above three-dimensional scanning path, the MIP technology is synchronously used to obtain the MIP two-dimensional projection images under each scanning sequence, and the MIP fracture reconstruction image data set K'' of each of the MIP two-dimensional projection images is obtained, including:

[0200] Starting the MIP scan according to the pre-set three-dimensional scanning path and its corresponding scanning parameters;

[0201] Perform perspective scanning along the body surface where the fracture site is located, sequentially obtain MIP two-dimensional projection images under different scanning sequences, and bind each of the MIP two-dimensional projection images to the corresponding scanning sequence ID;

[0202] Performing denoising and contrast enhancement processing on the MIP two-dimensional projection image;

[0203] After processing, traverse and identify the idle data nodes in the Hadoop distributed file system - datanode (x);

[0204] According to the scanning sequence ID, each of the MIP two-dimensional projection images is stored in the corresponding data node -data node (x), and coexists with the fracture medical film k bound to the scanning sequence ID;

[0205] Through the name node name node, the MIP two-dimensional projection image in each data node -data node (x) is traversed and extracted according to the scan sequence ID to obtain the MIP fracture reconstruction image data set K''.

[0206] When acquiring MIP (maximum intensity projection) 2D projection images under different scanning sequences, the following steps are usually involved:

[0207] 1. Understand the principle of MIP:

[0208] MIP is a widely used post-processing technology for CT and MR images.

[0209] It produces a two-dimensional image by calculating the maximum density of pixels encountered along each ray of the scanned object.

[0210] When the light beam passes through the original image of a section of tissue, the pixels with the highest density in the image are retained and projected onto a two-dimensional plane to form a MIP reconstructed image.

[0211] 2. Set the scanning sequence:

[0212] Different scanning sequences are set according to specific diagnostic needs. This usually involves selecting appropriate parameters such as layer thickness, scanning range, and resolution.

[0213] Before performing MIP reconstruction, it should be ensured that the parameters of the scanning sequence are set to capture the maximum density information of the region of interest.

[0214] 3. MIP reconstruction process:

[0215] In vtk (Visualization Toolkit), MIP reconstruction focuses on the vtkImageSlabReslice class. This class is derived from vtkImageReslice and is used to re-slice data.

[0216] MIP implementation requires two steps:

[0217] Set the direction to be cut, and set the reslicing axis direction cosines through the SetResliceAxes or SetResliceAxesDirectionCosines method.

[0218] Set the thickness of the slice, which can be achieved with the SlabThickness parameter.

[0219] The effect of MIP reconstruction can be optimized by adjusting parameters such as SlabResolution (thickness sampling spacing) and NumBlendSamplePoints (number of slices).

[0220] 4. Get the MIP two-dimensional projection image:

[0221] After completing the MIP reconstruction, the obtained two-dimensional projection image is saved or displayed.

[0222] The MIP image can be further processed and analyzed as needed, such as adjusting the window width and window position, and performing image fusion.

[0223] By optimizing parameter settings and post-processing, high-quality MIP images can be obtained, providing strong support for clinical diagnosis. In this embodiment, the parameter debugging of the MIP image of the fracture site obtained by MIP can be completed on-site by medical staff.

[0224] We still use the Hadoop distributed file system to store MIP impact data, so as to facilitate the call and management of MIP images.

[0225] The MIP images and fracture medical films k corresponding to each scanning sequence can be found using the Hadoop distributed file system, making it easy to manage.

[0226] As a preferred embodiment of the present invention, the three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model includes:

[0227] Obtaining the fracture three-dimensional MIP data to be reconstructed: through the name node, sequentially reading the corresponding MIP two-dimensional projection images according to the scanning sequence ID, to obtain the MIP fracture reconstruction image data set K'';

[0228] Sequentially traverse and calculate the MIP values ​​of each of the MIP two-dimensional projection images in the MIP fracture reconstruction image data set K'', and generate corresponding MIP fracture reconstruction images through projection;

[0229] According to the above steps, the MIP two-dimensional projection images under all scanning sequences are projected in a preset three-dimensional space to generate the corresponding second fracture three-dimensional model;

[0230] After post-processing the second fracture three-dimensional model, the model file of the second fracture three-dimensional model is stored in a background database.

[0231] MIP uses perspective to obtain a two-dimensional image, which is generated by calculating the maximum density pixel encountered along each ray of the scanned object. When the fiber bundle passes through the original image of a section of tissue, the pixel with the highest density in the image is retained and projected onto a two-dimensional plane, thus forming a MIP reconstructed image.

[0232] The steps of MIP image reconstruction are mainly based on the principle of maximum density projection, which generates a two-dimensional image by calculating the maximum density pixels encountered on each ray. The following is an overview of the steps of MIP image reconstruction:

[0233] 1. Data preparation: First, you need to obtain the 3D data volume to be reconstructed, which usually comes from medical scanning equipment such as CT or MRI. These data volumes contain a series of 2D slice images, each of which represents a layer in the 3D space.

[0234] 2. Ray casting: Next, multiple rays are simulated and cast from the reconstruction perspective. These rays pass through the 3D data volume, and each ray corresponds to a pixel or voxel in the image.

[0235] 3. Density calculation: For each ray, calculate the maximum density value among all pixels or voxels it passes through. This maximum density value represents the maximum intensity or brightness of the ray in the three-dimensional data volume.

[0236] 4. Projection imaging: Map the maximum density value of each ray onto a two-dimensional plane to form the final MIP reconstructed image. In this process, parameters such as grayscale and contrast can be adjusted as needed to optimize the image display effect.

[0237] MIP reconstructed images have a better display effect for tissues and structures with relatively high density, such as developed blood vessels and bones.

[0238] In addition, in order to improve the quality and efficiency of MIP reconstruction, it is also proposed to use methods such as MIP reconstruction based on the local maximum average value and the use of shear deformation algorithm to accelerate the reconstruction process, which can solve problems such as bone fracture and slow reconstruction speed to a certain extent and improve the accuracy and practicality of MIP reconstructed images.

[0239] Please understand the method of reconstructing a three-dimensional model using MIP two-dimensional projection in conjunction with the principle of MIP projection.

[0240] Using MIP (maximum intensity projection) technology to reconstruct a 3D model mainly involves the concepts of image processing and computer graphics. The following are the basic steps for reconstructing a 3D model using MIP projection:

[0241] 1. MIP technology introduction:

[0242] MIP (Maximum Intensity Projection) is a widely used post-processing technology for CT and MR images. It uses the perspective method to obtain a two-dimensional image, that is, it is generated by calculating the maximum density pixels encountered along each ray of the scanned object.

[0243] When the fiber bundle passes through the original image of a section of tissue, the pixels with the highest density in the image are retained and projected onto a two-dimensional plane to form a MIP reconstructed image.

[0244] MIP can reflect the X-ray attenuation value of the corresponding pixel, and smaller density changes can also be displayed on the MIP image, which makes MIP particularly suitable for displaying vascular stenosis, dilation, filling defects and distinguishing calcification on the vessel wall from contrast agents in the vascular cavity.

[0245] 2. MIP projection reconstruction of 3D model steps:

[0246] Data acquisition: First, the original three-dimensional image data needs to be acquired through medical imaging technologies such as CT or MR scanning.

[0247] MIP processing: Then, MIP processing is performed on these raw data. This involves calculating and projecting the maximum density pixels on each ray to generate a two-dimensional MIP image.

[0248] 3D reconstruction: Next, based on these 2D MIP images, computer graphics techniques such as volume rendering or surface rendering are used to reconstruct the 3D model.

[0249] Post-processing and optimization: Finally, the reconstructed 3D model can be post-processed, such as smoothing, denoising, and refinement, to further improve the quality and visual effect of the model.

[0250] 3. Application of MIP projection to reconstruct 3D models:

[0251] The 3D models reconstructed by MIP projections have a wide range of applications in the medical field, such as fracture reconstruction, etc. These 3D models can provide doctors with more intuitive and accurate diagnostic information, helping them to better understand the structure and location of lesions.

[0252] As a preferred embodiment of the present invention, the fusing of the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrating the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model, comprises:

[0253] Reading the model file, projecting the first fracture three-dimensional model and the second fracture three-dimensional model into the same three-dimensional space and performing rendering processing;

[0254] Taking a scanning vector of a scanning sequence in the three-dimensional scanning path as a reference, and positioning the first three-dimensional fracture model and the second three-dimensional fracture model on the reference, so as to achieve model overlap and fusion processing under the same reference;

[0255] Performing surface structure analysis on the overlapped and fused models, and trimming the first three-dimensional fracture model based on the surface of the second three-dimensional fracture model, to obtain the three-dimensional fracture model after trimming;

[0256] The fracture three-dimensional model is exported and input into an AR background, and the fracture three-dimensional model is AR instantiated and integrated on the AR background to generate and save the corresponding three-dimensional fracture AR model.

[0257] The rendering processing of the three-dimensional model can perform the rendering operation of the model surface in the corresponding three-dimensional space, such as the three-dimensional space of some three-dimensional modeling software (UG, SW).

[0258] Scan vectors in a scanning sequence, such as Figure 5 A reference direction shown is a vector with corresponding angle and orientation (and coordinates in space). It can also be a reference point. In the same three-dimensional space, the origin of the current coordinate system is used as the reference point (or Z axis), and the two models are overlapped and fused. If the model surfaces are inconsistent, the corresponding rendered parts will be highlighted. Therefore, the model can be further adjusted and corrected according to the fused parts and the unfused parts to obtain the final three-dimensional model.

[0259] The model is corrected by overlapping the two fracture 3D models to generate the final patient fracture 3D model to ensure the accuracy and quality of the model. The following are the key steps:

[0260] 1. Model import and alignment:

[0261] First, import the 3D model that needs to be corrected into the corresponding software or platform.

[0262] A reference model (usually the accurate or ideal model) is aligned with the model that needs correction using a model alignment technique such as the ICP algorithm. This helps determine the differences and similarities between the two models.

[0263] 2. Overlap analysis:

[0264] Through overlap analysis, areas or features that require correction are identified. This may involve comparing the geometric differences between the two models in the overlap area, such as shape, size or position.

[0265] 3. Mesh repair and surface fitting:

[0266] If defects or errors are found in the model during the overlap analysis, mesh repair techniques can be used to automatically or semi-automatically detect and repair these problems. For example, algorithms such as Poisson reconstruction and Marching cubes can be used to repair holes in the mesh, fill defects, or remove overlaps.

[0267] Surface fitting techniques can be used to convert a set of point cloud data into a continuous, smooth surface model, especially in the absence of geometric information. This can be achieved through algorithms such as point cloud fitting, surface fitting, and volume rendering.

[0268] 4. Point cloud filtering:

[0269] When processing 3D models, point cloud data may contain noise and outliers. Using point cloud filtering techniques (such as Gaussian filtering, median filtering, and adaptive filtering) can eliminate these noises and improve data accuracy.

[0270] 5. Model splitting and reconstruction:

[0271] If you need to process the model more finely, you can use model splitting technology to decompose large 3D models into small sub-models. This can improve the model's handleability and editability.

[0272] Surface reconstruction technology can convert discrete point cloud data into continuous surface models, which is particularly useful for reconstructing surface models from 3D scanning data.

[0273] 6. Correction process:

[0274] Based on the results of the overlap analysis and the areas that need correction, use the appropriate tools and techniques to correct the model. This may involve adjusting the shape, size, position or orientation of the model.

[0275] The correction process may require multiple iterations and optimizations to ensure the accuracy and quality of the model.

[0276] 7. Verification and export:

[0277] After the correction is completed, the model is validated using the validation tool to ensure that the model meets the expected accuracy and quality.

[0278] Finally, export the corrected model to the desired format for use in subsequent work or projects.

[0279] If the constructed three-dimensional model needs to be displayed on AR, it is also necessary to perform AR instantiation integration on the fracture three-dimensional model on the AR background to generate the corresponding three-dimensional fracture AR model.

[0280] The background management software (AR background) of each AR device is determined by the AR device type selected by the user, and its background can be deployed on the background server.

[0281] Performing AR instantiation integration on the fracture 3D model on the AR background to generate the corresponding 3D fracture AR model usually involves the following steps:

[0282] 1. Model preparation:

[0283] Obtain a 3D model of the fracture (the patient's 3D fracture model has been corrected previously).

[0284] 2. Model optimization:

[0285] According to the performance requirements of AR applications, 3D models are optimized, such as reducing the number of polygons and compressing textures, to improve rendering efficiency on mobile devices or VR / AR devices.

[0286] 3. Import the model into the AR development environment:

[0287] Import the optimized 3D model into the AR development environment (such as Unity, Unreal Engine, etc.).

[0288] Further adjustments and testing of the model are performed in the development environment to ensure that it displays and interacts correctly.

[0289] 4. AR instantiation integration:

[0290] In the AR development environment, add AR interactive features to the fracture 3D model. This may include allowing users to interact with the model through gestures, voice, or other input methods.

[0291] Integrate recognition and tracking technology to identify and track fracture models in AR applications.

[0292] 5. Testing and debugging:

[0293] The AR application was tested on different devices and in different environments to ensure that the fracture 3D model could be displayed and interacted with correctly under different conditions.

[0294] Perform necessary debugging and optimization based on the test results.

[0295] 6. Release and Application:

[0296] Once the tests are passed and all requirements are met, the AR application can be published to the corresponding platform (such as AppStore, Google Play, etc.).

[0297] In the medical field, this AR application can be used in education, diagnosis, surgical planning and other aspects to improve the efficiency and accuracy of medical services.

[0298] During specific use, the instantiated three-dimensional fracture AR model can be saved in the AR background and interacted with the front-end AR device. When the user requests, the AR background will send the instantiated three-dimensional fracture AR model to the AR device for visual display, and it can also interact according to the instantiated scene.

[0299] The background instantiation of AR is set by the administrator according to the needs of the user. The specific application scenarios and AR interactions are determined according to the interaction scenarios set in the instantiation.

[0300] By using AR positioning to display the three-dimensional fracture AR model of the fracture site, doctors can visually see the three-dimensional structure of the fracture on the AR device, thereby obtaining in-depth fracture parameters, deepening their understanding of the fracture, and helping doctors locate and diagnose parameters such as the fracture location. This can reduce the experience and professional requirements for doctors, is suitable for clinical use by a wide range of doctors, and can speed up the clinical treatment time for fracture patients.

[0301] Example 2

[0302] Based on the description of Example 1, on the other hand, a device for spatially locating fracture fragments based on a two-dimensional fracture image using augmented reality technology is provided, and the device for spatially locating fracture fragments based on a two-dimensional fracture image using augmented reality technology is used to implement the above-mentioned method for spatially locating fracture fragments based on a two-dimensional fracture image using augmented reality technology, such as Figure 3 As shown, the device comprises:

[0303] An X-ray image acquisition module is used to perform a three-dimensional scan along the body surface where the fracture site is located, obtain the corresponding fracture medical X-ray image data set K and perform distributed storage;

[0304] The parsing module is used to parse the fracture medical X-ray image data set to obtain the fracture medical film k in each scanning sequence:

[0305] K = ∏ n i=1 SCAN(k i ) I ,

[0306] I represents the projection intensity of X-rays,

[0307] SCAN represents the scanning function.

[0308] SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting the i-th scanning sequence under the projection intensity I i ,

[0309] K represents the fracture medical film k under several scanning sequences i The product is obtained;

[0310] An X-film reconstruction module, for obtaining the back-projected images of each of the fracture medical films k based on the FDK algorithm, and obtaining a fracture medical three-dimensional image dataset K' composed of each of the back-projected images, and performing three-dimensional reconstruction of the fracture site based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model;

[0311] A MIP image acquisition module is used to synchronously use the MIP technology to acquire MIP two-dimensional projection images under each scanning sequence based on the above three-dimensional scanning path, and obtain a MIP fracture reconstruction image data set K'' from each of the MIP two-dimensional projection images;

[0312] An MIP reconstruction module is used to perform three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model;

[0313] The AR background is used to fuse the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, integrate the three-dimensional fracture model with AR technology, generate a corresponding three-dimensional fracture AR model, and superimpose the three-dimensional fracture AR model on a preset AR device;

[0314] AR equipment is used for three-dimensional display and positioning and display of the three-dimensional model structure of the fracture site.

[0315] The functions and interactions of the above modules can be understood in conjunction with the corresponding implementation steps and interaction principles in Example 1, and will not be described in detail in this example.

[0316] The composition structure of the device can be combined with the attached Figure 3 shown.

[0317] Example 3

[0318] Figure 6 is a schematic diagram of a structure provided by an embodiment of the present invention, such as Figure 6 As shown, the fracture fragment spatial positioning device may include the above Figure 3 The device for spatially locating bone fragments based on a two-dimensional fracture image of an augmented reality technique is shown. Optionally, the device for spatially locating bone fragments 410 may include a first processor 2001 .

[0319] Optionally, the bone fragment spatial positioning device 410 may further include a memory 2002 and a transceiver 2003 .

[0320] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0321] Combine the following Figure 6 The components of the bone fragment spatial positioning device 410 are specifically introduced:

[0322] The first processor 2001 is the control center of the fracture fragment spatial positioning device 410, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0323] Optionally, the first processor 2001 can perform various functions of the bone fragment spatial positioning device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0324] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 6 CPU0 and CPU1 are shown in FIG.

[0325] In a specific implementation, as an embodiment, the fracture fragment spatial positioning device 410 may also include multiple processors, such as Figure 6 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0326] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0327] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently and access the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0328] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0329] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 6 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0330] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0331] It should be noted that Figure 6 The structure of the fracture fragment spatial positioning device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0332] In addition, the technical effects of the bone fragment spatial positioning device 410 can refer to the technical effects of the bone fragment spatial positioning method based on the two-dimensional fracture image of the augmented reality technology described in the above method embodiment, and will not be repeated here.

[0333] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0334] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0335] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0336] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0337] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0338] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0339] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0340] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0341] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0342] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0343] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0344] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0345] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology, characterized in that: The method comprises: Perform a three-dimensional scan along the body surface where the fracture site is located, obtain the corresponding fracture medical X-ray image dataset K and perform distributed storage; Parse the fracture medical X-ray image dataset to obtain the fracture medical film k in each scanning sequence: K=∏ n i=1 SCAN(k i ) I , I represents the projection intensity of X-rays, SCAN represents the scanning function. SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting the i-th scanning sequence under the projection intensity I i , K represents the fracture medical film k under several scanning sequences i The product is obtained; Based on the FDK algorithm, the back-projected images of each of the fracture medical films k are obtained, and a fracture medical three-dimensional image dataset K' composed of each of the back-projected images is obtained, and the fracture site is three-dimensionally reconstructed based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model; Based on the above three-dimensional scanning path, the MIP technology is synchronously used to obtain the MIP two-dimensional projection images under each scanning sequence, and the MIP fracture reconstruction image data set K'' is obtained from each of the MIP two-dimensional projection images, and the fracture site is three-dimensionally reconstructed based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model; fusing the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrating the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model; The three-dimensional fracture AR model is superimposed on a preset AR device, and three-dimensionally displayed in the AR device to locate and display the three-dimensional model structure of the fracture site.

2. The method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology according to claim 1, characterized in that: The three-dimensional scanning is performed along the body surface position where the fracture site is located, and the corresponding fracture medical X-ray image data set K is obtained and distributedly stored, including: Presetting the three-dimensional scanning path of the fracture site and its corresponding scanning parameters; Start X-ray scanning, perform three-dimensional scanning along the body surface where the fracture site is located, sequentially obtain fracture medical films k under different scanning sequences, and bind each fracture medical film k to the corresponding scanning sequence ID; Performing denoising and contrast enhancement processing on the fracture medical film k; After processing, traverse and identify the idle data nodes in the Hadoop distributed file system - data node (x); Each of the fracture medical films k is stored in the corresponding data node - data node (x) in sequence according to the scanning sequence, and the scanning sequence ID is written into the name node of the Hadoop distributed file system; Through the name node name node, the fracture medical film k in each data node -data node (x) is traversed and extracted according to the scan sequence ID to obtain the fracture medical X-image dataset K.

3. The method for spatial positioning of fracture fragments based on augmented reality fracture two-dimensional images according to claim 2, characterized in that: The method of obtaining the back-projected images of each fracture medical film k based on the FDK algorithm, and obtaining a fracture medical three-dimensional image data set K' composed of each back-projected image, and performing three-dimensional reconstruction of the fracture site based on the fracture medical three-dimensional image data set K' to generate a corresponding first fracture three-dimensional model includes: Through the name node, read the corresponding fracture medical films k in order according to the scan sequence ID; Based on the FDK algorithm, the fracture medical film k is back-projected according to its corresponding scanning sequence to generate the corresponding back-projection image; Performing three-dimensional voxel assignment on the back-projected images of all the fracture medical films k respectively, to obtain the fracture medical three-dimensional image data set K' composed of the back-projected images of all the fracture medical films k; Importing the fracture medical three-dimensional image data set K' into a preset three-dimensional coordinate system, performing three-dimensional reconstruction of the fracture site, and generating the first fracture three-dimensional model; After post-processing the first three-dimensional fracture model, the model file of the first three-dimensional fracture model is stored in a background database.

4. The method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology according to claim 2, characterized in that: Based on the above three-dimensional scanning path, the MIP technology is synchronously used to obtain the MIP two-dimensional projection images under each scanning sequence, and the MIP fracture reconstruction image data set K'' of each of the MIP two-dimensional projection images is obtained, including: Starting the MIP scan according to the pre-set three-dimensional scanning path and its corresponding scanning parameters; Perform perspective scanning along the body surface where the fracture site is located, sequentially obtain MIP two-dimensional projection images under different scanning sequences, and bind each of the MIP two-dimensional projection images to the corresponding scanning sequence ID; Performing denoising and contrast enhancement processing on the MIP two-dimensional projection image; After processing, traverse and identify the idle data nodes in the Hadoop distributed file system - data node (x); According to the scanning sequence ID, each of the MIP two-dimensional projection images is stored in the corresponding data node -datanode(x), and coexists with the fracture medical film k bound to the scanning sequence ID; Through the name node name node, the MIP two-dimensional projection image in each data node -data node (x) is traversed and extracted according to the scan sequence ID to obtain the MIP fracture reconstruction image data set K''.

5. The method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology according to claim 4, characterized in that: The three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model includes: Obtaining the fracture three-dimensional MIP data to be reconstructed: through the name node, sequentially reading the corresponding MIP two-dimensional projection images according to the scanning sequence ID, to obtain the MIP fracture reconstruction image data set K''; Sequentially traverse and calculate the MIP values ​​of each of the MIP two-dimensional projection images in the MIP fracture reconstruction image data set K'', and generate corresponding MIP fracture reconstruction images through projection; According to the above steps, the MIP two-dimensional projection images under all scanning sequences are projected in a preset three-dimensional space to generate the corresponding second fracture three-dimensional model; After post-processing the second fracture three-dimensional model, the model file of the second fracture three-dimensional model is stored in a background database.

6. The method for spatial positioning of fracture fragments based on augmented reality fracture two-dimensional images according to claim 2, characterized in that: The fusing the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrating the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model, including: Reading the model file, projecting the first fracture three-dimensional model and the second fracture three-dimensional model into the same three-dimensional space and performing rendering processing; Taking a scanning vector of a scanning sequence in the three-dimensional scanning path as a reference, and positioning the first three-dimensional fracture model and the second three-dimensional fracture model on the reference, so as to achieve model overlap and fusion processing under the same reference; Performing surface structure analysis on the overlapped and fused models, and trimming the first three-dimensional fracture model based on the surface of the second three-dimensional fracture model, to obtain the three-dimensional fracture model after trimming; The fracture three-dimensional model is exported and input into an AR background, and the fracture three-dimensional model is AR instantiated and integrated on the AR background to generate and save the corresponding three-dimensional fracture AR model.

7. A device for spatially locating bone fragments based on two-dimensional fracture images using augmented reality technology, wherein the device for spatially locating bone fragments based on two-dimensional fracture images using augmented reality technology is used to implement the method for spatially locating bone fragments based on two-dimensional fracture images using augmented reality technology as claimed in any one of claims 1 to 6, characterized in that: The device comprises: An X-ray image acquisition module is used to perform a three-dimensional scan along the body surface where the fracture site is located, obtain the corresponding fracture medical X-ray image data set K and perform distributed storage; The parsing module is used to parse the fracture medical X-ray image data set to obtain the fracture medical film k in each scanning sequence: K=∏ n i=1 SCAN(k i ) I , I represents the projection intensity of X-rays, SCAN represents the scanning function. SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting the i-th scanning sequence under the projection intensity I i , K represents the fracture medical film k under several scanning sequences i The product is obtained; An X-film reconstruction module, for obtaining the back-projected images of each of the fracture medical films k based on the FDK algorithm, and obtaining a fracture medical three-dimensional image dataset K' composed of each of the back-projected images, and performing three-dimensional reconstruction of the fracture site based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model; A MIP image acquisition module is used to synchronously use the MIP technology to acquire MIP two-dimensional projection images under each scanning sequence based on the above three-dimensional scanning path, and obtain a MIP fracture reconstruction image data set K'' from each of the MIP two-dimensional projection images; An MIP reconstruction module is used to perform three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model; The AR background is used to fuse the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, integrate the three-dimensional fracture model with AR technology, generate a corresponding three-dimensional fracture AR model, and superimpose the three-dimensional fracture AR model on a preset AR device; AR equipment is used for three-dimensional display and positioning and display of the three-dimensional model structure of the fracture site.

8. A bone fragment spatial positioning device, characterized in that: The bone fragment spatial positioning device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images