Method for spatially locating fracture fragments in two-dimensional fracture images based on augmented reality technology

Abstract

The present invention discloses a method for spatially locating fracture fragments in two-dimensional fracture images based on augmented reality technology. By performing three-dimensional scanning along the body surface position where the fracture site is located, the corresponding fracture medical X-ray image dataset K is obtained and distributed for storage, facilitating centralized retrieval of image pictures and distributed processing of image pictures, and reducing the computing pressure on the server. Based on the FDK algorithm and the MIP fracture reconstruction technology, corresponding fracture models are respectively constructed and model fusion is carried out. Then, using the AR augmented reality technology, the fused fracture models are integrated to generate a three-dimensional fracture AR model that displays the fracture site through AR positioning, enabling doctors to visually observe the three-dimensional structure of the fracture on the AR device, thereby obtaining in-depth fracture parameters, deepening the understanding of fractures, helping doctors locate and diagnose fracture position and other parameters, being able to reduce the requirements for doctors' experience and professionalism, being suitable for clinical use by a large number of doctors, and accelerating the clinical treatment time for fracture patients.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent medicine, and particularly to a method for spatially positioning fracture fragments based on an enhanced reality technology for fracture two-dimensional images, a device for spatially positioning fracture fragments based on an enhanced reality technology for fracture two-dimensional images, and a device for spatially positioning fracture fragments. Background Art

[0002] Fracture positioning is an important step in fracture treatment, and its main purpose is to determine the specific location and type of the fracture in order to select an appropriate treatment method.

[0003] Fracture positioning usually relies on medical imaging examinations such as X-ray films, CT scans, or MRIs. These examinations can provide detailed images of the fracture site, helping doctors determine the accurate location of the fracture, the orientation of the fracture line, the size and displacement of the fracture fragments, etc.

[0004] Once the fracture location is determined, doctors can formulate targeted treatment plans. This may include conservative treatment measures such as manual reduction, fixation, traction, or more complex treatment methods such as surgical reduction and internal fixation.

[0005] Therefore, the accuracy of fracture positioning is crucial for the treatment effect. If the positioning is inaccurate, it may lead to improper treatment, affecting fracture healing and the patient's recovery. Therefore, after a fracture occurs, medical treatment should be sought as soon as possible to undergo professional medical imaging examinations to ensure the accuracy of fracture positioning and the effectiveness of treatment.

[0006] In the existing clinical fracture positioning, the traditional method is mainly to take X-ray pictures (X pictures) of the spatial position of the bone, and doctors judge the fracture position based on experience. However, the traditional fracture positioning method has the following technical defects:

[0007] First, it requires doctors to have rich clinical experience and be able to quickly find the fracture site and its creases on the X picture. However, not all orthopedic doctors have rich clinical experience. Especially for novice doctors, lacking such experience, they need to spend a long time to determine, consult senior doctors, or refer to materials, etc. This will undoubtedly delay the clinical treatment time for fracture patients.

[0008] Second, as shown in the attached Figure 1 In the two-dimensional picture shown in the X picture, only the frontal fracture creases and sites can be shown. For deep creases, depths, fracture surfaces, etc., they cannot be presented to doctors. Especially for some clinical symptoms with adhesions after fractures, exact three-dimensional fracture data cannot be provided, and it is impossible to better assist doctors in comprehensively understanding the overall situation of the fracture site.

[0009] Thirdly, there are numerous X images, which is inconvenient for subsequent processing and the calling of each X image. Centralized storage and processing will increase the computing pressure on the server. Summary of the Invention

[0010] In order to solve the technical problems existing in the prior art, the present invention provides the following technical solutions:

[0011] On the one hand, a method for spatial positioning of fracture fragments in fracture two-dimensional images based on augmented reality technology is provided. This method is implemented by a fracture fragment spatial positioning device, and the method includes:

[0012] The method includes:

[0013] Perform three-dimensional scanning along the body surface position where the fracture site is located, obtain the corresponding fracture medical X-ray image dataset K and perform distributed storage;

[0014] Analyze the fracture medical X-ray image dataset to obtain the fracture medical film k in each scanning sequence:

[0015] K = ∏ n i=1 SCAN(k i ) I ,

[0016] I represents the projection intensity of the X-ray,

[0017] SCAN represents the scanning function,

[0018] SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting in the i-th scanning sequence under the projection intensity I i ,

[0019] K represents the product of fracture medical films k in a number of scanning sequences i obtained;

[0020] Based on the FDK algorithm, obtain the back-projection images of each fracture medical film k, and obtain the fracture medical three-dimensional image dataset K' composed of each back-projection image, and perform three-dimensional reconstruction on the fracture site based on the fracture medical three-dimensional image dataset K' to generate the corresponding first fracture three-dimensional model;

[0021] Based on the above three-dimensional scanning path, synchronously use the MIP technology to obtain the MIP two-dimensional projection images in each scanning sequence, and obtain the MIP fracture reconstruction image dataset K'' composed of each MIP two-dimensional projection image, and perform three-dimensional reconstruction on the fracture site based on the MIP fracture reconstruction image dataset K'' to generate the corresponding second fracture three-dimensional model;

[0022] Fuse the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrate the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model;

[0023] Overlay the three-dimensional fracture AR model on a preset AR device for three-dimensional display in the AR device to locate and display the three-dimensional model structure of the fracture site.

[0024] As a preferred implementation of the present invention, the three-dimensional scanning along the body surface position where the fracture site is located to obtain a corresponding fracture medical X-ray image dataset K and perform distributed storage includes:

[0025] Preset the three-dimensional scanning path of the fracture site and its corresponding scanning parameters;

[0026] Start the X-ray image scanning, perform three-dimensional scanning along the body surface position where the fracture site is located, orderly obtain fracture medical films k under different scanning sequences, and bind each of the fracture medical films k to the corresponding scanning sequence ID;

[0027] Perform denoising and contrast enhancement processing on the fracture medical film k;

[0028] After the processing is completed, traverse and identify the idle data nodes - datanode(x) in the Hadoop distributed file system;

[0029] Deposit each of the fracture medical films k into the corresponding data node - datanode(x) in sequence according to the scanning sequence, and at the same time write its scanning sequence ID into the name node of the Hadoop distributed file system;

[0030] Through the name node, traverse and extract the fracture medical films k in each of the data nodes - data node(x) according to the scanning sequence ID to obtain the fracture medical X-ray image dataset K.

[0031] As a preferred implementation of the present invention, based on the FDK algorithm, obtain the back-projection images of each of the fracture medical films k, and obtain a fracture medical three-dimensional image dataset K' composed of each of the back-projection images, and perform three-dimensional reconstruction on the fracture site based on the fracture medical three-dimensional image dataset K' to generate a corresponding first three-dimensional fracture model, including:

[0032] Through the name node, orderly read the corresponding fracture medical films k according to the scanning sequence ID;

[0033] Based on the FDK algorithm, perform back-projection processing on the fracture medical film k according to its corresponding scanning sequence to generate the corresponding back-projected image;

[0034] Perform three-dimensional voxel assignment on the back-projected images of all the fracture medical films k respectively to obtain the fracture medical three-dimensional image dataset K' composed of the back-projected images of all the fracture medical films k;

[0035] Import the fracture medical three-dimensional image dataset K' into a preset three-dimensional coordinate system, perform three-dimensional reconstruction of the fracture site to generate the first fracture three-dimensional model;

[0036] After post-processing the first fracture three-dimensional model, store the model file of the first fracture three-dimensional model in the background database.

[0037] As a preferred implementation of the present invention, based on the above three-dimensional scanning path, synchronously use the MIP technology to obtain the MIP two-dimensional projection images under each scanning sequence, and obtain the MIP fracture reconstruction image dataset K'' composed of each of the MIP two-dimensional projection images, including:

[0038] Start MIP scanning according to the preset three-dimensional scanning path and its corresponding scanning parameters;

[0039] Perform fluoroscopic scanning along the body surface position where the fracture site is located, orderly obtain the MIP two-dimensional projection images under different scanning sequences, and bind each of the MIP two-dimensional projection images to the corresponding scanning sequence ID;

[0040] Perform denoising and contrast enhancement processing on the MIP two-dimensional projection images;

[0041] After processing, traverse and identify the idle data nodes - datanode(x) in the Hadoop distributed file system;

[0042] According to the scanning sequence ID, store each of the MIP two-dimensional projection images in the corresponding data node - data node(x), and coexist with the fracture medical film k bound to the scanning sequence ID;

[0043] Through the name node, traverse and extract the MIP two-dimensional projection images in each of the data nodes - data node(x) according to the scanning sequence ID to obtain the MIP fracture reconstruction image dataset K''.

[0044] As a preferred embodiment of the present invention, the three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image dataset K'' to generate a corresponding second three-dimensional fracture model includes:

[0045] Obtain the three-dimensional MIP data of the fracture to be reconstructed: Through the name node, read each corresponding MIP two-dimensional projection map in order according to the scan sequence ID to obtain the MIP fracture reconstruction image dataset K'';

[0046] Traverse and calculate the MIP values of each MIP two-dimensional projection map in the MIP fracture reconstruction image dataset K'' in sequence, and generate a corresponding MIP fracture reconstruction image through projection;

[0047] According to the above steps, project all the MIP two-dimensional projection maps under all scan sequences in a preset three-dimensional space to generate the corresponding second three-dimensional fracture model;

[0048] After post-processing the second three-dimensional fracture model, store the model file of the second three-dimensional fracture model in the background database.

[0049] As a preferred embodiment of the present invention, the fusion of the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model and the integration of the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model includes:

[0050] Read the model file, place the first three-dimensional fracture model and the second three-dimensional fracture model in the same three-dimensional space and perform rendering processing;

[0051] Take the scan vector under a certain scan sequence in the three-dimensional scan path as a reference, and position the first three-dimensional fracture model and the second three-dimensional fracture model on the reference to achieve model overlap and fusion processing under the same reference;

[0052] Perform surface structure analysis on the overlapped and fused model, and trim the first three-dimensional fracture model based on the surface of the second three-dimensional fracture model. After trimming, obtain the three-dimensional fracture model;

[0053] Export the three-dimensional fracture model and input it to the AR background. Perform AR instantiation integration on the three-dimensional fracture model on the AR background to generate the corresponding three-dimensional fracture AR model and save it.

[0054] On the other hand, a fracture fragment spatial positioning device for fracture two-dimensional images based on augmented reality technology is provided. The fracture fragment spatial positioning device for fracture two-dimensional images based on augmented reality technology is used to implement the above-mentioned fracture fragment spatial positioning method for fracture two-dimensional images based on augmented reality technology. The device includes:

[0055] An X-ray image acquisition module, which is used to perform three-dimensional scanning along the body surface position where the fracture site is located, acquire the corresponding fracture medical X-ray image dataset K, and perform distributed storage;

[0056] An analysis module, which is used to analyze the fracture medical X-ray image dataset to obtain the fracture medical film k in each scanning sequence:

[0057] K = ∏ n i=1 SCAN(k i ) I ,

[0058] where I represents the projection intensity of the X-ray,

[0059] SCAN represents the scanning function,

[0060] SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting under the projection intensity I and through the i-th scanning sequence i ,

[0061] K represents the product of the fracture medical films k in a number of scanning sequences i ;

[0062] An X-ray film reconstruction module, which is used to obtain the back-projection images of each of the fracture medical films k based on the FDK algorithm, obtain the fracture medical three-dimensional image dataset K' composed of each of the back-projection images, and perform three-dimensional reconstruction on the fracture site based on the fracture medical three-dimensional image dataset K' to generate the corresponding first fracture three-dimensional model;

[0063] A MIP image acquisition module, which is used to synchronously acquire the MIP two-dimensional projection images in each scanning sequence based on the above three-dimensional scanning path, and obtain the MIP fracture reconstruction image dataset K'' composed of each of the MIP two-dimensional projection images;

[0064] A MIP reconstruction module, which is used to perform three-dimensional reconstruction on the fracture site based on the MIP fracture reconstruction image dataset K'' to generate the corresponding second fracture three-dimensional model;

[0065] An AR background, which is used to fuse the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, integrate the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model, and superimpose the three-dimensional fracture AR model on a preset AR device;

[0066] An AR device, which is used for three-dimensional display and positioning to display the three-dimensional model structure of the fracture site.

[0067] On the other hand, a fracture block spatial positioning device is provided. The fracture block spatial positioning device includes: a processor; a memory, on which computer-readable instructions are stored. When the computer-readable instructions are executed by the processor, any one of the methods in the above-mentioned fracture block spatial positioning method for a fracture two-dimensional image based on augmented reality technology is implemented.

[0068] On the other hand, a computer-readable storage medium is provided. At least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the methods in the above-mentioned fracture block spatial positioning method for a fracture two-dimensional image based on augmented reality technology.

[0069] The beneficial effects brought by the technical solutions provided in the embodiments of the present invention at least include:

[0070] Based on the implementation of the present invention, the present invention performs three-dimensional scanning along the body surface position where the fracture site is located, obtains the corresponding fracture medical X-ray image dataset K and stores it distributively, which is convenient for centralized retrieval of image pictures and distribution processing of image pictures, reducing the computing pressure on the server. Based on the FDK algorithm and the MIP fracture reconstruction technology, corresponding fracture models are respectively constructed and model fusion is performed. Then, using AR augmented reality technology, the fused fracture model is integrated to generate a three-dimensional fracture AR model that displays the fracture site through AR positioning, enabling doctors to visually observe the three-dimensional structure of the fracture on the AR device, thereby obtaining in-depth fracture parameters, deepening the understanding of the fracture, helping doctors locate and diagnose fracture position and other parameters, being able to reduce the requirements for doctors' experience and professionalism, being suitable for clinical use by a large number of doctors, and accelerating the clinical treatment time for fracture patients. Description of the Drawings

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0072] Figure 1 It is a two-dimensional fracture schematic diagram shown in the X-ray picture;

[0073] Figure 2 It is a flowchart of a method for spatially locating fracture fragments in a fracture two-dimensional image based on augmented reality technology provided by an embodiment of the present invention (the formula of k is not shown in step S2);

[0074] Figure 3 It is a block diagram of a device for spatially locating fracture fragments in a fracture two-dimensional image based on augmented reality technology provided by an embodiment of the present invention;

[0075] Figure 4 It is a schematic diagram of the functional architecture of a Hadoop distributed file system provided by an embodiment of the present invention;

[0076] Figure 5 It is a schematic diagram of model location fusion under a benchmark provided by an embodiment of the present invention;

[0077] Figure 6 It is a schematic diagram of the structure of a device for spatially locating fracture fragments provided by an embodiment of the present invention. Detailed implementation manners

[0078] Next, the technical solutions in the present invention will be described with reference to the accompanying drawings.

[0079] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of the word "example" is intended to present concepts in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two.

[0080] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, the meanings they express are the same. "(of)", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, the meanings they express are the same.

[0081] In the embodiments of the present invention, sometimes subscripts such as W 1 may be miswritten as non-subscript forms such as W1. When the difference is not emphasized, the meanings they express are the same.

[0082] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0083] An embodiment of the present invention provides a method for spatially positioning fracture fragments in a fracture two-dimensional image based on augmented reality technology. This method can be implemented by a fracture fragment spatial positioning device, which can be a terminal or a server.

[0084] As Figure 2 and Figure 3 shown in the flowchart of the method for spatially positioning fracture fragments in a fracture two-dimensional image based on augmented reality technology, the processing flow of this method can include the following steps:

[0085] On the one hand, a method for spatially positioning fracture fragments in a fracture two-dimensional image based on augmented reality technology is provided. This method is implemented by a fracture fragment spatial positioning device and includes:

[0086] S1. Perform three-dimensional scanning along the body surface position where the fracture site is located, obtain the corresponding fracture medical X-ray image dataset K, and perform distributed storage;

[0087] S2. Analyze the fracture medical X-ray image dataset to obtain the fracture medical film k in each scan sequence:

[0088] K = ∏ n i=1 SCAN(k i ) I ,

[0089] I represents the projection intensity of the X-ray,

[0090] SCAN represents the scanning function,

[0091] SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting under the projection intensity I in the i-th scan sequence i ,

[0092] K represents the product of fracture medical films k in a number of scan sequences i ;

[0093] S3. Based on the FDK algorithm, obtain the back-projection images of each fracture medical film k, and obtain a fracture medical three-dimensional image dataset K' composed of each back-projection image. Then, perform three-dimensional reconstruction on the fracture site based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model;

[0094] S4. Based on the above three-dimensional scanning path, synchronously adopt the MIP technology to obtain the MIP two-dimensional projection images under each scanning sequence, and obtain the MIP fracture reconstruction image dataset K'' composed of each of the MIP two-dimensional projection images, and perform three-dimensional reconstruction on the fracture site based on the MIP fracture reconstruction image dataset K'', and generate a corresponding second fracture three-dimensional model;

[0095] S5. Integrate the first fracture three-dimensional model and the second fracture three-dimensional model to obtain a fracture three-dimensional model, and integrate the fracture three-dimensional model with AR technology to generate a corresponding three-dimensional fracture AR model;

[0096] S6. Superimpose the three-dimensional fracture AR model on a preset AR device, and perform three-dimensional display on the AR device to locate and display the three-dimensional model structure of the fracture site.

[0097] In this case, the X-ray images are converted into a three-dimensional space through AR technology to assist doctors in positioning.

[0098] Converting X-ray images into a three-dimensional space through AR technology is an innovative and practical application. This technology combines the advantages of augmented reality (AR) and medical imaging, providing doctors, researchers, and patients with a more intuitive and comprehensive way to observe and understand X-ray images.

[0099] First, it is necessary to understand the basic principles of AR technology and X-ray images. AR technology is a technology that can integrate virtual information with the real world. After virtual information such as text, images, and three-dimensional models generated by a computer is simulated, it is applied to the real world to achieve the "enhancement" of the real world. X-ray images, on the other hand, are images obtained after X-rays penetrate human tissues and are subjected to imaging processing, which can show the internal structure and pathological conditions of the human body.

[0100] Combining the two can achieve the three-dimensional space conversion of X-ray images through the following steps:

[0101] 1. X-ray image acquisition and processing: First, obtain the patient's X-ray images and perform necessary preprocessing, such as denoising and contrast enhancement, to improve the image quality.

[0102] 2. Three-dimensional reconstruction: Use computer vision and image processing technologies to perform three-dimensional reconstruction on X-ray images. This usually involves steps such as image segmentation and surface reconstruction to extract three-dimensional structure information from two-dimensional images.

[0103] 3. AR integration: Integrate the reconstructed three-dimensional model with AR technology. This includes aligning the three-dimensional model with the spatial position of the real world for correct display on the AR device.

[0104] The integration of 3D models and AR technology is a forward-looking combination of technologies in the contemporary digital field. This integration not only greatly enriches the way users interact with digital content, but also brings innovative applications to multiple industries.

[0105] A 3D model is a digital object created by a specific software tool. It can be displayed in three dimensions on a computer and can be viewed from any angle. This model can accurately reflect the shape, structure, texture and other characteristics of an object, so it has a wide range of applications in design, manufacturing, visualization and other fields.

[0106] AR technology, or augmented reality technology, is a technology that cleverly integrates virtual information with the real world. It uses computer-generated text, images, three-dimensional models and other virtual information to present it to users through specific display devices (such as AR glasses, smartphones, etc.), allowing users to perceive this virtual information in the real world.

[0107] When 3D models are combined with AR technology, the effect is amazing. This integration makes 3D models no longer limited to display on computer screens or specific devices, but can be directly integrated into the user's real environment. Users can interact with these 3D models in the real world through AR devices to gain a more intuitive and realistic experience.

[0108] This integration has broad application prospects in many fields. For example, in the field of architectural design, designers can use the integration of 3D models and AR technology to present design plans to customers in a more realistic way, helping them better understand and feel the design effects. In the field of education and training, teachers can combine 3D models with AR technology to create a more vivid and intuitive learning environment for students and improve teaching effectiveness. In the field of entertainment and games, this integration can bring users a more immersive experience and enhance the fun and attractiveness of games.

[0109] Of course, to achieve the effective integration of 3D models and AR technology, some technical challenges need to be overcome. For example, with the continuous expansion of progress and application scenarios, it will bring more convenience and innovation to our lives in the future.

[0110] 4. AR display and interaction: Through AR devices, such as AR glasses or smartphones, the 3D model is superimposed on the real world and allows users to interact with it. Doctors or researchers can rotate, scale, move, and perform other operations on the 3D model through gestures, voice, etc., so as to observe and understand the lesion more comprehensively.

[0111] Through this technology, doctors can more intuitively understand the pathological conditions of patients, improving the accuracy and efficiency of diagnosis. At the same time, patients can also better understand their own conditions and communicate more effectively with doctors. In addition, this technology can also be applied to medical education and training to help students and doctors better learn and master medical imaging knowledge.

[0112] Therefore, this solution adopts a three-dimensional scanning method, uses X-rays to obtain three-dimensional scanning data of the fracture site, and uses the FDK algorithm to perform three-dimensional reconstruction on the three-dimensional scanning data to generate a first fracture three-dimensional model of the fracture site generated under X-ray scanning; using the same three-dimensional scanning path, then using the mip scanning technology, and using the MIP two-dimensional projection to reconstruct the second fracture three-dimensional model of the same part. Subsequently, the fracture three-dimensional model of the fracture site is obtained through the fusion of the two fracture three-dimensional models, so that the model constructed by the MIP two-dimensional projection corrects the three-dimensional model generated by scanning the fracture site under X-rays. The three-dimensional model of the fracture site is deeply constructed through the method of model fusion to improve the reconstruction accuracy of the fracture site.

[0113] Using augmented reality technology to generate a virtual simulation model of the corresponding model in the AR background and perform three-dimensional display of the fracture site in the AR device. Therefore, doctors can more intuitively view the three-dimensional shape structure and parameters of the fracture site. Through the AR display, doctors can more intuitively and with a sense of spatial hierarchy understand the three-dimensional morphology of the current patient's fracture site, which is convenient for better formulating diagnostic and treatment plans.

[0114] The three-dimensional scanning of X medical imaging images plays a crucial role in the medical field, as it can provide doctors with detailed three-dimensional information about the internal structures and pathologies of patients. The following is a detailed explanation of the scanning paths, parameters, and methods for three-dimensional scanning of X medical imaging images:

[0115] (I) Scanning Path

[0116] In X medical imaging, the three-dimensional scanning path generally refers to how the scanning device (such as CT or MRI) traverses the patient's body to obtain image data of multiple layers. These paths can be customized according to the scanning site and purpose, but generally follow the following basic principles:

[0117] 1. Continuous layer scanning: The device continuously scans multiple layers along the patient's body axis (such as the sagittal plane, coronal plane, or transverse plane) to obtain complete three-dimensional data.

[0118] 2. Multi-angle scanning: In some cases, in order to obtain more comprehensive information, multi-angle scanning may be required, such as oblique scanning or rotational scanning.

[0119] (II) Scanning Parameters

[0120] Scanning parameters are crucial settings that control how a scanning device operates. They directly affect the quality and resolution of the images. The following are some common scanning parameters:

[0121] 1. Slice thickness: The thickness of each scanned slice, usually measured in millimeters. A smaller slice thickness can achieve higher spatial resolution but will increase the scanning time and radiation dose (for CT).

[0122] 2. Inter-slice gap: The distance between adjacent slices. In some cases, to save time or reduce radiation dose, an inter-slice gap larger than the slice thickness may be chosen.

[0123] 3. Scanning range: The body area covered by the scanning device. This can be adjusted as needed to include or exclude specific anatomical structures.

[0124] 4. Reconstruction algorithm: A mathematical method used to convert the raw scan data into a three-dimensional image. Different reconstruction algorithms may have different effects on the image resolution, contrast, and noise characteristics.

[0125] 5. Tube voltage and tube current (projection intensity): For CT scans, the tube voltage and tube current determine the energy and dose of the X-rays. Higher voltage and current can improve image quality but also increase the radiation dose.

[0126] In X-ray scanning, which set of data is used for scanning can be specifically selected by choosing the corresponding scanning function (performing three-dimensional scanning along the body surface position where the fracture is located, such as the sagittal plane). Specifically, the user can perform the scanning according to various types of scanning functions provided by the system (including the corresponding scanning path, parameters, and scanning methods).

[0127] (III) Scanning methods

[0128] In X-ray medical imaging, there are multiple methods to obtain three-dimensional image data. The most commonly used ones are computed tomography (CT) and magnetic resonance imaging (MRI):

[0129] 1. Computed tomography (CT):

[0130] The human body is scanned using an X-ray beam, and the X-rays passing through the body are received by detectors and converted into electrical signals.

[0131] The electrical signals are converted into digital data through an analog / digital converter and input into a computer for processing.

[0132] The computer uses a reconstruction algorithm to convert the digital data into a three-dimensional image.

[0133] 2. Magnetic resonance imaging (MRI):

[0134] Utilize magnetic fields and radio waves to excite hydrogen atoms in the human body and detect the signals generated by them.

[0135] These signals are converted into digital data and input into a computer for processing.

[0136] The computer uses specific algorithms to convert the digital data into three-dimensional images.

[0137] The three-dimensional scanning path, parameters, and methods of X-ray medical imaging are crucial for ensuring high-quality three-dimensional images. Doctors and technicians need to select appropriate scanning paths, parameters, and methods according to the specific conditions of the patient and the scanning purpose. With the continuous progress of technology, more advanced three-dimensional scanning techniques may emerge in the future to further improve image quality and diagnostic accuracy.

[0138] The principle of the present invention will be further described below.

[0139] As a preferred embodiment of the present invention, performing three-dimensional scanning along the body surface position where the fracture site is located, acquiring the corresponding fracture medical X-ray image dataset K and performing distributed storage, includes:

[0140] Preset the three-dimensional scanning path of the fracture site and its corresponding scanning parameters;

[0141] Start X-ray image scanning, perform three-dimensional scanning along the body surface position where the fracture site is located, orderly acquire fracture medical films k under different scanning sequences, and bind each of the fracture medical films k with the corresponding scanning sequence ID;

[0142] Perform denoising and contrast enhancement processing on the fracture medical films k;

[0143] After processing is completed, traverse and identify the idle data nodes - datanode(x) in the Hadoop distributed file system;

[0144] Deposit each of the fracture medical films k into the corresponding data node - datanode(x) in sequence according to the scanning sequence, and at the same time write its scanning sequence ID into the name node of the Hadoop distributed file system;

[0145] Through the name node, traverse and extract the fracture medical films k in each of the data nodes - data node(x) according to the scanning sequence ID to obtain the fracture medical X-ray image dataset K.

[0146] Such as Figure 3As shown, the three-dimensional scanning path and scanning parameters of the fracture site are set by the user according to the operation method of ordinary X-ray three-dimensional scanning. It is only necessary to perform three-dimensional scanning on the fracture site and obtain the corresponding three-dimensional scanning data. Specifically, the parameters, path setting and operation are carried out on the X-ray imaging scanning system, which will not be elaborated in this solution and will be handled by the user himself.

[0147] The three-dimensional scanning path for the fracture part can be saved and used to guide the subsequent VIP scanning.

[0148] In the three-dimensional scanning sequence, a corresponding fracture medical film will be obtained in each sequence. Therefore, in order to process each fracture medical film, this solution binds the scanning sequence ID with the corresponding obtained fracture medical film, which is convenient for subsequent traversing and retrieving each fracture medical film by means of convenient scanning of virtual IDs, and is used for other film processing steps to improve the film processing efficiency.

[0149] The preprocessing methods for noise reduction and contrast enhancement of the film can refer to the noise reduction and contrast enhancement processing methods on conventional films and can be completed by the medical imaging system.

[0150] In order to avoid the problem that the system is difficult to manage the storage and scheduling use of background three-dimensional data, a Hadoop distributed file system is set up on the background system to store the fracture medical films under each sequence. The name node can manage the storage status of each data node and identify the idle status of the data node. If it is idle, it is used for data storage. Specifically, the use of the Hadoop distributed file system and the method of identifying the data nodes in the empty display state and using them to store the fracture medical films under different scanning sequences can be implemented according to the working principle of the distributed file system, which will not be elaborated in this solution.

[0151] Subsequently, a film request for the corresponding sequence can be sent to the name node, so that the name node retrieves the fracture medical film of the corresponding sequence from the corresponding data node and feeds it back to the subsequent film processing module for the next step of processing.

[0152] Such as Figure 4 The schematic diagram of the architecture of the Hadoop distributed file system shown.

[0153] The nodes in the Hadoop distributed file system (HDFS) are mainly divided into two categories: the master node (Master Node) or the name node (Name Node), and the slave node (Slave Node) or the data node (Data Node). The following is a detailed description of these two types of nodes:

[0154] 1. Master Node or Name Node (Administrator)

[0155] Function: Manages the namespace of the file system, maintains the file system tree, and all files and directories on the entire tree.

[0156] Data Structure:

[0157] FsImage: A snapshot of the file system tree and metadata of all files and folders in the file tree.

[0158] EditLog: Records all operations on files such as creation, deletion, and renaming.

[0159] Record Information: The location information of the data nodes where each block in each file is located (including the number of replicas and user operations on HDFS), but the block location information is not permanently saved. This information will be reconstructed based on the data node information when the system starts.

[0160] Name Node in HA (High Availability) Cluster: Hadoop supports high-availability (HA) configurations, which involve an Active Name Node and a Standby Name Node. When the Active Name Node fails, the Standby Name Node can take over the service. Zookeeper and ZKFC (Failure Detection and Control) are used to ensure the smooth progress of this transition.

[0161] 2. Slave Node or Data Node

[0162] Function: Stores the split data blocks (Blocks) and provides read and write requests from the file system client.

[0163] Communication Mechanism: When starting the DN thread, it reports block information to the NN and maintains contact with it by sending heartbeats through the NN (default 3 seconds). If the NN does not receive the heartbeat from the DN for a long time, it will consider it to have failed and may copy the blocks on it to other DNs.

[0164] Data Block Replication Policy:

[0165] The first replica: Placed in the DN where the file is uploaded; if it is submitted outside the cluster, a node with not-too-full disk and not-too-busy CPU is randomly selected.

[0166] The second replica: Placed on a node in a different rack from the first replica.

[0167] The third replica: On a node in the same rack as the second replica.

[0168] More replicas: Randomly select nodes.

[0169] Data nodes in the HA cluster: In the HA cluster, all Data Nodes must be configured with the addresses of two Name Nodes and send data block location information and heartbeats to both of them.

[0170] In summary, the nodes in the Hadoop distributed file system work together to ensure high availability, fault tolerance, and scalability of data.

[0171] Therefore, the present invention uses the Hadoop distributed file system to store the fracture medical films k obtained by scanning under each scanning sequence. Each fracture medical film k is stored in the idle data node - data node of the Hadoop distributed file system according to the scanning sequence ID, and at the same time, the scanning sequence ID is stored in the name node, facilitating the administrator to read the fracture medical film k stored in the corresponding Data Node according to the scanning sequence ID, which is convenient for subsequent calls and processing of each X - ray image, and avoids the operation pressure on the server caused by centralized storage and processing (otherwise, it will be stored in the background database together, and it is necessary to retrieve the library or use SQL to retrieve and call data, and the calculation is relatively complex and troublesome).

[0172] As a preferred implementation of the present invention, based on the FDK algorithm, back - projection images of each fracture medical film k are obtained, and a fracture medical three - dimensional image dataset K' composed of each back - projection image is obtained, and three - dimensional reconstruction of the fracture site is performed based on the fracture medical three - dimensional image dataset K', generating a corresponding first fracture three - dimensional model, including:

[0173] Through the name node, each corresponding fracture medical film k is read in order according to the scanning sequence ID;

[0174] Based on the FDK algorithm, back - projection processing is performed on the fracture medical film k according to its corresponding scanning sequence to generate the corresponding back - projection image;

[0175] Three - dimensional voxel assignment is performed on the back - projection images of all fracture medical films k respectively to obtain the fracture medical three - dimensional image dataset K' composed of the back - projection images of all fracture medical films k;

[0176] The fracture medical three - dimensional image dataset K' is imported into a preset three - dimensional coordinate system for three - dimensional reconstruction of the fracture site to generate the first fracture three - dimensional model;

[0177] After post-processing the first three-dimensional fracture model, store the model file of the first three-dimensional fracture model in the background database.

[0178] This solution performs three-dimensional reconstruction of the fracture site through the FDK algorithm.

[0179] The FDK algorithm, also known as the Feldkamp-Davis-Kress algorithm, is an algorithm widely used in computer tomography (CT) image reconstruction. Its core principle is an image reconstruction method based on filtered backprojection. During CT scanning, the FDK algorithm first projects the sample to be measured from multiple angles using X-rays to obtain a series of two-dimensional projection data. Subsequently, Fourier transforms and filtering are performed on these projection data to remove low-frequency signals and other noises. Finally, the processed data is converted into an internal structure image of the three-dimensional object through backprojection technology.

[0180] An important feature of the FDK algorithm is that it can approximately regard all cone-beam projection data that does not pass through the geometric center plane as the fan-beam passing through the geometric center plane after being tilted by an angle. This approximate processing enables the FDK algorithm to more accurately restore the internal structure of the object during image reconstruction.

[0181] The FDK algorithm is a commonly used method in computer tomography (CT) image reconstruction, which can recover the structure of a three-dimensional object from a series of two-dimensional projection images. The following are the basic steps for reconstructing a three-dimensional model using the FDK algorithm:

[0182] 1. Projection data acquisition: First, multiple two-dimensional projection images of the object to be reconstructed need to be obtained. These projection images are usually obtained by rotating the object and scanning it using X-rays or other radiation sources.

[0183] 2. Preprocessing: Preprocess the acquired projection data, including removing noise, correcting distortion, etc., to improve the reconstruction quality.

[0184] 3. Fourier transform: Perform a Fourier transform on each projection image to convert it from the spatial domain to the frequency domain. This step helps to apply filtering operations in subsequent steps.

[0185] 4. Filtering: Filter the projection data in the frequency domain. The purpose of filtering is to remove high-frequency noise introduced by the blurring effect during the projection process while retaining low-frequency information to improve the quality of the reconstructed image.

[0186] 5. Inverse Fourier transform: Convert the filtered frequency domain data back to the spatial domain through an inverse Fourier transform to obtain the filtered projection image.

[0187] 6. Back-projection: The filtered projection images are back-projected, that is, they are re-projected back into the three-dimensional space. This step is achieved by rotating each projection image in the opposite direction along its corresponding rotation angle and superimposing them.

[0188] 7. 3D voxel assignment: During the back-projection process, voxels in 3D space are assigned values ​​based on the projection data. The value of each voxel can be determined by comprehensively considering the contribution of all projection images.

[0189] 8. Post-processing: Perform post-processing on the reconstructed 3D model, such as smoothing and segmentation, to further optimize the reconstruction results.

[0190] The steps of 3D reconstruction using the FDK algorithm are as follows:

[0191] 1. Data acquisition: First, a series of two-dimensional image data needs to be acquired through X-ray technology. These images are usually continuous tomographic images that can show the internal structure of the object.

[0192] 2. Data cleaning and processing: After obtaining the original image data, a series of processing tasks are required, including denoising and filtering, to improve the quality and clarity of the image and lay the foundation for subsequent reconstruction work.

[0193] 3. Data reconstruction and merging: This is the core step of 3D reconstruction. By using specific reconstruction algorithms, such as the FDK algorithm (Feldkamp-Davis-Kress algorithm), the 2D image data is converted into a 3D model. At the same time, the 2D images at different angles and levels need to be accurately registered and merged to form a complete 3D structure.

[0194] 4. Structural analysis and visualization: Finally, the reconstructed 3D model is subjected to structural analysis and visualization, which includes rendering, measuring, and cutting the model to more intuitively display the internal structure of the object and provide doctors or researchers with detailed diagnosis and analysis basis.

[0195] Specifically, please combine the FDK steps and the corresponding fracture medical 3D image data set K' to complete the reconstruction and obtain the first fracture 3D model.

[0196] The implementation of the FDK algorithm requires certain conditions to be met, such as the detector and the source must be fixed, the scanned object must spin around the vertical axis, and the scanned object must be completely within the cone beam of the X-ray source. At the same time, the X-ray source must be a point light source, and the reconstruction result is usually represented by a voxel model.

[0197] Specifically, data collection can be completed in combination with the above-mentioned scanning path setting.

[0198] Next, a three-dimensional model of a fracture will be reconstructed in combination with the MIP technology and used for model fusion to improve the accuracy of the three-dimensional fracture model.

[0199] As a preferred embodiment of the present invention, based on the above three-dimensional scanning path, the MIP two-dimensional projection images under each scanning sequence are synchronously acquired by using the MIP technology, and the MIP fracture reconstruction image dataset K'' composed of the MIP two-dimensional projection images is obtained, including:

[0200] Start the MIP scan according to the preset three-dimensional scanning path and its corresponding scanning parameters;

[0201] Perform fluoroscopic scanning along the body surface position where the fracture site is located, orderly acquire the MIP two-dimensional projection images under different scanning sequences, and bind each of the MIP two-dimensional projection images to the corresponding scanning sequence ID;

[0202] Perform denoising and contrast enhancement processing on the MIP two-dimensional projection images;

[0203] After the processing is completed, traverse and identify the idle data nodes - datanode(x) in the Hadoop distributed file system;

[0204] According to the scanning sequence ID, store each of the MIP two-dimensional projection images in the corresponding data node - data node(x), and coexist with the fracture medical film k bound to the scanning sequence ID;

[0205] Through the name node, traverse and extract the MIP two-dimensional projection images in each of the data nodes - data node(x) according to the scanning sequence ID to obtain the MIP fracture reconstruction image dataset K''.

[0206] When acquiring the MIP (maximum intensity projection) two-dimensional projection images under different scanning sequences, the following steps are usually involved:

[0207] 1. Understand the MIP principle:

[0208] MIP is a widely used CT and MR image post-processing technology.

[0209] It generates a two-dimensional image by calculating the maximum density pixels encountered along each ray of the scanned object.

[0210] When the light beam passes through the original image of a section of tissue, the pixels with the maximum density in the image are retained and projected onto a two-dimensional plane to form the MIP reconstruction image.

[0211] 2. Set the scanning sequence:

[0212] Set different scanning sequences according to specific diagnostic requirements. This usually involves selecting parameters such as appropriate slice thickness, scanning range, resolution, etc.

[0213] Before performing MIP reconstruction, ensure that the parameter settings of the scanning sequence can capture the maximum density information of the region of interest.

[0214] 3. MIP reconstruction process:

[0215] In vtk (Visualization Toolkit), MIP reconstruction mainly focuses on the class vtkImageSlabReslice. This class is derived from vtkImageReslice and is used to reslice data.

[0216] Two steps are required to implement MIP:

[0217] Set the direction to slice. Set the reslice axis direction cosines through the SetResliceAxes or SetResliceAxesDirectionCosines method.

[0218] Set the slice thickness, which can be achieved through the SlabThickness parameter.

[0219] The effect of MIP reconstruction can be optimized by adjusting parameters such as SlabResolution (sampling spacing of the thickness) and NumBlendSamplePoints (number of slices).

[0220] 4. Obtain the MIP two-dimensional projection image:

[0221] After completing MIP reconstruction, save or display the obtained two-dimensional projection image.

[0222] According to needs, the MIP image can be further processed and analyzed, such as adjusting window width and window level, performing image fusion, etc.

[0223] By optimizing parameter settings and post-processing, high-quality MIP images can be obtained, providing strong support for clinical diagnosis. In this embodiment, the parameter debugging for obtaining the MIP image of the fracture site through MIP can be completed on-site by medical staff.

[0224] Here, the Hadoop distributed file system is still used to store the MIP image data, so as to facilitate the calling and management of MIP images.

[0225] For each MIP image and fracture medical film k corresponding to a scanning sequence, they can be found correspondingly using the Hadoop distributed file system, which is convenient for management.

[0226] As a preferred implementation of the present invention, performing three-dimensional reconstruction on the fracture site based on the MIP fracture reconstruction image dataset K'', and generating a corresponding second fracture three-dimensional model, including:

[0227] Obtain the three-dimensional MIP data of the fracture to be reconstructed: through the name node, orderly read each corresponding MIP two-dimensional projection map according to the scanning sequence ID, and obtain the MIP fracture reconstruction image dataset K'';

[0228] Traverse and calculate the MIP values of each MIP two-dimensional projection map in the MIP fracture reconstruction image dataset K'' in turn, and generate a corresponding MIP fracture reconstruction image through projection;

[0229] According to the above steps, project all the MIP two-dimensional projection maps under all scanning sequences in a preset three-dimensional space, and generate the corresponding second fracture three-dimensional model;

[0230] After post-processing the second fracture three-dimensional model, store the model file of the second fracture three-dimensional model in the background database.

[0231] MIP uses perspective to obtain a two-dimensional image, that is, it is generated by calculating the maximum density pixels encountered along each ray of the object being scanned. When the fiber bundle passes through the original image of a section of tissue, the pixels with the maximum density in the image are retained and projected onto a two-dimensional plane, thus forming a MIP reconstruction image.

[0232] The steps of MIP reconstruction image are mainly based on the principle of maximum density projection, and generate a two-dimensional image by calculating the maximum density pixels encountered along each ray. The following is an overview of the MIP reconstruction image steps:

[0233] 1. Data preparation: First, it is necessary to obtain the three-dimensional data volume to be reconstructed, which usually comes from medical scanning devices such as CT or MRI. These data volumes contain a series of two-dimensional slice images, and each slice image represents a layer in three-dimensional space.

[0234] 2. Ray casting: Next, from the reconstruction perspective, simulate and project multiple rays. These rays pass through the three-dimensional data volume, and each ray corresponds to a pixel or voxel in the image.

[0235] 3. Density calculation: For each ray, calculate the maximum density value among all the pixels or voxels it passes through. This maximum density value represents the maximum intensity or brightness of the ray in the three-dimensional data volume.

[0236] 4. Projection imaging: Map the maximum density value of each ray onto a two-dimensional plane to form the final MIP reconstruction image. During this process, parameters such as gray levels and contrast can be adjusted as needed to optimize the display effect of the image.

[0237] The MIP reconstruction image has a good display effect for tissues and structures with relatively high density, such as developed blood vessels, bones, etc.

[0238] In addition, to improve the quality and efficiency of MIP reconstruction, methods such as MIP reconstruction based on local maximum average values and using the shearing transformation algorithm to accelerate the reconstruction process can be utilized to solve problems such as fracture breaks and slow reconstruction speed to a certain extent, and improve the accuracy and practicality of the MIP reconstruction image.

[0239] The method of reconstructing a three-dimensional model using MIP two-dimensional projection should be understood in combination with the MIP projection principle.

[0240] Using the MIP (Maximum Intensity Projection) technology to reconstruct a three-dimensional model mainly involves concepts in image processing and computer graphics. The following are the basic steps for reconstructing a three-dimensional model using MIP projection:

[0241] 1. Introduction to MIP technology:

[0242] MIP (Maximum Intensity Projection) is a widely used post-processing technology for CT and MR images. It uses perspective projection to obtain a two-dimensional image, that is, it is generated by calculating the maximum density pixels encountered along each ray of the scanned object.

[0243] When the fiber bundle passes through the original image of a section of tissue, the pixels with the maximum density in the image are retained and projected onto a two-dimensional plane, thus forming the MIP reconstruction image.

[0244] MIP can reflect the X-ray attenuation value of the corresponding pixels, and even small density changes can be displayed on the MIP image. This makes MIP particularly suitable for showing vascular stenosis, dilation, filling defects, and distinguishing calcification on the vascular wall from the contrast agent in the vascular lumen.

[0245] 2. Steps for reconstructing a three-dimensional model using MIP projection:

[0246] Data acquisition: First, it is necessary to obtain the original three-dimensional image data through medical imaging techniques such as CT or MR scans.

[0247] MIP Processing: Then, perform MIP processing on these raw data. This involves calculating and projecting the maximum density pixels on each ray to generate a two-dimensional MIP image.

[0248] Three-dimensional Reconstruction: Next, based on these two-dimensional MIP images, use computer graphics techniques such as volume rendering or surface rendering to reconstruct a three-dimensional model.

[0249] Post-processing and Optimization: Finally, the reconstructed three-dimensional model can be post-processed, such as smoothing, denoising, and refinement, to further improve the quality and visual effect of the model.

[0250] 3. Applications of the Three-dimensional Model Reconstructed by MIP Projection:

[0251] The three-dimensional model reconstructed by MIP projection has a wide range of applications in the medical field, such as fracture reconstruction. These three-dimensional models can provide doctors with more intuitive and accurate diagnostic information, helping doctors better understand the structure and location of the lesions.

[0252] As a preferred implementation of the present invention, fusing the first fracture three-dimensional model and the second fracture three-dimensional model to obtain a fracture three-dimensional model, and integrating the fracture three-dimensional model with AR technology to generate a corresponding three-dimensional fracture AR model includes:

[0253] Read the model file, place the first fracture three-dimensional model and the second fracture three-dimensional model in the same three-dimensional space and perform rendering processing;

[0254] Take the scanning vector in a certain scanning sequence in the three-dimensional scanning path as a reference, and position the first fracture three-dimensional model and the second fracture three-dimensional model on the reference to achieve model overlap and fusion processing under the same reference;

[0255] Perform surface structure analysis on the overlapped and fused model, and trim the first fracture three-dimensional model based on the surface of the second fracture three-dimensional model. After trimming, the fracture three-dimensional model is obtained;

[0256] Export the fracture three-dimensional model and input it into the AR background. Perform AR instantiation integration on the fracture three-dimensional model in the AR background to generate the corresponding three-dimensional fracture AR model and save it.

[0257] The rendering processing of the three-dimensional model can perform model surface rendering operations in the corresponding three-dimensional space, such as the three-dimensional space of some three-dimensional modeling software (UG, SW).

[0258] The scanning vector under the scanning sequence, such as attachedFigure 5 A reference direction shown is a vector with corresponding angles and azimuths (there are also coordinates in space). Here, it can also be a reference point. In the same three-dimensional space, with the origin of the current coordinate system as the reference point (or the Z-axis), two models are overlapped for model fusion. If the model surfaces are inconsistent, the corresponding rendered parts will stand out. Therefore, the model can be further adjusted and corrected according to the fused parts and the unfused parts to obtain the final three-dimensional model.

[0259] The model is corrected by overlapping two three-dimensional models of fractures to generate the three-dimensional fracture model of the final patient to ensure the accuracy and quality of the model. The following are the key steps:

[0260] 1. Model import and alignment:

[0261] First, import the three-dimensional model to be corrected into the corresponding software or platform.

[0262] Use model alignment techniques (such as the ICP algorithm) to align the reference model (usually an accurate or ideal model) with the model to be corrected. This helps to determine the differences and similarities between the two models.

[0263] 2. Overlap analysis:

[0264] Through overlap analysis, identify the areas or features that need to be corrected. This may involve comparing the geometric differences between the two models in the overlapping area, such as shape, size, or position, etc.

[0265] 3. Mesh repair and surface fitting:

[0266] If defects or errors are found in the model during overlap analysis, mesh repair techniques can be used to automatically or semi-automatically detect and repair these problems. For example, use algorithms such as Poisson reconstruction, Marching cubes, etc. to repair holes in the mesh, fill defects, or remove overlaps, etc.

[0267] Surface fitting techniques can be used to convert a set of point cloud data into a continuous and smooth surface model, especially in the case of lack of geometric information. This can be achieved through algorithms such as point cloud fitting, surface fitting, and volume rendering, etc.

[0268] 4. Point cloud filtering:

[0269] When processing three-dimensional models, the point cloud data may contain noise and outliers. Using point cloud filtering techniques (such as Gaussian filtering, median filtering, and adaptive filtering, etc.) can eliminate this noise and improve the accuracy of the data.

[0270] 5. Model splitting and reconstruction:

[0271] If more refined processing of the model is required, model splitting techniques can be used to decompose a large 3D model into small sub-models. This can improve the processability and editability of the model.

[0272] Surface reconstruction techniques can convert discrete point cloud data into a continuous surface model, which is particularly useful for reconstructing surface models from 3D scan data.

[0273] 6. Correction process:

[0274] Based on the results of the overlap analysis and the area to be corrected, use the corresponding tools and techniques to correct the model. This may involve adjusting the shape, size, position, or orientation of the model, etc.

[0275] Multiple iterations and optimizations may be required during the correction process to ensure the accuracy and quality of the model.

[0276] 7. Verification and export:

[0277] After the correction is completed, use verification tools to verify the model to ensure that the model meets the expected accuracy and quality.

[0278] Finally, export the corrected model to the required format for use in subsequent work or projects.

[0279] For the constructed 3D model to be displayed on AR, it is also necessary to perform AR instantiation integration on the fracture 3D model on the AR backend to generate the corresponding 3D fracture AR model.

[0280] The background management software (AR backend) of each AR device is determined by the type of AR device selected by the user, and its backend can be deployed on a backend server.

[0281] Performing AR instantiation integration on the fracture 3D model on the AR backend to generate the corresponding 3D fracture AR model generally involves the following steps:

[0282] 1. Model preparation:

[0283] Obtain the 3D model of the fracture (the corrected 3D model of the patient's fracture has been obtained previously).

[0284] 2. Model optimization:

[0285] According to the performance requirements of the AR application, optimize the 3D model, such as reducing the number of polygons, texture compression, etc., to improve the rendering efficiency on mobile devices or VR / AR devices.

[0286] 3. Import the model into the AR development environment:

[0287] Import the optimized 3D model into the AR development environment (such as Unity, Unreal Engine, etc.).

[0288] Further adjust and test the model in the development environment to ensure its correct display and interaction.

[0289] 4. AR instantiation and integration:

[0290] In the AR development environment, add AR interaction functions to the 3D fracture model. This may include allowing users to interact with the model through gestures, voice, or other input methods.

[0291] Integrate recognition and tracking technologies to identify and track the fracture model in the AR application.

[0292] 5. Testing and debugging:

[0293] Test the AR application on different devices and in different environments to ensure that the 3D fracture model can be correctly displayed and interacted with under different conditions.

[0294] Perform necessary debugging and optimization according to the test results.

[0295] 6. Release and application:

[0296] Once the test is passed and all requirements are met, the AR application can be released to the corresponding platforms (such as App Store, Google Play, etc.).

[0297] In the medical field, such AR applications can be used in multiple aspects such as education, diagnosis, and surgical planning to improve the efficiency and accuracy of medical services.

[0298] Specifically, when in use, the instantiated 3D fracture AR model can be saved in the AR background and interact with the AR device at the front end. When the user requests, the AR background sends the instantiated 3D fracture AR model to the AR device for visual display, and at the same time, interaction can be carried out according to the instantiated scenario.

[0299] The background instantiation of AR is set by the administrator according to the needs of the users. The specific application scenarios and AR interactions are determined according to the interaction scenarios set in the instantiation.

[0300] Through AR positioning to display the 3D fracture AR model of the fracture site, enabling doctors to directly view the three-dimensional structure of the fracture on the AR device, thereby obtaining in-depth fracture parameters, deepening the understanding of the fracture, helping doctors to locate and diagnose fracture position and other parameters, which can reduce the requirements for doctors' experience and professionalism, is suitable for the clinical use of the majority of doctors, and can speed up the clinical treatment time of fracture patients.

[0301] Example 2

[0302] Based on the description of Example 1, on the other hand, a fracture fragment spatial positioning device for fracture two-dimensional images based on augmented reality technology is provided. The fracture fragment spatial positioning device for fracture two-dimensional images based on augmented reality technology is used to implement the above-mentioned fracture fragment spatial positioning method for fracture two-dimensional images based on augmented reality technology. As Figure 3 shown, the device includes:

[0303] An X-ray image acquisition module, which is used to perform three-dimensional scanning along the body surface position where the fracture site is located, acquire the corresponding fracture medical X-ray image dataset K and store it in a distributed manner;

[0304] An analysis module, which is used to analyze the fracture medical X-ray image dataset to obtain the fracture medical film k in each scanning sequence:

[0305] K = ∏ n i=1 SCAN(k i ) I ,

[0306] where I represents the projection intensity of the X-ray,

[0307] SCAN represents the scanning function,

[0308] SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting through the i-th scanning sequence under the projection intensity I, i ,

[0309] K represents the product of the fracture medical films k in a number of scanning sequences, i ;

[0310] An X-ray film reconstruction module, which is used to obtain the back-projection images of each fracture medical film k based on the FDK algorithm, obtain the fracture medical three-dimensional image dataset K' composed of each back-projection image, and perform three-dimensional reconstruction on the fracture site based on the fracture medical three-dimensional image dataset K' to generate the corresponding first fracture three-dimensional model;

[0311] A MIP image acquisition module, which is used to synchronously obtain the MIP two-dimensional projection images in each scanning sequence based on the above three-dimensional scanning path and obtain the MIP fracture reconstruction image dataset K'' composed of each MIP two-dimensional projection image;

[0312] A MIP reconstruction module, which is used to perform three-dimensional reconstruction on the fracture site based on the MIP fracture reconstruction image dataset K'' to generate the corresponding second fracture three-dimensional model;

[0313] The AR background is used to fuse the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, integrate the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model, and superimpose the three-dimensional fracture AR model on a preset AR device;

[0314] The AR device is used for three-dimensional display and positioning to display the three-dimensional model structure of the fracture site.

[0315] For the functions and interactions of the above-mentioned various modules, please refer to the corresponding implementation steps and interaction principles in Embodiment 1 for understanding, and will not be elaborated in this embodiment.

[0316] The composition structure of the device can be combined with the attached Figure 3 as shown.

[0317] Embodiment 3

[0318] Figure 6 is a schematic structural diagram provided by an embodiment of the present invention. As Figure 6 shown, the fracture fragment spatial positioning device may include the above-mentioned Figure 3 fracture fragment spatial positioning device for fracture two-dimensional images based on augmented reality technology. Optionally, the fracture fragment spatial positioning device 410 may include a first processor 2001.

[0319] Optionally, the fracture fragment spatial positioning device 410 may further include a memory 2002 and a transceiver 2003.

[0320] Among them, the first processor 2001, the memory 2002, and the transceiver 2003 may be connected through a communication bus, for example.

[0321] Next, the various components of the fracture fragment spatial positioning device 410 will be specifically introduced in conjunction with Figure 6 :

[0322] Among them, the first processor 2001 is the control center of the fracture fragment spatial positioning device 410, which may be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0323] Optionally, the first processor 2001 may execute various functions of the fracture fragment spatial positioning device 410 by running or executing software programs stored in the memory 2002 and invoking data stored in the memory 2002.

[0324] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 6 the CPU0 and CPU1 shown in

[0325] In a specific implementation, as an embodiment, the fracture fragment spatial positioning device 410 may also include multiple processors, such as Figure 6 the first processor 2001 and the second processor 2004 shown in

[0326] Among them, the memory 2002 is used to store software programs for implementing the solution of the present invention and is controlled by the first processor 2001 for execution. The specific implementation manner may refer to the above method embodiments and will not be elaborated here.

[0327] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and is coupled to the first processor 2001 through an interface circuit ( Figure 6 not shown in

[0328] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0329] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 6 not shown separately). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0330] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and is coupled to the first processor 2001 through an interface circuit ( Figure 6 not shown) of the fracture block space positioning device 410. The embodiments of the present invention do not make specific limitations on this.

[0331] It should be noted that Figure 6 the structure of the fracture block space positioning device 410 shown in

[0332] does not constitute a limitation to the router. The actual knowledge structure recognition device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0333] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0334] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0335] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0336] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0337] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0338] It should be understood that in various embodiments of the present invention, the magnitudes of the serial numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0339] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0340] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the devices, apparatuses, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0341] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0342] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0343] In addition, the functional units in various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0344] If the above-described functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0345] As described above, the above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology, characterized in that: The method comprises: Perform a three-dimensional scan along the body surface where the fracture site is located, obtain the corresponding fracture medical X-ray image dataset K and perform distributed storage; Parse the fracture medical X-ray image dataset to obtain the fracture medical film k in each scanning sequence: K=∏ n i=1 SCAN(k i ) I , I represents the projection intensity of X-rays, SCAN represents the scanning function. SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting the i-th scanning sequence under the projection intensity I i , K represents the fracture medical film k under several scanning sequences i The product is obtained; Based on the FDK algorithm, the back-projected images of each of the fracture medical films k are obtained, and a fracture medical three-dimensional image dataset K' composed of each of the back-projected images is obtained, and the fracture site is three-dimensionally reconstructed based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model; Based on the above three-dimensional scanning path, the MIP technology is synchronously used to obtain the MIP two-dimensional projection images under each scanning sequence, and the MIP fracture reconstruction image data set K'' is obtained from each of the MIP two-dimensional projection images, and the fracture site is three-dimensionally reconstructed based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model; fusing the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrating the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model; The three-dimensional fracture AR model is superimposed on a preset AR device, and three-dimensionally displayed in the AR device to locate and display the three-dimensional model structure of the fracture site.

2. The method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology according to claim 1, characterized in that: The three-dimensional scanning is performed along the body surface position where the fracture site is located, and the corresponding fracture medical X-ray image data set K is obtained and distributedly stored, including: Presetting the three-dimensional scanning path of the fracture site and its corresponding scanning parameters; Start X-ray scanning, perform three-dimensional scanning along the body surface where the fracture site is located, sequentially obtain fracture medical films k under different scanning sequences, and bind each fracture medical film k to the corresponding scanning sequence ID; Performing denoising and contrast enhancement processing on the fracture medical film k; After processing, traverse and identify the idle data nodes in the Hadoop distributed file system - data node (x); Each of the fracture medical films k is stored in the corresponding data node - data node (x) in sequence according to the scanning sequence, and the scanning sequence ID is written into the name node of the Hadoop distributed file system; Through the name node name node, the fracture medical film k in each data node -data node (x) is traversed and extracted according to the scan sequence ID to obtain the fracture medical X-image dataset K.

3. The method for spatial positioning of fracture fragments based on augmented reality fracture two-dimensional images according to claim 2, characterized in that: The method of obtaining the back-projected images of each fracture medical film k based on the FDK algorithm, and obtaining a fracture medical three-dimensional image data set K' composed of each back-projected image, and performing three-dimensional reconstruction of the fracture site based on the fracture medical three-dimensional image data set K' to generate a corresponding first fracture three-dimensional model includes: Through the name node, read the corresponding fracture medical films k in order according to the scan sequence ID; Based on the FDK algorithm, the fracture medical film k is back-projected according to its corresponding scanning sequence to generate the corresponding back-projection image; Performing three-dimensional voxel assignment on the back-projected images of all the fracture medical films k respectively, to obtain the fracture medical three-dimensional image data set K' composed of the back-projected images of all the fracture medical films k; Importing the fracture medical three-dimensional image data set K' into a preset three-dimensional coordinate system, performing three-dimensional reconstruction of the fracture site, and generating the first fracture three-dimensional model; After post-processing the first three-dimensional fracture model, the model file of the first three-dimensional fracture model is stored in a background database.

4. The method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology according to claim 2, characterized in that: Based on the above three-dimensional scanning path, the MIP technology is synchronously used to obtain the MIP two-dimensional projection images under each scanning sequence, and the MIP fracture reconstruction image data set K'' of each of the MIP two-dimensional projection images is obtained, including: Starting the MIP scan according to the pre-set three-dimensional scanning path and its corresponding scanning parameters; Perform perspective scanning along the body surface where the fracture site is located, sequentially obtain MIP two-dimensional projection images under different scanning sequences, and bind each of the MIP two-dimensional projection images to the corresponding scanning sequence ID; Performing denoising and contrast enhancement processing on the MIP two-dimensional projection image; After processing, traverse and identify the idle data nodes in the Hadoop distributed file system - data node (x); According to the scanning sequence ID, each of the MIP two-dimensional projection images is stored in the corresponding data node -datanode(x), and coexists with the fracture medical film k bound to the scanning sequence ID; Through the name node name node, the MIP two-dimensional projection image in each data node -data node (x) is traversed and extracted according to the scan sequence ID to obtain the MIP fracture reconstruction image data set K''.

5. The method for spatial positioning of fracture fragments based on two-dimensional fracture images using augmented reality technology according to claim 4, characterized in that: The three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model includes: Obtaining the fracture three-dimensional MIP data to be reconstructed: through the name node, sequentially reading the corresponding MIP two-dimensional projection images according to the scanning sequence ID, to obtain the MIP fracture reconstruction image data set K''; Sequentially traverse and calculate the MIP values ​​of each of the MIP two-dimensional projection images in the MIP fracture reconstruction image data set K'', and generate corresponding MIP fracture reconstruction images through projection; According to the above steps, the MIP two-dimensional projection images under all scanning sequences are projected in a preset three-dimensional space to generate the corresponding second fracture three-dimensional model; After post-processing the second fracture three-dimensional model, the model file of the second fracture three-dimensional model is stored in a background database.

6. The method for spatial positioning of fracture fragments based on augmented reality fracture two-dimensional images according to claim 2, characterized in that: The fusing the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, and integrating the three-dimensional fracture model with AR technology to generate a corresponding three-dimensional fracture AR model, including: Reading the model file, projecting the first fracture three-dimensional model and the second fracture three-dimensional model into the same three-dimensional space and performing rendering processing; Taking a scanning vector of a scanning sequence in the three-dimensional scanning path as a reference, and positioning the first three-dimensional fracture model and the second three-dimensional fracture model on the reference, so as to achieve model overlap and fusion processing under the same reference; Performing surface structure analysis on the overlapped and fused models, and trimming the first three-dimensional fracture model based on the surface of the second three-dimensional fracture model, to obtain the three-dimensional fracture model after trimming; The fracture three-dimensional model is exported and input into an AR background, and the fracture three-dimensional model is AR instantiated and integrated on the AR background to generate and save the corresponding three-dimensional fracture AR model.

7. A device for spatially locating fracture fragments based on two-dimensional fracture images using augmented reality technology, wherein the device for spatially locating fracture fragments based on two-dimensional fracture images using augmented reality technology is used to implement the method for spatially locating fracture fragments based on two-dimensional fracture images using augmented reality technology as claimed in any one of claims 1 to 6, characterized in that: The device comprises: An X-ray image acquisition module is used to perform a three-dimensional scan along the body surface where the fracture site is located, obtain the corresponding fracture medical X-ray image data set K and perform distributed storage; The parsing module is used to parse the fracture medical X-ray image data set to obtain the fracture medical film k in each scanning sequence: K=∏ n i=1 SCAN(k i ) I , I represents the projection intensity of X-rays, SCAN represents the scanning function. SCAN(k i ) I represents the fracture medical film k obtained by scanning and projecting the i-th scanning sequence under the projection intensity I i , K represents the fracture medical film k under several scanning sequences i The product is obtained; An X-film reconstruction module, for obtaining the back-projected images of each of the fracture medical films k based on the FDK algorithm, and obtaining a fracture medical three-dimensional image dataset K' composed of each of the back-projected images, and performing three-dimensional reconstruction of the fracture site based on the fracture medical three-dimensional image dataset K' to generate a corresponding first fracture three-dimensional model; A MIP image acquisition module is used to synchronously use the MIP technology to acquire MIP two-dimensional projection images under each scanning sequence based on the above three-dimensional scanning path, and obtain a MIP fracture reconstruction image data set K'' from each of the MIP two-dimensional projection images; An MIP reconstruction module is used to perform three-dimensional reconstruction of the fracture site based on the MIP fracture reconstruction image data set K'' to generate a corresponding second fracture three-dimensional model; The AR background is used to fuse the first three-dimensional fracture model and the second three-dimensional fracture model to obtain a three-dimensional fracture model, integrate the three-dimensional fracture model with AR technology, generate a corresponding three-dimensional fracture AR model, and superimpose the three-dimensional fracture AR model on a preset AR device; AR equipment is used for three-dimensional display and positioning and display of the three-dimensional model structure of the fracture site.

8. A bone fragment spatial positioning device, characterized in that: The bone fragment spatial positioning device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images