A method and device for converting CT and MRI images into three-dimensional models

Abstract

A method and device for converting CT and MRI images into three-dimensional models, which relate to the field of medical image processing. In this method, the original image data is read and a CT image or an MRI image is generated according to the original image data, and the contour information of the CT image or the MRI image is extracted; the first target image in the CT image or the MRI image is selected as the reference plane, the position of the first vertex is generated according to the contour information of the first target image, and the position of the second vertex is calculated by interpolating the first target image and the second target image; triangular faces are divided according to the positions of the vertices to obtain triangular face information, the normal information of the vertices is calculated according to the triangular face information, and the two-dimensional texture coordinates of the vertices are calculated according to the positions of the vertices; a three-dimensional mesh model is created according to the positions of the vertices, the normal information, the two-dimensional texture coordinates and the triangular face information. Implementing the technical solution provided by this application provides a more comprehensive and intuitive three-dimensional model display.

Description

Technical Field

[0001] This application relates to the technical field of medical image processing, and particularly relates to a method, system, electronic device, and storage medium for converting CT and MRI images into three-dimensional models. Background Art

[0002] In the medical field, computed tomography (CT) and magnetic resonance imaging (MRI) technologies are widely used in the diagnosis and treatment of diseases.

[0003] However, traditional CT and MRI images are usually two-dimensional. For doctors, understanding and analyzing these two-dimensional images requires rich experience and spatial imagination. Currently, although there are some three-dimensional modeling technologies, when dealing with medical images, they often face problems such as inaccurate data and model distortion. In addition, existing three-dimensional modeling methods are often inefficient when dealing with a large amount of medical image data and cannot meet the real-time requirements of clinical practice.

[0004] Therefore, a method that can accurately and efficiently convert CT and MRI images into three-dimensional models is needed. Summary of the Invention

[0005] This application provides a method and device for converting CT and MRI images into three-dimensional models, which can convert CT and MRI images into high-quality three-dimensional models, enabling doctors and technicians to more accurately understand and analyze the condition.

[0006] In the first aspect of this application, a method for converting CT and MRI images into three-dimensional models is provided, which is applied to a CT and MRI image processing platform. The method includes:

[0007] Reading the original image data and generating a CT image or an MRI image according to the original image data, and extracting the contour information of the CT image or the MRI image;

[0008] Selecting a first target image in the CT image or the MRI image as a reference plane, and generating the position of a first vertex according to the contour information of the first target image, and interpolating and calculating the position of a second vertex according to the first target image and a second target image. The first target image is any one of the CT image or the MRI image, the second target image is the other image in the CT image or the MRI image except the first target image, and the second vertex is the other vertex except the first vertex;

[0009] Dividing triangular faces according to the positions of the vertices and obtaining triangular face information, calculating the normal information of the vertices according to the triangular face information, and calculating the two-dimensional texture coordinates of the vertices according to the positions of the vertices. The vertices include the first vertex and the second vertex;

[0010] Create a three-dimensional mesh model based on the positions of the vertices, normal information, two-dimensional texture coordinates, and the triangular face information.

[0011] By adopting the above technical solution, converting two-dimensional CT and MRI images into three-dimensional models enables doctors to more intuitively observe and analyze the morphology, position, size of the lesion, and its relationship with the surrounding tissues. The provision of this three-dimensional perspective helps doctors make more accurate diagnoses and reduces the risks of misdiagnosis and missed diagnosis. The visualization effect of the three-dimensional model allows doctors to more easily explain the condition and treatment plan to the patients and their families. This intuitive communication method helps enhance the understanding and trust between doctors and patients. In the fields of medical research and teaching, the three-dimensional model provides rich data sources and tools for researchers. By analyzing and comparing the three-dimensional models of different diseases, researchers can deeply explore the pathogenesis and evolution laws of diseases, providing new ideas and methods for the treatment and prevention of diseases. Through steps such as automated contour extraction, vertex calculation, and mesh generation in this application, the efficiency of converting image data into three-dimensional models is significantly improved. This helps reduce the time consumption of doctors in processing image data, enabling them to focus more on the diagnosis and treatment of diseases. By interpolating to calculate the position of the second vertex and calculating the normal information, two-dimensional texture coordinates, etc. of the vertices, this method can generate three-dimensional models with higher details and realism. These models not only help doctors make accurate diagnoses but can also be used in applications such as virtual reality and augmented reality to provide patients with a more immersive treatment experience.

[0012] Optionally, the reading of the original image data and generating a CT image or an MRI image based on the original image data includes:

[0013] Read the data structure storing the original image data, obtain the data information in the header file of the data structure, and use the data information to generate a CT image or an MRI image. The data information includes data length, data image width, data image height, single data sampling length, and color channel type.

[0014] By adopting the above technical solution, reading the data structure storing the original image data, and obtaining the key data information in its header file (such as data length, data image width, height, single data sampling length, and color channel type), the system can efficiently and accurately locate and process the image data. This method avoids blindly traversing the entire data set, thereby improving the efficiency and accuracy of data processing. Using the data information extracted from the data structure, the system can accurately reconstruct CT images or MRI images. These information are crucial for the size, resolution, and color performance of the images, ensuring a high degree of consistency between the generated images and the original data. Since the system can identify and process data of different color channel types, it enables the processing of various types of medical image data, including grayscale images and color images. This flexibility allows the system to be widely applied to different medical image processing scenarios. By reading key information such as data length, the system can process large-scale original image data. This is particularly important for processing high-resolution or long-time series medical image data, helping to maintain the stability of processing speed and performance. The accurately generated CT images or MRI images provide a solid foundation for subsequent processing steps such as three-dimensional model reconstruction, image analysis, and lesion detection. High-quality image input can significantly improve the accuracy and reliability of subsequent processing steps.

[0015] Optionally, the extracting the contour information of the CT image or the MRI image includes:

[0016] Reading eight pixel points around the target pixel point, marking the pixel points with a black color value as a first value, marking the pixel points with a white color value as a second value, calculating the total value of the eight pixel points, and determining whether the total value is equal to a preset value;

[0017] When the total value is not equal to the preset value, defining the target pixel point as an edge pixel point, and determining the contour information of the CT image or the MRI image according to the edge pixel point.

[0018] By adopting the above technical solution, by reading eight pixel points around the target pixel point and making markings and calculations based on the color values (black or white), the present application can accurately identify the edge pixel points in the image. This method of comparing color values based on the local neighborhood is highly sensitive to the common gray-scale changes (such as tissue boundaries) in medical images, which helps to accurately extract contour information. By simply reading eight pixel points around the target pixel point and performing simple numerical calculations and judgments, it can be determined whether the pixel point is an edge pixel point. This local processing method avoids the need for global scanning or complex algorithms, thus improving the calculation efficiency. Since the present application relies on the comparison of color values (black and white), it has a certain degree of adaptability to CT or MRI images obtained under different imaging conditions and devices. As long as there are significant differences in color values between the target area and the background area in the image, the present application can effectively extract contour information.

[0019] Optionally, the calculating the position of the second vertex by interpolating according to the first target image and the second target image includes:

[0020] Using a feature matching algorithm to find matching feature points between the first target image and the second target image, and calculating a transformation matrix from the first target image to the second target image based on the feature points;

[0021] Mapping the position of the first vertex into the second target image according to the transformation matrix to obtain the target position of the first vertex in the second target image, and calculating the position of the second vertex by interpolation according to the target position.

[0022] By adopting the above technical solution, the matching feature points between the first target image and the second target image can be found through the feature matching algorithm, and the transformation relationship (i.e., the transformation matrix) between the two images can be accurately calculated. This transformation calculation based on feature points is more accurate than simple pixel-level transformation and can better reflect the geometric relationship between images. Therefore, when using the transformation matrix to map the first vertex position to the second target image and perform interpolation calculation, a more accurate second vertex position can be obtained, thereby improving the accuracy of the finally generated three-dimensional model. In three-dimensional model reconstruction, the vertex positions between adjacent images need to maintain a certain continuity to ensure a smooth transition on the model surface. Through feature matching and transformation matrix calculation, it can be ensured that the target position of the first vertex in the second target image is geometrically consistent with its position in the first target image, and thus the second vertex position obtained through interpolation calculation can also maintain this continuity. This helps to reduce the unevenness and fracture phenomena on the model surface and improve the visual effect and realism of the model. This application does not depend on specific image content or imaging conditions. As long as there are sufficient matchable feature points between images, the position of the second vertex can be interpolated by calculating the transformation matrix. Therefore, it has strong adaptability and versatility and can be applied to different types of CT or MRI image data. When dealing with scenes containing complex structures or irregular shapes, traditional pixel- or voxel-based methods may be difficult to accurately reconstruct three-dimensional models. However, this application can better handle the geometric transformation relationship between images through feature matching and transformation matrix calculation, thus supporting the three-dimensional model reconstruction of more complex scenes.

[0023] Optionally, the dividing the triangular faces according to the positions of the vertices and obtaining the triangular face information includes:

[0024] For the polygon at the mesh sealing, determine whether the polygon is a simple polygon, where a simple polygon refers to a polygon whose sides do not intersect each other, and the polygon is composed of multiple vertices;

[0025] When the polygon is a simple polygon, determine whether there are ear points in the polygon;

[0026] When there are ear points in the polygon, connect the two vertices adjacent to the ear point, and remove the ear point and the edges adjacent to the ear point from the polygon until the polygon is divided into multiple triangles;

[0027] Determine the triangular face information according to the position information of the three vertices of the triangle.

[0028] By adopting the above technical solution, it is ensured that the polygon at the sealing position is correctly divided into triangles, which can avoid problems such as shape distortion, overlap or voids, thereby improving the overall quality and realism of the mesh. Compared with other polygon division methods (such as dividing into quadrilaterals and then converting to triangles), directly dividing the polygon into triangles based on the vertex positions can simplify the calculation process. Especially by identifying simple polygons and ear points, the number of polygons to be processed can be gradually reduced, thereby improving the execution efficiency of the algorithm. By dividing the polygon into triangles, the topological structure of the mesh can be enhanced. Triangles are the basic units for constructing 3D meshes, and they can be flexibly connected and combined to adapt to various complex shapes and structures. In addition, triangle meshes also have good stability and scalability, facilitating subsequent editing and modification. During 3D modeling and rendering, the data processing of triangle meshes is relatively simple. The three vertices of a triangle can be directly used to define the geometric shape and surface properties (such as color, texture, etc.) of the mesh. In addition, triangle meshes are also convenient for performing advanced graphics processing tasks such as lighting calculation, shadow generation, and collision detection. Compared with other polygon division methods, triangle meshes usually have fewer vertices and edges. This means that less memory space can be occupied when storing and transmitting mesh data.

[0029] Optionally, calculating the normal information of the vertices according to the triangular face information includes:

[0030] Performing a cross product calculation on the positions of the three vertices corresponding to the target triangular face to obtain a normal vector, and processing the normal vector to obtain the normal of the target triangular face, where the target triangular face is any triangular face;

[0031] Determining all triangular faces containing the target vertex, and calculating the average value of the normals of all the triangular faces to obtain the normal of the target vertex, where the target vertex is any vertex.

[0032] By adopting the above technical solution, by calculating the normal vectors of each triangular face and further calculating the normal of each vertex based on these normal vectors, the orientation and curvature of the model surface can be described more accurately. This is crucial for rendering processes such as lighting calculation, shadow generation, and texture mapping, as they all rely on surface normals to determine the lighting direction and reflection / refraction effects. Therefore, this application helps to improve the authenticity and visual quality of model surface rendering. Since the normal of each vertex is calculated based on the average value of the normals of all triangular faces containing that vertex, it can better reflect the geometric details around the vertex. This smoothly transitioning normal information helps to reduce jagged edges and unnatural shading variations during the rendering process, making the model surface appear smoother and more delicate. Although calculating the normal of each vertex requires a certain amount of computational effort, this preprocessing step can be completed when the model is loaded or initialized and reused during subsequent rendering processes. Compared with real-time normal calculation, this application can significantly improve the rendering efficiency, especially when dealing with complex models and large-scale scenes. By providing accurate normal information for each vertex, this method supports the rendering of dynamic lighting and shadow effects. When the position or direction of the light source changes, the lighting intensity and shadow distribution can be calculated in real time based on the normal information of the vertices, thus generating realistic lighting and shadow effects. By directly calculating vertex normals based on triangular face information, this application simplifies the model processing flow. It does not require additional geometric processing steps (such as smoothing, subdivision, etc.) to generate smooth normal information, thereby reducing the processing time and computational resource usage.

[0033] Optionally, the creating of the three-dimensional mesh model according to the vertex position, normal information, two-dimensional texture coordinates, and the triangular face information includes:

[0034] Form a vertex array based on the vertex position, normal information, and two-dimensional texture coordinates, and form an index array based on the triangular face information;

[0035] Use a graphics application programming interface to load the vertex array and the index array into the GPU and configure the rendering state, where the rendering state includes a shader program, texture binding, and vertex attribute pointers;

[0036] Call a rendering command using the indices in the index array to render the mesh according to the rendering state to generate a three-dimensional mesh model.

[0037] By adopting the above technical solutions, vertex positions are the basis of a 3D mesh model, and they directly determine the geometric shape of the model. By precisely defining the positions of each vertex, a highly accurate and realistic 3D model can be created. Normal information is crucial for the lighting and shadow effects on the model surface. They determine how light interacts with the model surface, thus generating realistic lighting effects and shadows. Correct normal information can significantly enhance the realism and detail level of the model. 2D texture coordinates are used to map 2D images (such as texture maps) onto the 3D model surface. By specifying texture coordinates for each vertex, precise texture mapping can be achieved, increasing the detail and richness of the model. By storing vertex data and index data separately and using an index array to specify the connection relationships of vertices, the number of vertices to be processed during the rendering process can be significantly reduced. This data organization method helps improve the rendering efficiency, especially when dealing with complex models with a large number of repeated vertices. Loading vertex arrays and index arrays onto the GPU using graphics application programming interfaces (such as OpenGL, DirectX, etc.) and utilizing the powerful parallel processing capabilities of the GPU for rendering calculations can greatly enhance the rendering speed. By configuring the rendering state (including shader programs, texture bindings, vertex attribute pointers, etc.), fine control over the rendering process can be achieved. This flexibility enables developers to adjust the rendering effects according to needs to meet different visual requirements. By combining multiple 3D mesh models and using an index array to specify the connection relationships between them, a complex scene can be constructed. This ability allows developers to create virtual environments with rich details and high interactivity.

[0038] In the second aspect of the present application, a system for converting CT and MRI images into 3D models is provided, which is characterized by including a contour module, a position module, a calculation module, and a conversion module, where:

[0039] The contour module is configured to read the original image data and generate a CT image or an MRI image based on the original image data, and extract the contour information of the CT image or the MRI image;

[0040] The position module is configured to select a first target image in the CT image or the MRI image as a reference plane, and generate the position of a first vertex based on the contour information of the first target image, and interpolate and calculate the position of a second vertex based on the first target image and a second target image. The first target image is any one of the CT image or the MRI image, the second target image is any other image in the CT image or the MRI image except the first target image, and the second vertex is any other vertex except the first vertex;

[0041] A calculation module, configured to divide triangular faces according to the positions of vertices and obtain triangular face information, calculate the normal information of vertices according to the triangular face information, and calculate the two-dimensional texture coordinates of vertices according to the positions of vertices, where the vertices include the first vertex and the second vertex;

[0042] A conversion module, configured to create a three-dimensional mesh model according to the positions of vertices, normal information, two-dimensional texture coordinates, and the triangular face information.

[0043] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method described in any one of the above.

[0044] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method described in any one of the above is executed.

[0045] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0046] 1. By converting CT and MRI images into three-dimensional models, doctors can observe and analyze the internal structures of patients from multiple angles. This three-dimensional visualization significantly improves the accuracy and reliability of diagnosis;

[0047] 2. Patients can more intuitively understand their own conditions through the three-dimensional model, which helps to enhance their confidence and understanding of the treatment. In addition, the three-dimensional model can also be used for educational purposes to help patients and their families better understand complex medical processes;

[0048] 3. By automatically extracting contour information, calculating vertex positions, dividing triangular faces, etc. from the original image data, the speed and efficiency of data processing are significantly improved. This reduces the need for manual intervention, reduces the risk of human errors, and speeds up the model generation process;

[0049] 4. By extracting the contour information of CT and MRI images and combining interpolation calculations to obtain the positions of the second vertices, the present application can generate three-dimensional models with high realism and detail. At the same time, by calculating the normal information and two-dimensional texture coordinates of vertices, the visual effect and fidelity of the model can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic flowchart of a method for converting CT and MRI images into three-dimensional models disclosed in the embodiments of the present application;

[0051] Figure 2 It is a schematic diagram of the DICOM data structure disclosed in the embodiments of the present application;

[0052] Figure 3 It is a schematic diagram of obtaining image contour data disclosed in the embodiments of the present application;

[0053] Figure 4 It is a schematic diagram of selecting a reference plane and interpolating to generate grid vertex position information disclosed in the embodiments of the present application;

[0054] Figure 5 It is a schematic diagram of the ear cutting method disclosed in the embodiments of the present application;

[0055] Figure 6 It is a schematic diagram of angle inspection disclosed in the embodiments of the present application;

[0056] Figure 7 It is a schematic diagram of internal inspection disclosed in the embodiments of the present application;

[0057] Figure 8 It is a schematic diagram of the final three-dimensional grid model obtained in the embodiments of the present application;

[0058] Figure 9 It is a schematic diagram of the modules of the system for converting CT and MRI images into three-dimensional models disclosed in the embodiments of the present application;

[0059] Figure 10 It is a schematic diagram of the structure of an electronic device disclosed in the embodiments of the present application.

[0060] Explanation of reference numerals: 901, contour module; 902, position module; 903, calculation module; 904, conversion module; 1001, processor; 1002, communication bus; 1003, user interface; 1004, network interface; 1005, memory. Detailed implementation manners

[0061] In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0062] In the description of the embodiments of the present application, words such as "for example" or "for instance" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for instance" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "for example" or "for instance" aims to present relevant concepts in a specific manner.

[0063] In the description of the embodiments of the present application, the term "plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the technical features indicated. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0064] This embodiment discloses a method for converting CT and MRI images into a three-dimensional model, which is applied to a CT and MRI image processing platform. Figure 1 It is a schematic flowchart of the method for converting CT and MRI images into a three-dimensional model disclosed in the embodiments of the present application, as Figure 1 shown, the method includes the following steps:

[0065] S110. Read the original image data, generate a CT image or an MRI image according to the original image data, and extract the contour information of the CT image or the MRI image;

[0066] The original image data usually comes from medical imaging devices such as CT scanners or MRI machines, which store detailed information about the internal structure of the patient's body in digital form. However, before being processed, these data may not be intuitive to humans. Specialized software or hardware interfaces are used to access and load these original image data. These data may be stored in various formats, such as DICOM (Digital Imaging and Communications in Medicine standard), which is a widely used standard format in the field of medical imaging. According to the original image data, complex algorithms are used for image reconstruction to generate visual CT or MRI images. For CT, this usually involves back-projecting multiple X-ray projection data back into three-dimensional space to reconstruct the cross-sectional images of the patient's body internal. For MRI, it involves processing the received radio frequency signals to generate images reflecting tissue characteristics. On the generated CT or MRI images, contour detection is required to extract the boundary information of the region of interest. This is usually achieved through image processing techniques such as edge detection algorithms (such as the Canny edge detector), threshold segmentation, region growing, etc. When the edges or contours are detected, the precise position information of these contours needs to be extracted from the image. This usually involves converting the contours into a series of points or line segments, which precisely describe the shape and position of the contours.

[0067] Optionally, the reading the original image data and generating a CT image or an MRI image according to the original image data includes:

[0068] Read the data structure storing the original image data, obtain the data information in the header file of the data structure, and generate a CT image or an MRI image using the data information. The data information includes data length, data image width, data image height, single data sampling length, and color channel type.

[0069] Access the data structure storing the original image data. In the field of medical imaging, this typically means accessing files that conform to the DICOM (Digital Imaging and Communications in Medicine) standard or other proprietary formats. These files contain a series of metadata (i.e., data about data) and the actual image data. Figure 2 is a schematic diagram of the DICOM data structure disclosed in the embodiments of the present application, as Figure 2 shown, a DICOM file typically contains the following parts:

[0070] Preamble: Located at the beginning of the file, usually 128 bytes, does not contain actual information and can be skipped;

[0071] Prefix: Immediately following the preamble, is a 4-byte string "DICM" used to identify the DICOM file;

[0072] Data elements: Starting from after the prefix until the end of the file, are a collection of data elements, and the data elements are arranged in ascending order according to the Tag (label) value.

[0073] File metadata: Is a set of special data elements used to describe the overall attributes and transfer syntax of the DICOM file. File metadata is usually located at the beginning of the file, but does not appear immediately after the preamble and prefix, but before the normal data elements.

[0074] Each DICOM data element contains the following four fields:

[0075] Tag (label): Is the unique identifier of the data element, composed of a 16-bit group number and a 16-bit element number. For example, (0008, 0020) represents "Study Date". The value of the Tag is predefined in the DICOM standard and is used to identify different data contents.

[0076] VR (Value Representation): Represents the data type and format of the data element value, similar to data types in software (such as int, string). VR can be explicit or implicit, depending on the transfer syntax of the DICOM file. Explicit VR directly includes the VR field in the data element, while implicit VR infers the VR through the data dictionary and Tag.

[0077] Value Length: Represents the byte length of the data element value (Value Field). This is a fixed-length field, usually 2 bytes or 4 bytes, depending on the VR and transfer syntax.

[0078] Value Field: Contains the actual value of the data element, and its length is specified by the Value Length field. The content of the value field varies according to the VR and can be a string, an integer, a floating-point number, binary data, etc.

[0079] In a DICOM file or other files in a similar format, there are usually one or more header parts that contain detailed information about the file content and structure. This information is crucial for correctly parsing and displaying images. The process of reading the header file is the process of extracting this metadata. The data length field represents the total length of the actual image data in the file (in bytes). It helps to determine the position and range of the data in the file. The data image width field represents the number of pixels of the image in the horizontal direction. It is the basis for setting the image width when generating an image. The data image height field represents the number of pixels of the image in the vertical direction. Together with the data image width, these two fields define the resolution of the image. In medical images, each pixel may consist of multiple sampling values (such as grayscale values, RGB values, etc.). The single data sampling length field specifies the number of bytes required for each sampling value (for example, 8 bits, 16 bits, etc.). The color channel type field is used to indicate whether the image contains color channels and the type of color channels (such as RGB, YCbCr, etc.).

[0080] Based on the data information obtained from the header file, the actual image data can be parsed. This usually involves converting the data from the binary format of the file into a format that can be displayed on a computer screen (such as a bitmap). Using the parsed image data and other information provided in the header file (such as image width, height, etc.), a complete image can be constructed. For CT and MRI images, this usually means generating a grayscale image where the brightness value of each pixel corresponds to the sampling value in the original data. When performing image processing, numerical comparison can be used to eliminate some unusable noise data.

[0081] By reading the data information in the header file, it can be ensured that when processing the original image data, the structure, size, and format of the data can be accurately known. This helps to avoid data corruption, loss, or misunderstanding, thus ensuring the integrity and accuracy of the generated CT or MRI images. The acquisition of information such as data length, image width, image height, and single data sampling length enables the image processing software to directly allocate memory, set the image size, and parse the data according to these parameters. This targeted processing method improves the speed and efficiency of image generation. Adhering to the data structure of DICOM or other medical imaging standards enables the generated CT or MRI images to be seamlessly docked and exchanged with other medical information systems. This helps to achieve the sharing and interoperability of medical information, improving the quality and efficiency of medical services. The generated CT or MRI images can not only be directly used for clinical diagnosis and treatment planning but also serve as the basis for subsequent image post-processing and analysis. For example, operations such as filtering, enhancement, and segmentation can be performed on the images to extract more useful information or conduct more in-depth research. Accurately and efficiently generating high-quality CT or MRI images helps doctors to more clearly observe the internal structure and abnormalities of patients, thereby improving the accuracy and efficiency of medical diagnosis. This is of great significance for the early detection, treatment, and rehabilitation of diseases.

[0082] Optionally, the extracting of the contour information of the CT image or the MRI image includes:

[0083] Read the eight pixel points around the target pixel point, mark the pixel points with a black color value as the first numerical value, mark the pixel points with a white color value as the second numerical value, calculate the total numerical value of the eight pixel points, and determine whether the total numerical value is equal to a preset numerical value;

[0084] When the total numerical value is not equal to the preset numerical value, define the target pixel point as an edge pixel point, and determine the contour information of the CT image or the MRI image according to the edge pixel point.

[0085] Figure 3 It is a schematic diagram of obtaining image contour data disclosed in an embodiment of the present application, as Figure 3As shown, for each pixel point (i.e., the target pixel point) in a CT or MRI image, the eight pixel points around it are read. These eight points are usually located above, below, to the left, to the right of the target pixel point, and on the four diagonals. This can be achieved by traversing each pixel in the image and using a coordinate system to access the values of the surrounding pixels. In the simplified scenario of this application, it is assumed that the image has been binarized, where black pixel points represent low gray values (such as 0), and white pixel points represent high gray values (such as 255). The black pixel points can be directly marked as -1, and the white pixel points as 1. The marked values of the eight pixel points around the target pixel point are added together to obtain a sum. If the sum does not meet the preset condition (i.e., is not equal to -8 or 8), the target pixel point can be considered to be on the edge of the image and is defined as an edge pixel point. When the judgment condition is met, a certain attribute of the target pixel point (such as an additional edge marker) is set to true or it is added to the set of edge pixel points. The positions of all pixel points marked as edge pixel points are collected, and this information is used to construct or describe the contour of the image. This may involve connecting the edge pixel points to form continuous lines or boundaries, or using these points to generate a contour map of the image.

[0086] By checking the color values of the eight pixel points around the target pixel point and simplifying them into two numerical values (black as the first value, such as -1; white as the second value, such as 1), this method can efficiently identify the edge regions in the image. Compared with traditional edge detection algorithms, this method is computationally simple, easy to implement, and can quickly process a large amount of data. Since this application determines the edge based on the overall color distribution of the area around the pixel point, it can reduce the influence of image noise on edge detection to a certain extent. When the pixel color distribution in the local area is relatively uniform (i.e., the total value is close to the preset extreme values of -8 or 8), this area is considered a non-edge area, thus avoiding misjudgment caused by individual noise points. By defining the target pixel point as an edge pixel point (when the total value of the surrounding pixel points is not equal to the preset value), this application can accurately outline the contour of the image. These edge pixel points together form the contour information of the image, providing an important basis for subsequent image analysis, processing, and recognition. Although the above description specifically mentions CT images and MRI images, this method is also applicable to other types of medical images and non-medical images. As long as the edges in the image can be identified by differences in color or gray values, this method can effectively extract the contour information. The extracted contour information can be used as the input for subsequent image processing steps, such as image segmentation, feature extraction, and image recognition. These steps usually need to be based on the contour or edge of the image, so accurately extracting the contour information is of great significance for improving the accuracy and efficiency of subsequent processing.

[0087] S120. Select the first target image from the CT image or the MRI image as the reference plane, generate the position of the first vertex according to the contour information of the first target image, and calculate the position of the second vertex by interpolating the first target image and the second target image. The first target image is any one of the CT image or the MRI image, and the second target image is other images in the CT image or the MRI image except the first target image. The second vertex is other vertices except the first vertex;

[0088] From a series of CT images or MRI images, select an image as the reference plane, and this image is called the first target image. The selection can be arbitrary, but usually it is based on factors such as image quality, clarity, and lesion display. Perform contour extraction on the selected first target image, and based on the extracted contour information, generate the initial vertex positions in three-dimensional space. These vertices will serve as the starting points for a three-dimensional model or structure for subsequent shape construction. The remaining CT images or MRI images can all be used as the second target image, and by using the information between the first target image and the second target image, calculate the positions of the remaining vertices through an interpolation method. The interpolation method can be linear, non-linear, or based on a more complex mathematical model. These methods usually consider factors such as the spatial relationship, gray value change, and shape deformation between the two images. Through interpolation calculation, a series of new vertex positions can be obtained, and these vertices together with the first vertex constitute a more complete representation of the three-dimensional model or structure. The positions of these vertices are determined based on the continuity and variability between the two images, so they can more accurately reflect the three-dimensional structure in the original image sequence.

[0089] Optionally, the calculating the position of the second vertex by interpolating the first target image and the second target image includes:

[0090] Use a feature matching algorithm to find matching feature points between the first target image and the second target image, and calculate the transformation matrix from the first target image to the second target image based on the feature points;

[0091] Map the position of the first vertex into the second target image according to the transformation matrix to obtain the target position of the first vertex in the second target image, and calculate the position of the second vertex by interpolation according to the target position.

[0092] In a CT image or an MRI image sequence, although the first target image and the second target image represent adjacent or similar anatomical structures, there may be certain differences between the two images due to factors such as imaging conditions, patient movement, and respiratory movement. In order to perform effective interpolation calculations between these images, it is first necessary to find their common points or matching points, that is, feature points. Feature matching algorithms are an important technique in image processing for finding similar feature structures between two or more images. These feature points are usually points in the image with high contrast, easy to identify, and can appear stably in different images, such as corner points, edge intersection points, texture feature points, etc. In this application, the feature matching algorithm will first detect feature points in the first target image and the second target image respectively, and extract their descriptors (such as feature vectors generated by algorithms like SIFT, SURF, ORB, etc.). Then, the algorithm will compare these descriptors to find the best match between the two sets of feature points. These matched feature point pairs will be used in subsequent steps to calculate the transformation relationship between the images. When the matched feature point pairs are found, they can be used to calculate the transformation matrix from the first target image to the second target image. This transformation matrix describes how to map the points in the first target image to the corresponding positions in the second target image, usually including transformations such as translation, rotation, scaling, and possible deformation. The calculation of the transformation matrix can be achieved through various methods, such as the least squares method, the RANSAC (Random Sample Consensus) algorithm, etc. The RANSAC algorithm is especially suitable for situations with noise and incorrect matching points. It can find the optimal transformation matrix through an iterative method, so that most of the matching points can satisfy this transformation relationship. After obtaining the transformation matrix, the position of the first vertex can be mapped from the first target image to the second target image to obtain the target position of the vertex in the second target image. This mapping process is calculated by substituting the coordinates of the first vertex into the transformation matrix. However, since the purpose of interpolation calculation is usually to generate more vertices to construct a more refined three-dimensional model or structure, simply mapping the position of the first vertex is not enough. Next, according to the mapped target position and the spatial relationship between the first target image and the second target image, interpolation calculation is required to obtain the position of the second vertex. The specific method of interpolation calculation depends on the complexity and accuracy requirements of the required three-dimensional model or structure. A common method is to use linear interpolation or higher-order interpolation methods (such as bilinear interpolation, bicubic interpolation, etc.) to generate a series of interpolation points as the second vertex near the mapped target position. The positions and quantities of these interpolation points can be adjusted according to needs to meet specific modeling requirements. Figure 4 It is a schematic diagram of selecting a reference plane and interpolating to generate grid vertex position information disclosed in an embodiment of the present application.

[0093] Use the feature matching algorithm to find matching feature points between two images. These feature points are usually stable and easily recognizable parts in the images, such as corner points, edges, etc. By matching these feature points, the relative position and pose between the two images, that is, the transformation matrix, can be calculated more accurately. This feature point-based positioning method is more accurate than traditional pixel-level matching, can reduce error accumulation, and improve the accuracy of subsequent vertex calculations. The feature matching algorithm usually has a certain robustness to local changes in the images (such as illumination, noise, small-scale deformation, etc.). This means that even if there are certain differences between the two images, a sufficient number of matching feature points can still be found, so as to calculate a reliable transformation matrix. This robustness makes this method more reliable and stable when dealing with actual medical images. By mapping the position of the first vertex to the second target image through the transformation matrix, the corresponding position (i.e., the target position) of this vertex in the second image can be obtained. Then, based on this target position and the information of the surrounding pixels, the position of the second vertex is calculated through interpolation. This interpolation method can achieve a smooth transition between vertex positions while maintaining image details, avoiding sudden changes or jumps that may be caused by direct calculation. During the 3D reconstruction process, the accurate position of the vertex is crucial for the quality of the reconstruction result. The position of the second vertex calculated by the above method is more accurate and reasonable, and can better reflect the 3D structure information in the original image sequence. This helps to improve the accuracy and fidelity of the 3D reconstruction, providing doctors with more accurate and intuitive diagnostic basis. Although the feature matching and transformation matrix calculation itself may involve relatively complex mathematical operations, once the transformation matrix is obtained, it can be quickly applied to the position mapping and interpolation calculation of multiple vertices. This batch processing method can significantly improve the calculation efficiency and shorten the time of 3D reconstruction.

[0094] S130. Divide the triangular faces according to the positions of the vertices and obtain the triangular face information. Calculate the normal information of the vertices according to the triangular face information. Calculate the 2D texture coordinates of the vertices according to the positions of the vertices. The vertices include the first vertex and the second vertex;

[0095] In a 3D model, vertices are connected by edges to form polygonal faces, and triangular faces are often used as basic building blocks due to their simplicity and good geometric properties. Given the positions of vertices (including the first vertex and the second vertex), these vertices can be divided into triangular faces through specific algorithms (such as the greedy algorithm, Delaunay triangulation, etc.). Triangulation algorithm: Select a suitable algorithm to partition the set of vertices into a series of non-overlapping triangles. These algorithms aim to minimize the shape distortion of the triangles and ensure the quality of the triangulation result. Triangular face information: For each generated triangular face, record the vertex indices it contains (usually the index values in the vertex list), as well as other possible information (such as the normal of the face, area, etc.). This information is crucial for subsequent calculations and rendering processes. The normal of a vertex refers to the vector perpendicular to the average plane of all the triangular faces where the vertex lies. The normal information is crucial for lighting and shadow calculations because it determines the angle between the light and the surface. For each vertex, find all the triangular faces that contain this vertex. Calculate the normal of each triangular face (usually obtained by cross-multiplying two edge vectors). Perform a weighted average of the normals of all the triangular faces that contain this vertex (the weights can be the area of the face or other factors) to obtain the normal of this vertex. Normal normalization: Ensure that the calculated normal vector is a unit vector, that is, its length is 1. Texture coordinates (also known as UV coordinates) are used to map a 2D image (texture) onto the surface of a 3D model. They define the correspondence between each pixel in the texture image and each vertex on the 3D model. Create a 2D texture coordinate system, usually a unit square from (0,0) to (1,1). Based on the relative positions of the vertices on the 3D model (for example, based on a certain parameterization scheme, such as planar unfolding, cylindrical unfolding, etc.), assign a 2D texture coordinate to each vertex. Ensure that the texture coordinates of adjacent vertices change continuously to avoid gaps or distortions during texture mapping. Adjust and optimize the texture coordinates as needed to achieve the best texture mapping effect.

[0096] Optionally, the dividing the triangular faces according to the positions of the vertices and obtaining the triangular face information includes:

[0097] For the polygon at the mesh closure, determine whether the polygon is a simple polygon. A simple polygon refers to a polygon where the edges do not intersect each other, and the polygon is composed of multiple vertices;

[0098] When the polygon is a simple polygon, determine whether there are ear points in the polygon;

[0099] When there are ear points in the polygon, connect the two vertices adjacent to the ear point, and remove the ear point and the edges adjacent to the ear point from the polygon until the polygon is divided into multiple triangles;

[0100] Determine the triangular surface information based on the position information of the three vertices of the triangle.

[0101] An elevation generally refers to the vertical or side parts in a mesh model, which are composed of a series of horizontal or approximately horizontal line segments (or edges) and the vertices between them. For the triangulation of an elevation, the main concern is how to connect the vertices on the elevation into triangles to cover the entire elevation area. This triangulation is relatively simple because the elevation usually has a regular structure, such as a regular grid or continuous lines. The triangulation of an elevation can form triangles by connecting the upper and lower vertices with the next vertex. Sealing refers to the top, bottom, or other edge parts in a mesh model, which may form one or more closed polygons. Sealing is crucial for creating a complete and hole-free model surface. The triangulation of the seal is more complex than that of the elevation because the seal may have irregular shapes and vertex distributions. To triangulate the seal into triangles, more complex algorithms are needed, such as the ear-cutting method (or the "ear clipping" algorithm). The ear-cutting method gradually triangulates a polygon by finding and removing the "ear points" (i.e., convex vertices that meet specific conditions) of the polygon. This process requires careful consideration of the positions of the vertices, the relationships between adjacent edges, and the point distribution inside the polygon.

[0102] Specifically, when dealing with the polygon at the mesh seal, it is first necessary to confirm whether the polygon is a simple polygon. A simple polygon refers to a closed path planar figure composed of a series of edges that do not intersect each other. This step is a prerequisite to ensure that the subsequent triangulation process can proceed correctly. When it is confirmed that the polygon is a simple polygon, the next step is to determine whether there are ear points in the polygon. An ear point is a special vertex whose line segment formed with its two adjacent vertices is completely inside the polygon and does not intersect any other edges of the polygon. The existence of ear points is the key for the ear-cutting method triangulation to proceed. When there are ear points in the polygon, connect the two vertices adjacent to the ear point to form a triangle. Then, remove this triangle from the polygon (i.e., remove the ear point and the adjacent edges), and the remaining part is still a simple polygon, but the number of vertices and edges has decreased. This process is repeated, and each time an ear point is found and removed until the entire polygon is completely divided into multiple triangles. Figure 5 It is a schematic diagram of the ear-cutting method disclosed in the embodiment of the present application.

[0103] To determine whether a vertex is an ear point, two main checks are required:

[0104] (1) Angle check: Ensure that the angle formed by the vertex and its two adjacent vertices is less than 180 degrees. This can be judged by calculating the vectors of two adjacent edges and using the cross product of the vectors to determine whether the angle is acute or right. Figure 6It is a schematic diagram of angle inspection disclosed in an embodiment of the present application.

[0105] (2) Interiority check: Ensure that the triangle formed by this vertex and its two adjacent vertices does not contain other points inside the polygon. This can also be achieved through the cross product of vectors. By checking whether all other points inside the polygon are on one side (usually the left or right side) of this triangle, it can be determined whether they are inside the triangle. Figure 7 It is a schematic diagram of interiority check disclosed in an embodiment of the present application. As Figure 7 shown, the cross product of vectors can be used to determine whether point P is on the left or right side of vector AB. When point P is inside the triangle and in the counterclockwise direction, then point P is always on the left side of vectors AB, BC, and CA; when point P is outside the triangle, then point P must be on the right side of one of the vectors AB, BC, or CA.

[0106] Finally, according to the position information of the three vertices of each triangle obtained by the division, the specific information of the triangular face is determined. This information usually includes the vertex index of the triangle (the position in the mesh vertex list), and may also include the normal of the triangle and other possible attributes (such as color, material, etc.). Triangular face information is an indispensable part of the 3D rendering and processing process, and they define the basic shape and structure of the 3D mesh model.

[0107] By identifying the ear points in a simple polygon and using the ear points for triangulation, this method can efficiently divide the polygon into multiple triangles. Compared with other complex triangulation algorithms, the ear clipping method has a lower computational complexity and a faster execution speed, and is particularly suitable for processing large-scale grid data. Since the ear clipping method strictly follows the definition of a simple polygon and the judgment conditions of ear points, it can ensure that the generated triangles are accurate and effective. These triangles not only cover all regions of the original polygon, but also do not overlap or have gaps with each other, thus ensuring the integrity and accuracy of the grid model. This application is applicable not only to polygons with regular shapes, but also to polygons with complex shapes and irregular boundaries. As long as the polygon is simple (i.e., the sides do not intersect each other), it can be triangulated by the ear clipping method. This flexibility makes the method widely applicable in practical applications. During the three-dimensional modeling and rendering process, the grid model often needs to be modified and adjusted multiple times. Since the ear clipping method triangulates based on the vertex positions, when the vertex positions of the grid model change, this method can be reapplied for triangulation to adapt to the new grid structure. The triangular face information obtained through triangulation is crucial for the three-dimensional rendering process. This information not only defines the basic shape and structure of the grid model, but also affects effects such as lighting, shadow, and texture mapping during the rendering process. Since the ear clipping method can generate a high-quality triangular mesh, it can optimize the rendering performance and improve the realism and efficiency of the rendering results.

[0108] Optionally, the calculating the normal information of the vertex according to the triangular face information includes:

[0109] Performing a cross product calculation on the positions of the three vertices corresponding to the target triangular face to obtain a normal vector, and processing the normal vector to obtain the normal of the target triangular face, where the target triangular face is any triangular face;

[0110] Determining all the triangular faces containing the target vertex, and calculating the average value of the normals of all the triangular faces to obtain the normal of the target vertex, where the target vertex is any vertex.

[0111] For any triangular face (the target triangular face), its normal vector can be obtained by taking the cross product (cross multiplication) of the position vectors of the three vertices corresponding to the triangular face. The result of the cross product is a vector that is perpendicular to the plane formed by these three vertices, i.e., the normal vector of the triangular face. However, the normal vector directly obtained by the cross product may not always point in the desired direction (for example, it may point inside or outside the triangle), so it may be necessary to further process the normal vector (such as taking the inverse) to ensure the consistency of its direction. In a 3D mesh, a vertex may belong to multiple triangular faces. To obtain the normal information of the vertex, it is necessary to consider all the triangular faces that contain the vertex (the target vertex). Specifically, first determine all the triangular faces that contain the target vertex, and then calculate the average value of the normals of these triangular faces. This average vector is the normal vector of the target vertex. By calculating the normals of the vertices, a unified information representing the local surface direction can be provided for each vertex. This is particularly important for lighting calculations because the lighting effect usually depends on the direction of the surface normal. For example, when calculating the illumination intensity of a light source on a certain vertex, it is necessary to know the normal direction of the surface where the vertex is located in order to determine the angle between the light and the surface, and then calculate the correct lighting effect.

[0112] The normal information of vertices plays a crucial role in 3D rendering. They determine the directionality of lighting and shadows, thus affecting the realism of the rendering result. By accurately calculating the normals of each vertex, it can be ensured that the lighting and shadow effects are more realistic and the overall rendering quality is improved. Obtaining the normal vector by taking the cross product of the positions of the three vertices of the target triangular face can accurately reflect the orientation of the triangular face. Obtaining the normal of the vertex by calculating the average value of the normals of all the triangular faces that contain the target vertex takes into account the shape changes in the local area around the vertex, further improving the accuracy of normal calculation. When calculating the vertex normal, using the average value of the normals of all adjacent triangular faces as the vertex normal, this smoothing process helps to reduce jagged edges and unevenness in the rendering. Especially in areas where the surface curvature of the model changes greatly, this method can generate a smoother normal transition and make the rendering result more natural. Although calculating the normals of each vertex involves the normal calculation of multiple triangular faces, this process usually only needs to be carried out once in the preprocessing stage. During the rendering process, these pre-computed normal information can be directly used without repeated calculation. This optimization can significantly improve the rendering performance, especially when dealing with large-scale mesh models. Accurate vertex normal information is crucial for implementing various lighting models. Whether it is simple parallel light, point light source or complex ambient light, global illumination, etc., all rely on the normal information of the vertex to accurately calculate the lighting effect. Therefore, this technical process provides a solid foundation for subsequent lighting processing and rendering effects.

[0113] S140. Create a 3D mesh model based on the vertex positions, normal information, 2D texture coordinates, and the triangular face information.

[0114] Vertex positions are the basis of a 3D mesh model. They define the geometric shape of the model in space. Each vertex is a point in 3D space with three coordinate values: x, y, and z. By connecting these vertices to form edges and faces, the framework of the entire mesh model is ultimately constructed. Normal information is important data that describes the orientation of vertices or triangular faces. It defines the direction of the model surface at a specific point and is crucial for lighting and shadow calculations. Each vertex or triangular face can have a normal vector that is perpendicular to the surface and points outward. By calculating the average of the normals of all adjacent triangular faces around a vertex, smoother vertex normals can be obtained, thereby enhancing the realism of the rendering. 2D texture coordinates (also known as UV coordinates) are used to map a 2D image (such as a texture map) onto the surface of a 3D mesh model. Each vertex is associated with a set of UV coordinates that specify the position of the corresponding pixel in the texture image. By interpolating the UV coordinates between vertices, the texture image can be smoothly covered over the entire model surface during the rendering process, adding detail and realism to the model. Triangular face information defines the basic building blocks of the mesh model - triangles. Each triangle consists of three vertices, and the surface of the mesh model is formed by connecting these vertices. Triangular face information not only includes vertex indices but may also contain other face-related attributes such as color, material, etc. This information is crucial for lighting calculations, texture mapping, and shadow generation during the rendering process. When creating a 3D mesh model, first, the positions of the vertices need to be determined according to the design requirements, and the basic shape of the model is constructed. Then, the normal information of each vertex or triangular face is calculated based on the geometric characteristics of the model surface. Next, a texture map is specified for the model, and the corresponding UV coordinates are assigned to each vertex. Finally, the triangular face information is generated based on the connection relationships of the vertices, and the vertex, normal, texture coordinate, and triangular face information are combined to form the complete mesh model data.

[0115] Optionally, the creating a 3D mesh model based on the vertex positions, normal information, 2D texture coordinates, and the triangular face information includes:

[0116] Form a vertex array based on the vertex positions, normal information, 2D texture coordinates, and form an index array based on the triangular face information;

[0117] Use a graphics application programming interface to load the vertex array and the index array into the GPU and configure the rendering state, where the rendering state includes a shader program, texture binding, and vertex attribute pointers;

[0118] Call a rendering command using the indices in the index array to render the mesh according to the rendering state to generate a 3D mesh model.

[0119] The vertex array is a collection that contains all vertex data. Each vertex data typically includes a position, a normal (for lighting calculations), and two-dimensional texture coordinates (UV coordinates, for texture mapping). This data is stored in a structured manner to facilitate efficient access and processing by the GPU. The index array stores the indices of the triangle vertices that make up the mesh model. These indices point to specific vertices in the vertex array, through which the topology of the mesh can be efficiently defined. Using an index array can significantly reduce the number of duplicate vertices, thereby improving rendering efficiency. Use graphics application programming interfaces (such as the glBufferData function in OpenGL, the ID3D11Buffer::Create function in DirectX, etc.) to load the data of the vertex array and the index array into the GPU's memory. In this way, the GPU can directly access this data during the rendering process without having to read it from the CPU memory every time. The shader program is the program code that runs on the GPU and is used to process vertex data and pixel data. Before rendering a 3D mesh model, it is necessary to configure the vertex shader (to process vertex data) and the fragment shader (to process pixel data). These shader programs define the appearance of the model, lighting effects, texture mapping, etc. If the mesh model needs to use texture mapping, the texture image needs to be bound to the corresponding texture unit before rendering. In this way, during the rendering process, the GPU can sample color values from the texture image according to the UV coordinates of the vertices. The vertex attribute pointer is used to tell the GPU how to read specific vertex data (such as position, normal, texture coordinates, etc.) from the vertex array. These pointers define the format, stride, and offset of the data to ensure that the GPU can correctly parse the data in the vertex array. Use the indices in the index array to call rendering commands (such as the glDrawElements function in OpenGL, the ID3D11DeviceContext::DrawIndexed function in DirectX, etc.) to render the mesh model according to the previously configured rendering state. During the rendering process, the GPU reads vertex data from the vertex array according to the indices in the index array, then processes this data through the shader program, and finally generates and displays the 3D mesh model on the screen. Figure 8 It is a schematic diagram of the 3D mesh model finally obtained in the embodiment of this application.

[0120] By loading the vertex array and index array into the GPU and leveraging the parallel processing power of the GPU for rendering, the rendering performance of 3D mesh models can be significantly improved. The GPU is designed specifically for graphics processing and can handle a large amount of data simultaneously, thus greatly accelerating the rendering speed. The use of the index array reduces the redundant storage of vertex data because the same vertex can be shared by multiple triangles. This means that when loading vertex data into the GPU, less data needs to be transmitted, thereby reducing the demand for memory bandwidth. This is particularly important for processing large-scale mesh models. Through the configuration of rendering states such as shader programs, texture bindings, and vertex attribute pointers, the appearance, lighting effects, and texture mapping of 3D mesh models can be flexibly controlled. This flexibility enables the technology to adapt to different rendering requirements and scenarios and also provides the possibility for future expansion and optimization. Accurate vertex positions and normal information ensure the accuracy of lighting and shadow calculations, thereby enhancing the realism of the rendering results. At the same time, the use of 2D texture coordinates enables complex texture mapping to be applied to the surface of the mesh model, increasing the details and richness of the model. By reasonably configuring rendering states, such as using appropriate shader programs and optimizing the use of texture resources, the utilization of GPU resources can be further optimized. This helps to reduce resource consumption and energy consumption while maintaining high performance.

[0121] Users run the client on Windows. The client runs the algorithm for converting CT or MRI images into 3D models, converting CT and MRI images into high-quality 3D models. The algorithm utilizes the data of CT and MRI images, extracts key features and performs 3D reconstruction to generate complete 3D models. The system for converting CT and MRI images into 3D models includes a front-end web page where users can create new patients and cases, and upload CT / MRI images as well as 3D models and associate them with the cases. During the upload process, the system compresses the 3D models to reduce data transmission and the rendering pressure on the browser. After the upload is completed, users can view the cases and their corresponding CT / MRI images and 3D models. The system also provides a series of tools to help users better observe and operate the 3D models. For example, users can freely rotate the perspective to observe the model from all directions, users can freely move, rotate, and scale the model, and users can also perform operations such as annotating and coloring the 3D models to help doctors better understand and analyze the condition. The system also includes a back-end server for storing CT / MRI images, 3D models, patient and case information, and providing necessary back-end interfaces. The back-end server is responsible for data storage, management, and transmission to ensure the normal operation of the system and data security.

[0122] Further explain the model compression part.

[0123] The model compression part uses C++ to implement the edge-collapse mesh simplification algorithm. This algorithm can be used to reduce the complexity of 3D meshes for more efficient processing in calculations and rendering.

[0124] The implementation of this algorithm is based on the following steps:

[0125] 1. Initialization: Load the original mesh into memory and calculate the initial quadratic equations for each vertex. The quadratic equation for each vertex describes the mesh topology and shape around it, and these quadratic equations are used to calculate the error metric of the vertex. The calculation process of the quadratic equation is as follows: For each vertex v, initialize a zero matrix Q(v) as the quadratic equation of the vertex. Traverse each triangular face Fi adjacent to vertex v. For each face Fi, calculate the normal vector n of the face and the area area of the face. Use the normal vector n and the area area of the face to update the quadratic equation Q(v) of vertex v. First, calculate a symmetric matrix S, which is the transpose of the normal vector n multiplied by itself, and then multiplied by the area area of the face. S = n * n T * area. Next, add the matrix S to the quadratic equation Q(v) of vertex v. Q(v) = Q(v) + S. Repeat this process until all triangular faces adjacent to vertex v have been traversed.

[0126] 2. Priority queue: Create a priority queue that contains all the edges. The priority of each edge is determined by the sum of the error metrics of its two vertices. The error metric of an edge can be obtained by calculating the sum of the quadratic equations of the two vertices.

[0127] 3. Merge vertices: Select the edge with the lowest priority from the priority queue and merge its two vertices. The process of merging vertices includes the following steps:

[0128] a. Update the quadratic equations of adjacent vertices: After merging vertices, it is necessary to update the quadratic equations of its adjacent vertices. This can be achieved by adding the quadratic equations of the adjacent vertices to the quadratic equation of the merged vertex.

[0129] b. Recalculate the error metric of the vertex: After merging vertices, it is necessary to recalculate its error metric. This can be achieved by applying the quadratic equation of the merged vertex to the new position and then calculating the error between the result and the original vertex position.

[0130] Repeat step 3 until the desired level of simplification is reached or the mesh reaches the minimum complexity that can be tolerated.

[0131] After the algorithm is written, convert the C++ code to WebAssembly. The process of converting a C++ program to WebAssembly (Wasm) involves the following steps:

[0132] 1. Install necessary tools: To convert a C++ program to Wasm, you need to install the Emscripten toolchain. Emscripten is a tool for compiling C / C++ code into Wasm and JavaScript. You can download and install the version suitable for your operating system from the official Emscripten website.

[0133] 2. Compile the C++ program into Wasm: Use the em++ command of the Emscripten toolchain to compile the C++ program into Wasm.

[0134] After the above steps, a wasm module that can compress the model can be obtained. Next, when running, you can use JavaScript to load and call the generated Wasm module to compress the model when uploading the model in the web page.

[0135] This embodiment also discloses a system for converting CT and MRI images into a three-dimensional model. Figure 9 It is a schematic diagram of the module for converting CT and MRI images into a three-dimensional model disclosed in the embodiment of the present application. As Figure 9 shown, the system includes a contour module 901, a position module 902, a calculation module 903, and a conversion module 904, where:

[0136] The contour module 901 is configured to read the original image data and generate a CT image or an MRI image according to the original image data, and extract the contour information of the CT image or the MRI image;

[0137] The position module 902 is configured to select a first target image in the CT image or the MRI image as a reference plane, and generate the position of the first vertex according to the contour information of the first target image, and interpolate and calculate the position of the second vertex according to the first target image and the second target image. The first target image is any one of the CT image or the MRI image, the second target image is any other image in the CT image or the MRI image except the first target image, and the second vertex is any other vertex except the first vertex;

[0138] The calculation module 903 is configured to divide triangular faces according to the positions of the vertices and obtain triangular face information, calculate the normal information of the vertices according to the triangular face information, and calculate the two-dimensional texture coordinates of the vertices according to the positions of the vertices. The vertices include the first vertex and the second vertex;

[0139] The conversion module 904 is configured to create a three-dimensional mesh model according to the positions of the vertices, the normal information, the two-dimensional texture coordinates, and the triangular face information.

[0140] Optionally, the contour module 901 is configured to:

[0141] Read the data structure storing the original image data, obtain the data information in the header file of the data structure, and generate a CT image or an MRI image using the data information, where the data information includes data length, data image width, data image height, single data sampling length, and color channel type.

[0142] Optionally, the contour module 901 is configured to:

[0143] Read eight pixel points around the target pixel point, mark the pixel points with a black color value as a first numerical value, mark the pixel points with a white color value as a second numerical value, calculate the total numerical value of the eight pixel points, and determine whether the total numerical value is equal to a preset numerical value;

[0144] When the total numerical value is not equal to the preset numerical value, define the target pixel point as an edge pixel point, and determine the contour information of the CT image or the MRI image based on the edge pixel point.

[0145] Optionally, the position module 902 is configured to:

[0146] Use a feature matching algorithm to find matching feature points between the first target image and the second target image, and calculate a transformation matrix from the first target image to the second target image based on the feature points;

[0147] Map the position of the first vertex into the second target image according to the transformation matrix to obtain the target position of the first vertex in the second target image, and perform interpolation calculation based on the target position to obtain the position of the second vertex.

[0148] Optionally, the calculation module 903 is configured to:

[0149] For the polygon at the grid seal, determine whether the polygon is a simple polygon, where a simple polygon refers to a polygon whose sides do not intersect each other, and the polygon is composed of multiple vertices;

[0150] When the polygon is a simple polygon, determine whether there is an ear point in the polygon;

[0151] When there is an ear point in the polygon, connect the two vertices adjacent to the ear point, and remove the ear point and the edges adjacent to the ear point from the polygon until the polygon is divided into multiple triangles;

[0152] Determine the triangular surface information according to the position information of the three vertices of the triangle.

[0153] Optionally, the computing module 903 is configured to:

[0154] Perform a cross product calculation on the positions of the three vertices corresponding to the target triangular face to obtain a normal vector, and process the normal vector to obtain the normal of the target triangular face, where the target triangular face is any triangular face;

[0155] Determine all the triangular faces containing the target vertex, and calculate the average value of the normals of all these triangular faces to obtain the normal of the target vertex, where the target vertex is any vertex.

[0156] Optionally, the conversion module 904 is configured to:

[0157] Form a vertex array based on the positions of the vertices, normal information, and two-dimensional texture coordinates, and form an index array based on the triangular face information;

[0158] Use the graphics application programming interface to load the vertex array and the index array into the GPU, and configure the rendering state, where the rendering state includes a shader program, texture binding, and vertex attribute pointers;

[0159] Call a rendering command using the indices in the index array to render the mesh according to the rendering state to generate a three-dimensional mesh model.

[0160] It should be noted that: when the device provided in the above embodiment realizes its functions, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process can be seen in the method embodiment, which will not be elaborated here.

[0161] This embodiment also discloses an electronic device. Referring to Figure 10 , the electronic device may include: at least one processor 1001, at least one communication bus 1002, a user interface 1003, a network interface 1004, and at least one memory 1005.

[0162] Among them, the communication bus 1002 is used to realize the connection and communication between these components.

[0163] Among them, the user interface 1003 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface.

[0164] Among them, the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0165] Among them, the processor 1001 may include one or more processing cores. The processor 1001 connects various parts within the entire server through various interfaces and lines, and executes various functions of the server and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling the data stored in the memory 1005. Optionally, the processor 1001 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 1001 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 1001 and may be implemented separately by a single chip.

[0166] Among them, the memory 1005 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store the data involved in the above-mentioned various method embodiments. Optionally, the memory 1005 may also be at least one storage device located far from the aforementioned processor 1001. As Figure 10 shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for converting CT and MRI images into three-dimensional models.

[0167] In Figure 10In the electronic device shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain the data input by the user; while the processor 1001 can be used to call the application program stored in the memory 1005 that converts CT and MRI images into three-dimensional models. When executed by one or more processors 1001, the electronic device executes the method of one or more of the above embodiments.

[0168] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be adopted in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0169] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0170] In the several embodiments provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0171] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] In addition, in each embodiment of this application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0173] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 1005 and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of this application. The aforementioned memory 1005 includes: various media such as USB flash drives, mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0174] The above are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other implementation schemes of the present disclosure. This application aims to cover any variations, uses, or adaptive changes of the present disclosure, and these variations, uses, or adaptive changes follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for converting CT and MRI images into a three-dimensional model, characterized in that: Applied to CT and MRI image processing platforms, the method comprises: Reading original image data and generating a CT image or an MRI image according to the original image data, and extracting contour information of the CT image or the MRI image; Selecting a first target image in the CT image or the MRI image as a reference plane, generating a position of a first vertex according to contour information of the first target image, and calculating a position of a second vertex by interpolation according to the first target image and the second target image, wherein the first target image is any one of the CT image or the MRI image, the second target image is an image other than the first target image in the CT image or the MRI image, and the second vertex is a vertex other than the first vertex; Divide the triangular surface according to the positions of the vertices and obtain triangular surface information, calculate the normal information of the vertices according to the triangular surface information, and calculate the two-dimensional texture coordinates of the vertices according to the positions of the vertices, wherein the vertices include the first vertex and the second vertex; Create a three-dimensional mesh model based on the vertex positions, normal information, two-dimensional texture coordinates and the triangular surface information. The extracting the contour information of the CT image or the MRI image comprises: Read eight pixels around the target pixel, mark the pixel with a black color value as a first value, mark the pixel with a white color value as a second value, calculate the total value of the eight pixels, and determine whether the total value is equal to a preset value; When the total value is not equal to a preset value, the target pixel point is defined as an edge pixel point, and contour information of the CT image or the MRI image is determined according to the edge pixel point.

2. The method for converting CT and MRI images into three-dimensional models according to claim 1, characterized in that: The reading of the original image data and generating a CT image or an MRI image according to the original image data comprises: Read the data structure storing the original image data, obtain the data information in the header file of the data structure, and use the data information to generate a CT image or an MRI image, wherein the data information includes data length, data image width, data image height, single data sampling length and color channel type.

3. The method for converting CT and MRI images into three-dimensional models according to claim 1, characterized in that: The position of the second vertex is calculated by interpolating the first target image and the second target image. include: Using a feature matching algorithm to find matching feature points between the first target image and the second target image, and calculating a transformation matrix from the first target image to the second target image based on the feature points; The position of the first vertex is mapped to the second target image according to the transformation matrix to obtain the target position of the first vertex in the second target image, and the position of the second vertex is obtained by performing interpolation calculation according to the target position.

4. The method for converting CT and MRI images into three-dimensional models according to claim 1, characterized in that: The method of dividing the triangular surface according to the positions of the vertices and obtaining the triangular surface information comprises: For a polygon at the mesh end, determine whether the polygon is a simple polygon, wherein the simple polygon refers to a polygon whose edges do not intersect each other and the polygon is composed of multiple vertices; When the polygon is a simple polygon, determining whether there is an ear point in the polygon; When an ear point exists in the polygon, two vertices adjacent to the ear point are connected, and the ear point and the edge adjacent to the ear point are removed from the polygon until the polygon is divided into a plurality of triangles; The triangle surface information is determined based on the position information of the three vertices of the triangle.

5. The method for converting CT and MRI images into three-dimensional models according to claim 1, characterized in that: The step of calculating the normal information of the vertices according to the triangular surface information includes: Performing a cross product calculation on the positions of three vertices corresponding to a target triangular face to obtain a normal vector, and processing the normal vector to obtain a normal of the target triangular face, where the target triangular face is any triangular face; All triangular faces including a target vertex are determined, and the average values ​​of the normals of all the triangular faces are calculated to obtain the normal of the target vertex, where the target vertex is any vertex.

6. The method for converting CT and MRI images into three-dimensional models according to claim 1, characterized in that: The creating of a three-dimensional mesh model according to the positions of vertices, normal information, two-dimensional texture coordinates and the triangular surface information comprises: Form a vertex array according to the positions of the vertices, the normal information, and the two-dimensional texture coordinates, and form an index array according to the triangle surface information; Using a graphics application program interface, the vertex array and the index array are loaded into a GPU, and a rendering state is configured, wherein the rendering state includes a shader program, a texture binding, and a vertex attribute pointer; The rendering command is called using the index in the index array to render the mesh according to the rendering state to generate a three-dimensional mesh model.

7. A system for converting CT and MRI images into three-dimensional models, characterized in that: It includes contour module, position module, calculation module and conversion module, among which: A contour module, configured to read the original image data and generate a CT image or an MRI image according to the original image data, and extract contour information of the CT image or the MRI image; a position module, configured to select a first target image in the CT image or the MRI image as a reference plane, generate a position of a first vertex according to contour information of the first target image, and calculate a position of a second vertex by interpolation according to the first target image and the second target image, wherein the first target image is any one of the CT image or the MRI image, the second target image is an image other than the first target image in the CT image or the MRI image, and the second vertex is a vertex other than the first vertex; A calculation module, configured to divide the triangular surface according to the positions of the vertices and obtain triangular surface information, calculate the normal information of the vertices according to the triangular surface information, and calculate the two-dimensional texture coordinates of the vertices according to the positions of the vertices, wherein the vertices include the first vertex and the second vertex; A conversion module is configured to create a three-dimensional mesh model according to the positions of the vertices, the normal information, the two-dimensional texture coordinates and the triangular surface information, The extracting the contour information of the CT image or the MRI image comprises: Read eight pixels around the target pixel, mark the pixel with a black color value as a first value, mark the pixel with a white color value as a second value, calculate the total value of the eight pixels, and determine whether the total value is equal to a preset value; When the total value is not equal to a preset value, the target pixel point is defined as an edge pixel point, and contour information of the CT image or the MRI image is determined according to the edge pixel point.

8. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 6 is performed.

Images