A simulated medical image visualization system and method based on MR technology

Abstract

The present invention discloses a simulated medical image visualization system and method based on MR technology, and the present invention relates to the field of medical simulation teaching, including a medical image data acquisition module, a medical image data processing module, a simulated medical image dynamic adjustment module and a device real-time monitoring module. The present invention ensures the quality of medical images used for mixed reality simulation by screening the quality of medical images of pre-visualized medical images, improves the realism of mixed reality scenes, and dynamically adjusts virtual environment light parameters by real-time monitoring of real environment light parameters, making the virtual environment more changeable and flexible. By monitoring the real-time transmission parameters of MR equipment, the stability and immediacy of the user's use are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of medical simulation teaching, and in particular to a simulation medical picture visualization system and method based on MR technology. Background Art

[0002] Traditional medical education relies on textbooks, lectures and limited laboratory practice, while MR technology can provide a more intuitive and interactive learning experience by simulating medical images, helping users better understand and remember complex medical knowledge, helping to improve medical quality, reduce medical errors, and promote continuous progress in the medical field.

[0003] Existing technologies, such as the invention patent with announcement number: CN117789563B, are virtual-reality fusion simulation medical teaching methods and systems based on AI and MR technology, which involve the field of training assistance technology. This virtual-reality fusion simulation medical teaching system based on AI and MR technology uses MR mixed reality technology to combine the virtual human model with the actual space scene and the actual rubber simulator to achieve the effect of combining virtual and real. The user performs real physical operations on the rubber simulator and real-time force feedback. The data is processed through an algorithm and fed back to the virtual human model in real time. The virtual human model can dynamically adjust according to real-time data and interact with the operator, and supports simultaneous operation and interaction of multiple devices and multiple people to achieve a simulated disaster rescue scene. AI intelligent algorithms are introduced to enable the virtual human model to have autonomous learning and decision-making capabilities.

[0004] The prior art, such as the invention patent with announcement number: CN113269995B, is a simulation training system for B-ultrasound-guided puncture peritoneal dialysis catheterization based on MR, which belongs to the field of mixed reality technology and is used to solve the problem that traditional teaching methods cannot enable doctors to quickly and standardizedly master the surgical process of B-ultrasound-guided puncture peritoneal dialysis catheterization. The present invention uses three-dimensional software to create 3D models of human bodies and surgical equipment. After wearing mixed reality glasses (MR), 3D virtual images can be presented in the real space, and gestures can be used for interaction, so that users can simulate and practice the entire B-ultrasound-guided puncture peritoneal dialysis catheterization according to the prompt information in the system, thereby indirectly improving the practical effect of users in actual surgery.

[0005] Based on the above scheme, it can be seen that the existing technology often only focuses on improving the user's interactive experience in the medical virtual environment. In actual applications, since MR presents medical virtual images in the real space, unlike traditional VR, if the virtual generated visualization images are not realistic enough, it will produce a strong sense of separation, seriously affecting the user's experience. In the current medical virtual image construction, usually only direct medical virtual image construction is performed, and the sophistication and detail completion of the medical virtual image are limited by the data source. Therefore, directly constructing the medical virtual image will result in insufficient image accuracy or incomplete details. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a simulated medical image visualization system and method based on MR technology. To achieve the above purpose, the present invention is implemented through the following technical solutions: A simulated medical image visualization system based on MR technology, comprising:

[0007] The medical image data acquisition module acquires the medical image of the pre-visualized medical screen and simultaneously extracts the quality parameters of each frame of the medical image. Based on the quality parameters of each frame of the medical image, the module performs analysis and processing to obtain the quality assessment value of each frame of the medical image and performs comparison to obtain the quality assessment results of each frame of the medical image, wherein the quality assessment results of each frame of the medical image are qualified or unqualified.

[0008] The specific process of extracting the quality parameters of each frame of medical image in the medical image is as follows:

[0009] The medical images of the pre-visualized medical images are segmented according to a preset frame rate, the medical images of each frame of the medical image are obtained, and the quality parameters of each frame of the medical image are extracted, including the resolution, clarity, number of pixels and signal-to-noise ratio of each frame of the medical image.

[0010] The analyzing and processing based on the quality parameters of each frame of medical image to obtain the quality evaluation value of each frame of medical image specifically includes:

[0011] Extracting a set of quality parameter comparison values ​​of medical images of pre-visualized medical images from a database, specifically including: a resolution comparison value, a clarity comparison value, a pixel number comparison value, and a signal-to-noise ratio comparison value;

[0012] Based on the quality parameters of each frame of medical images and the set of quality parameter comparison values ​​of the medical images, the quality assessment value of each frame of medical images is obtained through analysis and processing.

[0013] The comparison to obtain the quality assessment results of each frame of medical images specifically includes:

[0014] The quality assessment value of each frame of medical image is compared with the quality assessment value threshold preset in the database. If the quality assessment value of a frame of medical image is greater than or equal to the quality assessment value threshold, the frame is marked as a qualified frame. If the quality assessment value of a frame of medical image is less than the quality assessment value threshold, the frame is marked as an unqualified frame.

[0015] The medical image data processing module counts the medical image frames with qualified medical image quality assessment results as valid medical images, and imports them into the data processing center for image enhancement. The enhanced valid medical images are segmented, registered and three-dimensionally reconstructed, and the three-dimensional feature parameters of the pre-visualized medical images after three-dimensional reconstruction are extracted to configure the virtual environment for simulated medical image visualization.

[0016] The step of importing the images into the data processing center for image enhancement and performing segmentation, registration and three-dimensional reconstruction on each enhanced valid medical image specifically includes:

[0017] Count the frames of medical images with qualified medical image quality assessment results from each frame of medical image and record them as valid medical images. Extract the quality assessment value of each valid medical image from the quality assessment value of each frame of medical image. Based on the quality assessment value of each valid medical image, map and match the image enhancement parameters corresponding to each quality assessment value interval pre-set by the data processing center to obtain the image enhancement parameters of each valid medical image, and perform image enhancement processing on each valid medical image.

[0018] After segmentation and registration processing is performed on each enhanced valid medical image, the two-dimensional image sequence of each processed valid medical image is extracted and converted into three-dimensional volume data, and three-dimensional reconstruction and image rendering operations are performed.

[0019] The dynamic adjustment module of the simulated medical image monitors the visualized simulated medical image in real time, obtains the real-time light parameters of the virtual environment, and monitors the real-time light parameters of the real environment through optical sensors. Based on the comparison and analysis results of the real-time light parameters of the real environment and the real-time light parameters of the virtual environment, the virtual environment light intensity and RGB values ​​are dynamically adjusted.

[0020] As a preferred technical solution, the analysis process obtains the quality assessment value of each frame of medical image, and the specific analysis conditions are:

[0021] ;

[0022] in, is the quality assessment value of the i-th frame of medical image, is the resolution of the i-th frame of medical image, is the definition of the i-th frame of medical image, is the number of pixels of the i-th frame of medical image, is the signal-to-noise ratio of the i-th frame of medical image, is the resolution reference value of medical images, is the clarity comparison value of medical images, is the pixel number comparison value of medical images, It is the signal-to-noise ratio comparison value of medical images. The softplus function is a built-in function in Pytnon. .

[0023] As a preferred technical solution, the extraction of three-dimensional feature parameters of the pre-visualized medical image after three-dimensional reconstruction configures a virtual environment to simulate the visualization of the medical image, specifically including:

[0024] The three-dimensional characteristic parameters of the pre-visualized medical picture after three-dimensional reconstruction include surface smoothness, three-dimensional model grid density and reconstruction algorithm type.

[0025] Based on the surface smoothness and 3D model mesh density corresponding to each reconstruction algorithm type, matching is performed to screen out the surface smoothness limit value and 3D model mesh density limit value corresponding to the reconstruction algorithm type of the pre-visualized medical image, and matching and comparison are performed with the surface smoothness and 3D model mesh density of the pre-visualized medical image after 3D reconstruction to obtain the 3D feature value of the pre-visualized medical image.

[0026] Based on the three-dimensional feature values ​​of the pre-visualized medical picture, a mapping and matching is performed with a set of virtual environment configuration parameters corresponding to each three-dimensional feature value interval pre-stored in a data processing center to obtain a set of virtual environment configuration parameters of the pre-visualized medical picture and configure the virtual environment accordingly, wherein the set of virtual environment configuration parameters of the pre-visualized medical picture includes an initial viewing angle, an initial zoom ratio, and virtual initial environment lighting parameters of the pre-visualized medical picture.

[0027] As a preferred technical solution, the visualized simulated medical images are monitored in real time to obtain real-time light parameters of the virtual environment, and the real-time light parameters of the real environment are monitored by optical sensors, wherein:

[0028] The virtual environment real-time light parameters include the virtual environment real-time light intensity, the virtual environment real-time color temperature and the virtual environment real-time RGB value.

[0029] The real-time environment light parameters include real-time environment light intensity, real-time environment color temperature and real-time environment RGB value.

[0030] As a preferred technical solution, the virtual environment light intensity and RGB value are dynamically adjusted based on the comparison and analysis results of the real environment real-time light parameters and the virtual environment real-time light parameters. The specific processing conditions are:

[0031] The real-time light parameters of the virtual environment are compared and analyzed with the real-time light parameters of the real environment to obtain the real-time verification value of the ambient light. Based on the real-time verification value of the ambient light, the virtual ambient light intensity adjustment value and the RGB real-time adjustment parameter corresponding to each real-time verification value interval of the ambient light preset by the data processing center are mapped and matched to obtain the virtual ambient light intensity adjustment value and the RGB real-time adjustment parameter of the virtual environment, so as to dynamically adjust the virtual environment.

[0032] The RGB real-time adjustment parameters include an R value adjustment value, a G value adjustment value, and a B value adjustment value.

[0033] As a preferred technical solution, a real-time equipment monitoring module is also included, which is used to count the connected devices used for the simulation of medical image visualization, recorded as MR devices, and to monitor each MR device in real time, specifically including:

[0034] Real-time monitoring of data transmission parameters between MR devices, including real-time transmission bandwidth, real-time transmission flow, real-time transmission delay and real-time transmission packet loss rate of each MR device, analysis to obtain application performance evaluation values ​​of each MR device, and obtaining monitoring results of each MR device based on the application performance evaluation values ​​of each MR device, wherein the monitoring results of each MR device are specifically:

[0035] If the application performance evaluation value of a certain MR device is greater than or equal to the application performance evaluation value threshold, it is determined that the real-time transmission result of the MR device is normal transmission.

[0036] If the application performance evaluation value of a certain MR device is less than the application performance evaluation value threshold, the real-time transmission result of the MR device is determined to be abnormal transmission and an early warning is issued.

[0037] In addition, the present invention also provides a method for visualizing simulated medical images based on MR technology, comprising:

[0038] S1, obtaining a medical image of a pre-visualized medical picture, and synchronously extracting a quality parameter of each frame of the medical image in the medical image, analyzing and processing to obtain a quality assessment value of each frame of the medical image based on the quality parameter of each frame of the medical image, and performing comparison to obtain a quality assessment result of each frame of the medical image, wherein the quality assessment result of each frame of the medical image is qualified or unqualified;

[0039] S2. Count the medical image frames with qualified medical image quality assessment results as valid medical images, import them into the data processing center for image enhancement, segment and register the enhanced valid medical images, and perform three-dimensional reconstruction, extract the three-dimensional feature parameters of the pre-visualized medical images after three-dimensional reconstruction, configure the virtual environment, and simulate medical image visualization;

[0040] S3. Monitor the visualized simulated medical images in real time to obtain the real-time light parameters of the virtual environment, and monitor the real-time light parameters of the real environment through optical sensors. Based on the comparison and analysis results of the real-time light parameters of the real environment and the real-time light parameters of the virtual environment, dynamically adjust the light intensity and RGB value of the virtual environment.

[0041] Compared with the prior art, the embodiments of the present invention have at least the following beneficial effects:

[0042] The present invention provides a simulated medical image visualization system and method based on MR technology, which ensures the quality of medical images used for mixed reality simulation and improves the realism of mixed reality scenes by screening the quality of medical images of pre-visualized medical images. By evaluating the quality of overall and individual medical images respectively, delays caused by image quality problems during MR simulation can be avoided, saving time and medical resources.

[0043] The present invention monitors the real environment light parameters in real time and dynamically adjusts the virtual environment light parameters to match the lighting of the virtual environment with the real environment, thereby improving the sense of immersion. The real-time adjustment can adapt to different ambient light conditions, such as different time periods and actual space conditions, making the virtual environment more varied and flexible.

[0044] The present invention monitors the real-time transmission parameters of the MR device to ensure stability and immediacy during user use, optimize data transmission efficiency and device response speed, more reasonably allocate network and computing resources, improve resource utilization efficiency, and help to quickly diagnose device failures and repair them in a timely manner;

[0045] Of course, any product implementing the present invention does not necessarily need to achieve all of the above advantages at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the description of the present invention, it should be understood that the terms "opening", "upper", "lower", "thickness", "top", "middle", "length", "inside", "all around" and the like indicating orientation or positional relationship are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the components or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0049] See also Figure 1 As shown, the embodiment of the present invention provides a simulated medical image visualization system based on MR technology, which specifically includes:

[0050] The medical image data acquisition module acquires the medical image of the pre-visualized medical screen and simultaneously extracts the quality parameters of each frame of the medical image. Based on the quality parameters of each frame of the medical image, the module performs analysis and processing to obtain the quality assessment value of each frame of the medical image and performs comparison to obtain the quality assessment results of each frame of the medical image, wherein the quality assessment results of each frame of the medical image are qualified or unqualified.

[0051] The specific process of extracting the quality parameters of each frame of medical image in the medical image is as follows:

[0052] Segmenting the medical images of the pre-visualized medical images according to a preset frame rate, obtaining the medical images of each frame of the medical images through an image data processing and analysis tool, and extracting quality parameters of each frame of the medical images, including resolution, clarity, number of pixels, and signal-to-noise ratio of each frame of the medical images;

[0053] Resolution determines the fineness of the image. The higher the resolution, the clearer the image and the more details.

[0054] Clarity evaluates whether the image is clear and easy to identify, which is affected by factors such as resolution, contrast, and noise;

[0055] The number of pixels is the total number of pixels in the image, which is directly related to the resolution. The more pixels there are, the richer the image information and the more complete the details;

[0056] The signal-to-noise ratio is the ratio of the signal intensity to the noise intensity in the image, which reflects the quality of the image. The higher the signal-to-noise ratio, the clearer the image, the less noise, and the less interference with diagnosis.

[0057] The analyzing and processing based on the quality parameters of each frame of medical image to obtain the quality evaluation value of each frame of medical image specifically includes:

[0058] Extracting a set of quality parameter comparison values ​​of medical images of pre-visualized medical images from a database, specifically including: a resolution comparison value, a clarity comparison value, a pixel number comparison value, and a signal-to-noise ratio comparison value;

[0059] Based on the quality parameters of each frame of medical images and the set of quality parameter comparison values ​​of the medical images, the quality assessment value of each frame of medical images is obtained through analysis and processing.

[0060] The analysis process obtains the quality assessment value of each frame of medical image, and the specific analysis conditions are:

[0061] ;

[0062] in, is the quality assessment value of the i-th frame of medical image, is the resolution of the i-th frame of medical image, is the definition of the i-th frame of medical image, is the number of pixels of the i-th frame of medical image, is the signal-to-noise ratio of the i-th frame of medical image, is the resolution reference value of medical images, is the clarity comparison value of medical images, is the pixel number comparison value of medical images, is the signal-to-noise ratio comparison value of the medical image. It should be understood that the softplus function is a built-in function in Pytnon. .

[0063] It should be noted that there is a certain correlation between the parameters of resolution, clarity, number of pixels and signal-to-noise ratio. The higher the resolution, the more pixels the image contains, the clearer the image is, and the richer the details are. Resolution is one of the important factors affecting clarity. The higher the signal-to-noise ratio, the less noise in the image and the higher the clarity. Noise will reduce the clarity of the image and make the image blurry.

[0064] The comparison to obtain the quality assessment results of each frame of medical images specifically includes:

[0065] The quality assessment value of each frame of medical image is compared with the quality assessment value threshold preset in the database. If the quality assessment value of a frame of medical image is greater than or equal to the quality assessment value threshold, the frame is marked as a qualified frame. If the quality assessment value of a frame of medical image is less than the quality assessment value threshold, the frame is marked as an unqualified frame.

[0066] The ratio between the number of qualified medical image frames and the total number of medical image frames is counted and recorded as the qualified ratio value of the medical image. If the proportion of qualified medical image frames to the total number of frames of the medical image is greater than the preset qualified ratio value threshold, the qualified medical image frames of the medical image after eliminating the unqualified medical image frames are imported into the data processing center. If the proportion of qualified medical image frames to the total number of frames of the medical image is less than or equal to the qualified ratio value threshold, the medical image quality is determined to be non-compliant and an early warning is issued.

[0067] The qualified ratio value threshold is processed as follows: in a specific embodiment, the qualified ratio value threshold is set directly in the database during the development of the simulated medical image visualization system involved in the embodiment of the present invention. There are many methods for setting the qualified ratio value threshold, such as obtaining it through statistical analysis, including counting the qualified ratio values ​​under the historical high qualified ratio state, and averaging the qualified ratio values ​​obtained multiple times to obtain the qualified ratio value threshold, or experts judge the boundary between the high qualified ratio value and the low qualified ratio value through the corresponding historical data to set the threshold, thereby adjusting the threshold.

[0068] The medical image data processing module counts the medical image frames with qualified medical image quality assessment results as valid medical images, imports them into the data processing center for image enhancement, performs segmentation, registration and 3D reconstruction on the enhanced valid medical images, extracts the 3D feature parameters of the pre-visualized medical images after 3D reconstruction, configures the virtual environment, and performs simulated medical image visualization;

[0069] Each frame of medical image with a qualified statistical medical image quality assessment result is recorded as a valid medical image, and is imported into a data processing center for image enhancement. Each enhanced valid medical image is segmented, registered and three-dimensionally reconstructed, specifically including:

[0070] Extracting the quality evaluation value of each valid medical image from the quality evaluation value of each frame of medical image, mapping and matching the quality evaluation value of each valid medical image with the image enhancement parameters corresponding to each quality evaluation value interval pre-set by the data processing center, obtaining the image enhancement parameters of each valid medical image, and performing image enhancement processing on each valid medical image;

[0071] The image enhancement parameters include contrast adjustment value, brightness adjustment value, sharpening value and grayscale quantity adjustment value. Based on the image enhancement parameters of each valid medical image, contrast adjustment, brightness adjustment, sharpening and grayscale quantity adjustment are performed on each valid medical image to achieve image enhancement.

[0072] Contrast refers to the difference between the lightest and darkest parts of an image. The contrast adjustment value is a parameter used to change the contrast level in an image. Increasing contrast can make details in an image more distinct, increase the dynamic range of an image, and make the image appear to have more depth and three-dimensionality. Decreasing contrast can make an image appear softer.

[0073] Brightness refers to the overall lightness or darkness of an image. The brightness adjustment value is a parameter used to change the brightness of an image. Increasing brightness can make an image appear brighter, while decreasing brightness makes it darker. Adjusting brightness can help improve the visual effects of an image in different lighting conditions.

[0074] Sharpening is a technique to enhance the edges and details of an image. The sharpening value is a parameter used to control the degree of image sharpening. Through sharpening, the edges and details of an image can be enhanced, making the image look clearer and more focused.

[0075] Grayscale refers to the number of different gray levels in a black and white image. Grayscale adjustment value is a parameter used to change the number of gray levels in an image. Increasing the number of gray levels can improve the dynamic range and detail performance of the image, making the image more delicate.

[0076] After segmentation and registration processing is performed on each enhanced valid medical image, the two-dimensional image sequence of each processed valid medical image is extracted and converted into three-dimensional volume data, and three-dimensional reconstruction and image rendering operations are performed.

[0077] The two-dimensional image sequence of each valid medical image refers to two-dimensional images with the same spatial resolution, different time points or different spatial positions. In an embodiment of the present invention, for example, in a CT scan or an MRI scan, a series of images continuously collected along a specific direction (for example, from the head to the feet) form a two-dimensional image sequence, which is used to construct a three-dimensional image or observe the details of a specific part.

[0078] Three-dimensional reconstruction and image rendering are used to convert a two-dimensional image sequence into a three-dimensional visualization model and present it visually. Among them, three-dimensional reconstruction refers to the process of extracting three-dimensional information from a two-dimensional image sequence and constructing a three-dimensional model. In another embodiment of the invention, a surface reconstruction method is used to extract surface information from three-dimensional volume data to construct a three-dimensional surface model to display the surface morphology of the three-dimensional structure. Image rendering refers to the process of converting a three-dimensional model into a two-dimensional image, such as color rendering, that is, using different colors and lighting effects to make the three-dimensional model look more realistic. Image rendering can make the three-dimensional model more realistic and enhance the visual effect.

[0079] The step of extracting the three-dimensional characteristic parameters of the pre-visualized medical image after three-dimensional reconstruction and configuring a virtual environment to simulate the visualization of the medical image specifically includes:

[0080] The three-dimensional characteristic parameters of the pre-visualized medical image after three-dimensional reconstruction include surface smoothness, three-dimensional model grid density and reconstruction algorithm type;

[0081] Surface smoothness refers to the smoothness of the surface of a 3D model, which reflects the fineness of the model surface details. A higher smoothness can make the model surface look smoother and more realistic, reducing the jagged effect. Smoothing can reduce the noise on the model surface, making the image easier to observe and analyze. A lower smoothness can reduce the number of triangles on the model surface, thereby reducing the amount of calculation and rendering time.

[0082] The mesh density of a 3D model refers to the number and size of triangular units on the surface of a 3D model. The higher the mesh density, the finer the model surface, but the more computational effort required. A higher mesh density can more accurately express the detailed features of the model, and a higher mesh density can improve the visual effect of the model, making it look more realistic. However, a higher mesh density will increase computational effort and rendering time.

[0083] The type of reconstruction algorithm refers to the method used to construct a three-dimensional model from a two-dimensional image sequence. Different types of reconstruction algorithms have different advantages and disadvantages, which will affect the quality and accuracy of the reconstruction results. Different reconstruction algorithms have different computational complexities, which will affect the reconstruction speed and computing resource requirements. The types of reconstruction algorithms include volume reconstruction algorithms, surface reconstruction algorithms, feature reconstruction algorithms, and fusion reconstruction algorithms. For example, a volume reconstruction algorithm (Volume ReconstructionAlgorithm) refers to the process of converting a series of two-dimensional image data into three-dimensional volume data, while a surface reconstruction algorithm (Surface Reconstruction Algorithms) refers to the process of inferring the surface shape of an object from a series of two-dimensional image data through an algorithm.

[0084] Extracting the surface smoothness limit value and the three-dimensional model mesh density limit value of the corresponding reconstruction algorithm type from the database, and performing matching and comparison processing with the three-dimensional feature parameters of the pre-visualized medical image after three-dimensional reconstruction to obtain the three-dimensional feature value of the pre-visualized medical image;

[0085] The processing obtains the three-dimensional feature value of the pre-visualized medical image, and the specific processing conditions are:

[0086] ;

[0087] in, is the three-dimensional feature value of the pre-visualized medical image, To pre-visualize the surface smoothness of medical images, To pre-visualize the 3D model mesh density of medical images, is the surface smoothness limit value corresponding to the reconstruction algorithm type of the pre-visualized medical image, The limit value of the mesh density of the three-dimensional model for the reconstruction algorithm type of pre-visualization medical images.

[0088] It should be noted that there is a certain correlation between the surface smoothness of the pre-visualized medical image and the mesh density of the three-dimensional model, and the two parameters are positively correlated. The higher the surface smoothness of the pre-visualized medical image, the greater the mesh density of the three-dimensional model, and vice versa.

[0089] Based on the three-dimensional feature values ​​of the pre-visualized medical picture, a mapping and matching is performed with a set of virtual environment configuration parameters corresponding to each three-dimensional feature value interval pre-stored in a data processing center to obtain a set of virtual environment configuration parameters of the pre-visualized medical picture and configure the virtual environment accordingly, wherein the set of virtual environment configuration parameters of the pre-visualized medical picture includes an initial viewing angle, an initial zoom ratio, and virtual initial environment lighting parameters of the pre-visualized medical picture.

[0090] The initial viewing angle is set to ensure the best viewing position. The initial viewing angle determines the user's perspective when they first see the medical image in the virtual environment. Selecting a suitable initial viewing angle ensures that key features can be accurately identified and observed when they are first presented. By setting a suitable initial viewing angle, the user's attention can be directed to the most important or most attention-grabbing areas in the image, improving usage efficiency.

[0091] The initial zoom determines the level of detail that users see in the virtual environment. The right zoom ensures that the image is neither too coarse to lose detail nor too detailed to make it difficult to grasp the overall structure when it is first presented. Different screen sizes and resolutions require different zooms to achieve the best display effect. Setting the right initial zoom helps provide a consistent viewing experience across devices. By setting the initial zoom, users can first understand the structure of the image as a whole, and then further zoom in to observe details as needed.

[0092] Virtual initial environment lighting parameters are used to enhance visibility. Appropriate lighting parameters can enhance the contrast and clarity of medical images, making details more obvious. By simulating the lighting conditions in the real environment, users can be provided with an observation environment that is closer to the actual situation. The lighting effect can highlight the key features in the image, helping users to conduct more in-depth analysis.

[0093] In summary, these initial parameters are the basis for the configuration of the medical imaging virtual environment, ensuring that the virtual environment can present medical images in the most appropriate way, allowing users to observe and analyze more effectively. By mapping and matching the pre-stored three-dimensional feature value intervals, these parameters can be automatically adjusted to adapt to different image characteristics and diagnostic requirements.

[0094] The dynamic adjustment module of the simulated medical image monitors the visualized simulated medical image in real time, obtains the real-time light parameters of the virtual environment, and monitors the real-time light parameters of the real environment through optical sensors. Based on the comparison and analysis results of the real-time light parameters of the real environment and the real-time light parameters of the virtual environment, the virtual environment light intensity and RGB values ​​are dynamically adjusted.

[0095] The visualized simulated medical images are monitored in real time to obtain the real-time light parameters of the virtual environment, and the real-time light parameters of the real environment are monitored through optical sensors, where:

[0096] The virtual environment real-time light parameters include the virtual environment real-time light intensity, the virtual environment real-time color temperature, and the virtual environment real-time RGB value;

[0097] The real-time RGB value of the virtual environment includes a real-time R value of the virtual environment, a real-time G value of the virtual environment, and a real-time B value of the virtual environment.

[0098] The real-time light intensity of the virtual environment determines the brightness of the lighting in the virtual environment. It affects the brightness of the scene, as well as the contrast between the bright and shadow parts of the object surface. By adjusting the light intensity, the lighting effect in the virtual environment can be made closer to the real environment, thus providing a more realistic and immersive experience.

[0099] The real-time color temperature of the virtual environment describes the color quality of the light, which determines whether the color of the light is warm (such as yellow, orange) or cool (such as blue). Adjusting the color temperature can change the color presentation of objects in the virtual environment, making them look more natural and realistic under different lighting conditions.

[0100] The real-time RGB value of the virtual environment represents the intensity distribution of light in the three color channels of red, green and blue, which together determine the color of the light. By adjusting the RGB value, the color of the light in the virtual environment can be precisely controlled to match the lighting conditions in a specific environment or to create a specific atmosphere.

[0101] The real-time environment light parameters include real-time environment light intensity, real-time environment color temperature and real-time environment RGB value.

[0102] The real-time RGB value of the real environment includes a real-time R value, a real-time G value and a real-time B value of the real environment.

[0103] The real-time light intensity of the real environment reflects the brightness of the light in the real world and is an important indicator for measuring the level of ambient lighting. By measuring the light intensity of the real environment, a reference can be provided for the virtual environment to ensure that the lighting effect of the virtual environment matches the real environment.

[0104] Real-time color temperature of the real environment describes the color quality of light in the real world and affects people's perception of the environment atmosphere. Knowing the color temperature of the real environment helps to reproduce the same lighting effect in the virtual environment, making the virtual environment more realistic.

[0105] The real-time RGB values ​​of the real environment record the specific values ​​of the real light in the three color channels of red, green and blue, which are used to accurately describe the color of the light. These values ​​can be used as a benchmark for adjusting the lighting of the virtual environment to ensure that the color performance in the virtual environment is consistent with the real environment.

[0106] Based on the comparison and analysis results of the real-time light parameters of the real environment and the real-time light parameters of the virtual environment, the virtual environment light intensity and RGB value are dynamically adjusted. The specific processing conditions are:

[0107] Compare and analyze the real-time light parameters of the virtual environment with the real-time light parameters of the real environment to obtain the real-time verification value of the ambient light, and map and match the virtual ambient light intensity adjustment value and the RGB real-time adjustment parameter corresponding to each ambient light real-time verification value interval preset by the data processing center based on the real-time verification value of the ambient light to obtain the virtual ambient light intensity adjustment value and the RGB real-time adjustment parameter of the virtual environment, so as to dynamically adjust the virtual environment;

[0108] The RGB real-time adjustment parameters include an R value adjustment value, a G value adjustment value, and a B value adjustment value.

[0109] The real-time verification value of the ambient light is specifically processed as follows:

[0110] ;

[0111] in, It is the real-time calibration value of ambient light. is the real-time light intensity of the virtual environment, Real-time color temperature for virtual environments, Real-time R value for the virtual environment, Real-time G value for the virtual environment, is the real-time B value of the virtual environment, The real-time light intensity of the real environment. Real-time color temperature for real environment, It is the real-time R value in the real environment. It is the real-time G value of the real environment. It is the real-time B value in the real environment. is the light intensity weight factor, is the color temperature weight factor, is the RGB value weight factor, and e is a natural constant.

[0112] It should be noted that the light intensity weight factor, color temperature weight factor and RGB value weight factor all have a value range between 0 and 1 and satisfy The light intensity weight factor is an influence factor corresponding to the ambient light real-time verification value preset in the database, indicating the numerical value of the influence degree of light intensity on the ambient light real-time verification value. The color temperature weight factor is an influence factor corresponding to the ambient light real-time verification value preset in the database, indicating the numerical value of the influence degree of color temperature on the ambient light real-time verification value. The RGB value weight factor is an influence factor corresponding to the ambient light real-time verification value preset in the database, indicating the numerical value of the influence degree of RGB value on the ambient light real-time verification value. When used, the light intensity weight factor, color temperature weight factor and RGB value weight factor can be directly obtained from the database, and the corresponding relationship is a pre-set mapping relationship. For example, light intensity, color temperature and RGB value respectively form a mapping set with the corresponding light intensity weight factor, color temperature weight factor and RGB value weight factor. The real-time light intensity, color temperature and RGB value are input into the mapping set to obtain the light intensity weight factor, color temperature weight factor and RGB value weight factor, wherein the mapping relationship is one-to-one correspondence.

[0113] It should also be noted that there is a certain correlation between the real-time light intensity of the virtual environment, the real-time color temperature of the virtual environment, and the real-time RGB value of the virtual environment. Light intensity usually refers to the brightness level of the light, while the RGB value represents the color component of the light. When the light intensity increases, the brightness of the RGB value will also increase accordingly, and vice versa. For example, if the light intensity increases, then RGB (255, 255, 255) will become brighter; if the light intensity decreases, the same RGB value will appear darker. Color temperature describes the color quality of light, which can be simulated by adjusting the RGB value. Light with low color temperature is warmer (for example, more red and yellow), while light with high color temperature is cooler (for example, more blue). Therefore, when the color temperature changes, the RGB value will also be adjusted accordingly. For example, a light source with a low color temperature may have a higher R value (red) and a lower B value (blue). Light intensity and color temperature jointly affect visual perception. The same color temperature will have different visual effects under different light intensities. For example, a high-intensity warm light usually looks very dazzling, while a low-intensity warm light usually looks soft and comfortable. Changing any one parameter may affect the perceived effect of the other parameters. For example, increasing the light intensity may make the cool and warm effects of color temperature more obvious. These three parameters together determine the final effect of lighting in the virtual environment. By adjusting these parameters, different atmospheres can be created.

[0114] The simulated medical image visualization system based on MR technology in the embodiment of the present invention further includes counting the connected devices, which are recorded as MR devices, and performing real-time monitoring on the MR devices, specifically including:

[0115] The real-time data transmission parameters between the MR devices are monitored in real time through the network flow meter, including the real-time transmission bandwidth, real-time transmission flow, real-time transmission delay and real-time transmission packet loss rate of each MR device.

[0116] Each MR device includes a head-mounted display, a tracking sensor, an input acquisition device, a computing platform, a wireless network connection module and an optical sensor.

[0117] Real-time transmission bandwidth refers to the amount of data transmitted in real time by the network. It reflects the data transmission capacity of the current network. Understanding bandwidth helps determine whether the network can support the amount of data required by MR applications and ensures that high-quality simulation image transmission will not be blocked or degraded due to bandwidth limitations.

[0118] Real-time traffic refers to the amount of real-time data passing through the network. Real-time traffic can help monitor network usage. By monitoring traffic, network congestion or abnormal traffic can be discovered in a timely manner, so that measures can be taken to optimize network performance or prevent network attacks.

[0119] Real-time transmission delay refers to the time required for data to be transmitted from the source to the destination. The real-time transmission delay parameter is an important indicator for measuring the network response speed. In MR applications, low latency is crucial to provide a smooth interactive experience. High latency will lead to unresponsive operations and affect the user experience.

[0120] The real-time transmission packet loss rate refers to the proportion of data packets lost during network transmission. The real-time transmission packet loss rate parameter can reflect the reliability of the network. A high packet loss rate will result in incomplete data and affect the normal operation of MR applications. Monitoring the packet loss rate helps diagnose network problems and take measures to reduce data loss.

[0121] The real-time transmission bandwidth benchmark value, real-time transmission flow benchmark value, real-time transmission delay threshold value and real-time transmission packet loss rate threshold value are extracted from the database for analysis to obtain the application performance evaluation value of each MR device. The specific processing conditions are as follows:

[0122] ;

[0123] in, For the The application performance evaluation value of each MR device, For the The real-time transmission bandwidth of each MR device, For the The real-time transmission flow of each MR device, For the The real-time transmission delay of each MR device, For the The real-time transmission packet loss rate of each MR device, is the real-time transmission bandwidth benchmark value. To transmit traffic benchmark value in real time, is the real-time transmission delay threshold, is the real-time transmission packet loss rate threshold, and e is a natural constant.

[0124] It should be noted that the parameters of real-time transmission bandwidth, real-time transmission traffic, real-time transmission delay and real-time transmission packet loss rate are related to each other. Bandwidth is the upper limit of network transmission capacity, while traffic is the actual usage. Traffic can fill the bandwidth but will not exceed the bandwidth limit. If the traffic is close to or reaches the bandwidth limit, network performance may degrade, resulting in increased latency and increased packet loss rate. If the real-time transmission traffic is continuously higher than the available bandwidth, it will cause network congestion, which will affect other parameters. Higher bandwidth can generally support faster transmission speeds, thereby reducing latency. However, if other factors in the network limit the transmission speed, latency may not be reduced even with higher bandwidth. In the case of sufficient bandwidth, latency is mainly affected by other network conditions. When network traffic increases, high traffic will cause network node processing capacity to overload, resulting in packet processing delays. Excessive traffic is one of the common causes of increased packet loss rate. Increased latency may be a precursor to increased packet loss rate, and latency will cause packets to time out and be discarded during transmission. High latency will also cause network protocols to retransmit packets, further increasing the network burden and causing more packet loss.

[0125] The monitoring results of each MR device are obtained based on the application performance evaluation value of each MR device. The monitoring results of each MR device are specifically:

[0126] If the application performance evaluation value of a certain MR device is greater than or equal to the application performance evaluation value threshold, then the real-time transmission result of the MR device is determined to be normal transmission;

[0127] If the application performance evaluation value of a certain MR device is less than the application performance evaluation value threshold, the real-time transmission result of the MR device is determined to be abnormal transmission, and the abnormal transmission result of the MR device is sent to the user interaction display terminal to give an early warning to the user.

[0128] The application performance evaluation value threshold is processed as follows: in a specific embodiment, the application performance evaluation value threshold is set directly in the database during the development of the simulated medical image visualization system based on MR technology involved in the embodiment of the present invention. There are many methods for setting the application performance evaluation value threshold, such as obtaining it through statistical analysis, including counting the application performance evaluation values ​​of each MR device under the historical normal transmission state (for example, the delay time is less than 10ms), and averaging the application performance evaluation values ​​obtained multiple times to obtain the application performance evaluation value threshold, or experts judge the boundary between high application performance evaluation values ​​and low application performance evaluation values ​​through corresponding historical data to set the threshold, thereby adjusting the threshold.

[0129] In this embodiment, the present invention further provides a method for visualizing simulated medical images based on MR technology, which specifically includes:

[0130] S1. Obtain a medical image of a pre-visualized medical picture, and simultaneously extract a quality parameter of each frame of the medical image in the medical image, and based on the quality parameter of each frame of the medical image, analyze and process to obtain a quality assessment value of each frame of the medical image and compare to obtain a quality assessment result of each frame of the medical image;

[0131] S2. Count the qualified frames of medical images and import them into the data processing center for image enhancement, segment and register the enhanced frames of medical images and perform 3D reconstruction, extract the 3D feature parameters of the pre-visualized medical images after 3D reconstruction, configure the virtual environment, and perform simulated medical image visualization;

[0132] S3. Monitor the visualized simulated medical images in real time to obtain the real-time light parameters of the virtual environment, and monitor the real-time light parameters of the real environment through optical sensors. Based on the comparison and analysis results of the real-time light parameters of the real environment and the real-time light parameters of the virtual environment, dynamically adjust the light intensity and RGB value of the virtual environment.

[0133] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0134] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that technicians in the relevant technical field can understand and use the present invention well. As long as they do not deviate from the structure of the present invention or exceed the scope defined by the present invention, they should all belong to the protection scope of the present invention.

Claims

1. A simulated medical image visualization system based on MR technology, characterized in that: include: A medical image data acquisition module acquires the medical image of the pre-visualized medical screen, and simultaneously extracts the quality parameters of each frame of the medical image in the medical image, and based on the quality parameters of each frame of the medical image, analyzes and processes to obtain the quality assessment value of each frame of the medical image and compares and obtains the quality assessment result of each frame of the medical image, wherein the quality assessment result of each frame of the medical image is qualified or unqualified; The specific process of extracting the quality parameters of each frame of medical image in the medical image is as follows: Segmenting the medical images of the pre-visualized medical images according to a preset frame rate, obtaining the medical images of each frame of the medical image and extracting the quality parameters of each frame of the medical image, including the resolution, clarity, number of pixels and signal-to-noise ratio of each frame of the medical image; The analyzing and processing based on the quality parameters of each frame of medical image to obtain the quality evaluation value of each frame of medical image specifically includes: Extracting a set of quality parameter comparison values ​​of medical images of pre-visualized medical images from a database, specifically including: a resolution comparison value, a clarity comparison value, a pixel number comparison value, and a signal-to-noise ratio comparison value; Based on the quality parameters of each frame of medical images and the set of quality parameter comparison values ​​of the medical images, analyzing and processing to obtain the quality assessment value of each frame of medical images; The comparison to obtain the quality assessment results of each frame of medical images specifically includes: The quality assessment value of each frame of medical image is compared with the quality assessment value threshold preset in the database. If the quality assessment value of a frame of medical image is greater than or equal to the quality assessment value threshold, the frame is marked as a qualified frame. If the quality assessment value of a frame of medical image is less than the quality assessment value threshold, the frame is marked as an unqualified frame. The medical image data processing module counts the medical image frames with qualified medical image quality assessment results as valid medical images, imports them into the data processing center for image enhancement, performs segmentation, registration and 3D reconstruction on the enhanced valid medical images, extracts the 3D feature parameters of the pre-visualized medical images after 3D reconstruction, configures the virtual environment, and performs simulated medical image visualization; The step of importing the images into the data processing center for image enhancement and performing segmentation, registration and three-dimensional reconstruction on each enhanced valid medical image specifically includes: Counting each frame of medical images whose medical image quality assessment results are qualified from each frame of medical images and recording them as each valid medical image, extracting the quality assessment value of each valid medical image from the quality assessment value of each frame of medical images, mapping and matching the quality assessment value of each valid medical image with the image enhancement parameters corresponding to each quality assessment value interval pre-set by the data processing center, obtaining the image enhancement parameters of each valid medical image, and performing image enhancement processing on each valid medical image; After segmenting and registering each enhanced valid medical image, extract the two-dimensional image sequence of each processed valid medical image and convert it into three-dimensional volume data, and perform three-dimensional reconstruction and image rendering operations; The simulated medical image dynamic adjustment module monitors the visualized simulated medical image in real time, obtains the real-time light parameters of the virtual environment, and monitors the real-time light parameters of the real environment through optical sensors. Based on the comparison and analysis results of the real-time light parameters of the real environment and the real-time light parameters of the virtual environment, the virtual environment light intensity and RGB value are dynamically adjusted; The analysis process obtains the quality evaluation value of each frame of medical image, and the specific analysis conditions are: ； in, is the quality assessment value of the i-th frame of medical image, is the resolution of the i-th frame of medical image, is the definition of the i-th frame of medical image, is the number of pixels of the i-th frame of medical image, is the signal-to-noise ratio of the i-th frame of medical image, is the resolution reference value of medical images, is the clarity comparison value of medical images, is the pixel number comparison value of medical images, It is the signal-to-noise ratio comparison value of medical images. The softplus function is a built-in function in Pytnon. .

2. The MR-based simulated medical image visualization system according to claim 1, characterized in that: The step of extracting the three-dimensional characteristic parameters of the pre-visualized medical image after three-dimensional reconstruction and configuring a virtual environment to simulate the visualization of the medical image specifically includes: The three-dimensional characteristic parameters of the pre-visualized medical image after three-dimensional reconstruction include surface smoothness, three-dimensional model grid density and reconstruction algorithm type; Based on the surface smoothness and 3D model mesh density corresponding to each reconstruction algorithm type, a surface smoothness limit value and a 3D model mesh density limit value corresponding to the reconstruction algorithm type of the pre-visualized medical image are matched and screened out, and a matching and comparison process is performed with the surface smoothness and 3D model mesh density of the pre-visualized medical image after 3D reconstruction to obtain a 3D feature value of the pre-visualized medical image; Based on the three-dimensional feature values ​​of the pre-visualized medical picture, a mapping and matching is performed with a set of virtual environment configuration parameters corresponding to each three-dimensional feature value interval pre-stored in a data processing center to obtain a set of virtual environment configuration parameters of the pre-visualized medical picture and configure the virtual environment accordingly, wherein the set of virtual environment configuration parameters of the pre-visualized medical picture includes an initial viewing angle, an initial zoom ratio, and virtual initial environment lighting parameters of the pre-visualized medical picture.

3. The MR-based simulated medical image visualization system according to claim 1, characterized in that: The visualized simulated medical images are monitored in real time to obtain real-time light parameters of the virtual environment, and the real-time light parameters of the real environment are monitored by optical sensors, wherein: The virtual environment real-time light parameters include the virtual environment real-time light intensity, the virtual environment real-time color temperature, and the virtual environment real-time RGB value; The real-time environment light parameters include real-time environment light intensity, real-time environment color temperature and real-time environment RGB value.

4. The MR-based simulated medical image visualization system according to claim 3, characterized in that: Based on the comparison and analysis results of the real-time light parameters of the real environment and the real-time light parameters of the virtual environment, the virtual environment light intensity and RGB value are dynamically adjusted. The specific processing conditions are: Compare and analyze the real-time light parameters of the virtual environment with the real-time light parameters of the real environment to obtain the real-time verification value of the ambient light, and map and match the virtual ambient light intensity adjustment value and the RGB real-time adjustment parameter corresponding to each ambient light real-time verification value interval preset by the data processing center based on the real-time verification value of the ambient light to obtain the virtual ambient light intensity adjustment value and the RGB real-time adjustment parameter of the virtual environment, so as to dynamically adjust the virtual environment; The RGB real-time adjustment parameters include an R value adjustment value, a G value adjustment value, and a B value adjustment value.

5. The MR-based simulated medical image visualization system according to claim 1, characterized in that: It also includes a device real-time monitoring module, which is used to count the connected devices used for the simulation of medical image visualization, recorded as MR devices, and perform real-time monitoring on each MR device, specifically including: Real-time monitoring of data transmission parameters between MR devices, including real-time transmission bandwidth, real-time transmission flow, real-time transmission delay and real-time transmission packet loss rate of each MR device, analysis to obtain application performance evaluation values ​​of each MR device, and obtaining monitoring results of each MR device based on the application performance evaluation values ​​of each MR device, wherein the monitoring results of each MR device are specifically: If the application performance evaluation value of a certain MR device is greater than or equal to the application performance evaluation value threshold, then the real-time transmission result of the MR device is determined to be normal transmission; If the application performance evaluation value of a certain MR device is less than the application performance evaluation value threshold, the real-time transmission result of the MR device is determined to be abnormal transmission and an early warning is issued.

6. A method for applying to the MR technology-based simulated medical image visualization system according to any one of claims 1 to 5, characterized in that: include: S1, obtaining a medical image of a pre-visualized medical picture, and synchronously extracting a quality parameter of each frame of the medical image in the medical image, analyzing and processing to obtain a quality assessment value of each frame of the medical image based on the quality parameter of each frame of the medical image, and performing comparison to obtain a quality assessment result of each frame of the medical image, wherein the quality assessment result of each frame of the medical image is qualified or unqualified; S2. Count the medical image frames with qualified medical image quality assessment results as valid medical images, import them into the data processing center for image enhancement, segment and register the enhanced valid medical images, and perform three-dimensional reconstruction, extract the three-dimensional feature parameters of the pre-visualized medical images after three-dimensional reconstruction, configure the virtual environment, and simulate medical image visualization; S3. Monitor the visualized simulated medical images in real time to obtain the real-time light parameters of the virtual environment, and monitor the real-time light parameters of the real environment through optical sensors. Based on the comparison and analysis results of the real-time light parameters of the real environment and the real-time light parameters of the virtual environment, dynamically adjust the light intensity and RGB value of the virtual environment.

Images