Model display method, device and system, computer equipment and storage medium

By using interactive devices to detect collision events of three-dimensional models in the metaverse space and perform visual enhancement processing, the problem of misoperation in the display of three-dimensional models of human organs is solved, and the accuracy and efficiency of analysis and processing are improved.

CN119091091APending Publication Date: 2024-12-06WUHAN UNITED IMAGING HEALTHCARE SURGICAL TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202310661673.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-05
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the prior art, the three-dimensional model of human organs shows that it is difficult to clearly display the boundary lines between adjacent organs, resulting in misoperation problems and affecting the analysis and processing effects.

Method used

By displaying three-dimensional models in the metaverse space, using interactive rays from interactive devices to detect collision events, identify target models and perform visual enhancement processing, including highlights, breathing lights and marquee effects to avoid misoperation.

Benefits of technology

It effectively avoids misoperation, improves the interaction efficiency and display effect of the three-dimensional model, and enhances the accuracy of analysis and processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091091A_ABST
    Figure CN119091091A_ABST
Patent Text Reader

Abstract

The invention relates to a model display method, device and system, computer equipment and a storage medium. The method comprises the following steps: acquiring three-dimensional data of a target part, and displaying a three-dimensional model corresponding to the three-dimensional data based on element cosmic space; if it is detected that interaction rays emitted by interaction equipment collide with the three-dimensional model, determining the three-dimensional model colliding with the interaction rays as a target model, and determining a visual enhancement area of the target model; and according to a preset vision enhancement effect, performing vision enhancement processing on the vision enhancement region to obtain a vision-enhanced target model, and displaying the vision-enhanced target model. By adopting the method, visual enhancement can be performed on the target model in the three-dimensional model, and misoperation on the target model is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model display method, device, system, computer equipment and storage medium. Background Art

[0002] With the development of virtual reality technology, the application of virtual reality in the medical field can more comprehensively display the physiological characteristics of the human body by constructing virtual three-dimensional models of human organs, playing an important auxiliary role in teaching, clinical diagnosis and treatment.

[0003] In the prior art, a 3D model of human organs can be displayed through a head display, and after selecting a target part of the human organ from the 3D model, the target part can be analyzed and processed. However, since the growth positions of human organs are usually close, it is difficult to display clear boundaries between adjacent organs in the 3D model. In the process of analyzing and processing the target part, it is easy to cause misoperation, which affects the effect of analysis and processing.

[0004] Therefore, there is a problem in the current model display technology that is prone to causing misoperation. Summary of the invention

[0005] Based on this, it is necessary to provide a model display method, device, system, computer equipment, computer-readable storage medium and computer program product that can avoid misoperation in response to the above technical problems.

[0006] In a first aspect, the present application provides a model display method. The method comprises:

[0007] Acquire three-dimensional data of a target part, and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space;

[0008] If it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model, the three-dimensional model colliding with the interaction ray is determined as a target model, and a visual enhancement area of ​​the target model is determined;

[0009] According to the preset visual enhancement effect, the visual enhancement area is subjected to visual enhancement processing to obtain a visually enhanced target model, and the visually enhanced target model is displayed.

[0010] In one of the embodiments, if it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model, determining the three-dimensional model that collides with the interaction ray as the target model includes:

[0011] When the interaction ray collides with the three-dimensional model, a collision monitoring event corresponding to the three-dimensional model is triggered;

[0012] In response to a trigger operation for the collision monitoring event, a three-dimensional model corresponding to the collision monitoring event is determined as the target model.

[0013] In one embodiment, determining the visual enhancement area of ​​the target model includes:

[0014] Transforming the vertices of the target model into a clipping space to obtain vertex coordinates of the target model in the clipping space;

[0015] Determining a model edge of the target model in the clipping space according to the vertex coordinates;

[0016] The edge of the model is extended to obtain a visual enhancement area of ​​the target model.

[0017] In one embodiment, performing visual enhancement processing on the visual enhancement area according to a preset visual enhancement effect to obtain a visually enhanced target model includes:

[0018] Generating first configuration information of the visual enhancement area according to the visual enhancement effect;

[0019] Generate a material ball according to the first configuration information and the second configuration information of the target model;

[0020] The target model and the visual enhancement area are rendered by the material ball to obtain the visually enhanced target model.

[0021] In one of the embodiments, the visual enhancement effect includes a breathing light effect;

[0022] The step of rendering the target model and the visual enhancement area by using the material ball to obtain the visually enhanced target model includes:

[0023] Determining a rendering time when the material ball renders the visual enhancement area;

[0024] Determining the rendering brightness corresponding to the rendering moment according to a preset trigonometric function mapping relationship;

[0025] At the rendering moment, the material ball is controlled to render the visual enhancement area according to the rendering brightness.

[0026] In one embodiment, the visual enhancement effect includes a marquee effect;

[0027] The step of rendering the target model and the visual enhancement area by using the material ball to obtain the visually enhanced target model includes:

[0028] Determining a rendering time when the material ball renders the visual enhancement area;

[0029] According to a preset moving direction of the marquee, a display position of the marquee corresponding to the rendering moment is determined in the visual enhancement area;

[0030] Control the material ball to render the marquee display position at the rendering time.

[0031] In a second aspect, the present application also provides a model display device. The device comprises:

[0032] A model acquisition module, used to acquire three-dimensional data of a target part and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space;

[0033] A model determination module, configured to determine the three-dimensional model that collides with the interaction ray as a target model, and determine a visual enhancement area of ​​the target model if it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model;

[0034] The visual enhancement module is used to perform visual enhancement processing on the visual enhancement area according to a preset visual enhancement effect, obtain a visually enhanced target model, and display the visually enhanced target model.

[0035] In a third aspect, the present application also provides a model display system. The system includes a display device and an interactive device;

[0036] The display device is used to obtain three-dimensional data of the target part and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space;

[0037] The interactive device is used to emit interactive rays;

[0038] The display device is also used to, if it is detected that the interactive ray collides with the three-dimensional model, determine the three-dimensional model that collides with the interactive ray as the target model, determine the visual enhancement area of ​​the target model, perform visual enhancement processing on the visual enhancement area according to a preset visual enhancement effect, obtain the visually enhanced target model, and display the visually enhanced target model.

[0039] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0040] Acquire three-dimensional data of a target part, and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space;

[0041] If it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model, the three-dimensional model colliding with the interaction ray is determined as a target model, and a visual enhancement area of ​​the target model is determined;

[0042] According to the preset visual enhancement effect, the visual enhancement area is subjected to visual enhancement processing to obtain a visually enhanced target model, and the visually enhanced target model is displayed.

[0043] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0044] Acquire three-dimensional data of a target part, and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space;

[0045] If it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model, the three-dimensional model colliding with the interaction ray is determined as a target model, and a visual enhancement area of ​​the target model is determined;

[0046] According to the preset visual enhancement effect, the visual enhancement area is subjected to visual enhancement processing to obtain a visually enhanced target model, and the visually enhanced target model is displayed.

[0047] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0048] Acquire three-dimensional data of a target part, and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space;

[0049] If it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model, the three-dimensional model colliding with the interaction ray is determined as a target model, and a visual enhancement area of ​​the target model is determined;

[0050] According to the preset visual enhancement effect, the visual enhancement area is subjected to visual enhancement processing to obtain a visually enhanced target model, and the visually enhanced target model is displayed.

[0051] The above-mentioned model display method, device, system, computer equipment, storage medium and computer program product obtain three-dimensional data of the target part, and display the three-dimensional model corresponding to the three-dimensional data based on the metaverse space. If it is detected that the interactive ray emitted by the interactive device collides with the three-dimensional model, the three-dimensional model colliding with the interactive ray is determined as the target model, and the visual enhancement area of ​​the target model is determined. According to the pre-set visual enhancement effect, the visual enhancement area is visually enhanced to obtain the visually enhanced target model, and the visually enhanced target model is displayed; the three-dimensional model can be displayed based on the metaverse space, and after the target model is selected from the three-dimensional model through the interactive device, the target model is visually enhanced to avoid misoperation of the target model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 An application environment diagram of a model display method in an embodiment;

[0053] Figure 2 A schematic diagram of a flow chart of a model display method in one embodiment;

[0054] Figure 3 A schematic diagram of a metaverse space in one embodiment;

[0055] Figure 4 is a schematic diagram of a target model after visual enhancement in one embodiment;

[0056] Figure 5 A schematic diagram of a trigonometric function mapping relationship of a breathing light effect in an embodiment;

[0057] Figure 6 A schematic diagram of a marquee effect in one embodiment;

[0058] Figure 7 A schematic diagram of a process of adding visual enhancement effects to a three-dimensional model in a metaverse space in one embodiment;

[0059] Figure 8 is a structural block diagram of a model display device in an embodiment;

[0060] Fig. 9 A structural block diagram of a model display system in one embodiment;

[0061] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] The model display method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the display device 102 communicates with the interactive device 104; the display device 102 may be, but is not limited to, various 3D projectors, 3D displays, head-mounted display devices (HMDs for short), personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc., and portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc.; the interactive device 104 may be, but is not limited to, various data gloves, 3D mice, position trackers, motion capture devices, etc.

[0064] In one embodiment, Figure 2 As shown, a model display method is provided, which is applied to Figure 1 The display device in the example is used to illustrate, including the following steps:

[0065] Step S210, acquiring three-dimensional data of the target part, and displaying a three-dimensional model corresponding to the three-dimensional data based on the metaverse space.

[0066] The target part may be a human organ to be displayed, and the three-dimensional data may be medical data collected by medical equipment.

[0067] In a specific implementation, the three-dimensional data of the target part can be collected by a medical device, and the collected three-dimensional data can be input into a display device. After the display device obtains the three-dimensional data, a three-dimensional model is constructed in the metaverse space according to the three-dimensional data to display the three-dimensional model of the target part. The number of target parts can be one or more, and the number of three-dimensional models can also be one or more, which is not limited in the embodiment of the present application.

[0068] For example, Figure 1 As shown, 3D data of the human heart, liver and stomach are collected by CT (Computed Tomography) equipment, and the 3D data is input into a head display. The head display can jointly build a metaverse space based on Hololens 2 (a mixed reality head display developed by Microsoft) and Unity (a real-time 3D interactive content creation and operation platform) engine, and project a 3D model of the heart, liver and stomach according to the acquired 3D data.

[0069] Figure 3Provides a schematic diagram of the metaverse space. Figure 3 The metaverse space can build virtual objects based on the collected human body data, and display them through sound, pictures, text, etc., and interact through drawing, clicking, voice, etc., to achieve virtual-reality interaction between real people and digital people. For digital people in the metaverse space, virtual models can be built based on Unity, and the constructed virtual models can be displayed through Hololens2.

[0070] Step S220: if it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model, the three-dimensional model colliding with the interaction ray is determined as a target model, and a visual enhancement area of ​​the target model is determined.

[0071] The interactive ray may be a directional signal emitted by the interactive device. The target model may be a three-dimensional model that needs to be visually enhanced. The visual enhancement area may be, but is not limited to, the edge of the target model.

[0072] In a specific implementation, when it is necessary to select one or more three-dimensional models from the three-dimensional model of the target part for processing, an interactive ray can be emitted through the interactive device to point to the one or more three-dimensional models, and the display device detects all the three-dimensional models currently displayed. If a collision of interactive rays is detected, one or more three-dimensional models that collide with the interactive rays are determined from all the three-dimensional models currently displayed, determined as target models, and a visual enhancement area of ​​the target model is determined; otherwise, if the display device does not detect a collision of interactive rays, no processing is required.

[0073] For example, interactive rays are emitted to the three-dimensional model of the liver through the data gloves, and the head display detects that the interactive rays collide with the three-dimensional model, and further determines that the three-dimensional model of the liver collides, then the three-dimensional model of the liver is determined as the target model, and the edge of the three-dimensional model of the liver is determined as the visual enhancement area; interactive rays can also be emitted to the three-dimensional models of the liver and stomach in sequence through the data gloves, and the head display detects that the interactive rays collide with the three-dimensional model, and further determines that the three-dimensional models of the liver and stomach collide, then the three-dimensional models of the liver and stomach are determined as the target models, and the edge of the three-dimensional model of the liver and the edge of the three-dimensional model of the stomach are determined as the visual enhancement area.

[0074] Step S230, performing visual enhancement processing on the visual enhancement area according to the preset visual enhancement effect, obtaining a visually enhanced target model, and displaying the visually enhanced target model.

[0075] Among them, the visual enhancement effect can be a display effect of the visual enhancement area, including but not limited to at least one of highlight, magnification, breathing light and marquee.

[0076] In the specific implementation, after determining the visual enhancement area of ​​the target model, the visual enhancement area can be visually enhanced according to the preset visual enhancement effect on the basis of retaining the original display effect of the target model to obtain the visually enhanced target model, and the visually enhanced target model can be displayed in the metaverse space.

[0077] For example, the visual enhancement effect may be preset to highlight display. After determining that the liver 3D model is the target model and the edge of the liver 3D model is the visual enhancement area, the edge of the liver 3D model may be highlighted while retaining the original display effect of the liver 3D model.

[0078] Figure 4 A schematic diagram of the target model after visual enhancement is provided. Figure 4 ,When the head display detects that the interactive rays emitted by the data glove collides with the three-dimensional liver model, the edge of the three-dimensional liver model is highlighted.

[0079] The above-mentioned model display method obtains the three-dimensional model of the target part based on the metaverse space. If it is detected that the interactive ray emitted by the interactive device collides with the three-dimensional model, the three-dimensional model that collides with the interactive ray is determined as the target model, and the visual enhancement area of ​​the target model is determined. According to the pre-set visual enhancement effect, the visual enhancement area is visually enhanced to obtain the visually enhanced target model, and the visually enhanced target model is displayed; the three-dimensional model can be displayed based on the metaverse space, and after the target model is selected from the three-dimensional model through the interactive device, the target model is visually enhanced to avoid misoperation of the target model.

[0080] In one embodiment, the above step S220 may specifically include: when the interaction ray collides with the three-dimensional model, triggering a collision monitoring event corresponding to the three-dimensional model; in response to the triggering operation for the collision monitoring event, determining the three-dimensional model corresponding to the collision monitoring event as the target model.

[0081] The collision monitoring event may be a program that detects a collision between an interactive ray and a three-dimensional model.

[0082] In a specific implementation, the display device can detect all currently displayed three-dimensional models. If it is detected that an interactive ray collides with any of the three-dimensional models, a collision monitoring event corresponding to the colliding three-dimensional model is triggered. The collision monitoring event responds to the triggering operation and determines the colliding three-dimensional model as the target model.

[0083] In actual applications, you can use Unity and Hololens2 to build a human body metaverse space, load one or more 3D models of target parts, mount a script under each 3D model, add collision monitoring events of the corresponding 3D model to the script, and at the same time, you can also use the Unity editor to generate a material ball for rendering the 3D model, introduce the material ball into the script of the corresponding 3D model, and render the 3D model. Monitor whether there is a collision event between the hand ray and the 3D model in Hololens2. If there is a collision event, turn on the visual enhancement effect in the script mounted by the 3D model that collides with the hand ray. Otherwise, if there is no collision event, there is no need to turn on the visual enhancement effect in the script.

[0084] In this embodiment, when the interactive ray collides with the three-dimensional model, a collision monitoring event corresponding to the three-dimensional model is triggered; in response to the triggering operation of the collision monitoring event, the three-dimensional model corresponding to the collision monitoring event is determined as the target model, and the interactive ray can be sent to the three-dimensional model through the interactive device to determine the target model that needs visual enhancement from the three-dimensional model, thereby improving the interactive efficiency of the model display.

[0085] In one embodiment, the above step S220 may further specifically include: transforming the vertices of the target model to the clipping space to obtain the vertex coordinates of the target model in the clipping space; determining the model edge of the target model in the clipping space based on the vertex coordinates; and performing an outward extension process on the model edge to obtain a visual enhancement area of ​​the target model.

[0086] In a specific implementation, in a vertex shader, the vertex coordinates of the target model can be transformed from the model space to the clipping space to obtain the vertex coordinates of the target model in the clipping space. The line between the vertex coordinates is used as the model edge of the target model in the clipping space. The model edge is extended to obtain the visual enhancement area of ​​the target model.

[0087] For example, the original edge of the target model in the clipping space may be moved a certain distance outside the target model to obtain a moved edge, and the area between the original edge and the moved edge may be determined as the visual enhancement area.

[0088] In this embodiment, the vertex coordinates of the target model in the clipping space are obtained by transforming the vertices of the target model to the clipping space; the model edge of the target model in the clipping space is determined based on the vertex coordinates; the model edge is extended to obtain the visual enhancement area of ​​the target model. The visual enhancement area of ​​the target model can be efficiently determined based on the vertices of the target model, thereby improving the efficiency of model display.

[0089] In one embodiment, the above step S230 may specifically include: generating first configuration information of the visual enhancement area according to the visual enhancement effect; generating a material ball according to the first configuration information and the second configuration information of the target model; rendering the target model and the visual enhancement area through the material ball to obtain the visually enhanced target model.

[0090] The first configuration information may be a pass semantic block for rendering the visual enhancement area in the script.

[0091] The second configuration information may be a pass semantic block for rendering the target model in the script.

[0092] Among them, the pass semantic block is a semantic block used to set the display frame buffer write queue in the Shader rendering technology.

[0093] In a specific implementation, the first configuration information can be generated according to the visual enhancement effect, the second configuration information can be generated according to the original display effect of the target model, and a material ball can be generated according to the first configuration information and the second configuration information. The material ball can be used to render the visual enhancement area and the area where the target model is located, respectively, to obtain the visually enhanced target model.

[0094] In actual applications, in the script mounted by the target model, a first pass semantic block for rendering the visual enhancement area can be generated according to the visual enhancement effect, and a second pass semantic block for rendering the area where the target model is located can be generated according to the original display effect of the target model. A material ball is generated according to the first pass semantic block and the second pass semantic block, and the material ball is used to render the visual enhancement area and the area where the target model is located, respectively, to obtain the visually enhanced target model.

[0095] In this embodiment, first configuration information of the visual enhancement area is generated according to the visual enhancement effect; a material ball is generated according to the first configuration information and the second configuration information of the target model; the target model and the visual enhancement area are rendered by the material ball to obtain the visually enhanced target model. The visual enhancement area and the area where the target model is located can be rendered separately by one material ball, thereby improving the rendering efficiency.

[0096] In one embodiment, the visual enhancement effect includes a breathing light effect; the above-mentioned step of rendering the target model and the visual enhancement area through the material ball to obtain the visually enhanced target model may specifically include: determining the rendering time when the material ball renders the visual enhancement area; determining the rendering brightness corresponding to the rendering time according to a preset trigonometric function mapping relationship; at the rendering time, controlling the material ball to render the visual enhancement area according to the rendering brightness.

[0097] The trigonometric function mapping relationship may be, but is not limited to, a sine relationship or a cosine relationship.

[0098] In a specific implementation, at least one rendering moment at which a material ball renders a visual enhancement area can be determined, and the brightness corresponding to each rendering moment can be determined based on a trigonometric function mapping relationship between a preset rendering moment and a rendering brightness to obtain the rendering brightness. Afterwards, at each rendering moment, the material ball can be controlled to render the visual enhancement area based on the corresponding rendering brightness.

[0099] Figure 5 A schematic diagram of the trigonometric function mapping relationship of the breathing light effect is provided. Figure 5 , the trigonometric function mapping relationship between the rendering time and the rendering brightness is preset to be l=sin(t), wherein t represents the rendering time, and l represents the rendering brightness. For the rendering times t1, t3, t5, and t7, the rendering brightness can be obtained as l1 according to the trigonometric function mapping relationship. For the rendering times t2, t4, t6, and t8, the rendering brightness can be obtained as l2 according to the trigonometric function mapping relationship. The material ball is controlled to render the edge area of ​​the target model according to the corresponding rendering brightness at each rendering time, so that the visual enhancement of the breathing light effect can be obtained in the edge area.

[0100] In this embodiment, by determining the rendering time when the material ball renders the visual enhancement area; determining the rendering brightness corresponding to the rendering time according to the preset trigonometric function mapping relationship; at the rendering time, controlling the material ball to render the visual enhancement area according to the rendering brightness, a visual enhancement effect in the form of a breathing light in the visual enhancement area can be achieved, so that the target model can be visually enhanced.

[0101] In one embodiment, the visual enhancement effect includes a marquee effect; the above-mentioned step of rendering the target model and the visual enhancement area through the material ball to obtain the visually enhanced target model may also specifically include: determining the rendering time when the material ball renders the visual enhancement area; determining the marquee display position corresponding to the rendering time in the visual enhancement area according to the pre-set marquee movement direction; controlling the material ball to render the marquee display position at the rendering time.

[0102] In a specific implementation, at least one rendering moment at which the material ball renders the visual enhancement area and the direction in which the marquee moves can be determined. In the visual enhancement area, the marquee display position corresponding to each rendering moment is determined according to the direction in which the marquee moves. Then, at each rendering moment, the material ball can be controlled to render the corresponding marquee display position.

[0103] Figure 6 A schematic diagram of the marquee effect is provided. Figure 6After determining that the visual enhancement area is the edge area of ​​the target model, set the initial rendering time t0 and its position (x0, y0), and set the marquee moving direction and the marquee moving speed v as shown in the figure. At rendering time t1, the marquee moving distance can be calculated as v(t1-t0), and the marquee display position at this moment is determined to be (x1, y1) based on (x0, y0) and the marquee moving distance; at rendering time t2, the marquee moving distance can be calculated as v(t2-t0), and the marquee display position at this moment is determined to be (x2, y2) based on (x0, y0) and the marquee moving distance; and so on, the marquee display position corresponding to each rendering time can be determined, and the material ball can be controlled to render the edge area of ​​the target model according to the corresponding marquee display position at each rendering time, so that the visual enhancement of the marquee effect can be obtained in the edge area.

[0104] In this embodiment, by determining the rendering time when the material ball renders the visual enhancement area; determining the marquee display position corresponding to the rendering time in the visual enhancement area according to the pre-set marquee movement direction; and controlling the material ball to render the marquee display position at the rendering time, a visual enhancement effect in the form of a marquee in the visual enhancement area can be achieved, so that the target model can be visually enhanced.

[0105] In order to facilitate those skilled in the art to have a deeper understanding of the embodiments of the present application, a specific example will be described below.

[0106] Virtual-real interaction is the main way to enhance the immersive experience of the combination of virtual and real in the metaverse space. Virtual-real interaction includes drawing, clicking, and voice. This embodiment discloses a method for enhancing the visual effect of a three-dimensional model in a metaverse space, which uses a collision event between a specific hand ray and a three-dimensional model as a trigger action to enhance the visual display effect of the three-dimensional model in the metaverse space, and add a selection prompt with a breathing light effect to the three-dimensional model. Among them, the metaverse space is jointly constructed by Hololens2 and Unity engine, and relies on MRTK (Mixed Reality Tookit) for gesture recognition. For the selected state of the three-dimensional model in the metaverse space, a method is proposed by combining the sin periodic function with the timing function through the external extension of the vertex homogeneous clipping space, and adding a selection prompt with a breathing light and a marquee effect to the three-dimensional model in the metaverse space, so as to achieve the visual enhancement effect of individual models when multiple three-dimensional models are displayed simultaneously in the metaverse space.

[0107] Figure 7 A flowchart of a method for adding visual enhancement effects to a three-dimensional model in a metaverse space is provided. Figure 7 , a method for adding visual enhancement effects to a three-dimensional model in a metaverse space may include the following steps:

[0108] Step S310, loading the three-dimensional model, loading the three-dimensional model in the metaverse space constructed by Hololens2 and Unity engine according to local or network request data, and adding collision monitoring events to the three-dimensional model through scripts when loading the three-dimensional model.

[0109] Step S320, mount the rendering material on the three-dimensional model, generate a rendering material ball through the Unity editor using the Shader file that realizes the visual enhancement effect, and introduce it into the script through the control panel and mount it on the three-dimensional model that needs to be rendered.

[0110] Step S330, judging whether to turn on the visual enhancement effect according to the collision event between the hand ray and the model, judging whether there is a three-dimensional model that needs to turn on the visual enhancement effect by monitoring the collision event between the hand ray and the rendering model in Hololens2, and if a collision between the three-dimensional model and the hand ray is detected, setting the visual enhancement effect of the model to be turned on by obtaining the operation script mounted on the three-dimensional model.

[0111] Step S340, re-render the model and visual enhancement effect. After the script is set to turn on the visual enhancement effect of the three-dimensional model, the material ball will re-render the model, and use two pass semantic blocks to render the model's own material display and the visual enhancement effect area respectively. When drawing the visual enhancement effect area, the vertex of the model can be homogeneously clipped and extended in the vertex shader to display the visual enhancement effect in the extended area.

[0112] Step S350, draw the breathing light effect or marquee effect of the selected model in the fragment shader. In the fragment shader, the display color of the extension area can be calculated by the sin (sine) periodic function and the Time timing function, so that the visual enhancement effect display area of ​​the three-dimensional model extension has a periodic breathing light flashing effect. If you want to add a marquee effect to the selected model, you need to determine the current model display surface in the Shader, determine the current marquee moving direction, and calculate the current marquee display effect area with the Time timing function to add a marquee effect to the visual enhancement effect display area of ​​the three-dimensional model extension.

[0113] The above method of adding visual enhancement effects to three-dimensional models in the metaverse space uses multi-pass semantic blocks to set different configuration information to render the model display area and the visual enhancement effect display area respectively when drawing the visual enhancement effect of the three-dimensional model, and controls whether to draw the visual enhancement effect display area through business input parameters, thereby improving the drawing performance of model rendering. At the same time, when adding the breathing light flashing effect, a timing function is used in conjunction with a sin periodic function to simply and efficiently achieve the breathing light and marquee effects of periodically and evenly flashing in the visual enhancement area.

[0114] Moreover, the collision of hand rays with the surface of the 3D model is used as a trigger condition to improve the flexibility and convenience of the user's operation of the 3D model. At the same time, by rendering the visual enhancement area of ​​the model, the user's experience of operating the model is greatly improved in the scene where multiple 3D models exist at the same time in the metaverse space, making the reality enhancement advantage of the metaverse more significant. It provides a convenient solution for complex scenes where multiple models coexist in the metaverse space.

[0115] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0116] Based on the same inventive concept, the embodiment of the present application also provides a model display device and system for implementing the model display method involved above. The implementation scheme for solving the problem provided by the device and system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model display device and system embodiments provided below can refer to the limitations on the model display method above, and will not be repeated here.

[0117] In one embodiment, Figure 8 As shown, a model display device is provided, including: a model acquisition module 410, a model determination module 420 and a visual enhancement module 430, wherein:

[0118] A model acquisition module 410 is used to acquire three-dimensional data of a target part and display a three-dimensional model corresponding to the three-dimensional data based on a metaverse space;

[0119] A model determination module 420 is used to determine the three-dimensional model that collides with the interaction ray as a target model and determine a visual enhancement area of ​​the target model if it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model;

[0120] The visual enhancement module 430 is used to perform visual enhancement processing on the visual enhancement area according to a preset visual enhancement effect, obtain a visually enhanced target model, and display the visually enhanced target model.

[0121] In one embodiment, the above-mentioned model determination module 420 is also used to trigger a collision monitoring event corresponding to the three-dimensional model when the interaction ray collides with the three-dimensional model; in response to the triggering operation of the collision monitoring event, the three-dimensional model corresponding to the collision monitoring event is determined as the target model.

[0122] In one embodiment, the above-mentioned model determination module 420 is also used to transform the vertices of the target model to the clipping space to obtain the vertex coordinates of the target model in the clipping space; determine the model edge of the target model in the clipping space based on the vertex coordinates; and perform outward extension processing on the model edge to obtain the visual enhancement area of ​​the target model.

[0123] In one embodiment, the visual enhancement module 430 further includes:

[0124] A configuration information module, used for generating first configuration information of the visual enhancement area according to the visual enhancement effect;

[0125] A material ball module, used for generating a material ball according to the first configuration information and the second configuration information of the target model;

[0126] The target model module is used to render the target model and the visual enhancement area through the material ball to obtain the visually enhanced target model.

[0127] In one embodiment, the above-mentioned target model module is also used to determine the rendering time when the material ball renders the visual enhancement area; determine the rendering brightness corresponding to the rendering time according to a preset trigonometric function mapping relationship; at the rendering time, control the material ball to render the visual enhancement area according to the rendering brightness.

[0128] In one embodiment, the above-mentioned target model module is also used to determine the rendering time when the material ball renders the visual enhancement area; determine the marquee display position corresponding to the rendering time in the visual enhancement area according to the pre-set marquee movement direction; and control the material ball to render the marquee display position at the rendering time.

[0129] In one embodiment, Fig. 9 As shown, a model display system is provided, including a display device 102 and an interactive device 104;

[0130] The display device 120 is used to obtain three-dimensional data of the target part and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space;

[0131] The interactive device 104 is used to emit interactive rays;

[0132] The display device 102 is also used to, if it is detected that the interaction ray collides with the three-dimensional model, determine the three-dimensional model that collides with the interaction ray as the target model, determine the visual enhancement area of ​​the target model, perform visual enhancement processing on the visual enhancement area according to a preset visual enhancement effect, obtain the visually enhanced target model, and display the visually enhanced target model.

[0133] Each module in the above-mentioned model display device and system can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0134] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a model display method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0135] Those skilled in the art will understand that Fig.10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0136] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0137] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0138] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0141] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A model display method, characterized in that: The method comprises: Acquire three-dimensional data of a target part, and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space; If it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model, the three-dimensional model colliding with the interaction ray is determined as a target model, and a visual enhancement area of ​​the target model is determined; According to the preset visual enhancement effect, the visual enhancement area is subjected to visual enhancement processing to obtain a visually enhanced target model, and the visually enhanced target model is displayed.

2. The method according to claim 1, characterized in that If it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model, the three-dimensional model colliding with the interaction ray is determined as the target model, including: When the interaction ray collides with the three-dimensional model, a collision monitoring event corresponding to the three-dimensional model is triggered; In response to a trigger operation for the collision monitoring event, a three-dimensional model corresponding to the collision monitoring event is determined as the target model.

3. The method according to claim 1, characterized in that Determining the visual enhancement area of ​​the target model includes: Transforming the vertices of the target model into a clipping space to obtain vertex coordinates of the target model in the clipping space; Determining a model edge of the target model in the clipping space according to the vertex coordinates; The edge of the model is extended to obtain a visual enhancement area of ​​the target model.

4. The method according to claim 1, characterized in that: The step of performing visual enhancement processing on the visual enhancement area according to the preset visual enhancement effect to obtain a visually enhanced target model includes: Generating first configuration information of the visual enhancement area according to the visual enhancement effect; Generate a material ball according to the first configuration information and the second configuration information of the target model; The target model and the visual enhancement area are rendered by the material ball to obtain the visually enhanced target model.

5. The method according to claim 4, characterized in that The visual enhancement effect includes a breathing light effect; The step of rendering the target model and the visual enhancement area by using the material ball to obtain the visually enhanced target model includes: Determining a rendering time when the material ball renders the visual enhancement area; Determining the rendering brightness corresponding to the rendering moment according to a preset trigonometric function mapping relationship; At the rendering moment, the material ball is controlled to render the visual enhancement area according to the rendering brightness.

6. The method according to claim 4, characterized in that The visual enhancement effect includes a marquee effect; The step of rendering the target model and the visual enhancement area by using the material ball to obtain the visually enhanced target model includes: Determining a rendering time when the material ball renders the visual enhancement area; According to a preset moving direction of the marquee, a display position of the marquee corresponding to the rendering moment is determined in the visual enhancement area; The material ball is controlled to render the marquee display position at the rendering time.

7. A model display device, characterized in that: The device comprises: A model acquisition module, used to acquire three-dimensional data of a target part and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space; A model determination module, configured to determine the three-dimensional model that collides with the interaction ray as a target model and determine a visual enhancement area of ​​the target model if it is detected that the interaction ray emitted by the interaction device collides with the three-dimensional model; The visual enhancement module is used to perform visual enhancement processing on the visual enhancement area according to a preset visual enhancement effect, obtain a visually enhanced target model, and display the visually enhanced target model.

8. A model display system, characterized in that: The system includes a display device and an interactive device; The display device is used to obtain three-dimensional data of the target part and display a three-dimensional model corresponding to the three-dimensional data based on the metaverse space; The interactive device is used to emit interactive rays; The display device is also used to, if it is detected that the interactive ray collides with the three-dimensional model, determine the three-dimensional model that collides with the interactive ray as the target model, determine the visual enhancement area of ​​the target model, perform visual enhancement processing on the visual enhancement area according to a preset visual enhancement effect, obtain the visually enhanced target model, and display the visually enhanced target model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Citation Information

Cited By

  • Heart electrophysiology simulation interaction method based on VR and electronic equipment

    CN119273879A

  • VR-based cardiac electrophysiological simulation interactive method and electronic equipment

    CN119273879B