Interaction method and system of MR experience scene

Abstract

The embodiment of the invention relates to the technical field of playing interaction, and discloses an MR experience scene interaction method, which comprises the following steps: acquiring spatial position data of a corresponding user in an actual physical environment through a radar module; obtaining a coordinate mapping relationship between an actual environment coordinate system constructed based on the radar module and a virtual scene coordinate system constructed based on the display module, and determining a display virtual coordinate group corresponding to the spatial position data in the actual physical environment in the virtual scene coordinate system according to the coordinate mapping relationship; and generating a corresponding interactive display effect according to the display virtual coordinate group, and sending the interactive display effect to a display module for display. According to the embodiment of the invention, the accurate space mapping not only ensures the accuracy of interaction, but also enables the user to freely move in the virtual environment and perform seamless interaction with the virtual environment, thereby further enhancing the continuity and naturalness of the user experience.

Description

Technical Field

[0001] The present invention relates to the field of game interaction technology, and in particular to an interactive method and system for an MR experience scene. Background Art

[0002] In the current field of interactive play, although a variety of interactive forms have emerged to enhance tourists' sense of participation and immersion, generally speaking, these forms are still single and have limitations. Traditional interactive methods mainly rely on two core display media: one is to directly present visual content through the display screen, and the other is to build a virtual environment with the help of VR glasses to achieve a higher level of visual immersion. Although these two methods have brought novel experiences to tourists to a certain extent, they are still insufficient in exploring immersive experiences and it is difficult to fully touch the multi-sensory dimensions and psychological level of tourists' deep participation.

[0003] Although the display mode of the display screen is intuitive and easy to popularize, its single display makes it difficult to create an immersive sense of enclosure. VR glasses open a door to the virtual world for tourists by simulating three-dimensional space and perspective changes, but its closed wearing method may also lead to a sense of social isolation and pose a challenge to the comfort of long-term use. Therefore, designing a more diverse and deeper immersive play interaction solution has become a technical problem that technicians in this field need to solve urgently. Summary of the invention

[0004] In view of the above-mentioned defects, an embodiment of the present invention discloses an interactive method for MR experience scenes, which can improve the fun and accuracy of interaction.

[0005] A first aspect of an embodiment of the present invention discloses an interactive method for an MR experience scene, including:

[0006] Acquiring spatial position data of a corresponding user in an actual physical environment through a radar module, wherein the user performs scene interaction with a provided virtual interactive environment;

[0007] Acquire a coordinate mapping relationship between an actual environment coordinate system constructed based on the radar module and a virtual scene coordinate system constructed based on the display module, and determine a display virtual coordinate group corresponding to spatial position data in the actual physical environment in the virtual scene coordinate system according to the coordinate mapping relationship;

[0008] A corresponding interactive display effect is generated according to the display virtual coordinate group, and the interactive display effect is sent to a display module for display.

[0009] As an optional implementation manner, in the first aspect of the embodiment of the present invention, generating a corresponding interactive display effect according to the display virtual coordinate group includes:

[0010] Inputting the displayed virtual coordinates into a three-dimensional rendering engine to construct a three-dimensional display model of the corresponding user, wherein the three-dimensional display model includes the generation of geometric bodies, the specification of materials, and the simulation of lighting and shadows;

[0011] In the three-dimensional display model, each vertex is assigned a stack of UV coordinates, and the two-dimensional texture image is attached to the corresponding three-dimensional display model according to the UV coordinates according to the UV map, so as to present the pattern and color in the texture image on the three-dimensional display model; wherein the mapping parameters between the UV map and the three-dimensional display model can be adjusted, and the mapping parameters include one or more of a scaling parameter, a rotation parameter, an offset parameter, a tilt parameter and a distortion angle;

[0012] The corresponding display coordinate point is determined according to the coordinate point of the first direction in the three-dimensional display model, and the display coordinate point is data-associated with the pre-configured interactive display effect to facilitate subsequent tracking and display at the display module.

[0013] As an optional implementation, in the first aspect of the embodiment of the present invention, after attaching the two-dimensional texture image to the corresponding three-dimensional display model according to the UV coordinates according to the UV map to present the pattern and color in the texture image on the three-dimensional display model, it also includes:

[0014] By pixel mapping, the color value of each point in the two-dimensional texture image is associated with each vector in the corresponding force field image point by point, and when the vector in the force field image changes, the color value in the two-dimensional texture image will change accordingly;

[0015] The rendered picture and the corresponding vector field image are saved as corresponding image reference files, and the image reference files are stored in the corresponding RAM storage module.

[0016] As an optional implementation manner, in the first aspect of the embodiment of the present invention, before performing point-to-point data association between the color value of each point in the two-dimensional texture image and each vector in the corresponding vector field image, the method further includes:

[0017] Convert the RGB value of each point in the two-dimensional texture image into a corresponding RG value, and use the RG value as the color value of each point in the subsequent two-dimensional texture image;

[0018] The interactive method further includes:

[0019] When a change in the corresponding force field vector in the vector field image is detected, the change direction and intensity of the corresponding force field vector are determined;

[0020] The display parameters in the associated two-dimensional texture image are adjusted according to the change direction and strength of the force field vector, wherein the display parameters include the ratio or strength of red and green values.

[0021] As an optional implementation, in the first aspect of the embodiment of the present invention, the vector field image is detected based on a pressure sensor arranged at the display module or the vector field image is determined by the movement speed and direction detected by a radar module.

[0022] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the interaction method further includes:

[0023] The distance between the user and the screen is detected by a distance sensor arranged at the display module to determine a corresponding distance parameter, and the size change of the three-dimensional display model is adjusted according to the distance parameter.

[0024] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the interaction method further includes:

[0025] In response to the action combination parameters of the user during the interaction process, it is determined whether the action combination parameters match the pre-set action trigger logic. If they match, the pre-built special effect display mode is displayed.

[0026] A second aspect of an embodiment of the present invention discloses an interactive system for an MR experience scene, including:

[0027] An acquisition module: used to acquire spatial position data of a corresponding user in an actual physical environment through a radar module, wherein the user performs scene interaction with a provided virtual interactive environment;

[0028] Determination module: used to obtain the coordinate mapping relationship between the actual environment coordinate system constructed based on the radar module and the virtual scene coordinate system constructed based on the display module, and determine the display virtual coordinate group corresponding to the spatial position data in the actual physical environment in the virtual scene coordinate system according to the coordinate mapping relationship;

[0029] A generating module is used to generate a corresponding interactive display effect according to the display virtual coordinate group, and send the interactive display effect to the display module for display.

[0030] A third aspect of an embodiment of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the interactive method of the MR experience scene disclosed in the first aspect of the embodiment of the present invention.

[0031] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the interactive method of the MR experience scene disclosed in the first aspect of the embodiment of the present invention.

[0032] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0033] The interactive method of the MR experience scene in the embodiment of the present invention can accurately map the user's position in the physical space to the virtual scene by acquiring and applying the coordinate mapping relationship between the actual environment coordinate system and the virtual scene coordinate system. This precise spatial mapping not only ensures the accuracy of the interaction, but also enables the user to move freely in the virtual environment and interact with it seamlessly, further enhancing the consistency and naturalness of the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is a flowchart of an interactive method for an MR experience scene disclosed in an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of a process for performing three-dimensional rendering disclosed in an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of user interaction disclosed in an embodiment of the present invention;

[0038] Figure 4 It is a structural schematic diagram of an interactive system for an MR experience scene provided by an embodiment of the present invention;

[0039] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] It should be noted that the terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present invention and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] In the current field of interactive play, although a variety of interactive forms have emerged to enhance the sense of participation and immersion of tourists, generally speaking, these forms are still single and have limitations. Traditional interactive methods mainly rely on two core display media: one is to directly present visual content through a display screen, and the other is to build a virtual environment with the help of VR glasses to achieve a higher level of visual immersion. Although these two methods have brought novel experiences to tourists to a certain extent, they are still insufficient in exploring immersive experiences, and it is difficult to fully touch the multi-sensory dimensions and psychological depth of tourists. Based on this, the embodiment of the present invention discloses an interactive method, system, electronic device and storage medium for MR experience scenes, which can accurately map the user's position in the physical space to the virtual scene by obtaining and applying the coordinate mapping relationship between the actual environment coordinate system and the virtual scene coordinate system. This precise spatial mapping not only ensures the accuracy of the interaction, but also enables users to move freely in the virtual environment and interact with it seamlessly, further enhancing the consistency and naturalness of the user experience.

[0043] Embodiment 1

[0044] See also Figure 1 , Figure 1 It is a flow chart of the interactive method of the MR experience scene disclosed in the embodiment of the present invention. Among them, the execution subject of the method described in the embodiment of the present invention is an execution subject composed of software and / or hardware, and the execution subject can receive relevant information by wired or / and wireless means, and can send certain instructions. Of course, it can also have certain processing functions and storage functions. The execution subject can control multiple devices, such as a remote physical server or cloud server and related software, or it can be a local host or server and related software that performs related operations on a device placed somewhere. In some scenarios, multiple storage devices can also be controlled, and the storage devices can be placed in the same place or different places as the devices. For example Figure 1 As shown, the interactive method based on the MR experience scene includes the following steps:

[0045] S101: Acquiring spatial position data of a corresponding user in an actual physical environment through a radar module, wherein the user performs scene interaction with a provided virtual interactive environment;

[0046] S102: Acquire a coordinate mapping relationship between an actual environment coordinate system constructed based on the radar module and a virtual scene coordinate system constructed based on the display module, and determine a display virtual coordinate group corresponding to spatial position data in the actual physical environment in the virtual scene coordinate system according to the coordinate mapping relationship;

[0047] S103: Generate a corresponding interactive display effect according to the display virtual coordinate group, and send the interactive display effect to a display module for display.

[0048] The solution of the embodiment of the present invention uses the radar module to obtain the spatial position data of the user in the actual physical environment in real time. This method can ensure that the user's interaction in the MR environment is closely synchronized with their actions in the real world. This instant feedback and precise spatial positioning significantly enhance the user's sense of reality and immersion in the MR experience, making the integration of virtual and reality more natural and smooth.

[0049] This method can accurately map the user's position in the physical space to the virtual scene by obtaining and applying the coordinate mapping relationship between the actual environment coordinate system and the virtual scene coordinate system. This precise spatial mapping not only ensures the accuracy of the interaction, but also enables users to move freely in the virtual environment and interact with it seamlessly, further enhancing the consistency and naturalness of the user experience.

[0050] The interactive display effect is generated and updated in real time according to the user's location data, and sent to the display module for display immediately. This process ensures the real-time and dynamic nature of the interactive effect. Users can immediately see the changes caused by their actions in the virtual environment. This instant feedback mechanism greatly increases the fun and attractiveness of the interaction.

[0051] Since this method is based on radar modules and coordinate mapping technology, it can be flexibly applied to various MR experience scenarios, including but not limited to education, entertainment, games, medical care and other fields. By adjusting the content and layout of the virtual scene, a rich and diverse interactive experience can be created to meet the needs of different user groups. As an important part of MR technology, the successful application of this interactive method will promote the further development and innovation of MR technology. At the same time, as the technology continues to mature and popularize, this method is expected to be widely used in more fields, bringing more convenience and fun to people's lives.

[0052] Rendering the data scanned by radar in three dimensions and assigning the corresponding UV map is a complex process involving multiple steps and technologies. The following is a general operation flow: Data acquisition and processing, first, use the radar equipment to scan the target area to obtain the three-dimensional spatial data of the target. These data usually include information such as the shape, position, and speed of the target. Data preprocessing: Preprocess the collected raw radar data, including denoising, filtering, interpolation and other steps to improve the quality and accuracy of the data. Three-dimensional modeling, model construction: Build a three-dimensional model of the target in the three-dimensional modeling software based on the preprocessed radar data. This usually involves the generation of geometry, the specification of materials, and the simulation of lighting and shadows. Mesh optimization: Optimize the mesh of the three-dimensional model to reduce the complexity of the model and improve rendering efficiency. UV mapping, UV coordinate generation: In the three-dimensional model, UV coordinates are assigned to each vertex of the model. UV coordinates define how to map a two-dimensional image (texture) to the surface of the three-dimensional model. Texture design: Design texture images according to requirements, which can be colors, patterns, materials, etc. Texture mapping: Use UV mapping technology to accurately attach the designed texture image to the 3D model according to the UV coordinates. 3D rendering, rendering settings: Set rendering parameters in the 3D rendering software, including lighting, shadows, materials, camera perspectives, etc. Rendering calculation: The rendering engine calculates the propagation of light in the scene, as well as the reflection and refraction of light by the surface material of the object based on the set parameters and the data of the 3D model, to generate realistic images. Effect adjustment: Adjust the rendering effect as needed, such as color correction, blurring and sharpening, light and shadow enhancement, etc. Interactive assignment, data association: Associate the data obtained by radar scanning with the corresponding parts of the 3D model so that the state of the model can be updated in real time during the interaction process. Interaction design: Design interaction logic so that users can interact with the 3D model in some way (such as touch, gesture, voice, etc.) and see the changes in the model state in real time. Post-processing and output, post-processing: Post-process the rendered image, such as adding special effects, adjusting color balance, etc., to further enhance the visual effect. Output and display: Output the processed images or videos to display devices, or embed them into applications for user interaction. Figure 3 As shown, it is the corresponding interactive effect diagram.

[0053] More preferably, Figure 2 As shown, the generating of the corresponding interactive display effect according to the display virtual coordinate group includes:

[0054] S1031: inputting the displayed virtual coordinates into a three-dimensional rendering engine to construct a three-dimensional display model of the corresponding user, wherein the three-dimensional display model includes the generation of geometric bodies, the specification of materials, and the simulation of lighting and shadows;

[0055] S1032: In the three-dimensional display model, each vertex is assigned a stack of UV coordinates, and the two-dimensional texture image is attached to the corresponding three-dimensional display model according to the UV coordinates according to the UV map, so as to present the pattern and color in the texture image on the three-dimensional display model; wherein the mapping parameters between the UV map and the three-dimensional display model can be adjusted, and the mapping parameters include one or more of a scaling parameter, a rotation parameter, an offset parameter, a tilt parameter, and a distortion angle;

[0056] S1033: Determine corresponding display coordinate points according to the coordinate points in the first direction in the three-dimensional display model, and associate data between the display coordinate points and pre-configured interactive display effects to facilitate subsequent tracking and display at the display module.

[0057] The solution of the embodiment of the present invention constructs a 3D display model of the user through a 3D rendering engine, including fine generation of geometric bodies, precise specification of materials, and simulation of lighting and shadows, making the user image in the virtual environment more realistic and enhancing the visual sense of reality and immersion. This high visual fidelity helps to improve the overall quality of user experience.

[0058] Use UV mapping to attach 2D texture images to 3D display models, and allow the adjustment of mapping parameters between UV mapping and models (such as scaling, rotation, offset, tilt and distortion angle, etc.), which provides developers with great flexibility to customize and optimize texture effects. By adjusting these parameters, you can ensure that the texture image is perfectly integrated with the 3D model, further enhancing the visual expression and detail richness of the model.

[0059] By determining the coordinate points of the first direction in the 3D display model to associate the corresponding display coordinate points, and by associating these coordinate points with the pre-configured interactive display effects, accurate tracking and presentation of the interactive effects are achieved. This data association mechanism ensures that every action of the user in the virtual environment can trigger the corresponding interactive effects, and these effects can be displayed in real time and accurately on the display module, thereby enhancing the real-time and responsiveness of the interaction.

[0060] Integrating complex operations such as 3D rendering, texture mapping, and interactive effect tracking into an orderly process and improving processing efficiency by optimizing algorithms and data structures can help improve the performance and stability of the entire MR system. This can not only reduce system delays and freezes, but also improve the smoothness and satisfaction of the user experience.

[0061] This more preferred approach provides developers with more tools and options to innovate and customize the MR experience. By adjusting the parameters of the 3D model, the texture mapping method, and the configuration of the interactive effects, developers can create unique and attractive MR scenes to meet the personalized needs of different user groups. This high degree of customizability and innovation will help promote the continuous development and application expansion of MR technology.

[0062] More preferably, after attaching the two-dimensional texture image to the corresponding three-dimensional display model according to the UV coordinates according to the UV map so as to present the pattern and color in the texture image on the three-dimensional display model, the method further includes:

[0063] By pixel mapping, the color value of each point in the two-dimensional texture image is associated with each vector in the corresponding force field image point by point, and when the vector in the force field image changes, the color value in the two-dimensional texture image will change accordingly;

[0064] The rendered picture and the corresponding vector field image are saved as corresponding image reference files, and the image reference files are stored in the corresponding RAM storage module.

[0065] The method of the embodiment of the present invention performs point-to-point data association between the color values ​​in the two-dimensional texture image and the vectors in the force field image through pixel mapping, thereby realizing dynamic interaction between the texture image and the physical field (such as the force field). When the vectors in the force field image change, this change can be reflected in the color value of the two-dimensional texture image in real time, thereby presenting a dynamically changing visual effect on the three-dimensional display model. This technology not only increases the visual layering and richness, but also makes the virtual environment more vivid and realistic.

[0066] Since the color value of the texture image can change with the change of the force field vector, the user can intuitively feel the impact of the change of the physical field on the appearance of the virtual object when interacting with the virtual environment. This intuitive feedback mechanism enhances the user's immersion and interactive experience, allowing the user to integrate into the virtual environment more naturally.

[0067] Saving the rendered image and the corresponding vector field image as an image reference file and storing it in the RAM storage module can significantly improve rendering performance. RAM has high-speed reading and writing capabilities. Storing rendering results and vector field data in RAM can speed up data access and reduce delays and freezes during the rendering process. At the same time, the use of image reference files can also help reduce the number of repeated renderings, further improving rendering efficiency.

[0068] Supports real-time updates and dynamic adjustments: Since the rendering results and vector field data are stored in RAM, they can support real-time updates and dynamic adjustments. When the physical field or other parameters in the virtual environment change, the data in RAM can be updated immediately and the screen can be re-rendered to reflect these changes. This real-time update mechanism enables the virtual environment to more accurately simulate the dynamic changes in the real world.

[0069] By storing rendering results and vector field data separately and introducing the integration technology of pixel mapping and force field images, rendering and display problems in complex scenes can be handled more flexibly. This technology is not only suitable for simple 3D model rendering, but can also be extended to the processing of large-scale scenes, complex lighting conditions, multi-physics field coupling and other complex situations, providing strong support for the widespread application of MR technology.

[0070] The specific implementation methods of the above-mentioned 3D rendering, UV mapping, RG value association and parameter adjustment, and post-processing can be elaborated as follows:

[0071] 3D rendering and UV mapping, 3D rendering: Data input: First, the 3D data obtained by radar scanning is input into the 3D rendering engine. These data usually include information such as the geometric shape, position, and orientation of the object.

[0072] Model construction: The rendering engine constructs a 3D model based on the input data. This involves the generation of geometry, the specification of materials, and the simulation of lighting and shadows. Rendering calculation: The engine generates realistic images by calculating how light propagates in the scene, as well as the reflection and refraction of light by the surface material of the object.

[0073] UV Mapping: UV Coordinate Generation: In a 3D model, each vertex is assigned a pair of UV coordinates. These coordinates define how to map a 2D image onto the 3D model surface. Texture Mapping: Using UV mapping technology, a 2D image (texture) is accurately attached to a 3D model according to the UV coordinates. In this way, the model surface will show the details and colors in the texture image. Adjusting Parameters: By adjusting the parameters of UV mapping (such as scaling, rotation, offset, etc.), you can achieve fine control over the texture image on the model surface to achieve the desired visual effect.

[0074] RG value association and parameter adjustment, RG value association: Color space conversion: In some cases, it is necessary to convert the image in the RGB color space into RG value (red and green value) form for processing. This usually involves discarding the information of the blue channel and retaining only the red and green channels. Point-to-point association: The converted RG value image is point-to-point associated with another image (such as a force field image). This can be achieved through specific algorithms in image processing software, such as pixel mapping, color mixing, etc.

[0075] Parameter adjustment: Dynamic effect control: By adjusting the parameters in the RG value image (such as the ratio and intensity of the red and green values), you can control the change direction and intensity of the effect in the associated image. This can achieve dynamic effects such as color gradient and shape distortion. Effect preview: In the process of adjusting parameters, you can use the preview function to view the change of the effect in real time for further optimization and adjustment.

[0076] Post-processing and rich effects, post-processing: Color correction: Use color correction tools to adjust the overall tone, brightness, contrast and other parameters of the image to improve the visual effect of the image. Blur and sharpen: Enhance the details of the image or reduce noise by applying blur or sharpening filters. Light and shadow enhancement: Use light and shadow effect plug-ins or algorithms to simulate the changes in light and shadow in the real world to make the image more three-dimensional and vivid. Rich effects: Special effects addition: Various special effects elements (such as flames, water waves, smoke, etc.) can be added to enrich the expressiveness of the image. Animation and interaction: Combine animation technology and interactive design to enable images to respond to user operations or automatically play animation sequences, thereby providing a richer interactive experience.

[0077] In summary, 3D rendering, UV mapping, RG value association and parameter adjustment, and post-processing are the key technical links to achieve high-quality interactive images. By comprehensively applying these technical means, realistic, vivid and attractive visual effects can be created.

[0078] In the context of image processing and interactive design, the concept of "vectors of force field change effects" can be understood as a series of operations or parameters that can affect the arrangement, shape, color and other properties of elements in the image, thereby creating a specific visual effect or interactive feedback. Specifically, these vectors can include the following meanings:

[0079] Definition of force field: A force field is a vector field in which each point is associated with a vector (i.e., force and direction). In image processing, we can regard an image as a two-dimensional force field, in which each pixel or pixel region can exert a force on other pixels. Application: By simulating the interaction in the force field, the deformation, distortion, stretching and other effects of the image can be achieved, creating a dynamic and layered visual experience.

[0080] Changing the vector of the effect, parameter adjustment: These vectors are usually achieved by adjusting a series of parameters, such as the magnitude of the force, direction, range of action, decay rate, etc. By changing these parameters, you can control the degree and way in which the force field affects the image.

[0081] Specific implementation: The magnitude of the force: determines the degree of deformation of the image element. The direction of the force: determines the direction of deformation of the image element. The scope of action: defines the size of the area affected by the force field. Attenuation speed: simulates the natural phenomenon that the force field weakens with increasing distance.

[0082] RG value UV map association, RG value: usually refers to the value of the red and green color channels, which can be used to represent the color information in the image. UV map: UV map is a technology in texture mapping, which is used to map a two-dimensional texture image to a three-dimensional model. Here, the UV map may be used to map the RG value to a specific area or pixel of the image.

[0083] Association: Through a specific algorithm or tool, the UV map of the RG value is associated point-to-point with another image (such as a force field image). In this way, when the vector in the force field image changes, the RG value image will also change accordingly, thereby achieving a change in visual effect.

[0084] Post-processing, effect distortion and dot matrix processing: After the force field is applied, the image may need to be further distorted or dot matrixed to enhance the visual effect or meet specific interactive needs. Specific components: These processing steps may require the use of specific image processing software, plug-ins or algorithms to achieve.

[0085] Enriching interactive images, Purpose: Through the above series of processing steps, more vivid, interesting and interactive images can be created. Application scenarios: This technology can be widely used in game development, virtual reality, augmented reality, advertising creativity and other fields to provide users with an immersive visual experience

[0086] In summary, the vector of the force field change effect affects the visual performance of the image by simulating the interaction in the physical force field, and enriches the effect of the interactive picture through a series of parameter adjustments and subsequent processing.

[0087] More preferably, before performing point-to-point data association between the color value of each point in the two-dimensional texture image and each vector in the corresponding vector field image, the method further includes:

[0088] Convert the RGB value of each point in the two-dimensional texture image into a corresponding RG value, and use the RG value as the color value of each point in the subsequent two-dimensional texture image;

[0089] The interactive method further includes:

[0090] When a change in the corresponding force field vector in the vector field image is detected, the change direction and intensity of the corresponding force field vector are determined;

[0091] The display parameters in the associated two-dimensional texture image are adjusted according to the change direction and strength of the force field vector, wherein the display parameters include the ratio or strength of red and green values.

[0092] The embodiment of the present invention converts the RGB value of each point in the two-dimensional texture image into an RG value, thereby removing the information of the blue channel. This step can significantly reduce the amount of data processing and improve processing efficiency. In some application scenarios, the information of the blue channel may not be necessary, or its influence can be ignored. Therefore, this conversion not only simplifies the data processing process, but also makes subsequent data association and analysis more efficient and accurate.

[0093] When the force field vector changes in the vector field image are detected, the direction and strength of these changes can be determined in real time, and the display parameters (such as the red-green value ratio or intensity) in the two-dimensional texture image can be dynamically adjusted based on this information. This mechanism provides users with intuitive visual feedback, allowing users to more clearly observe the changing trend and strength of the force field, thereby enhancing the interactive experience between users and image data.

[0094] In the MR (mixed reality) screen interaction method, the choice of using RG values ​​(red and green values) instead of the complete RGB (red, green and blue) parameter adjustment method may be based on the following considerations: simplifying calculations and improving efficiency. RGB contains three color channels, each of which needs to be processed independently, while RG values ​​only involve two channels, so the calculation is simpler. In real-time interactive systems, simplifying calculations can significantly improve the system's response speed and efficiency, reduce latency, and improve user experience.

[0095] Data processing volume: Reducing one color channel means reducing the amount of data that needs to be processed, which is especially important for MR systems that need to process large amounts of data at high speed. Smaller amounts of data can reduce the demand for hardware resources and reduce system costs. In some application scenarios, more emphasis may be placed on the contrast and recognition of visual effects rather than the integrity and richness of colors. By emphasizing the two main color channels of red and green, the RG value can create a sharper visual contrast, which helps to improve the communication effect of specific information.

[0096] Innovation of visual effects: Using RG values ​​instead of full RGB parameters can provide new possibilities for innovation of visual effects. By limiting the color channels, designers and developers can be inspired to create unique visual effects and interactive experiences within a limited color range. Environmental adaptability: Under different lighting conditions and environments, the performance of the three RGB color channels may be different. RG values ​​may be more adaptable to certain specific ambient lighting conditions, thus providing more stable and consistent visual effects.

[0097] In summary, the choice of using RG values ​​instead of the complete RGB parameter adjustment method is based on comprehensive considerations of computing efficiency, visual effect requirements, system compatibility, and specific application scenarios. In actual applications, the appropriate color representation method can be selected according to specific needs and scene characteristics.

[0098] More preferably, the vector field image is detected by a pressure sensor arranged at the display module or the vector field image is determined by the movement speed and direction detected by a radar module.

[0099] Combining the data from the pressure sensor and radar module can achieve the fusion and complementarity of multi-source data. The pressure sensor provides local pressure distribution information, while the radar module provides global movement speed and direction information. The combination of the two can construct a more comprehensive and accurate vector field image, providing strong support for subsequent data analysis and processing.

[0100] When performing point-to-point data association with a 2D texture image, the display parameters of the 2D texture image (such as the red-green value ratio or intensity) can be adjusted more accurately based on the real-time and accurate vector field image. This dynamic adjustment not only enhances the visual feedback effect, but also enables users to more intuitively understand the physical meaning behind the data, improving the interactive experience.

[0101] More preferably, the interactive method further comprises:

[0102] The distance between the user and the screen is detected by a distance sensor arranged at the display module to determine a corresponding distance parameter, and the size change of the three-dimensional display model is adjusted according to the distance parameter.

[0103] The solution of the embodiment of the present invention detects the distance between the user and the screen in real time, and the system can automatically adjust the size of the three-dimensional display model to ensure that the user can obtain the best visual effect at different distances. This personalized adjustment makes the user experience smoother and more comfortable, and enhances the sense of interaction between the user and the displayed content. When the user is close to the screen, the system can appropriately enlarge the three-dimensional display model so that the user can see the details of the model more clearly; when the user is away from the screen, the system will automatically reduce the model to avoid information overload or visual confusion. This intelligent adjustment method helps users receive and understand the displayed content more effectively.

[0104] More preferably, the interactive method further comprises:

[0105] In response to the action combination parameters of the user during the interaction process, it is determined whether the action combination parameters match the pre-set action trigger logic. If they match, the pre-built special effect display mode is displayed.

[0106] The embodiments of the present invention allow users to trigger special effects display through specific action combinations, greatly increasing the user's participation and immersion in the interactive process. Users are no longer just passively receiving information, but can influence and control the displayed content through their own actions. This initiative and control make the interactive experience more vivid and interesting.

[0107] Different users may have different action combination habits and preferences. By identifying and responding to these personalized action combination parameters, the system can provide each user with a unique special effect display method. This personalized experience not only enhances user satisfaction, but also helps to build a differentiated brand image and market competitiveness.

[0108] The introduction of action combination parameters makes the interaction more flexible and diverse. Developers can define multiple action combinations and corresponding trigger logic according to actual needs to achieve rich interaction effects. In addition, with the continuous advancement of technology and the continuous changes in user needs, this interaction method also has good scalability and can support more action combinations and special effect display methods.

[0109] By presetting the action trigger logic and constructing the special effect display method, the system can quickly respond and display the corresponding special effects after the user completes a specific action combination. This instant feedback mechanism ensures the consistency and smoothness of the user experience and avoids negative experiences caused by delays or freezes.

[0110] Allowing users to trigger special effects through action combinations also provides a new way for creative expression. Users can design unique action combinations and special effects display methods based on their imagination and creativity, thereby achieving personalized creative expression and artistic presentation. This innovative interactive method helps to stimulate users' creativity and imagination, and promote the development and innovation of related fields.

[0111] The scheme of the embodiment of the present invention specifically includes: 1. Game mode: When the experiencer approaches the scanning area in front of the screen, the visual effect content of the screen can change according to the area corresponding to the position of the experiencer. 2. Interaction implementation method: The radar scanning data is used as the interaction point, and after three-dimensional conversion and material assignment, it is rendered as a part of the visual special effects processing, and used as an image reference for effect generation. 3. Visual effect implementation method: In terms of effect implementation, the data scanned by the radar is subjected to three-dimensional rendering processing so that it can be used as a preliminary interactive assignment and assigned a corresponding UV map. The UV map that references the RG value enables the image to be point-to-point associated with another received image, and combined with parameter adjustment, it is used as a vector for changing the effect in the force field, so that the desired effect is obtained in image processing according to the corresponding calculation, and for subsequent effect distortion and dot matrix processing, specific components are used for post-processing to further enrich the effect of the interactive screen.

[0112] The interactive method of the MR experience scene in the embodiment of the present invention can accurately map the user's position in the physical space to the virtual scene by acquiring and applying the coordinate mapping relationship between the actual environment coordinate system and the virtual scene coordinate system. This precise spatial mapping not only ensures the accuracy of the interaction, but also enables the user to move freely in the virtual environment and interact with it seamlessly, further enhancing the consistency and naturalness of the user experience.

[0113] Embodiment 2

[0114] See also Figure 4 , Figure 4 Schematic diagram of the structure of the interactive system of the MR experience scene disclosed in the embodiment of the present invention. Figure 4 As shown, the interactive system of the MR experience scene may include:

[0115] Acquisition module 21: used to acquire spatial position data of a corresponding user in an actual physical environment through a radar module, wherein the user performs scene interaction with a provided virtual interactive environment;

[0116] Determination module 22: used to obtain the coordinate mapping relationship between the actual environment coordinate system constructed based on the radar module and the virtual scene coordinate system constructed based on the display module, and determine the display virtual coordinate group corresponding to the spatial position data in the actual physical environment in the virtual scene coordinate system according to the coordinate mapping relationship;

[0117] The generating module 23 is used to generate a corresponding interactive display effect according to the display virtual coordinate group, and send the interactive display effect to the display module for display.

[0118] The interactive method of the MR experience scene in the embodiment of the present invention can accurately map the user's position in the physical space to the virtual scene by acquiring and applying the coordinate mapping relationship between the actual environment coordinate system and the virtual scene coordinate system. This precise spatial mapping not only ensures the accuracy of the interaction, but also enables the user to move freely in the virtual environment and interact with it seamlessly, further enhancing the consistency and naturalness of the user experience.

[0119] Embodiment 3

[0120] See also Figure 5 , Figure 5 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may be a computer, a server, etc. Of course, in certain circumstances, it may also be a smart device such as a mobile phone, a tablet computer, a monitoring terminal, and an image acquisition device with processing functions. Figure 5 As shown, the electronic device may include:

[0121] A memory 510 storing executable program codes;

[0122] a processor 520 coupled to the memory 510;

[0123] The processor 520 calls the executable program code stored in the memory 510 to execute part or all of the steps in the interactive method of the MR experience scene in the first embodiment.

[0124] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute part or all of the steps in the interactive method of MR experience scene in embodiment 1.

[0125] The embodiment of the present invention further discloses a computer program product, wherein when the computer program product is run on a computer, the computer is enabled to execute part or all of the steps in the interactive method of the MR experience scene in the first embodiment.

[0126] An embodiment of the present invention further discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps in the interactive method of the MR experience scene in embodiment one.

[0127] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0128] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0129] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the method described in each embodiment of the present invention.

[0131] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0132] A person of ordinary skill in the art can understand that some or all of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0133] The interactive method, system, electronic device and storage medium of the MR experience scene disclosed in the embodiments of the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An interactive method for MR experience scene, characterized in that: include: Acquiring spatial position data of a corresponding user in an actual physical environment through a radar module, wherein the user performs scene interaction with a provided virtual interactive environment; Acquire a coordinate mapping relationship between an actual environment coordinate system constructed based on the radar module and a virtual scene coordinate system constructed based on the display module, and determine a display virtual coordinate group corresponding to spatial position data in the actual physical environment in the virtual scene coordinate system according to the coordinate mapping relationship; A corresponding interactive display effect is generated according to the display virtual coordinate group, and the interactive display effect is sent to a display module for display.

2. The interactive method of MR experience scene according to claim 1, characterized in that: Generating a corresponding interactive display effect according to the display virtual coordinate group includes: Inputting the displayed virtual coordinates into a three-dimensional rendering engine to construct a three-dimensional display model of the corresponding user, wherein the three-dimensional display model includes the generation of geometric bodies, the specification of materials, and the simulation of lighting and shadows; In the three-dimensional display model, each vertex is assigned a stack of UV coordinates, and the two-dimensional texture image is attached to the corresponding three-dimensional display model according to the UV coordinates according to the UV map, so as to present the pattern and color in the texture image on the three-dimensional display model; wherein the mapping parameters between the UV map and the three-dimensional display model can be adjusted, and the mapping parameters include one or more of a scaling parameter, a rotation parameter, an offset parameter, a tilt parameter and a distortion angle; The corresponding display coordinate point is determined according to the coordinate point of the first direction in the three-dimensional display model, and the display coordinate point is data-associated with the pre-configured interactive display effect to facilitate subsequent tracking and display at the display module.

3. The interactive method of MR experience scene according to claim 2, characterized in that: After attaching the two-dimensional texture image to the corresponding three-dimensional display model according to the UV coordinates based on the UV map to present the pattern and color in the texture image on the three-dimensional display model, the method further includes: By pixel mapping, the color value of each point in the two-dimensional texture image is associated with each vector in the corresponding force field image point by point, and when the vector in the force field image changes, the color value in the two-dimensional texture image will change accordingly; The rendered picture and the corresponding vector field image are saved as corresponding image reference files, and the image reference files are stored in the corresponding RAM storage module.

4. The interactive method of MR experience scene according to claim 3, characterized in that: Before the point-to-point data association of the color value of each point in the two-dimensional texture image with each vector in the corresponding vector field image, the method further includes: Convert the RGB value of each point in the two-dimensional texture image into a corresponding RG value, and use the RG value as the color value of each point in the subsequent two-dimensional texture image; The interactive method further includes: When a change in the corresponding force field vector in the vector field image is detected, the change direction and intensity of the corresponding force field vector are determined; The display parameters in the associated two-dimensional texture image are adjusted according to the change direction and strength of the force field vector, wherein the display parameters include the ratio or strength of red and green values.

5. The interactive method of MR experience scene according to claim 3, characterized in that: The vector field image is detected based on a pressure sensor arranged at the display module or the vector field image is determined by the movement speed and direction detected by a radar module.

6. The interactive method of MR experience scene according to claim 2, characterized in that: The interactive method further includes: The distance between the user and the screen is detected by a distance sensor arranged at the display module to determine a corresponding distance parameter, and the size change of the three-dimensional display model is adjusted according to the distance parameter.

7. The interactive method of MR experience scene according to claim 1, characterized in that: The interactive method further includes: In response to the action combination parameters of the user during the interaction process, it is determined whether the action combination parameters match the pre-set action trigger logic. If they match, the pre-built special effect display mode is displayed.

8. An interactive system for MR experience scenes, characterized in that: include: An acquisition module: used to acquire spatial position data of a corresponding user in an actual physical environment through a radar module, wherein the user performs scene interaction with a provided virtual interactive environment; Determination module: used to obtain the coordinate mapping relationship between the actual environment coordinate system constructed based on the radar module and the virtual scene coordinate system constructed based on the display module, and determine the display virtual coordinate group corresponding to the spatial position data in the actual physical environment in the virtual scene coordinate system according to the coordinate mapping relationship; A generating module is used to generate a corresponding interactive display effect according to the display virtual coordinate group, and send the interactive display effect to the display module for display.

9. An electronic device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the interactive method of the MR experience scene according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the interactive method of the MR experience scene according to any one of claims 1 to 7.

Images