A virtual reality fusion method and device

Abstract

Embodiments of the present application provide a virtual reality fusion method and device for ensuring the rationality of virtual scene fusion real scene, comprising: obtaining real object feature information containing a plurality of objects in the real scene; obtaining virtual environment feature information in the virtual scene; obtaining virtual objects corresponding to the objects in the real scene in the virtual object library according to the real object feature information and the virtual environment feature information; calculating and fusing the virtual object feature information of the obtained virtual objects according to the real object feature information; rendering the virtual scene containing the virtual objects according to the virtual object feature information and the virtual scene. Real objects can be fused into virtual environments in a unified style. When users interact with real objects in virtual environments, this technology can make the appearance and behavior of real objects consistent with other elements in virtual environments, making it easier for users to immerse themselves in virtual environments.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, particularly to the field of interaction technology between virtual and real environments, and more specifically, to the field of virtual reality fusion technology. Background Technology

[0002] Virtual reality (VR) technology is a computer-simulated, three-dimensional virtual world that provides users with a simulated experience of sight and other senses, making them feel as if they are actually there, able to observe objects in the three-dimensional space in real time and without limitations. VR technology is a comprehensive technology integrating computer technology, sensor technology, human psychology, and physiology. It uses computer simulation systems to simulate the external environment, primarily simulating the environment, skills, sensing devices, and perception, providing users with a multi-information, three-dimensional, dynamic, and interactive simulation experience.

[0003] Currently, virtual reality (VR) technology is a comprehensive technology composed of three main categories: stereoscopic display technology, 3D modeling technology, and natural interaction technology. Based on the degree of "immersion" and interactivity, VR technology is classified into four typical types: immersive VR systems, desktop VR systems, augmented VR systems, and distributed VR systems.

[0004] Immersive virtual reality (VR) systems are one of the most common VR technologies, allowing users to be completely immersed in a virtual environment. This technology requires interaction using devices such as head-mounted displays (HMDs) and controllers. VR technology possesses three main characteristics: immersion, interactivity, and imaginative possibilities, and has been widely applied in fields such as gaming, education, training, and remote collaboration. Current virtual scenes provide users with a completely different virtual environment from the real world by constructing entirely virtual scenarios, offering a high degree of immersion. The inventors analyze the existing technology as follows:

[0005] Patent 1: CN 112346572 A A method, system, and electronic device for realizing virtual-real fusion. This invention discloses a method, system, and electronic device for realizing virtual-real fusion. This method uses digital twin technology to accurately replicate and construct a virtual 3D scene based on a real-world scene. Based on motion capture technology, it locates and tracks the position and movement of people / objects in the real-world scene in real time, integrating the acquired real-world images into the virtual scene to obtain a virtual-real fusion image, which is then presented to the user. Using the virtual-real fusion method provided by this invention, users can simultaneously experience a realistic interactive feel while obtaining a good sense of visual immersion.

[0006] Patent 2: CN 114419293 A A data processing method, apparatus, and device for augmented reality. This invention relates to a data processing method, apparatus, and device for augmented reality. The method includes the following steps: acquiring real-world image data of a target area in real time using a camera device; performing virtual-real fusion processing on the real-world image data to obtain a superimposed virtual-real scene; and displaying the superimposed virtual-real scene through a display device. This augmented reality data processing method acquires data from a real-world rescue or training site using a camera device, performs virtual fusion processing on the real-world image data to obtain a superimposed virtual scene, and displays the superimposed scene through a display device. This achieves AR display of the real-world virtual-real environment of a rescue or training site, allowing users to see a comprehensive scene of the real world superimposed with virtual content. This solves problems such as insufficient training scene coverage, low training efficiency, limited command methods, and high costs of simulating real-world scenarios in existing training operations.

[0007] Patent 3: CN 113850920 A A method, system, device, and storage medium for virtual-real fusion based on spatial positioning. This invention discloses a method, system, device, and storage medium for virtual-real fusion based on spatial positioning. The method includes: determining multiple spatial positioning base points in a real-world scene and determining the relative positional relationship between each spatial positioning base point and the object to be interacted with; constructing a first coordinate system in a virtual digital space and determining the first coordinate of each spatial positioning base point in the first coordinate system; modeling the real-world scene according to the first coordinate system to obtain a virtual digital model, and then determining the area of ​​the model to be fused in the virtual digital model according to the relative positional relationship and the first coordinate; and performing virtual-real fusion between the area of ​​the model to be fused and the object to be interacted with based on the spatial positioning base points. This invention, on the one hand, reduces the number and process of comparing and recognizing the object with the digital model, improving the efficiency of mixed reality display and interaction; on the other hand, it ensures the real-time and preparedness of virtual-real fusion, greatly enhancing the user's interactive experience, and can be widely applied in the field of mixed reality technology.

[0008] Patent 4: CN 110942511 A Method and Apparatus for Reconstructing Indoor Scene Models. This invention discloses a method and apparatus for reconstructing indoor scene models. The method includes: receiving a single panoramic image and preprocessing the single panoramic image; extracting structural feature points from the preprocessed single panoramic image and restoring the scene layout; detecting the main objects in the single panoramic image and matching the object models corresponding to the main objects in a model library; combining the scene layout and the object models to generate a reconstructed model of the entire indoor scene; and displaying the reconstructed model using a 3D modeling tool to enable interaction with the objects in the reconstructed model.

[0009] The way current technologies construct virtual scenes necessitates that users consider their safety in the real environment. Users need to pre-define a safe area to ensure they are not disturbed or triggered by external factors while interacting with the virtual environment. Technically, developers typically restrict the user's interaction area to a specific, object-free space before activating virtual interaction functions.

[0010] Currently, the existing solutions to the above problems are as follows: one approach is to directly circumvent the technical difficulties by capturing external images with a camera and transmitting them to the virtual environment. Another approach is for developers to place virtual objects corresponding to real objects within a large virtual environment, ensuring that the virtual objects maintain the same type and shape as the real objects, thus guaranteeing user safety and the interactivity of real objects within the virtual environment. However, neither of these existing solutions solves the technical problem of maintaining a consistent style within the virtual environment. Both of these virtual-real fusion methods easily disrupt the sense of immersion in many scenarios. For example, in a virtual beach environment, if a leather sofa appears in the user's field of vision, this presentation method easily breaks the user's sense of immersion.

[0011] In the process of realizing the concept of this invention, the inventors discovered that there are at least the following problems in the related technologies: there is a lack of uniformity in the presentation of virtual objects and virtual environments in terms of categories, scenes, styles, etc., which can easily break the user's sense of immersion. Summary of the Invention

[0012] In view of this, the technical problem to be solved by the present invention is to provide a virtual reality fusion method and device, which solves the problem of the current style of virtual environment and real object projection, which destroys the user's immersion.

[0013] This invention proposes a virtual reality fusion method to ensure the rationality of merging a virtual scene with a real scene, comprising: acquiring feature information of real objects containing multiple objects in the real scene; acquiring feature information of the virtual environment in the virtual scene; acquiring virtual objects corresponding to the objects in the real scene from a virtual object library based on the real object feature information and the virtual environment feature information; calculating and fusing virtual object feature information of the virtual objects based on the real object feature information; and rendering a virtual scene containing the virtual objects based on the virtual object feature information and the virtual scene.

[0014] The virtual reality fusion method proposed in this invention involves acquiring real-world object feature information containing multiple objects within a real-world scene. This real-world object feature information includes real-world individual feature information for each object, where the real-world individual feature information F corresponding to the i-th object... RO i Includes: object category Tro i Object pose P ro i Object size B ro i Functional feature set A ro i .

[0015] The virtual reality fusion method proposed in this invention includes: acquiring virtual environment feature information within a virtual scene; identifying the virtual scene; and acquiring the virtual environment feature information based on the identified virtual scene. The virtual environment feature information includes: environment category E. vs Environmental Style S vs .

[0016] The virtual reality fusion method proposed in this invention involves obtaining virtual objects corresponding to objects in the real scene from a virtual object library based on the feature information of the real objects and the feature information of the virtual environment. The virtual object library contains multiple virtual objects, and each virtual object has virtual individual feature information, including: object category T. vo Object size B vo Functional feature set A vo Environmental Category Set E ov Environmental style collection S ov .

[0017] The virtual reality fusion method proposed in this embodiment of the invention includes virtual individual feature information of the virtual objects, wherein the virtual individual feature information F of the k-th virtual object... vo k For: F vo k ={T vo k B vo k A vo k E ov k ,S ov k}, k = 1, 2…K; where: T vo k B is the object category of the k-th virtual object; vo k Let A be the size of the k-th virtual object; vo k Let A be the functional feature set of the k-th virtual object. vo k ={A vo k1 A vok2 A vo km}, m=1,2…M,A vo km E represents the m-th functional feature of the k-th virtual object; ov k Let E be the set of environmental categories for the k-th virtual object. ov k ={E ov k1 E ov k2 ,…,E ov kp}, p=1,2…P,E ov kp This indicates that the k-th virtual object can be used in the p-th environment category; S ov k S is the environmental style set of the k-th virtual object. ov k ={S ov k1 ,S ov k2 ,…,S ov kq},q=1,2…Q,S ov kq This indicates that the k-th virtual object can be used in the q-th environment style.

[0018] The virtual reality fusion method proposed in this invention involves obtaining virtual objects corresponding to objects in the real scene from a virtual object library based on the feature information of the real objects and the feature information of the virtual environment. The selection of virtual objects from the virtual object library corresponding to objects in the real scene is subject to the following constraints: in, The size of the i-th object in the real-world object feature information; The size of the k-th virtual object in the virtual object library; This is the set of functional features of the i-th object in the real-world object feature information; The set of functional features of the k-th virtual object in the virtual object library; The p-th environment category of the k-th virtual object in the virtual object library; E represents the p-th environment style of the k-th virtual object in the virtual object library. vs The environment category of the virtual environment feature information; S vsThe environment style is defined as the feature information of the virtual environment. That is, when the k-th virtual object and the i-th real object have the most identical functional features, and any environment category of the k-th virtual object is consistent with the virtual environment category, and any style of the k-th virtual object is consistent with the virtual environment style, then the k-th virtual object and the corresponding i-th real object have the greatest size similarity.

[0019] The virtual reality fusion method proposed in this invention, wherein calculating and fusing virtual object feature information of the virtual object based on the feature information of the real object includes: obtaining the object pose P of the object within the feature information of the real object. ro Set the pose information P of the virtual object in the virtual object feature information. vo =P ro .

[0020] The virtual reality fusion method proposed in this embodiment of the invention further includes, before rendering a virtual scene containing the virtual objects based on the virtual object feature information and the virtual scene: obtaining the origin of the coordinates of the real scene, aligning the origin of the coordinates of the virtual scene with the origin of the coordinates of the real scene in the virtual scene; obtaining the size of the objects in the real scene and the size of the corresponding virtual objects, and scaling the virtual objects to the same size as the objects in the real scene.

[0021] The virtual reality fusion method proposed in this invention includes rendering a virtual scene containing the virtual objects based on the virtual object feature information and the virtual scene. This includes rendering the virtual object corresponding to the object if the positions of the virtual objects in the virtual scene overlap with those of the objects in the real scene.

[0022] The virtual reality fusion method proposed in this invention includes rendering a virtual scene containing the virtual objects based on the virtual object feature information and the virtual scene, which includes: not rendering the virtual objects corresponding to walkable areas within the real scene.

[0023] The virtual reality fusion method proposed in this invention includes rendering a virtual scene containing the virtual objects based on the virtual object feature information and the virtual scene, comprising: rendering virtual objects corresponding to objects in non-walkable areas of the real scene and virtual objects in the virtual scene.

[0024] This invention also proposes a virtual reality fusion device to ensure the rationality of fusion between a virtual scene and a real scene, comprising: a real environment recognition module for acquiring feature information of real objects containing multiple objects in the real scene; a virtual environment recognition module for acquiring feature information of the virtual environment in the virtual scene; a virtual object selection module for acquiring virtual objects corresponding to the objects in the real scene from a virtual object library based on the real object feature information and the virtual environment feature information; a virtual-real object fusion module for calculating and fusing virtual object feature information of the virtual objects based on the real object feature information; and a rendering module for rendering a virtual scene containing the virtual objects based on the virtual object feature information and the virtual scene.

[0025] The virtual reality fusion device proposed in this embodiment of the invention includes a real environment recognition module, wherein the real object feature information includes real individual feature information of each object, wherein the real individual feature information F corresponding to the i-th object is... RO i Includes: object category T ro i Object pose P ro i Object size B ro i Functional feature set A ro i .

[0026] The virtual reality fusion device proposed in this embodiment of the invention includes a virtual environment recognition module comprising: a recognition unit for recognizing the virtual scene; and an acquisition unit for acquiring virtual environment feature information based on the recognized virtual scene, wherein the virtual environment feature information includes: environment category E. vs Environmental Style S vs .

[0027] The virtual reality fusion device proposed in this embodiment of the invention includes a virtual object selection module, wherein: the virtual object library contains multiple virtual objects, and each virtual object has virtual individual feature information, the virtual individual feature information including: object category T. vo Object size B vo Functional feature set A vo Environmental Category Set E ov Environmental style collection S ov .

[0028] The virtual reality fusion device proposed in this embodiment of the invention includes virtual individual feature information of the virtual objects, wherein the virtual individual feature information F of the k-th virtual object... vo k For: F vok ={T vo k B vo k A vo k E ov k ,S ov k}, k = 1, 2…K; where: T vo k B is the object category of the k-th virtual object; vo k Let A be the size of the k-th virtual object; vo k Let A be the functional feature set of the k-th virtual object. vo k ={A vo k1 A vo k2 A vo km}, m=1,2…M,A vo km E represents the m-th functional feature of the k-th virtual object; ov k Let E be the set of environmental categories for the k-th virtual object. ov k ={E ov k1 E ov k2 ,…,E ov kp}, p=1,2…P,E ov kp This indicates that the k-th virtual object can be used in the p-th environment category; S ov k S is the environmental style set of the k-th virtual object. ov k ={S ov k1 ,S ov k2 ,…,S ov kq},q=1,2…Q,S ov kq This indicates that the k-th virtual object can be used in the q-th environment style.

[0029] The virtual reality fusion device proposed in this embodiment of the invention includes a virtual object selection module, wherein: a virtual object corresponding to an object in the real scene is selected from a virtual object library, and the selection constraints are as follows: in, The size of the i-th object in the real-world object feature information; The size of the k-th virtual object in the virtual object library; This is the set of functional features of the i-th object in the real-world object feature information; The set of functional features of the k-th virtual object in the virtual object library; The p-th environment category of the k-th virtual object in the virtual object library; E represents the p-th environment style of the k-th virtual object in the virtual object library. vs The environment category of the virtual environment feature information; S vs The environment style is the characteristic information of the virtual environment.

[0030] The virtual reality fusion device proposed in this embodiment of the invention includes a virtual-real object fusion module comprising: a pose alignment unit, used to acquire the object pose P of the object within the feature information of the real object. ro Set the pose information P of the virtual object in the virtual object feature information. vo =P ro .

[0031] The virtual reality fusion device proposed in this embodiment of the invention further includes: an origin alignment module, used to obtain the coordinate origin of the real scene and align the coordinate origin of the virtual scene with the coordinate origin of the real scene in the virtual scene; and a scaling module, used to obtain the size of the objects in the real scene and the size of the corresponding virtual objects, and scale the virtual objects to the same size as the objects in the real scene.

[0032] The virtual reality fusion device proposed in this embodiment of the invention includes a rendering module comprising: a direct rendering unit, which renders the virtual object corresponding to the object if the positions of the virtual object in the virtual scene overlap with those of the object in the real scene.

[0033] The virtual reality fusion device proposed in this embodiment of the invention includes a rendering module comprising: a passive rendering unit, used to not render virtual objects corresponding to walkable areas within the real scene.

[0034] The virtual reality fusion device proposed in this embodiment of the invention includes a rendering module comprising: an active rendering unit, used to render virtual objects corresponding to objects in non-walkable areas of the real scene and virtual objects in the virtual scene.

[0035] Therefore, the virtual reality fusion method and device proposed in this patent, based on the virtual-real fusion method of object functional equivalence, integrates real objects into the virtual environment, ensuring user safety and real-environment interactivity while maintaining the consistency of the virtual environment style and improving user immersion.

[0036] Another aspect of the present invention provides an electronic device including one or more processors and a storage device, wherein the storage device is used to store executable instructions, which, when executed by the processor, implement the method of the present invention.

[0037] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method of the present invention.

[0038] Another aspect of the present invention provides a computer program, which includes computer-executable instructions that, when executed, implement the method of the present invention.

[0039] According to the above embodiments of the present invention, the virtual reality fusion method and apparatus proposed by the present invention, based on the virtual-real fusion method of object functional equivalence, can integrate real objects into the virtual environment, which can at least partially solve the problem of disjointed content style in the virtual environment presentation in related technologies, and thus achieve the technical effect of improving user immersion. It not only ensures user safety but also maintains the consistency of the virtual environment style while improving the interactivity of the real environment. When users interact with real objects in the virtual environment, this technology can make the appearance and behavior of real objects consistent with other elements in the virtual environment, thereby making it easier for users to immerse themselves in the virtual environment.

[0040] It should be understood that the above general description and the following specific embodiments are merely exemplary and illustrative, and do not limit the scope of the invention. Attached Figure Description

[0041] The accompanying drawings, which are part of the specification of this invention, illustrate exemplary embodiments of the invention. The drawings, together with the description in the specification, serve to illustrate the principles of the invention.

[0042] Figure 1 This is a flowchart of a virtual reality fusion method provided for a specific embodiment of the present invention.

[0043] Figure 2 This is a partial flowchart provided for another specific embodiment of the present invention.

[0044] Figure 3 This is a rendering strategy diagram provided for another specific embodiment of the present invention.

[0045] Figure 4 This is a schematic diagram of a virtual reality fusion device provided for a specific embodiment of the present invention.

[0046] Figure 5 This is a schematic diagram illustrating the virtual reality fusion effect of a specific embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the spirit of the contents disclosed in the present invention will be clearly explained below with reference to the accompanying drawings and detailed description. After understanding the embodiments of the present invention, any person skilled in the art can make changes and modifications based on the technology taught in the present invention without departing from the spirit and scope of the present invention.

[0048] The illustrative embodiments and descriptions of the present invention are used to explain the invention, but are not intended to limit the invention. Furthermore, elements / components using the same or similar reference numerals in the drawings and embodiments are used to represent the same or similar parts.

[0049] The terms "first," "second," etc., used in this document are not intended to specifically refer to order or sequence, nor are they intended to limit the invention. They are merely used to distinguish elements or operations described using the same technical terms.

[0050] The directional terms used in this article, such as up, down, left, right, front, or back, are for reference only when referring to the accompanying drawings. Therefore, the use of directional terms is for illustrative purposes and not to limit this work.

[0051] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.

[0052] The term "and / or" as used herein includes any or all of the things mentioned.

[0053] The term "multiple" in this article includes "two" and "more than two"; the term "multiple groups" in this article includes "two groups" and "more than two groups".

[0054] The terms "approximately," "about," etc., used herein are intended to modify any quantity or error that may vary slightly, but these slight variations or errors do not change the essence of the quantity or error. Generally, the range of slight variations or errors modified by such terms may be 20% in some embodiments, 10% in others, 5% in still others, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0055] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0056] When expressions such as "at least one of A, B, and C" are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). When expressions such as "at least one of A, B, or C" are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., "a system having at least one of A, B, or C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). Those skilled in the art should also understand that any conjunction and / or phrase that substantially arbitrarily indicates two or more optional items, whether in the specification, claims, or drawings, should be understood to indicate the possibility of including one of these items, either of these items, or both items. For example, the phrase “A or B” should be understood as including the possibility of “A” or “B”, or “A and B”.

[0057] Figure 1 This is a flowchart illustrating a virtual reality fusion method provided in a specific embodiment of the present invention. (In conjunction with...) Figure 1As shown, the virtual reality fusion method of this invention is used to ensure the rationality of fusion between virtual and real scenes, including: S100, obtaining feature information of real objects containing multiple objects in the real scene; S200, obtaining feature information of virtual environment in the virtual scene; S300, obtaining virtual objects corresponding to the objects in the real scene from a virtual object library based on the feature information of the real objects and the feature information of the virtual environment; S400, calculating and fusing virtual object feature information of the virtual objects based on the feature information of the real objects; S500, rendering a virtual scene containing the virtual objects based on the feature information of the virtual objects and the virtual scene.

[0058] Step S100: Obtain real-world object feature information containing multiple objects within the real-world scene. Considering typical application scenarios of virtual reality fusion technology, the number and type of objects within the scene are often not unique. The main purpose of this step is to obtain object information within the real-world scene; the information for each object is incorporated into the real-world object feature information. In other words, this real-world object feature information includes information on all objects within the virtual reality fusion technology's application scenario. Subsequent steps for summarizing, selecting, and processing this information are based on this step. This real-world object feature information serves as a selection criterion for subsequent processing and may include elements such as the type, size, location, and functional characteristics of each real-world object. During implementation, the real-world environment scan is performed locally on the device running this step.

[0059] Step S200: Obtain virtual environment feature information within the virtual scene. This step is used to obtain processing information about the virtual environment. This virtual environment feature information is the intrinsic representation information of the virtual scene. As an environment element directly created for user perception, the virtual scene's style, category, size, and other factors are all determined by the user. The virtual environment feature information, such as scene style, category, and size, can be used as fixed scene parameters. After obtaining the selectable parameters of the virtual scene, the style and other elements of the virtual objects to be added can be adjusted to unify the style between the virtual objects and the virtual scene. It should be emphasized that there is no specific execution order between step S200 and the aforementioned step S100 for obtaining real-world object feature information. In specific implementation, the execution order of steps S100 and S200 can be adjusted according to the specific hardware resource allocation; they can be executed sequentially or simultaneously.

[0060] Step S300: Based on the real-world object feature information and the virtual environment feature information, obtain virtual objects corresponding to the objects in the real-world scene from the virtual object library. After obtaining the real-world object feature information and virtual environment feature information from steps S100 and S200, virtual objects corresponding to the objects in the real-world scene can be filtered from the virtual object library according to the information parameters provided by the two, such as size, style, and category. It should be noted that the virtual object library is a data set encompassing multiple virtual objects, containing multiple virtual objects corresponding to real-world objects, each of which can be matched with multiple scenes. Taking a car as an example, the virtual object of the car has multiple scene modes, including mountains, city roads, deserts, etc. Those skilled in the art can add, delete, modify, and query data in the existing virtual object library according to actual conditions to adapt it to the matching situation with real-world scenes. After determining the virtual feature information, the virtual object matching it is filtered from the aforementioned virtual object library; this virtual object is the one corresponding to the real-world object. The virtual objects obtained in this step can then be further adjusted and rendered to better fit the scene of virtual reality integration.

[0061] Step S400: Based on the real-world object feature information, calculate and fuse the virtual object feature information of the virtual object. After the aforementioned step S300, all virtual objects corresponding to real-world objects in the real-world scene have been obtained. For these virtual objects, their relationship with the virtual scene has not yet been processed. This relationship may include positional relationships, size relationships, lighting rendering relationships, etc. The main purpose of this step S400 is to calculate and fuse the feature information of real-world objects with the feature information of virtual objects obtained in the previous steps, adjusting the position, size, and other virtual object features of the virtual objects to achieve unity between these virtual objects and real-world objects in terms of scene relationships. The technical method of this calculation and fusion is not limited to resetting some or all of the virtual object feature information by the program, or manually resetting some or all of the virtual object feature information by the user. The virtual object feature information includes the position, size, and other virtual object feature information of the virtual objects. When implementing this implementation, technicians can adjust the types, quantities, types, and data structures of specific parameters according to the actual needs of the aforementioned steps. The purpose of this step is to establish a one-to-one correspondence between virtual objects and real-world objects in the virtual environment and the real-world environment, respectively. Only in this way can subsequent user virtual reality interaction become possible.

[0062] Step S500: Based on the virtual object feature information and the virtual scene, a virtual scene containing the virtual objects is rendered. After obtaining virtual objects and a virtual scene that completely correspond to the real scene, a rendering program is used to render the virtual scene and virtual objects, presenting the rendered whole to the user. After rendering, the virtual scene perceived by the user and the virtual objects within the virtual scene achieve stylistic unity. The user no longer experiences the stylistic disjointedness of existing technologies, improving the user's immersion. This not only ensures user safety but also maintains the consistency of the virtual environment's style while enhancing the interactivity of the real environment. When the user interacts with real objects in the virtual environment, this technology allows the appearance and behavior of real objects to remain consistent with other elements in the virtual environment, making it easier for the user to immerse themselves in the virtual environment. When implementing this embodiment, step S100 can run locally; the information processing part can run locally or in the cloud; steps S200, S300, S400, and the virtual object library can run locally or in the cloud; step S500 can run locally or in the cloud, transmitting data to the virtual scene's real-world device for display.

[0063] The present invention also proposes another embodiment of the virtual reality fusion method, which differs from the foregoing embodiments in that:

[0064] In step S100 of another embodiment, real-world object feature information containing multiple objects in a real-world scene is obtained, wherein the real-world object feature information includes real-world individual feature information for each object, and the real-world individual feature information F corresponding to the i-th object is... RO i Includes: object category T ro i Object pose P ro i Object size B ro i Functional feature set A ro i A real-world scene typically contains multiple objects, and the individual characteristics of these objects collectively constitute the scene's feature information. The feature information of a real-world environment with N objects is R = {F1, F2, F3, ..., F...}. N}, where F NThis refers to the individual feature information of the Nth object. In this embodiment, the real-world individual feature information includes: object category, object pose, object size, and a set of functional features. The set of functional features is a collection of the object's functional features. These functional features refer to the possible behaviors the object provides to the user, including: sitting, lying down, being reliable, drinking water, eating, pulling, pushing, grasping, overcoming obstacles, walking, etc. In other words, an object can have multiple functional features, which are concentrated within the set of functional features. It is important to emphasize that this set of functional features A... ro i The object can be retrieved by searching a predefined mapping table based on its category. This predefined mapping table can be defined by technicians before use of the objects that may be involved.

[0065] In another embodiment, step S200, obtaining virtual environment feature information within the virtual scene, includes: step S210, identifying the virtual scene; step S220, obtaining the virtual environment feature information based on the identified virtual scene, wherein the virtual environment feature information includes: environment category E. vs Environmental Style S vs In step S210, the virtual scene is identified. The virtual scene to be identified can be set by the program or by the user. During implementation, technicians can also reserve interfaces for other programs and user interfaces, suitable for situations where the virtual scene is undefined or difficult to identify. In step S220, based on the identified virtual scene, virtual environment feature information is obtained, including: environment category E. vs Environmental Style S vs In this step, the environment category E of the virtual environment feature information is... vs It can be a beach, forest, conference room, etc., with an environment style of S. vs It can be in Chinese style, European style, etc. During the implementation process, technicians can label the pre-set virtual scene with its environment category, environment style, etc., as needed.

[0066] In another embodiment, step S300 involves obtaining virtual objects corresponding to objects in the real scene from a virtual object library based on the real object feature information and the virtual environment feature information. The virtual object library contains multiple virtual objects, and each virtual object has virtual individual feature information, including: object category T. vo Object size B vo Functional feature set A vo Environmental Category Set E ov Environmental style collection S ov In the virtual object library, the characteristic information of different virtual individuals corresponds differently, as follows: their object category Tvo To determine this, each virtual object corresponds to an object category T. vo Its object size B vo To ensure accuracy, the volume and area values ​​of each virtual object are fixed, but can be updated subsequently by the program or manually; its functional feature set A vo To determine, the functional feature set A vo The object's environment can be obtained by searching a predefined mapping table based on its category. This predefined mapping table can be defined by technicians before the virtual object is used; its environment category set E ov To determine, this environmental category set E ov This is a set of multiple environmental category characteristics, which may include: beach, forest, conference room, etc.; its environmental style set S ov To determine the environmental style set S ov It is a collection of multiple environmental style characteristics, which may include: Chinese style, European style, etc.

[0067] In another embodiment, a virtual object library contains multiple virtual objects, wherein the virtual individual feature information F of the k-th virtual object is... vo k For: F vo k ={T vo k B vo k A vo k E ov k ,S ov k}, k = 1, 2…K; where: T vo k B is the object category of the k-th virtual object; vo k Let A be the size of the k-th virtual object; vo k Let A be the functional feature set of the k-th virtual object. vo k ={A vo k1 A vo k2 A vo km}, m=1,2…M,A vo km E represents the m-th functional feature of the k-th virtual object; ov k Let E be the set of environmental categories for the k-th virtual object. ov k ={Eov k1 E ov k2 ,…,E ov kp}, p=1,2…P,E ov kp This indicates that the k-th virtual object can be used in the p-th environment category; S ov k S is the environmental style set of the k-th virtual object. ov k ={S ov k1 ,S ov k2 ,…,S ov kq},q=1,2…Q,S ov kq This indicates that the k-th virtual object can be used in the q-th environment style.

[0068] In step S400 of another embodiment, the virtual object corresponding to the object in the real scene is obtained from the virtual object library based on the real object feature information and the virtual environment feature information. The selection of the virtual object corresponding to the object in the real scene from the virtual object library is subject to the following constraints: in, The size of the i-th object in the real-world object feature information; The size of the k-th virtual object in the virtual object library; This is the set of functional features of the i-th object in the real-world object feature information; The set of functional features of the k-th virtual object in the virtual object library; The p-th environment category of the k-th virtual object in the virtual object library; E represents the p-th environment style of the k-th virtual object in the virtual object library. vs The environment category of the virtual environment feature information; S vs The virtual environment style is defined by the virtual environment feature information. In this step, the constraint for selecting virtual objects is to traverse virtual objects of the same category in the virtual object library and filter for the virtual object most similar to the target (the i-th real object). Similarly, this constraint is applied to the selection of environment category and environment style. Given that the real object feature information is determined, virtual objects of the same environment category and environment style are selected. Similarly, for the functional feature set, virtual objects with the same functional feature set A as the real object are selected. roThe virtual object with the highest overlap. By traversing multiple real objects in this way, multiple corresponding virtual objects can be obtained. To help those skilled in the art understand, the selection method for this step is further explained. When selecting the virtual object corresponding to the i-th real object in the virtual object library, the aforementioned constraints are defined as follows: Constraint 1 is to constrain the object size, that is, the difference between the size of the virtual object and the real object is minimized; Constraint 2 is to constrain the functional feature set, that is, the functional feature set of the virtual object has the most common features with the functional feature set of the real object; Constraint 3 is to constrain the environment category, that is, the environment category of the real object is consistent with one of the environment categories of the virtual object; Constraint 4 is to constrain the environment style, that is, the environment style of the real object is consistent with one of the environment styles of the virtual object. When those skilled in the art implement this step, the following strategies can be selected: (i) Functional feature priority: After satisfying constraint 3 and constraint 4, constraint 2 is satisfied first, and constraint 1 is satisfied second; (ii) Object size priority: After satisfying constraint 3 and constraint 4, constraint 1 is satisfied first, and constraint 2 is satisfied second. Here, taking the object size priority strategy as an example, the way to select virtual objects is as follows: when the k-th virtual object has an environment category and environment style that are consistent with the environment category and environment style of the i-th real object, and when the k-th virtual object and the i-th real object have the most identical functional features, the k-th virtual object and the corresponding i-th real object have the greatest size similarity.

[0069] In another embodiment, step S500, calculating and fusing virtual object feature information of the virtual object based on the real object feature information, includes: obtaining the object pose P of the object within the real object feature information. ro Set the pose information P of the virtual object in the virtual object feature information. vo =P ro .

[0070] Figure 2 A partial flowchart provided for another specific embodiment of the present invention, in conjunction with Figure 2 and Figure 1 This other embodiment is similar to Figure 1The embodiment differs further in that, in step S500, before rendering the virtual scene containing the virtual object based on the virtual object feature information and the virtual scene, the following steps are included: Step S510, obtaining the origin of the coordinates of the real scene, and aligning the origin of the coordinates of the virtual scene with the origin of the coordinates of the real scene in the virtual scene; Step S520, obtaining the size of the object in the real scene and the size of the corresponding virtual object, and scaling the virtual object to the same size as the object in the real scene. To achieve a similar volume between the virtual object and the real object, this step optimizes the pose and size of the virtual object corresponding to the real object, ensuring that the real object and the virtual object are most similar in size for the user in the same position. In step S510, adjustments and calibrations are made to ensure that O... r =O v O r O represents the origin of the coordinate system in the real environment. v This represents the origin of the coordinate system in the virtual environment. In step S520, the coordinates of the virtual objects are adjusted and calibrated to ensure that... in, The size of the i-th object in the real-world object feature information; Let be the size of the virtual object corresponding to the i-th object in the virtual object library.

[0071] In another embodiment, step S500, rendering a virtual scene containing the virtual object based on the virtual object feature information and the virtual scene, includes: if a virtual object in the virtual scene overlaps with an object in the real scene, then rendering the virtual object corresponding to that object. The origin O of the real environment is maintained during rendering. r And the origin O of the virtual environment v They are located at the same point in space.

[0072] In another embodiment, step S500, rendering a virtual scene containing the virtual object based on the virtual object feature information and the virtual scene, includes: not rendering virtual objects corresponding to walkable areas within the real scene. The origin O of the real environment's coordinates is maintained during rendering. r And the origin O of the virtual environment v They are located at the same point in space.

[0073] In another embodiment, step S500, rendering a virtual scene containing the virtual objects based on the virtual object feature information and the virtual scene, includes: rendering virtual objects corresponding to objects in non-walkable areas of the real scene and virtual objects in the virtual scene. The origin O of the real environment is maintained during rendering. rAnd the origin O of the virtual environment v They are located at the same point in space.

[0074] Figure 3 This is a rendering strategy diagram provided for another specific embodiment of the present invention. (Combined with...) Figure 3 And step S500. Another embodiment of the present invention provides the following rendering strategy: First, it is determined whether the area is walkable; if it is walkable, no virtual object is rendered at that location; if it is not walkable, it is further determined whether the area has a real object. If there is a real object, the virtual object corresponding to the real object is rendered at that location; if there is no real object, the original virtual object of the virtual environment is rendered at that location.

[0075] Figure 4 This is a schematic diagram of a virtual reality fusion device provided in a specific embodiment of the present invention, combined with... Figure 4 This invention also proposes a virtual reality fusion device to ensure the rationality of fusion between a virtual scene and a real scene, comprising: a real environment recognition module 100 for acquiring feature information of real objects containing multiple objects in the real scene; a virtual environment recognition module 200 for acquiring feature information of the virtual environment in the virtual scene; a virtual object selection module 300 for acquiring virtual objects corresponding to the objects in the real scene from a virtual object library based on the feature information of the real objects and the feature information of the virtual environment; a virtual-real object fusion module 400 for calculating and fusing virtual object feature information of the virtual objects based on the feature information of the real objects; and a rendering module 500 for rendering a virtual scene containing the virtual objects based on the feature information of the virtual objects and the virtual scene.

[0076] The virtual reality fusion device proposed in this embodiment of the invention includes a real environment recognition module 100, wherein the real object feature information includes real individual feature information of each object, wherein the real individual feature information F corresponding to the i-th object is... RO i Includes: object category T ro i Object pose P ro i Object size B ro i Functional feature set A ro i .

[0077] The virtual reality fusion device proposed in this embodiment of the invention includes a virtual environment recognition module 200, comprising: a recognition unit for recognizing the virtual scene; and an acquisition unit for acquiring virtual environment feature information based on the recognized virtual scene, the virtual environment feature information including: environment category E. vs Environmental Style S vs .

[0078] The virtual reality fusion device proposed in this embodiment of the invention includes a virtual object selection module 300, wherein: the virtual object library contains multiple virtual objects, and each virtual object has virtual individual feature information, which includes: object category T. vo Object size B vo Functional feature set A vo Environmental Category Set E ov Environmental style collection S ov .

[0079] Virtual individual feature information of virtual objects, where the virtual individual feature information F of the k-th virtual object vo k For: F vo k ={T vo k B vo k A vo k E ov k ,S ov k}, k = 1, 2…K; where: T vo k B is the object category of the k-th virtual object; vo k Let A be the size of the k-th virtual object; vo k Let A be the functional feature set of the k-th virtual object. vo k ={A vo k1 A vo k2 A vo km}, m=1,2…M,A vo km E represents the m-th functional feature of the k-th virtual object; ov k Let E be the set of environmental categories for the k-th virtual object. ov k ={E ovk1 E ov k2 ,…,E ov kp}, p=1,2…P,E ov kp This indicates that the k-th virtual object can be used in the p-th environment category; S ov k S is the environmental style set of the k-th virtual object. ov k ={S ov k1 ,S ov k2 ,…,S ov kq},q=1,2…Q,S ov kq This indicates that the k-th virtual object can be used in the q-th environment style.

[0080] The virtual reality fusion device proposed in this embodiment of the invention includes a virtual object selection module 300, wherein: a virtual object is selected from a virtual object library to obtain a virtual object corresponding to an object in the real scene, and the selection constraint relationship is as follows: in, The size of the i-th object in the real-world object feature information; The size of the k-th virtual object in the virtual object library; This is the set of functional features of the i-th object in the real-world object feature information; The set of functional features of the k-th virtual object in the virtual object library; The p-th environment category of the k-th virtual object in the virtual object library; E represents the p-th environment style of the k-th virtual object in the virtual object library. vs The environment category of the virtual environment feature information; S vs The environment style is the characteristic information of the virtual environment.

[0081] The virtual reality fusion device proposed in this embodiment of the invention includes a virtual-real object fusion module 400 comprising: a pose alignment unit, used to acquire the object pose P of the object within the feature information of the real object. ro Set the pose information P of the virtual object in the virtual object feature information. vo =P ro .

[0082] The virtual reality fusion device proposed in this embodiment of the invention further includes: an origin alignment module, used to obtain the coordinate origin of the real scene and align the coordinate origin of the virtual scene with the coordinate origin of the real scene in the virtual scene; and a scaling module, used to obtain the size of the objects in the real scene and the size of the corresponding virtual objects, and scale the virtual objects to the same size as the objects in the real scene.

[0083] The virtual reality fusion device proposed in this embodiment of the invention includes a rendering module 500 comprising: a direct rendering unit, which renders the virtual object corresponding to the object if the positions of the virtual object in the virtual scene overlap with those of the object in the real scene.

[0084] The virtual reality fusion device proposed in this embodiment of the invention includes a rendering module 500 comprising: a passive rendering unit, used to not render virtual objects corresponding to walkable areas within the real scene.

[0085] The virtual reality fusion device proposed in this embodiment of the invention includes a rendering module 500 comprising: an active rendering unit, used to render virtual objects corresponding to objects in non-walkable areas of the real scene and virtual objects in the virtual scene.

[0086] Figure 5 This is a schematic diagram illustrating the virtual reality fusion effect of a specific embodiment of the present invention. (In conjunction with...) Figure 5 As shown, after applying the virtual reality fusion method and device of this invention, a wardrobe in a real-world scene is rendered as a large tree in the virtual scene, a sofa as a sitable piece of wood, a table as a stone, and a chair as a sitable wooden stake. All of these rendering effects conform to the scene category and style of a forest scene. Users can integrate real objects into the virtual environment, which can at least partially solve the problem of disjointed content style in virtual environment presentation in related technologies, thus achieving the technical effect of improving user immersion. This not only ensures user safety but also maintains the consistency of the virtual environment style while improving the interactivity of the real environment. When users interact with real objects in the virtual environment, this technology can make the appearance and behavior of real objects consistent with other elements in the virtual environment, making it easier for users to immerse themselves in the virtual environment.

[0087] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. According to embodiments of the present invention, the electronic devices, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0088] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0089] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0090] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0091] Those skilled in the art will understand that the features described in the various embodiments and / or claims of the present invention can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention can be combined or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0092] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A virtual reality fusion method for ensuring the rationality of merging virtual scenes with real-world scenes, characterized in that, include: Obtain feature information of real-world objects containing multiple objects in a real-world scene; Obtain virtual environment feature information within a virtual scene; Based on the real-world object feature information and the virtual environment feature information, obtain virtual objects from the virtual object library that correspond to the objects in the real-world scene; Based on the real-world object feature information, calculate and fuse the virtual object feature information of the virtual object; Based on the virtual object feature information and the virtual scene, a virtual scene containing the virtual object is rendered. The step involves retrieving virtual objects corresponding to objects in the real-world scene from a virtual object library based on the real-world object feature information and the virtual environment feature information, wherein: The virtual object library contains multiple virtual objects, each possessing virtual individual characteristic information, including: object category T. vo Object size B vo Functional feature set A vo Environmental Category Set E ov Environmental style collection S ov ; The virtual individual feature information of the virtual object, wherein the virtual individual feature information F of the k-th virtual object vo k for: F vo k ={T vo k , B vo k , A vo k , E ov k , S ov k },k=1,2…K; in: T vo k The object category of the k-th virtual object; B vo k The size of the k-th virtual object; A vo k Let A be the functional feature set of the k-th virtual object. vo k ={A vo k1 A vo k2 , …, A vo km }, m=1,2…M,A vo km This represents the m-th functional feature of the k-th virtual object; E ov k Let E be the set of environmental categories for the k-th virtual object. ov k ={E ov k1 E ov k2 ,…,E ov kp }, p=1,2…P, E ov kp This indicates that the k-th virtual object can be used in the p-th environment category; S ov k S is the environmental style set of the k-th virtual object. ov k ={S ov k1 , S ov k2 , …, S ov kq }, q=1,2…Q,S ov kq This indicates that the k-th virtual object can be used in the q-th environment style; The step involves retrieving virtual objects from a virtual object library that correspond to the objects in the real-world scene, based on the real-world object feature information and the virtual environment feature information. Select the virtual object from the virtual object library that corresponds to the object in the real scene. The selected constraint relationship is as follows: ； ； ； in, The size of the i-th object in the real-world object feature information; The size of the k-th virtual object in the virtual object library; This is the set of functional features of the i-th object in the real-world object feature information; The set of functional features of the k-th virtual object in the virtual object library; The p-th environment category of the k-th virtual object in the virtual object library; The p-th environment style of the k-th virtual object in the virtual object library; The environment category is the virtual environment feature information. The environment style is the characteristic information of the virtual environment.

2. The virtual reality fusion method as described in claim 1, characterized in that, The step involves acquiring real-world object feature information containing multiple objects within a real-world scene. This real-world object feature information includes the real-world individual feature information for each object, where the real-world individual feature information F corresponding to the i-th object... RO i Includes: object category T ro i Object pose P ro i Object size B ro i Functional feature set A ro i .

3. The virtual reality fusion method as described in claim 1, characterized in that, The process involves acquiring virtual environment feature information within the virtual scene; Identify the virtual scene; Based on the identified virtual scene, the virtual environment feature information is obtained, including: environment category E. vs Environmental Style S vs .

4. The virtual reality fusion method as described in claim 1, characterized in that, The step of calculating and fusing virtual object feature information of the virtual object based on the real object feature information includes: Obtain the object pose P of the object within the feature information of the real object. ro Set the pose information P of the virtual object in the virtual object feature information. vo =P ro .

5. The virtual reality fusion method as described in claim 1, characterized in that, Before rendering a virtual scene containing the virtual object based on the virtual object feature information and the virtual scene, the method further includes: Obtain the origin of the coordinates of the real scene, and align the origin of the coordinates of the virtual scene with the origin of the coordinates of the real scene in the virtual scene; Obtain the size of the objects in the real scene and the size of the corresponding virtual objects, and scale the virtual objects to the same size as the objects in the real scene.

6. The virtual reality fusion method as described in claim 1, characterized in that, The step of rendering a virtual scene containing the virtual object based on the virtual object feature information and the virtual scene includes: If a virtual object in the virtual scene overlaps with an object in the real scene, then the virtual object corresponding to that object is rendered.

7. The virtual reality fusion method as described in claim 1, characterized in that, The step of rendering a virtual scene containing the virtual object based on the virtual object feature information and the virtual scene includes: Virtual objects within the walkable area of ​​the real-world scene are not rendered.

8. The virtual reality fusion method as described in claim 1, characterized in that, The step of rendering a virtual scene containing the virtual object based on the virtual object feature information and the virtual scene includes: Render virtual objects corresponding to objects in non-walkable areas of the real scene and virtual objects in the virtual scene.

9. A virtual reality fusion device for ensuring the rationality of merging virtual scenes with real-world scenes, characterized in that, include: The real-world environment recognition module is used to acquire feature information of real-world objects containing multiple objects in a real-world scene; The virtual environment recognition module is used to acquire virtual environment feature information within a virtual scene; The virtual object selection module is used to obtain virtual objects corresponding to objects in the real scene from the virtual object library based on the real object feature information and the virtual environment feature information. The virtual-real object fusion module is used to calculate and fuse virtual object feature information of the virtual object based on the feature information of the real object; The rendering module is used to render a virtual scene containing the virtual object based on the virtual object feature information and the virtual scene. The virtual object selection module, wherein: The virtual object library contains multiple virtual objects, and each virtual object has virtual individual feature information, which includes: object category Tvo, object size Bvo, functional feature set Avo, environment category set Eov, and environment style set Sov. The virtual individual feature information of the virtual object, wherein the virtual individual feature information F of the k-th virtual object vo k for: F vo k ={T vo k , B vo k , A vo k , E ov k , S ov k },k=1,2…K; in: T vo k The object category of the k-th virtual object; B vo k The size of the k-th virtual object; A vo k Let A be the functional feature set of the k-th virtual object. vo k ={A vo k1 A vo k2 , …, A vo km }, m=1,2…M,A vo km This represents the m-th functional feature of the k-th virtual object; E ov k Let E be the set of environmental categories for the k-th virtual object. ov k ={E ov k1 E ov k2 ,…,E ov kp }, p=1,2…P, E ov kp This indicates that the k-th virtual object can be used in the p-th environment category; S ov k S is the environmental style set of the k-th virtual object. ov k ={S ov k1 , S ov k2 , …, S ov kq }, q=1,2…Q,S ov kq This indicates that the k-th virtual object can be used in the q-th environment style; The virtual object selection module, wherein: Select the virtual object from the virtual object library that corresponds to the object in the real scene. The selected constraint relationship is as follows: ； ； ； in, The size of the i-th object in the real-world object feature information; The size of the k-th virtual object in the virtual object library; This is the set of functional features of the i-th object in the real-world object feature information; The set of functional features of the k-th virtual object in the virtual object library; The p-th environment category of the k-th virtual object in the virtual object library; The p-th environment style of the k-th virtual object in the virtual object library; The environment category is the virtual environment feature information. The environment style is the characteristic information of the virtual environment.

10. The virtual reality fusion device as described in claim 9, characterized in that, The real-world environment recognition module, wherein the real-world object feature information includes the real-world individual feature information of each object, wherein the real-world individual feature information F corresponding to the i-th object is... RO i Includes: object category T ro i Object pose P ro i Object size B ro i Functional feature set A ro i .

11. The virtual reality fusion device as described in claim 9, characterized in that, The virtual environment recognition module includes: The identification unit is used to identify the virtual scene; The acquisition unit is used to acquire the virtual environment feature information based on the identified virtual scene. The virtual environment feature information includes: environment category E. vs Environmental Style S vs .

12. The virtual reality fusion device as described in claim 9, characterized in that, The virtual-real object fusion module includes: The pose alignment unit is used to obtain the object pose P of the object within the feature information of the real object. ro Set the pose information P of the virtual object in the virtual object feature information. vo =P ro .

13. The virtual reality fusion device as described in claim 9, characterized in that, Also includes: The origin alignment module is used to obtain the coordinate origin of the real scene and align the coordinate origin of the virtual scene with the coordinate origin of the real scene in the virtual scene. The scaling module is used to obtain the size of the objects in the real scene and the size of the corresponding virtual objects, and to scale the virtual objects to the same size as the objects in the real scene.

14. The virtual reality fusion device as described in claim 9, characterized in that, The rendering module includes: The direct rendering unit renders the virtual object corresponding to the real-world object if the virtual object in the virtual scene overlaps with the position of the object in the real-world scene.

15. The virtual reality fusion device as described in claim 9, characterized in that, The rendering module includes: A negative rendering unit is used to prevent the rendering of virtual objects within the walkable area of ​​the real scene.

16. The virtual reality fusion device as described in claim 9, characterized in that, The rendering module includes: An active rendering unit is used to render virtual objects corresponding to objects in non-walkable areas of the real scene and virtual objects in the virtual scene.

Images