Industrial simulation system

Abstract

The disclosure provides an industrial simulation system, comprising a host computer and a head-mounted display device, the host computer and the head-mounted display device being communicatively connected, the head-mounted display device comprising a somatosensory controller, the host computer being configured to build a three-dimensional scene of a target object in response to a request for building the three-dimensional scene of the target object, wherein the three-dimensional scene comprises at least a three-dimensional model of the target object; the somatosensory controller being configured to determine three-dimensional spatial position information of a hand of a wearer of the head-mounted display device in a case where the wearer performs a target operation on the target object, and send the three-dimensional spatial position information of the hand to the host computer; and the host computer being configured to control the three-dimensional model to simulate the target operation in the three-dimensional scene according to the three-dimensional spatial position information of the hand.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent electronic device technology, and more specifically, to an industrial simulation system. Background Technology

[0002] Industrial simulation is now widely used by many companies in various aspects of industry. It plays an important role in improving development efficiency, enhancing data collection, analysis and processing capabilities, reducing decision-making errors, and lowering corporate risks.

[0003] Currently, the main form of industrial simulation is based on Virtual Reality (VR) technology. VR technology generates a simulated environment through computers, immersing users in that environment. Using this technology for industrial simulation, virtual scenes can be used in conjunction with specific display devices to create an immersive experience for the user. However, since it is entirely a computer-simulated world, the user's senses are separated from the real world by the device, making it impossible for them to perceive their surroundings, resulting in a low user experience. Summary of the Invention

[0004] One objective of this disclosure is to provide a new technical solution for an industrial simulation system.

[0005] According to a first aspect of the present disclosure, an industrial simulation system is provided, including a host computer and a head-mounted display device, wherein the host computer and the head-mounted display device are communicatively connected, and the head-mounted display device includes a motion controller.

[0006] The host is configured to respond to a request to build a three-dimensional scene for a target object, and to build a three-dimensional scene of the target object, wherein the three-dimensional scene includes at least a three-dimensional model of the target object;

[0007] The motion controller is used to determine the three-dimensional spatial position information of the wearer's hand when the wearer of the head-mounted display device performs a target operation on the target object, and to send the three-dimensional spatial position information of the hand to the host.

[0008] The host computer is used to control the three-dimensional model to simulate the target operation in the three-dimensional scene based on the three-dimensional spatial position information of the hand.

[0009] Optionally, the motion controller is used to determine the three-dimensional spatial position information of the wearer's hand when the wearer of the head-mounted display device performs a target operation on the target object, specifically including:

[0010] When the wearer of the head-mounted display device performs a target operation on the target object, the wearer's hand movement information is tracked;

[0011] Based on the hand movement information, the three-dimensional spatial position information of multiple key points of the hand is located;

[0012] Based on the three-dimensional spatial position information of multiple key points on the hand, the three-dimensional spatial position information of the wearer's hand is determined.

[0013] Optionally, the host is configured to construct a 3D scene of the target object in response to a request to do so, specifically including:

[0014] In response to a request to build a 3D scene for a target object, the initial 3D model of the target object and its corresponding initial scene model are obtained;

[0015] The initial 3D model is used to develop a 3D model of the target object.

[0016] Logical development is performed on the 3D model and the initial scene model to obtain the 3D scene.

[0017] Optionally, the model development of the initial 3D model includes: setting the materials of the initial 3D model and / or setting the UI controls of the initial 3D model.

[0018] Optionally, the wearer's hand is bound with a first collider, and the target object is provided with a second collider;

[0019] The motion controller is used to determine the three-dimensional spatial position information of the wearer's hand when the wearer of the head-mounted display device performs a target operation on the target object, and to send the three-dimensional spatial position information of the hand to the host.

[0020] The host is used to synchronize the three-dimensional spatial position information of the hand to the first collider, and execute logic code to display the operation result when the wearer of the head-mounted display device performs a target operation on the target object, causing the first collider and the second collider to collide.

[0021] Optionally, the head-mounted display device includes a display module.

[0022] The host computer is also used to render a virtual hand based on the three-dimensional spatial position information of the hand, and send the rendered virtual hand to the display module;

[0023] The display module is used to display the rendered virtual hand.

[0024] Optionally, the target object is provided with first identification information, and the head-mounted display device further includes a camera module.

[0025] The camera module is used to acquire scene images of the real scene in which the wearer is located, and send the scene images to the host.

[0026] The host computer is used to identify the first identification information in the scene image and obtain the three-dimensional spatial location information of the target object; and

[0027] The current state of the target object is determined based on its three-dimensional spatial location information.

[0028] Optionally, the first identification information includes two-color identification information.

[0029] Optionally, the host provides at least one of a training mode, a practice mode, and an assessment mode for the target object;

[0030] In the training mode, the host provides voice explanations of the operations performed on the target object;

[0031] In the practice mode, the host provides the order of operations performed on the target object;

[0032] In the assessment mode, the host provides a score for the operations performed on the target object.

[0033] Optionally, the target object is provided with second identification information, and the head-mounted display further includes a camera module and a display module.

[0034] The camera module is used to acquire scene images of the real scene in which the wearer is located, and send the scene images to the host.

[0035] The host computer is configured to identify the second identification information in the scene image, and fuse the scene image and the 3D scene according to the second identification information to obtain a fused image and output it to the display module.

[0036] The display module is used to display the fused image.

[0037] One beneficial effect of this disclosure is that the industrial simulation system it provides includes a host computer and a head-mounted display device. The head-mounted display device is equipped with a motion controller. The host computer can build a 3D scene for a target object, including a 3D model of the target object. The motion controller can determine the 3D spatial position information of the user's hands when the user performs a target operation on the target object and send it to the host computer. Based on this hand position information, the host computer controls the 3D model to simulate the target operation within the 3D scene. In other words, this industrial simulation system, by allowing operation in a virtual scene in conjunction with real components, significantly improves the realism of practical training, effectively overcomes the problems existing in real-world training environments, effectively improves training efficiency, and enhances the user experience and user engagement.

[0038] Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of this specification.

[0040] Figure 1 This is a schematic diagram of the hardware configuration of an industrial simulation system according to an embodiment of the present disclosure;

[0041] Figure 2 This is a schematic diagram of the operation of a host according to an embodiment of the present disclosure;

[0042] Figure 3 This is a schematic diagram of key hand points according to an embodiment of the present disclosure;

[0043] Figure 4 This is a schematic diagram of a rendered virtual hand according to an embodiment of the present disclosure;

[0044] Figure 5 This is a schematic diagram of the target object according to an embodiment of the present disclosure;

[0045] Figure 6 This is a schematic diagram of a target object according to another embodiment of the present disclosure;

[0046] Figure 7 This is a schematic diagram of the hardware configuration of an industrial simulation system according to another embodiment of the present disclosure;

[0047] Figure 8 This is a schematic diagram of a target object according to another embodiment of the present disclosure. Detailed Implementation

[0048] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.

[0049] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0050] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0051] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0052] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0053] <Equipment Example>

[0054] Please see Figure 1 This is a schematic diagram of the hardware structure of an industrial simulation system provided in an embodiment of this application. Figure 1 As shown, the industrial simulation system 1 includes a host 11 and a head-mounted display device 12, which are connected in communication. The head-mounted display device 12 includes a motion controller 121.

[0055] The head-mounted display device 12 can be an augmented reality (AR) device that uses Video See Through (VST) technology. VST involves the camera module on the VR device first acquiring real-time scene information of the actual scene, then combining it with virtual information generated by the host computer, and finally transmitting it to the VR device's display module for output to the wearer's eyes. VST technology ensures the wearer has a sufficient field of view for a good viewing experience; simultaneously, it effectively combines virtual and reality before the information reaches the eyes, thus solving the occlusion problem. Of course, the head-mounted display device 12 can also be other devices, which are not limited in this embodiment.

[0056] The head-mounted display device 12 is equipped with a Leap Motion controller 121, which can capture the hand movement information of the wearer of the head-mounted display device 12.

[0057] The host device 11 can be a mobile phone, tablet computer, laptop computer, etc. This host device 11 uses a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) as a computing device to implement various augmented reality applications through software algorithms. (See reference...) Figure 2 The host 11 is equipped with a VST development engine, which can build a 3D scene corresponding to the target object in the real scene. Simultaneously, the host 11 has a Leap Motion plugin installed, which manages the data transmitted by the motion controller 121, enabling data transfer between the VST development engine and the motion controller, thus allowing the virtual hand to correspond to the real hand. The VST development engine can also transfer data with the camera module to determine the position of the target object.

[0058] In this embodiment, the host 11 is configured to construct a 3D scene of the target object in response to a request to do so. The 3D scene includes at least a 3D model of the target object.

[0059] In one specific embodiment, host 11 is configured to, in response to a request to build a 3D scene for a target object, build the 3D scene for the target object, specifically including: in response to the request to build a 3D scene for the target object, obtaining an initial 3D model of the target object and its corresponding initial scene model; performing model development on the initial 3D model to obtain a 3D model of the target object; and performing logic development on the 3D model and the initial scene model to obtain the 3D scene.

[0060] The target object mentioned above can be an object in the real-world scene where the wearer of the head-mounted display device 12 is currently located, and this target object is the object that the wearer is actually operating. This target object can be a component such as a valve, a mold temperature controller, etc.

[0061] Specifically, refer to Figure 2 The host computer 11 is equipped with SolidWorks and 3DS Max software. First, SolidWorks allows for the creation of an initial 3D model of the target object and its corresponding initial scene model. Simultaneously, the number of faces in the model can be simplified based on the actual operation process. Then, 3DS Max is used to optimize the initial 3D model and its corresponding initial scene model, which are then imported into the VST development engine for development. The development based on the VST engine is divided into two phases: the first phase is model development, and the second phase is model logic development.

[0062] Model development involves the development of an initial 3D model and an initial scene model. Developing the initial 3D model includes setting its materials and / or its UI controls. It also involves creating animations, features, and voice narration for the 3D model. Developing the initial scene model includes setting its materials and creating animations and special effects. It is understood that in this embodiment, the initial 3D model of the target object after model development is referred to as the 3D model of the target object.

[0063] For logic development, this involves developing the logic of the target object's 3D model, such as establishing the correspondence between UI controls and the actual button controls of the target object in the real scene, and establishing the business logic relationships between UI controls. It also involves the logic development between the target object's 3D model and the initial scene model after model development, such as establishing the interaction relationships and business logic relationships between the target object's 3D model and the initial scene model after model development.

[0064] For example, taking target object 2 as... Figure 8 Taking the valve shown as an example, firstly, an initial 3D model of the valve, as well as initial 3D models of the pipe connected to the valve and other valves connected to the pipe, are created using SolidWorks. Then, the initial 3D models of the valve, the pipe connected to the valve, and other valves connected to the pipe are optimized using 3DS Max software. Finally, the initial 3D model of the valve is optimized to obtain its final 3D model, as are the initial 3D models of the pipe connected to the valve and other valves connected to the pipe. The UI controls and other interactive logic for the 3D model of the valve are developed, as well as the interaction logic between the 3D model of the valve and the optimized initial 3D models of the pipe connected to the valve and other valves connected to the pipe, thus obtaining the 3D scene of the valve.

[0065] In this embodiment, the motion controller 121 is used to determine the three-dimensional spatial position information of the wearer's hand when the wearer of the head-mounted display device performs a target operation on the target object, and send the three-dimensional spatial position information of the hand to the host.

[0066] In one specific embodiment, the motion controller 121 is used to determine the three-dimensional spatial position information of the wearer's hand when the wearer of the head-mounted display device performs a target operation on the target object. Specifically, this includes: tracking the wearer's hand movement information when the wearer of the head-mounted display device performs a target operation on the target object; locating the three-dimensional spatial position information of multiple key points of the hand based on the hand movement information; and determining the three-dimensional spatial position information of the wearer's hand based on the three-dimensional spatial position information of the multiple key points of the hand.

[0067] Specifically, when the wearer of the head-mounted display device 12 operates a target object in a real-world scene, the motion controller 121 can obtain hand movement information within its field of view, and based on this hand movement information, locate the three-dimensional spatial position information of 21 key hand points. The three-dimensional spatial position information of these 21 key hand points is referenced... Figure 3 As shown. Based on the three-dimensional spatial position information of these 21 key hand points, the three-dimensional spatial position information of the wearer's hand is determined.

[0068] In this embodiment, the host 11 is used to control the three-dimensional model to simulate the target operation in the three-dimensional scene based on the three-dimensional spatial position information of the hand.

[0069] In one specific embodiment, the wearer's hand is bound to a first collision element, and the target object is provided with a second collision element.

[0070] Specifically, the target objects include the target operation location and related target UI controls; correspondingly, a second collider may be bound to the target operation location and related target UI controls. This second collider can be... Figure 6 The cube colliders 2 and 3 are shown.

[0071] Specifically, a first impactor is attached to the fingertip of the wearer's hand. The shape of the first impactor matches the size of the fingertip. The first impactor can be... Figure 6 The spherical impactor 1 shown. For example, it could be a first impactor with a shape matching the size of the middle fingertip, attached to the tip of the wearer's middle finger.

[0072] In this embodiment, the motion controller 121 is used to determine the three-dimensional spatial position information of the wearer's hand when the wearer performs a target operation on the target object, and send the three-dimensional spatial position information of the hand to the host.

[0073] Specifically, when the wearer performs a target operation on a target object at a target location, the motion controller acquires the three-dimensional spatial position information of the wearer's hand, including the position information of the fingertips. This fingertip position information is then sent to the host computer.

[0074] In this embodiment, the host 11 is used to synchronize the three-dimensional spatial position information of the hand to the first collider, and execute logic code to display the operation result when the wearer of the head-mounted display device performs a target operation on the target object, causing the first collider and the second collider to collide.

[0075] Specifically, after the host receives the three-dimensional spatial position information of the wearer's hand, it will synchronize the fingertip position information to the first collider whose shape matches the size of the fingertip. That is, the host controls the position of the first collider according to the fingertip position information so that when the wearer performs the target operation on the target object and the first collider collides with the second collider at the target operation position, the host will perform logical judgment to display the operation result.

[0076] For example, taking target object 2 as... Figure 5 Taking the mold temperature controller as an example, as shown below... Figure 6 As shown, a cube collider 2 is added to the button that controls the heating function, and a cube collider 3 is added to the button that controls the cooling function.

[0077] For example, when a wearer clicks the cooling button on a temperature regulator in a real-world scenario using their real hand, the wearable motion controller 131 sends the three-dimensional position information of the wearer's hand to the host 11. The host 11 assigns the fingertip position information to a spherical collider of the same size as the fingertip, here the spherical collider is collider 1, ensuring that collider 1 is always synchronized with the coordinate position of the fingertip. When the fingertip comes into contact with the heating button, the host 11 performs collision detection, and if collider 1 collides with the cube collider 2 corresponding to the heating button, the collision logic is executed to increase the virtual temperature.

[0078] In this embodiment, refer to Figure 7 The head-mounted display device 12 also includes a display module 123.

[0079] The host 11 is also used to render a virtual hand based on the three-dimensional spatial position information of the hand, and send the rendered virtual hand to the display module.

[0080] The display module 123 is used to display the rendered virtual hand.

[0081] Continuing the example above, when the wearer clicks the cooling button on the temperature regulator of the mold temperature controller in a real scene with their real hand, the wearer's three-dimensional hand position information is sent to the host 11 via the motion-sensing controller 131. The host then renders a virtual hand based on the hand's three-dimensional spatial position information, and the rendered virtual hand is referenced... Figure 4 As shown.

[0082] According to an embodiment of this application, the provided industrial simulation system includes a host computer and a head-mounted display device. The head-mounted display device is equipped with a motion controller. The host computer can build a 3D scene for a target object, including a 3D model of the target object. The motion controller can determine the 3D spatial position information of the user's hands when the user performs a target operation on the target object and send this information to the host computer. Based on this hand position information, the host computer controls the 3D model to simulate the target operation within the 3D scene. In other words, this industrial simulation system, by allowing operation in a virtual scene in conjunction with real components, significantly improves the realism of practical training, effectively overcomes the problems existing in real-world training environments, effectively improves training efficiency, and enhances the user experience and user engagement.

[0083] In one embodiment, the target object 2 is further provided with first identification information, which may be two-color identification information. (Refer to...) Figure 8 The target object 2 is a valve. The first identification information of the valve can be a first color 21 and a second color 22 set on the valve. The first color and the second color are different colors. For example, the first color can be red and the second color can be blue. The first color and the second color can be used to distinguish the valve from other objects in the real scene, such as pipes and other valves, so as to facilitate tracking and positioning.

[0084] Reference Figure 7 The head-mounted display device 12 also includes a camera module 122, which may be a binocular camera.

[0085] In this embodiment, the camera module 122 is used to acquire scene images of the real scene in which the wearer is located, and send the scene images to the host.

[0086] In this embodiment, the host 11 is used to identify the first identification information in the scene image, obtain the three-dimensional spatial position information of the target object, and determine the current state of the target object based on the three-dimensional spatial position information of the target object.

[0087] For example, continue with Figure 8Taking the valve shown as an example, the camera module 122 acquires a scene image of the actual scene where the valve is located and sends it to the host 11. The host 11 detects and tracks the first identification information of the valve in the scene image, calculates the three-dimensional spatial position information of the valve, and then determines whether the wearer's operation is correct based on the three-dimensional spatial position information.

[0088] According to embodiments of this application, the invention employs manual marking for tracking, relying on specific colors and other features of the markings to achieve fast and accurate tracking. Identifying the pose of a physical object through manual marking significantly reduces software development costs compared to previous methods that used sensors and trackers for pose acquisition.

[0089] In one embodiment, the target object 2 is further provided with second identification information, which is different from the first identification information.

[0090] In this embodiment, the camera module 122 is used to acquire scene images of the real scene in which the wearer is located, and send the scene images to the host 11.

[0091] In this embodiment, the host 11 is used to identify the second identification information in the scene image, and to fuse the video image and the three-dimensional scene according to the second identification information to obtain a fused image and output it to the display module.

[0092] Specifically, 3D registration technology aligns the virtual world coordinate system with the real world coordinate system, mapping information from both environments into the same space. Common 3D registration methods include hardware sensor-based tracking registration; however, the hardware costs are high. This disclosure presents a 3D registration method based on artificial markers. This method involves the host computer recognizing second marker information in the scene image, and then mapping the video image and 3D scene information into the same space based on this second marker information to obtain a fused image, which is then output to the display module.

[0093] In this embodiment, the display module 123 is used to display the fused image.

[0094] According to embodiments of this disclosure, the industrial simulation system is based on video perspective technology, that is, by using video synthesis technology to fuse real-world images captured by the camera module with computer-generated digital images, achieving a wider field of view. Simultaneously, through VST technology, content is directly displayed via video, requiring low positioning accuracy and effectively combining virtual and reality before reaching the human eye, thus solving the occlusion problem.

[0095] In one embodiment, the host 11 also provides at least one of a training mode, a practice mode, and an assessment mode for the target object.

[0096] In the training mode, the host provides voice explanations of the operations performed on the target object;

[0097] In the practice mode, the host provides the order of operations performed on the target object;

[0098] In the assessment mode, the host provides a score for the operations performed on the target object.

[0099] In this embodiment, the host 11 provides a selection interface, and the wearer can select one of the training mode, practice mode and assessment mode as the target mode according to actual needs, so as to realize the interaction between the real scene and the virtual scene in the target mode.

[0100] According to the embodiments of this application, it can provide three different modes: training mode, practice mode, and assessment mode. In each mode, it can realize the interaction between real and virtual scenarios, achieving the purpose of direct and natural interaction, good immersive experience, and high operational efficiency.

[0101] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0102] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. An industrial simulation system, characterized in that, It includes a host computer and a head-mounted display device, which are communicatively connected. The head-mounted display device includes a motion controller. The host is configured to respond to a request to build a three-dimensional scene for the target object, and to build the three-dimensional scene of the target object, wherein the three-dimensional scene includes at least a three-dimensional model of the target object, and the target object is the actual object that the wearer of the head-mounted display device is currently operating in the real scene. The wearer's hand is bound with a first collision object, the shape of which matches the size of the fingertip, and a second collision object is provided on the target object; The motion controller is used to determine the three-dimensional spatial position information of the wearer's hand when the wearer of the head-mounted display device performs a target operation on the target object in the real scene using a real hand, and to send the three-dimensional spatial position information of the hand to the host; wherein, the three-dimensional spatial position information of the hand includes fingertip position information; The host is configured to control the position of the first collider in the virtual space based on the fingertip position information, so that the position of the first collider in the virtual space is synchronized with the fingertip position of the wearer's hand, and execute logic code to display the result of the collision between the first collider and the second collider in the virtual space when the wearer of the head-mounted display device performs a target operation on the target object, causing the first collider and the second collider to collide in the real scene.

2. The system according to claim 1, characterized in that, The motion controller is used to determine the three-dimensional spatial position information of the wearer's hand when the wearer of the head-mounted display device performs a target operation on the target object, specifically including: When the wearer of the head-mounted display device performs a target operation on the target object, the wearer's hand movement information is tracked; Based on the hand movement information, the three-dimensional spatial position information of multiple key points of the hand is located; Based on the three-dimensional spatial position information of multiple key points on the hand, the three-dimensional spatial position information of the wearer's hand is determined.

3. The system according to claim 1, characterized in that, The host is configured to respond to a request to build a 3D scene of the target object, and to build the 3D scene of the target object, specifically including: In response to a request to build a 3D scene for a target object, the initial 3D model of the target object and its corresponding initial scene model are obtained; The initial 3D model is used to develop a 3D model of the target object. Logical development is performed on the 3D model and the initial scene model to obtain the 3D scene.

4. The system according to claim 3, characterized in that, Model development for the initial 3D model includes setting the materials of the initial 3D model and / or setting the UI controls of the initial 3D model.

5. The system according to claim 4, characterized in that, The head-mounted display device includes a display module. The host computer is also used to render a virtual hand based on the three-dimensional spatial position information of the hand, and send the rendered virtual hand to the display module; The display module is used to display the rendered virtual hand.

6. The system according to claim 1, characterized in that, The target object is provided with first identification information, and the head-mounted display device also includes a camera module. The camera module is used to acquire scene images of the real scene in which the wearer is located, and send the scene images to the host. The host computer is used to identify the first identification information in the scene image and obtain the three-dimensional spatial position information of the target object; as well as, The current state of the target object is determined based on its three-dimensional spatial location information.

7. The system according to claim 6, characterized in that, The first identification information includes two-color identification information.

8. The system according to claim 1, characterized in that, The host provides at least one mode: a training mode, a practice mode, and an assessment mode for the target object. In the training mode, the host provides voice explanations of the operations performed on the target object; In the practice mode, the host provides the order of operations performed on the target object; In the assessment mode, the host provides a score for the operations performed on the target object.

9. The system according to claim 1, characterized in that, The target object is provided with second identification information, and the head-mounted display device further includes a camera module and a display module. The camera module is used to acquire scene images of the real scene in which the wearer is located, and send the scene images to the host. The host computer is configured to identify the second identification information in the scene image, and fuse the scene image and the 3D scene according to the second identification information to obtain a fused image and output it to the display module. The display module is used to display the fused image.

Images