Information display method and apparatus, and device and storage medium

Abstract

Provided in the embodiments of the present application are an information display method and apparatus, and a device and a storage medium. The method comprises: displaying a target virtual scene by means of a display generation component, wherein the target virtual scene comprises a target object; and in response to a preset operation instruction which is input by means of one or more input components, displaying, by means of the display generation component, a real environment where a user is currently located, and presenting the target object in the real environment. By means of the embodiments of the present application, accurate switching of displaying the target object from the target virtual scene to the real environment can be realized, so that the target object in the target virtual scene can enter the real environment to interact with the user, thereby ensuring the diversity of scenes of interaction between the user and the target object, enhancing the level of interest of interaction between the target object and the user, and improving the experience of immersive interaction between the user and the target object in the real environment.

Description

Information display method, device, equipment and storage medium

[0001] This application claims priority to the Chinese invention patent application entitled “Information Display Method, Apparatus, Device and Storage Medium” filed on August 14, 2023, with application number 2023110221885, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of data processing technology, and in particular to an information display method, apparatus, device, and storage medium. Background Art

[0003] In a virtual reality scenario, virtual reality technology can enable users to immersively watch various virtual scene images. For example, users can experience real virtual interactive scenes by wearing a head-mounted display (HMD).

[0004] Typically, the corresponding video stream can be directly played on the 3D video playback screen in the virtual space to present the corresponding virtual scene image in the virtual space, which makes the display effect of the virtual scene relatively simple and lacks certain interactive fun.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide an information display method, apparatus, device and storage medium to achieve accurate scene switching of a target object from a target virtual scene to a real environment, thereby ensuring the diversity of user interaction scenarios with the target object.

[0007] In a first aspect, an embodiment of the present application provides an information display method, which is applied to an electronic device that communicates with a display generation component and one or more input components, the method comprising: displaying a target virtual scene via the display generation component, wherein the target virtual scene includes a target object; in response to a preset operation instruction input via the one or more input components, displaying the real environment in which the user is currently located via the display generation component, and presenting the target object within the real environment.

[0008] In second aspect, an embodiment of the present application provides an information display device, which is configured on an electronic device that communicates with a display generation component and one or more input components, and the device includes: a virtual scene display module, which is used to display a target virtual scene via the display generation component, and the target virtual scene includes a target object; a real environment display module, which is used to respond to a preset operation instruction input via the one or more input components, display the real environment in which the user is currently located via the display generation component, and present the target object in the real environment.

[0009] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the information display method provided in the first aspect of the present application.

[0010] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, which enables a computer to execute the information display method provided in the first aspect of the present application.

[0011] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, which enables a computer to execute the information display method provided in the first aspect of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] FIG1 is a flow chart of an information display method provided in an embodiment of the present application;

[0014] FIG2 is a schematic diagram showing the principle of switching a target virtual scene to display a real environment via a display generation component according to an embodiment of the present application;

[0015] FIG3 is a flow chart of another information display method provided in an embodiment of the present application;

[0016] FIG4 is a schematic diagram showing the principle of dividing safety zones in a real environment according to an embodiment of the present application;

[0017] FIG5 is a flow chart of a method for determining a presentation position of a target object in a real environment according to an embodiment of the present application;

[0018] FIG6 is a flow chart of another method for determining the presentation position of a target object in a real environment provided by an embodiment of the present application;

[0019] FIG7 is a flowchart of a method for presenting a target object in a real environment according to an embodiment of the present application;

[0020] FIG8 is a schematic diagram of an information display device provided in an embodiment of the present application;

[0021] FIG9 is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0024] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or solution described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0025] Before introducing the specific technical solutions of this application, the application scenarios of this application are first described accordingly:

[0026] To achieve information display in a virtual reality scene, the present application may display the corresponding virtual scene through a display generation component that communicates with any electronic device. The electronic device may be any extended reality (XR) device, specifically including a virtual reality (VR) device, an augmented reality (AR) device, and a mixed reality (MR) device, etc., which is not limited in the present application.

[0027] The display generation component may be any display screen that has established a communication connection with the above-mentioned electronic device, etc. For example, the display generation component may be a display screen configured on a VR device, an AR device, or an MR device, which is not limited in this application.

[0028] Moreover, in order to achieve normal interaction of users in virtual reality scenes, the present application can initiate corresponding interactive operations on the virtual scene displayed by the display generation component through one or more input components that communicate with the above-mentioned electronic devices, thereby supporting users to perform various interactions in the virtual scene.

[0029] The one or more input components may be any control device or information collection module that has established a communication connection with the above-mentioned electronic device. For example, the one or more input components may be a handle configured on a VR device, an AR device, or an MR device, or a collection module for detecting hand operations and eye movement information, or a voice collector for collecting user voice information, etc., and this application does not limit this.

[0030] Currently, in order to solve the problem of a relatively single scene display effect after a user enters any virtual scene, the inventive concept of this application is that a scene switching function can be set for the target virtual scene displayed by the display generation component. Then, a preset operation instruction input through one or more input components can be used to switch the target virtual scene to the user's current real environment through the display generation component, and present the target object in the target virtual scene in the real environment, thereby achieving accurate switching display of the target object from the target virtual scene to the real environment, so that the target object in the target virtual scene can enter the real environment to interact with the user, ensuring the diversity of the user's interactive scenes with the target object.

[0031] Figure 1 is a flowchart of an information display method provided in an embodiment of the present application. The method can be applied to an electronic device that communicates with a display generation component and one or more input components, but is not limited thereto. The method can be executed by the information display device provided in the present application, wherein the information display device can be implemented by any software and / or hardware. For example, the information display device can be configured on an electronic device such as AR / VR / MR that can simulate a virtual scene. The present application does not impose any restrictions on the specific type of electronic device.

[0032] Specifically, as shown in FIG1 , the method may include the following steps: S110 , displaying a target virtual scene via a display generation component, wherein the target virtual scene includes a target object.

[0033] In this application, in order to realize the display of various information in the virtual reality scene, it is generally possible to support a display component connected to the electronic device to display the corresponding virtual reality scene. The display generating component can be any display screen, such as a display screen configured on a VR device, AR device, or MR device.

[0034] Moreover, the target virtual scene can be a corresponding virtual environment simulated for a certain interactive scene selected by any user, so as to display corresponding interactive operation information in the target virtual scene. For example, the target virtual scene can be a 3D live scene screen of any live broadcast room.

[0035] The target virtual scene may be a simulation environment of the real world, a semi-simulation and semi-fictional virtual scene, or a purely fictional virtual scene.

[0036] In order to achieve normal interaction between users in the target virtual scene, the target virtual scene may include multiple types of virtual objects. Among them, the target object in this application can be a virtual object in the target virtual scene that has specific interaction requirements with the user, such as a virtual person, virtual props, etc.

[0037] Typically, after enabling the virtual reality interactive function of the electronic device, the user may first select a virtual scene as a target virtual scene, and then display the target virtual scene through the display generation component, so that the user enters the target virtual scene.

[0038] Among them, various types of virtual objects can be presented in the target virtual scene, including but not limited to the target object in this application, to support the user to interact with the target object accordingly.

[0039] As an optional implementation scheme in the present application, the target virtual scene in the present application can be a live video stream of any live broadcast room, and the live video stream can be a 180° 3D video stream.

[0040] Then, for the target virtual scene displayed by the display generation component, if the user wants to watch the live content of a certain live broadcast room in the virtual space, he can select the live broadcast room from multiple live broadcast rooms in the virtual space and enter the live broadcast room to watch the live broadcast.

[0041] Therefore, when a selection command is detected for any live broadcast room in the virtual space, it indicates that the user currently wishes to view the live broadcast content of that live broadcast room in the virtual space. Therefore, in response to the selection command for any live broadcast room, the live video stream of that live broadcast room can be obtained. The live video stream of that live broadcast room can then be played via the display generation component to present the live broadcast content of that live broadcast room in the 3D playback field of the virtual space, allowing the user to enter the target virtual scene of the live broadcast room and immerse themselves in the three-dimensional spatial and stereoscopic sense of the target virtual scene.

[0042] In some implementable methods, in order to ensure the convenience of interaction within the target virtual scene, after the target virtual scene is displayed via the display generation component, a corresponding control screen and a public screen can be set for the user in the target virtual scene. The control screen can be provided with various types of interactive control controls, such as adjusting the volume, viewing the details of the live broadcast room, etc., so that the user can perform various control operations on the corresponding virtual objects in the target virtual scene by triggering the corresponding interactive control controls. The public screen can be used to display various messages such as bullet screens and comments sent by each audience in the target virtual scene to realize the interaction between each audience in the target virtual scene.

[0043] S120 , in response to a preset operation instruction input via one or more input components, display the real environment in which the user is currently located via a display generation component, and present a target object in the real environment.

[0044] After the target virtual scene is displayed through the display generation component, in order to avoid the problem of a relatively single interactive effect of the target virtual scene, the present application can set a scene switching function within the target virtual scene to support each user to switch from the target virtual scene to different real scenes to interact with the same target object.

[0045] For example, as shown in Figure 2, the present application can pre-set a scene switching control in the control screen or public screen within the target virtual scene. The scene switching control can be used to diversify the display effects of the target virtual scene, supporting switching from a single target virtual scene to the real scenes of different users, so that each user can immerse themselves in the real scene and interact with the same target object accordingly.

[0046] For a scene switching function set in a target real scene, a corresponding scene switching request may be initiated via a preset operation instruction inputted via one or more input components.

[0047] The preset operation instructions input via one or more input components may include at least one of the following operation instructions:

[0048] 1) A first operation instruction is generated by preset handle manipulation information input via a user handle.

[0049] The user can manipulate a specific button or joystick on any handle communicatively connected to the electronic device to send corresponding handle manipulation information to the electronic device. When the electronic device detects the preset handle manipulation information input via the user's handle, it indicates that the user currently wants to switch scenes of the target virtual scene, and thus generates a corresponding first operation instruction.

[0050] 2) A second operation instruction is generated by detecting a preset gesture initiated by the user's hand via the hand detection component.

[0051] The electronic device can be equipped with a corresponding hand detection component to detect various gestures made by the user's hand in real time. For example, the user's hand image can be captured in real time by a camera on the electronic device. The hand detection component then performs corresponding feature analysis on the hand image to identify various gestures initiated by the user's hand. When the hand detection component on the electronic device detects a preset gesture initiated by the user's hand, it indicates that the user currently wants to switch the target virtual scene, thereby generating a corresponding second operation instruction.

[0052] 3) A third operation instruction is generated by detecting preset eye movement information initiated by the user's eyes through the eye detection component and a preset gesture initiated by the hand detected by the hand detection component.

[0053] The electronic device can be configured with corresponding eye detection components and hand detection components to respectively detect various eye movement information initiated by the user's eyes and various gesture information made by the user's hands. For example, the user's eye image and the user's hand image can be collected in real time by the camera on the electronic device. Then, the eye detection component performs corresponding feature analysis on the eye image to identify various eye movement information initiated by the user's eyes. Moreover, the hand detection component performs corresponding feature analysis on the hand image to identify various gesture information initiated by the user's hands. When the eye detection component on the electronic device detects the preset eye movement information initiated by the user's eyes, and the hand detection component on the electronic device simultaneously detects the preset gesture initiated by the user's hand, it indicates that the user currently wants to switch the target virtual scene, thereby generating a corresponding third operation instruction.

[0054] 4) A fourth operation instruction generated by preset voice information collected by any voice collecting component.

[0055] Any voice collection component installed on the electronic device can collect audio data emitted by the user in real time and perform corresponding voice recognition processing on the audio data to identify it as corresponding voice information. When the electronic device recognizes the preset voice information collected by any voice collection component, it indicates that the user currently wants to switch the target virtual scene, and thus generates a corresponding fourth operation instruction.

[0056] It can be understood that any one of the above-mentioned first operation instruction, second operation instruction, third operation instruction and fourth operation instruction can be a trigger instruction for the scene switching function in the target real scene.

[0057] Then, after the target virtual scene is displayed via the display generation component, various operation instructions input via one or more input components can be detected in real time to determine whether there is a scene switching requirement for the target virtual scene.

[0058] When a preset operation command is detected via one or more input components, indicating that a scene switch is currently required in the target virtual scene, the electronic device can activate the Video See-Through (VST) function configured to capture the user's current surrounding environment in real time, thereby continuously capturing the real environment image within the user's real scene.

[0059] Then, the display generation component can stop displaying the target virtual scene and instead display various real environment images captured in real time within the real scene where the user is located, so that the target virtual scene is switched to be presented as the real environment where the user is currently located.

[0060] It should be understood that when the target virtual scene is switched to the real environment in which the user is currently located via the display generation component, in order to ensure the normal interaction of the user in the real environment, the control screen and public screen in the target virtual scene will usually continue to be presented in the real environment so that the user can continue to use the various control and interactive functions of the target virtual scene normally.

[0061] Among them, by real-time analysis of the user's location information in the real environment, and then in the real environment displayed by the display generation component, the control screen and public screen in the target virtual scene can be presented facing the user at a default relative position between the user, so that the control screen and public screen in the target virtual scene can always be presented in front of the user, to ensure the user's convenient control and interaction in the real environment.

[0062] After the target virtual scene is switched to the user's current real environment through the display generation component, in order to ensure the user's normal interaction in the real environment, the corresponding target object can first be determined from the various virtual objects in the target virtual scene according to the user's interaction needs.

[0063] In some possible implementations, as shown in FIG2 , the target object in the present application may at least include a character object within the target virtual scene, so that the user can interact with each character object in the real environment, such as the host of a live broadcast room.

[0064] In order to ensure that users can still interact with the target objects in the target virtual scene after the target virtual scene switches to the real environment, the present application can continuously project the target objects in the virtual live scene into the real environment displayed after the switch, so that the target objects in the target virtual scene can be additionally presented in the real environment. In other words, it can support the target object from the target virtual scene to enter the real environment currently located by different users, allowing each user to interact with the same target object in their respective real environment, ensuring the diversity of the user's interaction scenarios with the target object and enhancing the interactive fun between the target object and the user.

[0065] In some implementations, if the target virtual scene is a live video stream of any live broadcast room, then the target object may be an image of a target object in a real-time live frame of the live video stream.

[0066] Then, for the target object image, the present application can determine it in the following manner: perform matting processing on the target object in the real-time live frame of the live video stream to obtain the corresponding target object image.

[0067] That is, upon detecting a preset operation instruction input via one or more input components, the present application can capture the live frame at the current moment from the live video stream of any live broadcast room representing the target virtual scene, thereby obtaining the real-time live frame in the present application. Then, for each real-time live frame, the present application can perform a matte processing on the target object in the real-time live frame to obtain the corresponding target object image.

[0068] In one embodiment, if the live video stream of any live broadcast room is a binocular video stream captured in real time by a binocular camera of the live broadcast environment where the anchor is located, then the real-time live frames of the live broadcast room can be the left-eye video frame and the right-eye video frame captured by the left-eye camera and the right-eye camera at the same time in the binocular video stream. Therefore, the present application can start from detecting a preset operation instruction input via one or more input components, and perform image matting processing on the target object in the left-eye video frame and the right-eye video frame captured by the binocular camera at the same time in the live broadcast environment where the anchor is located, thereby obtaining a first target object image in the left-eye video frame and a second target object image in the right-eye video frame. Then, by fusing the first target object image and the second target object image at the same time, the corresponding target object image can be obtained, so that the target object image can be a three-dimensional stereo image.

[0069] According to the above method, the two target object images after the matting processing in the binocular video frame collected at each moment after the preset operation instruction input through one or more input components is detected are fused, and the target object image corresponding to each real-time live broadcast frame can be continuously obtained.

[0070] The technical solution provided by the embodiment of the present application can display a target virtual scene via a display generation component. When a preset operation instruction input via one or more input components is detected, the display generation component can switch the display from the target virtual scene to the user's current real environment, and display the target object in the target virtual scene in the real environment, thereby achieving accurate switching display of the target object from the target virtual scene to the real environment. This allows the target object in the target virtual scene to enter the real environment to interact with the user, ensuring the diversity of the user's interactive scenes with the target object, enhancing the interactive fun between the target object and the user, and improving the user's immersive interactive experience with the target object in the real environment.

[0071] As an optional implementation scheme in this application, after the target virtual scene is switched to the real environment in which the user is currently located through the display generation component, in order to ensure the accurate presentation of the target object in the real environment, this application needs to accurately analyze the presentation position of the target object in the real environment, so as to explain the specific presentation process of the target object in the real environment.

[0072] FIG3 is a flow chart of another information display method provided in an embodiment of the present application. As shown in FIG3 , the method may include the following steps:

[0073] S310: Displaying a target virtual scene via a display generation component, wherein the target virtual scene includes a target object.

[0074] S320: In response to a preset operation instruction input via one or more input components, the display generation component displays the current real environment where the user is located.

[0075] S330: Determine the relative display position of the target object in the target virtual scene.

[0076] After the target virtual scene is switched to a real environment via the display generation component, in order to ensure that the user can still interact normally with a target object in the target virtual scene, the target object in the target virtual scene needs to appear in the real environment to support the interaction between the user and the target object.

[0077] Therefore, starting from detecting the preset operation instructions input through one or more input components, the present application can analyze in real time the position information of the target object in the target virtual scene and the position information of the virtual camera in the target virtual scene to determine the relative position information between the target object in the target virtual scene and the virtual camera, as the relative display position of the target object in this application.

[0078] In some embodiments, if the target virtual scene is a live video stream of any live broadcast room, then the present application can analyze in real time the depth information of the target object in each real-time live frame obtained in the live video stream starting from the detection of a preset operation instruction to determine the relative position information of the target object in the target virtual scene and the virtual camera represented by the live camera, thereby obtaining the relative display position of the target object in the present application.

[0079] The relative display position of the target object may represent the relative position relationship between the target object and the live broadcast camera in the live broadcast scene where the anchor is located.

[0080] S340, determining the presentation position of the target object in the real environment based on the spatial origin and the relative display position in the real environment, where the spatial origin is the spatial position of the user in the real environment when the preset operation instruction is detected.

[0081] Whether in the target virtual scene displayed before the display generation component switches, or in the real environment displayed after the display generation component switches, the various virtual objects therein are generally viewed from the user's perspective, that is, the user can be equivalent to a virtual camera in the target virtual scene. Therefore, when a preset operation instruction inputted via one or more input components is detected, in addition to displaying the user's current real environment via the display generation component, the user's spatial position in the real environment at the time the preset operation instruction inputted via the one or more input components is also determined. Then, considering that after the real environment is displayed via the display generation component, the user views the target objects in the real environment from the user's perspective, and the target objects are displayed by the virtual camera in the target virtual scene, the user's spatial position in the real environment at the time the preset operation instruction is detected can be used as the spatial origin in the real environment to represent the position of the virtual camera in the target virtual scene.

[0082] By analyzing the relative display position of the target object, the relative positional relationship between the target object and the virtual camera can be determined. Then, based on this relative positional relationship, combined with the virtual camera position represented by the spatial origin within the real environment, the target object's presentation position within the real environment can be determined.

[0083] It is understood that the target object can move in real time within the target virtual scene, and when the virtual camera captures the real-time scene image of the target virtual scene, the position of the target object within the target virtual scene also changes relative to the virtual camera. After detecting a preset operation instruction input via one or more input components, the target object within the target virtual scene can be positioned within the real environment by analyzing the relative positional relationship between the target object and the virtual camera, as represented by the relative display position of the target object, based on the virtual camera position represented by the spatial origin within the real environment.

[0084] S350: Present the target object in the real environment according to the presentation position.

[0085] After determining the presentation position of the target object in the real environment, the target object can be directly placed at the presentation position in the real environment displayed by the display generation component, thereby accurately presenting the target object in the real environment.

[0086] S360: If the relative distance between the presentation position and the current position of the user in the real environment is less than a preset distance threshold, a model effect of the target object is presented in the real environment.

[0087] Since the target object may move in real time in the target virtual scene, and the user may also move in real time in the real environment, when the target object is presented in the real environment, the user and the target object may collide with each other.

[0088] Therefore, to enhance the interaction between the user and the target object, this application can determine the user's location in the real environment in real time, using this as the user's current location in the real environment. Then, by comparing the relative distance between the target object's location in the real environment and the user's current location with a preset distance threshold, it can be determined whether the user and the target object in the real environment are too close to each other and have touched each other.

[0089] If the relative distance between the target object's presentation location in the real environment and the user's current location is less than a preset distance threshold, it indicates that the user is too close to the target object and may touch it. Therefore, when presenting the target object at its presentation location in the real environment, this application also presents a mannequin effect for the target object. This mannequin effect can include gradually increasing the target object's transparency within the real environment, making it gradually transparent, and presenting a prompt message in the real environment that "the user is too close to the target object" to prompt the user to move away from the target object, thereby enhancing the virtual interaction effect.

[0090] S370: If the presentation position is on the boundary of the safety zone in the real environment, a fading effect of the target object is presented in the real environment.

[0091] Because the target virtual scene can be a relatively closed virtual space, such as the livestream room where the streamer broadcasts, the target virtual scene cannot include objects outside of this closed space. During the target object's movement, it may step out of the closed space, causing it to no longer exist in the target virtual scene and thus no longer be present in the real world. In this case, the user may suddenly lose sight of the target object in the real world and be unable to promptly understand the reason for its disappearance.

[0092] Therefore, in order to ensure the user's immersive interactive experience in the real environment, this application can divide the corresponding safe zone in the real environment according to the spatial size of the target virtual scene and use the spatial origin in the real environment to represent the virtual camera position in the target virtual scene.

[0093] Then, after presenting the target object at the presentation position in the real environment, it is possible to further determine whether the presentation position of the target object is on the safe zone boundary in the real environment, so as to analyze whether the target object has stepped out of the spatial environment corresponding to the target virtual scene.

[0094] If the presentation position of the target object is on the boundary of the safe zone in the real environment, it means that the target object will step out of the spatial environment corresponding to the target virtual scene, making it impossible to continue to present the target object in the real environment. Therefore, when presenting the target object at the presentation position in the real environment, the present application will further present a fading effect of the target object in the real environment. As shown in Figure 4, the fading effect can include gradually increasing the transparency of the target object in the real environment, so that the target object gradually becomes transparent, and presenting a prompt message in the real environment that "the target object has left and will be back soon" to prompt the user to wait for the target object to return to the target virtual scene, thereby ensuring the user's immersive interactive experience in the real environment.

[0095] In some implementations, the safety zone boundary in the real environment may be determined by the following steps: determining the safety zone boundary in the real environment based on the spatial boundary of the target virtual scene and the spatial origin in the real environment.

[0096] That is, upon detecting a preset operation instruction input via one or more input components, the present application can perform corresponding feature analysis on the real-time scene image of the target virtual scene at the current moment, thereby extracting scene boundary feature points within the real-time scene image. Then, based on the relative display position of each scene boundary feature point, the relative positional relationship between it and the virtual camera is represented, thereby determining the spatial boundary of the target virtual scene.

[0097] Based on the relative positional relationship between the scene boundary feature points and the virtual camera represented by the relative display positions of each scene boundary feature point within the spatial boundary of the target virtual scene, combined with the virtual camera position represented by the spatial origin in the real environment, the mapping point position of each scene boundary feature point in the real environment can be determined, thereby forming a corresponding safe zone in the real environment and obtaining the corresponding safe zone boundary.

[0098] The technical solution provided in the embodiment of the present application analyzes the presentation position of the target object in the real environment through the relative display position of the spatial origin and the target object in the real environment, so that the target object can be accurately presented at the presentation position in the real environment. This improves the accurate presentation of the target object in the real environment while ensuring the diversity of the interactive scenes, enhances the interactive fun between the user and the target object, and improves the user's immersive interactive experience with the target object in the real environment.

[0099] According to one or more embodiments of the present application, after the target object is accurately switched from the target virtual scene to the real environment through the display generation component, various real objects will be presented in the real environment displayed after the switch, and the target object (such as the anchor) will continue to move in the target virtual scene, so that the target object will also continue to move in the real environment.

[0100] This shows that the target object may overlap with the corresponding real object in the real environment. In order to ensure the authenticity of the target object in the real environment, it is required to avoid overlap between the target object and the real object in the real environment as much as possible.

[0101] Therefore, in order to prevent the positions of the target object and the real object in the real environment from overlapping, the present application can determine the calibration position of each real object in the real environment in response to the object calibration operation in the real environment.

[0102] That is, after the target virtual scene is switched to be displayed as a real environment via the display generation component, the user can be supported to perform spatial calibration on various real objects in the real environment by triggering the calibration control under the perspective function.

[0103] After detecting a trigger operation for a calibration control, the present application can control each real object in the real environment to enter an object calibration state, thereby supporting the user to perform corresponding object calibration operations on each real object through a physical controller such as a joystick. At this time, for each real object in the real environment, the present application can detect the object calibration operation performed by the user on the real object in real time, and perform position analysis on the object calibration operation to obtain the position of the real object in the real environment, which is used as the calibration position of the real object in the present application.

[0104] According to the same calibration method as above, the calibration position of each real object in the real environment can be determined.

[0105] Subsequently, when the target object in the target virtual scene needs to be presented in the real environment, the presentation position of the target object in the real environment can be determined by the following steps as shown in FIG5 :

[0106] S510: Determine the initial position of the target object in the real environment according to the spatial origin and the relative display position in the real environment.

[0107] To avoid overlap between the target object and various real objects when it is presented in the real environment, the present application can determine the relative display position of the target object with respect to the virtual camera. Then, based on the virtual camera position represented by the spatial origin in the real environment, combined with the relative positional relationship between the target object and the virtual camera represented by the relative display position of the target object, the initial position of the target object in the real environment can be preliminarily determined to determine whether the initial position of the target object overlaps with various real objects in the real environment.

[0108] S520 : Determine a presentation position of the target object in the real environment according to the initial position and the calibrated position of each real object in the real environment.

[0109] After determining the initial position of the target object in the real environment, the difference between the initial position and the calibrated position of each real object in the real environment is compared to determine whether the target object overlaps with the positions of the real objects when it is presented at the initial position.

[0110] If the target object does not overlap with any real object when presented at the initial position, then the initial position can be directly used as the presentation position of the target object in the real environment.

[0111] If the target object, when presented at the initial position, overlaps with a real object, the spatial range occupied by each real object in the real environment can be determined based on the calibrated positions of the real objects. Thus, the initial position of the target object can be adjusted accordingly to reposition the target object in the real environment outside the spatial range occupied by each real object, thereby preventing the target object from overlapping with the real objects when presented in the real environment.

[0112] In other embodiments, after the target object is presented in the real environment, in order to ensure normal interaction in the real environment, the user may be supported to perform corresponding drag operations on the target object to enhance the interaction diversity of the target object in the real environment.

[0113] Therefore, it can be seen that in the real environment, the target object will move accordingly under the user's drag operation. Therefore, as shown in Figure 6, the presentation position of the target object in the real environment can also be determined by the following steps:

[0114] S610 , in response to a drag operation of a target object, determining an initial position of the target object in the real environment according to a spatial origin and a relative display position in the real environment.

[0115] After the target object is continuously presented in the real environment, the user expresses the user's intention to present the target object in the real environment by dragging the target object presented at the previous moment, and the presentation position of the target object to be presented during the dragging process can be re-planned accordingly.

[0116] Therefore, upon detecting a user dragging a target object, the present application can determine the target object's relative display position relative to the virtual camera. Then, based on the virtual camera's position as represented by the spatial origin within the real environment, combined with the relative positional relationship between the target object and the virtual camera as represented by the target object's relative display position, the target object's initial position within the real environment can be preliminarily determined. This allows the target object's presentation position within the real environment to be replanned based on its initial position, in accordance with the presentation intent indicated by the user's dragging operation.

[0117] S620: Update the initial position according to the drag amplitude corresponding to the drag operation to determine the presentation position of the target object in the real environment.

[0118] After determining the initial position of the target object in the real environment, the drag amplitude corresponding to the drag operation can be determined by analyzing the user's drag operation on the target object. The drag amplitude can represent the degree of position change of the target object in the real environment. Then, based on the degree of position change represented by the drag amplitude, the initial position of the target object in the real environment can be adjusted to obtain the target object's presentation position in the real environment.

[0119] It is understood that the present application can also combine the above two solutions with respect to the presentation position of the target object in the real environment. That is, based on the drag amplitude corresponding to the drag operation of the target object and the calibrated positions of various real objects, the initial position of the target object in the real environment is adjusted to re-determine the presentation position of the target object in the real environment, thereby ensuring the accurate presentation of the target object in the real environment.

[0120] In the present application, after determining the presentation position of the target object in the real environment, the target object will be placed at the presentation position in the real environment to accurately present the target object in the real environment.

[0121] However, since the target object in the real environment is presented relative to the spatial origin used to represent the virtual camera in the real environment, and the user may be constantly moving in the real environment, when viewing the target object from the user's perspective in the real environment, the target object may appear distorted because it is not presented facing the user.

[0122] Therefore, in order to improve the authenticity of the target object when it is presented in the real environment, the present application can implement the following steps for the presentation of the target object in the real environment, as shown in FIG7 :

[0123] S710: Determine a presentation direction of the target object according to the presentation position and the current position of the user in the real environment.

[0124] After determining the presentation position of the target object in the real environment, since the target object is presented in the real environment relative to the spatial origin used to represent the virtual camera position in the real environment, in order to avoid distortion when viewing the target object from the user's perspective, the present application can determine the user's position in the real environment in real time as the user's current position in the present application.

[0125] Then, according to the relative relationship between the presentation position of the target object and the current position of the user, the target object can be controlled to transform from a direction toward the origin of the space to a direction toward the user, thereby determining the presentation direction of the target object.

[0126] In some possible implementations, the present application can determine the presentation direction of the target object in the following manner: determine the user's viewing angle rotation value for the target object based on the presentation position, the spatial origin, and the user's current position in the real environment; determine the presentation direction of the target object based on the viewing angle rotation value and a pre-set angle rotation threshold.

[0127] That is, after determining the presentation position of the target object in the real environment, the position point corresponding to the presentation position can be connected to the spatial origin and the position point corresponding to the user's current position in the real environment, respectively, to obtain a first line between the position point corresponding to the presentation position and the spatial origin, and a second line between the position point corresponding to the presentation position and the position point corresponding to the user's current position. Based on this, the angle between the first line and the second line can be determined as the rotation angle of the target object when it transforms from facing the spatial origin to facing the user's current position, that is, the user's viewing angle rotation value for the target object in this application.

[0128] If the viewing angle rotation value is small, then the difference between viewing the target object from the user's perspective and viewing the target object from the spatial origin is not significant. Therefore, to ensure efficient presentation of the target object, this application can pre-set a viewing angle rotation threshold to indicate the maximum rotation angle that can be supported when viewing the target object from the user's perspective without distortion.

[0129] Therefore, after determining the user's viewing angle rotation value for the target object, the present application can determine whether there will be distortion if the target object is presented toward the origin of space by comparing the viewing angle rotation value with the preset angle rotation threshold.

[0130] If the viewing angle rotation value is less than or equal to the preset viewing angle rotation threshold, it means that even if the target object is presented toward the origin of space, there will be no distortion when viewing the target object from the user's perspective. In this case, there is no need to change the orientation of the target object, and it can be determined that the presentation orientation of the target object is still toward the origin of space.

[0131] If the viewing angle rotation value is greater than a preset viewing angle rotation threshold, it means that if the target object is presented toward the spatial origin, then the target object will be distorted when viewed from the user's perspective. Therefore, the present application can control the target object to transform from a direction toward the spatial origin to a direction toward the user, thereby determining that the presentation direction of the target object is toward the user's current position.

[0132] S720: Determine a presentation height of the target object according to the current height of the user in the real environment.

[0133] Since the user may squat in the real environment, the user may look up or down at the target object presented in the real environment from the user's perspective, which will still cause viewing distortion of the target object.

[0134] Therefore, to ensure accurate presentation of the target object, this application can detect the user's various movements in the real environment in real time to determine the user's current height in the real environment. Then, by comparing the height of the target object with the user's current height, the target object can be controlled to be presented at a height that is flush with the user's horizontal line of sight. This determines the presentation height of the target object, preventing the user from looking up or down at the target object, and preventing the target object from being distorted.

[0135] S730: Present the target object at the presentation position in the real environment according to the presentation direction and the presentation height.

[0136] After determining the presentation direction and presentation height of the target object, the target object can be placed at the presentation position of the target object in the real environment according to the presentation direction and presentation height, so that the target object can face the direction of the user in the real environment and be presented at a height that is level with the user's horizontal line of sight, ensuring an immersive interactive effect when the target object is presented in the real environment.

[0137] The technical solution provided in the embodiment of the present application can display the target virtual scene via the display generation component. When a preset operation instruction input via one or more input components is detected, the display generation component can switch the display from the target virtual scene to the real environment in which the user is currently located, and display the target object in the target virtual scene in the real environment, thereby realizing accurate switching display of the target object from the target virtual scene to the real environment, so that the target object in the target virtual scene can enter the real environment to interact with the user, ensuring the diversity of the user's interaction scenes with the target object, enhancing the interactive fun between the target object and the user, and improving the user's immersive interactive experience with the target object in the real environment.

[0138] Figure 8 is a schematic diagram of an information display device provided in an embodiment of the present application. The information display device 800 can be configured on an electronic device that communicates with a display generation component and one or more input components. The information display device 800 includes: a virtual scene display module 810, which is used to display a target virtual scene via the display generation component, and the target virtual scene includes a target object; a real environment display module 820, which is used to respond to a preset operation instruction input via the one or more input components, display the real environment in which the user is currently located via the display generation component, and present the target object in the real environment.

[0139] In some possible implementations, the real environment display module 820 may include: a target object determination unit, used to determine the relative display position of the target object in the target virtual scene; a presentation position determination unit, used to determine the presentation position of the target object in the real environment based on the spatial origin in the real environment and the relative display position, wherein the spatial origin is the spatial position of the user in the real environment when the preset operation instruction is detected; and a target object presentation unit, used to present the target object in the real environment based on the presentation position.

[0140] In some possible implementations, the target object presentation unit may include: a presentation direction determination subunit, used to determine the presentation direction of the target object based on the presentation position and the current position of the user in the real environment; a presentation height determination subunit, used to determine the presentation height of the target object based on the current height of the user in the real environment; and a target object presentation subunit, used to present the target object at the presentation position in the real environment based on the presentation direction and the presentation height.

[0141] In some possible implementations, the presentation direction determination subunit can be specifically used to: determine the user's viewing perspective rotation value for the target object based on the presentation position, the spatial origin and the user's current position in the real environment; and determine the presentation direction of the target object based on the viewing perspective rotation value and a preset perspective rotation threshold.

[0142] In some implementations, the information display device 800 may further include: a real object calibration module, configured to determine a calibration position of each real object in the real environment in response to an object calibration operation in the real environment.

[0143] In some possible implementations, the presentation position determination unit can be specifically used to: determine the initial position of the target object in the real environment based on the spatial origin in the real environment and the relative display position; and determine the presentation position of the target object in the real environment based on the initial position and the calibrated position of each real object in the real environment.

[0144] In some possible implementations, the presentation position determination unit can also be specifically used to: in response to the drag operation of the target object, determine the initial position of the target object in the real environment based on the spatial origin in the real environment and the relative display position; update the initial position based on the drag amplitude corresponding to the drag operation to determine the presentation position of the target object in the real environment.

[0145] In some implementations, the information display device 800 may further include: a model-wearing effect presentation module, configured to present a model-wearing effect of the target object in the real environment if the relative distance between the presentation position and the user's current position in the real environment is less than a preset distance threshold.

[0146] In some implementations, the information display device 800 may further include: a fade-out special effect presentation module, configured to present a fade-out special effect of the target object image in the real environment if the presentation position is on the safety zone boundary in the real environment.

[0147] In some implementations, the information display device 800 may further include a safety zone determination module, which may be configured to determine a safety zone boundary within the real environment based on the spatial boundary of the target virtual scene and the spatial origin within the real environment.

[0148] In some possible implementations, the target virtual scene is a live video stream of any live broadcast room, and the live video stream is a 180° 3D video stream.

[0149] In some implementations, the target object is a target object image in a real-time live frame of the live video stream.

[0150] In some implementations, the information display device 800 may further include a target object image determination module. The target object image determination module may be used to:

[0151] Perform image matting processing on the target object in the real-time live frame of the live video stream to obtain a corresponding target object image.

[0152] In some implementations, the target object includes at least a character object in the target virtual scene.

[0153] In some possible implementations, the preset operation instructions input through the one or more input components include at least one of the following operation instructions: a first operation instruction generated by preset handle control information input through the user handle; a second operation instruction generated by a preset gesture initiated by the user's hand; a third operation instruction generated by preset eye movement information initiated by the user's eyes; and a fourth operation instruction generated by preset voice information collected by any voice collection component.

[0154] In an embodiment of the present application, a target virtual scene can be displayed via a display generation component. When a preset operation instruction inputted via one or more input components is detected, the display generation component can switch the display from the target virtual scene to the user's current real environment, and display the target object in the target virtual scene in the real environment, thereby realizing accurate switching display of the target object from the target virtual scene to the real environment. This enables the target object in the target virtual scene to enter the real environment to interact with the user, ensuring the diversity of the user's interactive scenes with the target object, enhancing the interactive fun between the target object and the user, and improving the user's immersive interactive experience with the target object in the real environment.

[0155] It should be understood that the device embodiment and the method embodiment in the present application may correspond to each other, and similar descriptions may refer to the method embodiment in the present application. To avoid repetition, they will not be described here.

[0156] Specifically, the device 800 shown in Figure 8 can execute any method embodiment provided in this application, and the aforementioned and other operations and / or functions of each module in the device 800 shown in Figure 8 are respectively for implementing the corresponding processes of the above-mentioned method embodiments. For the sake of brevity, they will not be repeated here.

[0157] The above-mentioned method embodiment of the embodiment of the present application is described from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in the form of hardware, can be implemented by instructions in the form of software, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above-mentioned method embodiment in conjunction with its hardware.

[0158] FIG9 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

[0159] As shown in Figure 9, the electronic device 900 may include a memory 910 and a processor 920. The memory 910 is used to store computer programs and transmit the program code to the processor 920. In other words, the processor 920 can call and run the computer program from the memory 910 to implement the method in the embodiment of the present application.

[0160] For example, the processor 920 may be configured to execute the above method embodiments according to instructions in the computer program.

[0161] In some embodiments of the present application, the processor 920 may include but is not limited to: a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0162] In some embodiments of the present application, the memory 910 includes, but is not limited to, volatile memory and / or non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0163] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 910 and executed by the processor 920 to implement the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program on the electronic device 900.

[0164] As shown in FIG. 9 , the electronic device may further include a transceiver 930 , which may be connected to the processor 920 or the memory 910 .

[0165] The processor 920 may control the transceiver 930 to communicate with other devices. Specifically, the processor 920 may send information or data to other devices or receive information or data sent by other devices. The transceiver 930 may include a transmitter and a receiver. The transceiver 930 may further include one or more antennas.

[0166] It should be understood that the various components in the electronic device 900 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0167] The present application also provides a computer storage medium having a computer program stored thereon, which enables the computer to perform the method of the above method embodiment when the computer program is executed by the computer.

[0168] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to perform the method of the above method embodiment.

[0169] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0170] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An information display method, comprising: Applied on an electronic device in communication with a display generation component and one or more input components, the method comprises: Displaying a target virtual scene via the display generation component, wherein the target virtual scene includes a target object; In response to a preset operation instruction input via the one or more input components, the real environment in which the user is currently located is displayed via the display generation component, and the target object is presented in the real environment.

2. The method according to claim 1, wherein presenting the target object in the real environment comprises: Determining a relative display position of the target object within the target virtual scene; Determining the presentation position of the target object in the real environment according to the spatial origin in the real environment and the relative display position, wherein the spatial origin is the spatial position of the user in the real environment when the preset operation instruction is detected; The target object is presented in the real environment according to the presentation position.

3. The method according to claim 2, wherein presenting the target object in the real environment according to the presentation position comprises: Determining a presentation direction of the target object according to the presentation position and a current position of a user in the real environment; Determining a presentation height of the target object according to a current height of the user in the real environment; The target object is presented at the presentation position in the real environment according to the presentation direction and the presentation height.

4. The method according to claim 3, wherein determining the presentation direction of the target object according to the presentation position and the current position of the user in the real environment comprises: Determining a viewing angle rotation value of the user with respect to the target object according to the presentation position, the spatial origin, and the current position of the user in the real environment; The presentation direction of the target object is determined according to the viewing angle rotation value and a preset viewing angle rotation threshold.

5. The method according to claim 2, wherein the method further comprises: In response to the object calibration operation in the real environment, a calibration position of each real object in the real environment is determined.

6. The method according to claim 5, wherein determining the presentation position of the target object in the real environment according to the spatial origin in the real environment and the relative display position comprises: Determining an initial position of the target object in the real environment according to the spatial origin in the real environment and the relative display position; The presentation position of the target object in the real environment is determined according to the initial position and the calibrated position of each real object in the real environment.

7. The method according to claim 2, wherein determining the presentation position of the target object in the real environment according to the spatial origin in the real environment and the relative display position further comprises: In response to the drag operation of the target object, determining an initial position of the target object in the real environment according to the spatial origin in the real environment and the relative display position; The initial position is updated according to the drag amplitude corresponding to the drag operation to determine the presentation position of the target object in the real environment.

8. The method according to claim 2, wherein after presenting the target object in the real environment according to the presentation position, the method further comprises: If the relative distance between the presentation position and the current position of the user in the real environment is less than a preset distance threshold, the model wearing effect of the target object is presented in the real environment.

9. The method according to claim 2, wherein after presenting the target object in the real environment according to the presentation position, the method further comprises: If the presentation position is on the boundary of the safety zone in the real environment, a fading effect of the target object is presented in the real environment.

10. The method according to claim 9, wherein the safety zone boundary in the real environment is determined by the following steps: The safety zone boundary in the real environment is determined according to the spatial boundary of the target virtual scene and the spatial origin in the real environment.

11. The method according to claim 1, wherein the target virtual scene is a live video stream of any live broadcast room, and the live video stream is a 180° 3D video stream.

12. The method according to claim 11, wherein the target object is a target object image in a real-time live frame of the live video stream.

13. The method according to claim 12, wherein the target object image is determined by the following steps: The target object in the real-time live frame of the live video stream is subjected to a matting process to obtain a corresponding target object image. The method according to claim 1 , wherein the target object comprises at least a character object in the target virtual scene.

15. The method according to claim 1, wherein the preset operation instructions input via the one or more input components include at least one of the following operation instructions: A first operation instruction generated by preset handle manipulation information inputted by a user handle; A second operation instruction generated by detecting a preset gesture initiated by a user's hand via a hand detection component; A third operation instruction generated by detecting preset eye movement information initiated by the user's eyes by the eye detection component and a preset gesture initiated by the hand by the hand detection component; The fourth operation instruction is generated by the preset voice information collected by any voice collecting component.

16. An information display device, configured on an electronic device communicating with a display generation component and one or more input components, the device comprising: A virtual scene display module, used for displaying a target virtual scene via the display generation component, wherein the target virtual scene includes a target object; The real environment display module is used to respond to the preset operation instructions input via the one or more input components, display the real environment where the user is currently located via the display generation component, and present the target object in the real environment.

17. An electronic device comprising: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the information display method described in any one of claims 1-15 by executing the executable instructions.

18. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the information display method according to any one of claims 1 to 15.

19. A computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the information display method according to any one of claims 1 to 15.

Images