AR image display method and device, electronic device and storage medium

Abstract

The present disclosure relates to an AR image display method and device, an electronic device, and a storage medium. The method displays at least one application portal of an AR application and determines that the corresponding AR application is a target application in response to the application portal being triggered. The method further displays a virtual object corresponding to the target application and application portals of other AR applications other than the target application. The embodiment of the present disclosure can start the corresponding AR application when the display terminal is in an AR display state, thereby starting the AR application in the metaverse space and enhancing the user's immersive experience.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an AR image display method and device, an electronic device, and a storage medium. Background Art

[0002] The current form of launching applications based on the spatial metaverse is to select the metaverse AR application to be launched through the traditional mobile phone interface and launch the AR application by clicking. This startup method provides poor immersion for users. Summary of the Invention

[0003] The present disclosure proposes an AR image display method and device, an electronic device, and a storage medium, which are intended to start applications in AR scenes and enhance user immersion.

[0004] According to a first aspect of the present disclosure, there is provided an AR image display method, comprising:

[0005] Displaying an application portal of at least one AR application;

[0006] In response to the application entry being triggered, determining the AR application corresponding to the application entry as a target application;

[0007] Display the virtual object corresponding to the target application and the application entrances of other AR applications other than the target application.

[0008] In a possible implementation, after displaying the virtual object corresponding to the target application, the method further includes:

[0009] In response to an application entry of the other AR application being triggered, determining the triggered other AR application as a new target application;

[0010] The virtual object corresponding to the new target application replaces the previously displayed virtual object and is displayed.

[0011] In a possible implementation, the application portal is displayed in at least one of text, image, video, and 3D image.

[0012] In a possible implementation, the application portal is triggered when receiving a portal opening request sent by the user in at least one of click, gaze, or voice.

[0013] In one possible implementation, each of the AR applications has a corresponding on effect and off effect, wherein the on effect is used to turn on the display of the virtual object with a corresponding visual effect, and the off effect is used to turn off the display of the virtual object with a corresponding visual effect.

[0014] In one possible implementation, displaying the application entry of at least one AR application includes:

[0015] In response to the display terminal being in an AR display state, acquiring a real scene image obtained by the display terminal;

[0016] Matching a corresponding 3D physical space model according to the real scene image, wherein the 3D physical space model has at least one corresponding AR application;

[0017] An application entrance of the at least one AR application is displayed through the display terminal.

[0018] In one possible implementation, the AR image display method is applied to a metaverse scene.

[0019] In a possible implementation, the application entry is triggered when:

[0020] Detecting that the user is looking at the application entrance for more than a preset time;

[0021] and / or, detecting that the application entry is clicked;

[0022] And / or, it is detected that the user calls the name or code corresponding to the application entrance.

[0023] According to a second aspect of the present disclosure, there is provided an AR image display device, comprising:

[0024] A first display module is configured to display an application entry of at least one AR application;

[0025] an application determination module, configured to, in response to the application entry being triggered, determine the AR application corresponding to the application entry as a target application;

[0026] The second display module is used to display the virtual object corresponding to the target application and the application entrances of other AR applications other than the target application.

[0027] In a possible implementation, after displaying the virtual object corresponding to the target application, the apparatus further includes:

[0028] an application updating module, configured to, in response to an application entry of the other AR application being triggered, determine the triggered other AR application as a new target application;

[0029] The object display module is configured to replace the previously displayed virtual object with the virtual object corresponding to the new target application and display the virtual object.

[0030] In a possible implementation, the application portal is displayed in at least one of text, image, video, and 3D image.

[0031] In a possible implementation, the application portal is triggered when receiving a portal opening request sent by the user in at least one of click, gaze, or voice.

[0032] In one possible implementation, each of the AR applications has a corresponding on effect and off effect, wherein the on effect is used to turn on the display of the virtual object with a corresponding visual effect, and the off effect is used to turn off the display of the virtual object with a corresponding visual effect.

[0033] In a possible implementation, the first display module includes:

[0034] An image acquisition submodule, configured to acquire a real scene image obtained by the display terminal in response to the display terminal being in an AR display state;

[0035] an application matching submodule, configured to match a corresponding 3D physical space model according to the real scene image, wherein the 3D physical space model has at least one corresponding AR application;

[0036] The display submodule is configured to display an application entry of the at least one AR application through the display terminal.

[0037] In one possible implementation, the AR image display method is applied to a metaverse scene.

[0038] In a possible implementation, the application entry is triggered when:

[0039] Detecting that the user is looking at the application entrance for more than a preset time;

[0040] and / or, detecting that the application entry is clicked;

[0041] And / or, it is detected that the user calls the name or code corresponding to the application entrance.

[0042] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0043] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0044] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, rather than limiting the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0046] Figure 1 A flowchart showing an AR image display method according to an embodiment of the present disclosure is shown;

[0047] Figure 2 A schematic diagram showing an AR image according to an embodiment of the present disclosure;

[0048] Figure 3 A schematic diagram showing another AR image according to an embodiment of the present disclosure;

[0049] Figure 4 A schematic diagram illustrating an AR image display device according to an embodiment of the present disclosure is shown;

[0050] Figure 5 A schematic diagram illustrating an electronic device according to an embodiment of the present disclosure is shown;

[0051] Figure 6 A schematic diagram illustrating another electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0052] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0053] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0054] The term "and / or" herein simply describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0055] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0056] In one possible implementation, the AR image display method of the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server. Among them, the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, and the like. The server can be a single server or a server cluster consisting of multiple servers. The electronic device can implement the AR image display method of the embodiment of the present disclosure by calling computer-readable instructions stored in a memory by a processor.

[0057] The embodiments of the present disclosure can be applied to any application scenario in which an AR application is enabled in an AR (Augmented Reality) image. For example, an AR game application scenario, or a metaverse scenario. As well as any application scenario in which an application portal of an AR application is displayed through an AR image, or an application scenario in which an AR application is switched in an AR image. Optionally, the display terminal that displays the AR image and the application portal of the AR application therein can be an electronic device that executes the AR image display method of the present disclosure, or a device that is communicatively connected to an electronic device that executes the AR image display method of the embodiments of the present disclosure.

[0058] Figure 1 FIG. 1 is a flow chart showing an AR image display method according to an embodiment of the present disclosure. Figure 1 As shown, the AR image display method of the embodiment of the present disclosure may include the following steps S10-S30.

[0059] Step S10: Display an application entry of at least one AR application.

[0060] In one possible implementation, the electronic device displays at least one application portal of an AR application through a built-in or connected display terminal. Optionally, the application portal of the AR application can be displayed in an AR image displayed in the display terminal. The AR image can include a real scene corresponding to at least part of the scene in the 3D physical space model, and the 3D physical space model can be obtained by performing traditional 3D reconstruction (SFM, Structure From Motion) based on the real scene.

[0061] Optionally, the 3D physical space model is a three-dimensional virtual scene that restores the real physical scene. The process of determining the 3D physical space model may include obtaining a continuous image sequence in the real physical scene, then establishing a three-dimensional physical model of the real physical scene, and obtaining a three-dimensional virtual physical scene corresponding to the real physical scene as the 3D physical space model. Among them, the real physical scene can be determined by a continuous image sequence obtained by collecting the real physical scene through an image / video acquisition device such as a smart phone or a camera. The three-dimensional physical model can be reconstructed by an SFM algorithm, that is, first extracting feature points from each frame of the continuous image sequence recording the real physical scene, and then matching adjacent feature points in multiple frames of images to obtain multiple three-dimensional points, and finally determining the three-dimensional virtual physical scene based on the three-dimensional dot matrix composed of multiple three-dimensional points as the 3D physical space model. The 3D physical space model can be deployed in electronic devices or in the cloud.

[0062] In one possible implementation, a 3D physical space model may have at least one corresponding AR application, each of which has an application portal that launches the corresponding AR application when triggered. Each application portal may have corresponding style attributes. The style attributes are used to define the display method and display effect of the AR application portal in the AR scene. Based on the corresponding style attributes, the application portal display method includes at least one of text, image, video, and 3D image. Furthermore, the application portal may also have corresponding first location information that indicates the display location of the corresponding application portal, enabling each application portal to be displayed by the electronic device. Optionally, each AR application may have multiple corresponding virtual objects that are displayed by the electronic device when the AR application is launched. Each virtual object can be a two-dimensional or three-dimensional object, such as a person, animal, object, or text symbol. Furthermore, the virtual object may also have corresponding style attributes and second location information. The style attributes are used to define the display effect of the corresponding virtual object in the AR image, and the second location information is used to indicate the display location of the corresponding virtual object in the 3D physical space model.

[0063] Furthermore, when the application entry of the AR application is displayed in the AR image, the AR image and the application entry of at least one of the AR applications can be displayed when the display terminal is in the AR display state. The AR display state of the display terminal can be triggered when a preset condition is met, for example, when the electronic device or the electronic device is connected to the display terminal and the built-in image acquisition device is turned on, it is determined that the display terminal is in the AR display state. For example, when the display terminal is a smart phone, the electronic device determines that the display terminal is in the AR display state when the scan function is turned on and the camera is turned on.

[0064] Optionally, multiple 3D physical space models can be stored in the electronic device or the cloud. When the display terminal is in an AR display state, the electronic device can match the 3D physical space model according to the real scene image obtained by the display terminal, and then display at least one AR application corresponding to the 3D physical space model according to the display terminal. That is, the electronic device can obtain the real scene image obtained by the display terminal in response to the display terminal being in the AR display state. According to the real scene image, the corresponding 3D physical space model is matched, and the 3D physical space model has at least one corresponding AR application. The application entrance of at least one AR application is displayed through the display terminal. Among them, the display terminal can display the application entrance of the AR application in the AR image, and in addition to the application entrance, the AR image also includes the acquired real scene image.

[0065] Figure 2 FIG. 1 is a schematic diagram showing an AR image according to an embodiment of the present disclosure. Figure 2 As shown, when the display terminal displays the application entrance of the AR application in the AR image, the display terminal is in the AR display state, and the AR image is displayed through the display terminal. The AR image includes a real scene image 20 as a background, and at least one application entrance 21 of the AR application. The application entrance 21 displayed in the AR image is the application entrance 21 of the AR application corresponding to the 3D physical space model matched with the real scene image 20. The display position of each application entrance 21 in the AR image can be randomly determined or displayed at a preset display position.

[0066] Step S20: In response to the application entry being triggered, determining the AR application corresponding to the application entry as the target application.

[0067] In one possible implementation, an application portal can open a corresponding AR application when triggered. When the current application portal is triggered, the AR application corresponding to the application portal is determined to be the target application. Optionally, the application portal can be triggered by any triggering method. For example, the application portal can be triggered when the electronic device receives a portal opening request sent by the user through at least one of clicking, gazing, or voice. That is, the user can trigger the application portal by clicking on the application portal displayed on the display terminal, or by gazing at the application portal through a VR (Virtual Reality) terminal connected to the display terminal or serving as a display terminal, or by sending the name of the application portal to be opened by voice, and the AR application corresponding to the triggered application portal is determined to be the target application. In other words, the electronic device can determine that the corresponding application portal is triggered when it detects that the user has gazed at the application portal for more than a preset time, and / or detects that the application portal has been clicked, and / or detects that the user has called the application portal by name or code, and the AR application corresponding to the application portal is the target application.

[0068] Step S30: Display the virtual object corresponding to the target application and the application entrances of other AR applications other than the target application.

[0069] In one possible implementation, the electronic device turns on the target application after determining the target application, so as to display the virtual object corresponding to the target application through the display terminal. The virtual object corresponding to the target application can be displayed at the position indicated by the corresponding second position information with preset style attributes. Optionally, the display terminal also displays application portals of other AR applications in addition to the target application, so as to enable switching to the target application when other AR application portals are triggered, and refresh the virtual object displayed in the display terminal. Optionally, the virtual object corresponding to the target application and the application portals of other AR applications can also be displayed in the AR image displayed by the display device.

[0070] Based on the above display method, the embodiments of the present disclosure can be applied to any application scenario for displaying AR images. For example, the AR image display method can be applied to metaverse scenarios, AR game scenarios, etc.

[0071] Figure 3 Schematic diagram of another AR image according to an embodiment of the present disclosure is shown. Figure 3As shown, when the display terminal displays the virtual objects of the target application and the application entrance of the AR application through the AR image, when the display terminal is in the AR display state, the AR image is displayed through the display terminal. Among them, when there is a target application that has been started in the AR image, the AR image includes a real scene image 30 as the background, application entrances 31 of other AR applications other than the target application, and at least one virtual object 32 corresponding to the target application. The application entrances 31 of other AR applications displayed in the AR image are application entrances 31 of other AR applications other than the target application in the AR application corresponding to the 3D physical space model matched by the real scene image 30. The display position of each application entrance 31 in the AR image can be randomly determined or displayed at a preset display position. Furthermore, the user can switch the currently opened target application by triggering the application entrance 31 in the AR image, and refresh the virtual objects 32 and application entrance 31 displayed in the AR image.

[0072] Furthermore, if a target application is already active, i.e., the display terminal is already displaying a virtual object corresponding to the target application, the electronic device can directly switch the active target application based on the application entry of another AR application displayed on the display terminal, i.e., close the current target application and open the other AR application, while refreshing the displayed virtual object. That is, while the display terminal is currently displaying the virtual object corresponding to the target application, the electronic device can continuously monitor whether the entry of the other application is triggered, and in response to the application entry of the other AR application being triggered, determine the triggered other AR application as the new target application, replace the previously displayed virtual object with the virtual object corresponding to the new target application, and display it. The other AR application is an AR application other than the target application corresponding to the currently displayed virtual object. That is, upon detecting that the application entry of an AR application other than the current target application has been triggered, the electronic device determines the newly triggered AR application as the new target application, closes the currently displayed virtual object and the application entry of the newly triggered AR application, and displays the virtual object corresponding to the updated target application and the application entry of the other AR application other than the updated target application in the AR image.

[0073] Optionally, each AR application has a corresponding on effect and off effect. The on effect is used to turn on the display of the virtual object with the corresponding visual effect, and the off effect is used to turn off the display of the virtual object with the corresponding visual effect. Among them, the on effect and off effect of each AR application can include at least one. The electronic device can select one of the at least one on effect and off effect to turn off or on the display of the virtual object according to preset rules each time the AR application is turned on or off. The on effect and off effect can be any effect, such as fade in and out, rotation in and out, and blinds, etc., to improve the user's visual effect and immersive experience during the application switching process.

[0074] In one possible implementation, the electronic device may also cancel the display of the application portal corresponding to the target application when the target application is opened, and add the display of the application portal corresponding to the target application when the target application is closed. Optionally, the opening and closing display of each application portal may also have a preset visual effect.

[0075] The disclosed embodiments enable launching corresponding AR applications while the display terminal is in AR display mode, enabling direct launching of AR applications within the metaverse space, as well as direct switching between launched AR applications within the metaverse space. This AR application launching and switching method can improve the efficiency of application usage within the metaverse space and enhance the user's immersive experience.

[0076] It is understood that the above-mentioned various method embodiments mentioned in this disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, this disclosure will not go into details. It is understood by those skilled in the art that in the above-mentioned methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0077] In addition, the present disclosure also provides an AR image display device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any AR image display method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method section and will not be repeated here.

[0078] Figure 4 A schematic diagram of an AR image display device according to an embodiment of the present disclosure is shown as follows: Figure 4 As shown, the device includes:

[0079] A first display module 40 is configured to display an application entry of at least one AR application;

[0080] an application determining module 41 for determining, in response to the application entry being triggered, the AR application corresponding to the application entry as a target application;

[0081] The second display module 42 is configured to display the virtual object corresponding to the target application and application entrances of other AR applications other than the target application.

[0082] In a possible implementation, after displaying the virtual object corresponding to the target application, the apparatus further includes:

[0083] an application updating module, configured to, in response to an application entry of the other AR application being triggered, determine the triggered other AR application as a new target application;

[0084] The object display module is configured to replace the previously displayed virtual object with the virtual object corresponding to the new target application and display the virtual object.

[0085] In a possible implementation, the application portal is displayed in at least one of text, image, video, and 3D image.

[0086] In a possible implementation, the application portal is triggered when receiving a portal opening request sent by the user in at least one of click, gaze, or voice.

[0087] In one possible implementation, each of the AR applications has a corresponding on effect and off effect, wherein the on effect is used to turn on the display of the virtual object with a corresponding visual effect, and the off effect is used to turn off the display of the virtual object with a corresponding visual effect.

[0088] In a possible implementation, the first display module 40 includes:

[0089] An image acquisition submodule, configured to acquire a real scene image obtained by the display terminal in response to the display terminal being in an AR display state;

[0090] an application matching submodule, configured to match a corresponding 3D physical space model according to the real scene image, wherein the 3D physical space model has at least one corresponding AR application;

[0091] The display submodule is configured to display an application entry of the at least one AR application through the display terminal.

[0092] In one possible implementation, the AR image display method is applied to a metaverse scene.

[0093] In a possible implementation, the application entry is triggered when:

[0094] Detecting that the user is looking at the application entrance for more than a preset time;

[0095] and / or, detecting that the application entry is clicked;

[0096] And / or, it is detected that the user calls the name or code corresponding to the application entrance.

[0097] This method has a specific technical connection with the internal structure of the computer system, and can solve the technical problem of how to improve the hardware computing efficiency or execution effect (including reducing the amount of data storage, reducing the amount of data transmission, increasing the hardware processing speed, etc.), thereby obtaining the technical effect of improving the internal performance of the computer system in accordance with the laws of nature.

[0098] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0099] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0100] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0101] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0102] The electronic device may be provided as a terminal, a server, or other forms of devices.

[0103] Figure 5 A schematic diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a terminal device such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, or a wearable device.

[0104] Reference Figure 5, the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0105] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0106] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0107] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0108] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0109] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0110] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0111] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect changes in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0112] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as wireless network (Wi-Fi), second generation mobile communication technology (2G), third generation mobile communication technology (3G), fourth generation mobile communication technology (4G), long term evolution (LTE) of universal mobile communication technology, fifth generation mobile communication technology (5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0113] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0114] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions. The computer program instructions can be executed by the processor 820 of the electronic device 800 to perform the above method.

[0115] The present disclosure relates to the field of augmented reality. By acquiring image information of a target object in a real-world environment, the relevant features, states, and attributes of the target object are detected or identified using various vision-related algorithms, thereby achieving an AR effect that combines virtual and real life and matches the specific application. For example, the target object may be a face, limbs, gestures, movements, etc. related to the human body, or an identifier or marker related to an object, or a sandbox, display area, or display items related to a venue or location. Vision-related algorithms may involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, key point extraction and tracking of objects, and object pose or depth detection. Specific applications can involve not only interactive scenarios such as guided tours, navigation, explanations, reconstruction, and virtual effect overlay displays related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, body beautification, special effects display, and virtual model display. Detection or identification of the relevant features, states, and attributes of the target object can be achieved using a convolutional neural network. The above-mentioned convolutional neural network is a network model obtained by model training based on a deep learning framework.

[0116] Figure 6 FIG2 is a schematic diagram showing another electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. Figure 6 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0117] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (Mac OSX TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open-source Unix-like operating system (FreeBSD TM ) or similar.

[0118] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0119] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0120] Computer-readable storage media can be a tangible device that can hold and store the instructions used by the instruction execution device. Computer-readable storage media can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. Computer-readable storage media used herein is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagated by waveguides or other transmission media (for example, light pulses by fiber optic cables), or electrical signals transmitted by wires.

[0121] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0122] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0123] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0124] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0125] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0126] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0127] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0128] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0129] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0130] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0131] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An AR image display method, characterized in that: The method comprises: Displaying an application portal of at least one AR application, wherein the at least one AR application corresponds to a 3D physical space model that restores a real physical scene, any one of the AR applications has a plurality of corresponding virtual objects for display by the electronic device when the AR application is opened, and any one of the virtual objects has corresponding second position information for representing a display position of the virtual object in the 3D physical space model; In response to the application entry being triggered, determining the AR application corresponding to the application entry as a target application; Display the virtual object corresponding to the target application and the application entrances of other AR applications other than the target application, and cancel the display of the application entrance corresponding to the target application when the target application is started.

2. The method according to claim 1, characterized in that After displaying the virtual object corresponding to the target application, the method further includes: In response to an application entry of the other AR application being triggered, determining the triggered other AR application as a new target application; The virtual object corresponding to the new target application replaces the previously displayed virtual object and is displayed.

3. The method according to claim 1 or 2, characterized in that The display mode of the application entrance includes at least one of text, image, video and 3D image.

4. The method according to claim 1 or 2, characterized in that The application portal is triggered when receiving a portal opening request sent by the user in at least one of click, gaze or voice.

5. The method according to claim 1 or 2, characterized in that Each of the AR applications has a corresponding on effect and off effect, the on effect being used to turn on the display of the virtual object with a corresponding visual effect, and the off effect being used to turn off the display of the virtual object with a corresponding visual effect.

6. The method according to claim 1 or 2, characterized in that Displaying at least one AR application application entry includes: In response to the display terminal being in an AR display state, acquiring a real scene image obtained by the display terminal; Matching a corresponding 3D physical space model according to the real scene image, wherein the 3D physical space model has at least one corresponding AR application; An application entrance of the at least one AR application is displayed through the display terminal.

7. The method according to claim 1 or 2, characterized in that The AR image display method is applied to the metaverse scene.

8. The method according to claim 1 or 2, characterized in that The situations where the application entry is triggered include: Detecting that the user is looking at the application entrance for more than a preset time; and / or, detecting that the application entry is clicked; And / or, it is detected that the user calls the name or code corresponding to the application entrance.

9. An AR image display device, characterized in that: The device comprises: a first display module, configured to display an application portal of at least one AR application, wherein the at least one AR application corresponds to a 3D physical space model that restores a real physical scene, each AR application having a plurality of corresponding virtual objects for display on the electronic device when the AR application is opened, each virtual object having corresponding second position information for representing a display position of the virtual object in the 3D physical space model; an application determination module, configured to, in response to the application entry being triggered, determine the AR application corresponding to the application entry as a target application; The second display module is used to display the virtual object corresponding to the target application and the application entrances of other AR applications other than the target application, and cancel the display of the application entrance corresponding to the target application when the target application is started.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images