Extended reality content display control method, device, electronic device and storage medium

Abstract

The embodiment of the present application discloses an extended reality content display control method, device, electronic device and storage medium, the method comprising: obtaining display status information of an extended reality device, the display status information indicating the display status of the extended reality device; if the display status representation satisfies the information display condition, then obtaining the information to be displayed; and controlling the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed. The display status is used as a condition for the extended reality device to display information, and the timing of the content display of the extended reality device is controlled to avoid the uncontrolled and unconstrained random display of the content of the extended reality device, thereby improving the effectiveness of the display of the content information of the extended reality device and improving the user experience.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of extended reality, and specifically to an extended reality content display control method, device, electronic device, and storage medium, wherein the storage medium includes a computer-readable storage medium. Background Art

[0002] Extended Reality (XR) is a comprehensive term that covers virtual reality (VR), augmented reality (AR) and mixed reality (MR). Extended reality technology expands human perception and interaction methods, interacting with the real world through digital technology to create virtual, augmented and mixed reality experiences.

[0003] Extended reality provides a more intuitive and convenient content presentation experience through virtual reality scenes. It has developed rapidly in the fields of entertainment, games, education, medical care, corporate production, and social communication, demonstrating great potential and influence.

[0004] However, currently the content presentation in virtual reality scenes is still mainly an uncontrolled, random, and unconstrained display method. After the device is started, the extended reality content is immediately presented in the virtual reality scene, and when the content to be displayed is detected, it will also be displayed without constraint. However, this method will result in too much content being presented, or presenting some content that the user is not interested in, which will affect the user's attention to important content. Although there is also a switch startup control to control the display of content, the switch-based startup control method makes the content presentation control operation complicated and the user experience needs to be improved. Summary of the invention

[0005] The embodiments of the present application provide an extended reality content display control method, device, electronic device and computer-readable storage medium, which can accurately control the timing of content display, automatically control the display of extended reality content, and improve user experience.

[0006] The present application provides an extended reality content display control method, which is applied to an extended reality device. The method includes:

[0007] Acquire display status information of the extended reality device, where the display status information indicates a display status of the extended reality device;

[0008] If the display status representation satisfies the information display condition, obtaining the information to be displayed;

[0009] Controlling the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed.

[0010] Accordingly, an embodiment of the present application further provides an extended reality content display control device, which is applied to an extended reality device, and the device includes:

[0011] A first acquisition module, configured to acquire display status information of the extended reality device, wherein the display status information indicates a display status of the extended reality device;

[0012] A second acquisition module, configured to acquire information to be displayed if the display status representation satisfies an information display condition;

[0013] The display module is used to control the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed.

[0014] Optionally, in some embodiments of the present application, the first acquisition module includes:

[0015] An information determining unit, used to determine usage scenario information of the extended reality device;

[0016] A state determination unit, configured to determine a display state of the extended reality device according to the usage scenario information and a preset trigger reference condition;

[0017] The state information generating unit is used to generate display state information according to the display state.

[0018] In some embodiments of the present application, the state determination unit includes:

[0019] a scene recognition subunit, configured to recognize a scene type corresponding to the usage scene information, and determine a target trigger reference condition from at least one preset trigger reference condition according to the scene type;

[0020] a factor determination subunit, configured to determine a display control reference factor according to the target trigger reference condition, wherein the display control reference factor includes at least one of time, location, posture, user manipulation, power, user emotion, or program running state;

[0021] The state setting subunit is used to determine the real-time reference information of the display control reference factor under the usage scenario information, and if the real-time reference information meets the target trigger reference condition, the satisfied information display condition is set as a display state.

[0022] In some embodiments of the present application, the second acquisition module includes:

[0023] An environmental information acquisition unit, used to acquire attribute information of a target user holding the extended reality device and environmental scene information of the target user;

[0024] A display information determination unit is used to determine the information to be displayed according to the environmental scene information and the attribute information.

[0025] In some embodiments of the present application, the display module includes:

[0026] a priority determination unit, configured to determine a display priority according to the attribute information and the information type corresponding to the information to be displayed;

[0027] a depth information determining unit, configured to determine display depth information of the information to be displayed according to the display priority;

[0028] A content display unit is used to present the extended reality content corresponding to the information to be displayed according to the display depth information through the optical display module.

[0029] Among them, in some embodiments of the present application, the content display unit includes:

[0030] A posture information acquisition subunit, used to acquire the posture information of the extended reality device;

[0031] a position information determining subunit, configured to determine display position information of the information to be displayed according to the posture information and the display priority;

[0032] The content display subunit is used to present the extended reality content corresponding to the information to be displayed according to the display depth information and the display position information through the optical display module.

[0033] In some embodiments of the present application, the content display subunit is specifically used for:

[0034] Determining display style information of the information to be displayed according to the display priority;

[0035] The optical display module presents the extended reality content corresponding to the information to be displayed according to the display depth information, the display position information and the display style information.

[0036] In some embodiments of the present application, the content display subunit is specifically used for:

[0037] Collecting a preview image previewed through the optical display module through the camera of the extended reality device;

[0038] Determining background content information corresponding to the information to be displayed from the preview screen according to the display position information;

[0039] Adjusting the initial display style information of the information to be displayed according to the content tone of the background content information to obtain target display style information;

[0040] The information to be displayed is presented through the optical display module according to the display depth information, the display position information and the target display style information.

[0041] In some embodiments of the present application, the display information determination unit includes:

[0042] An intention determination subunit, used to determine the behavior intention of the target user according to the environmental scene information and the attribute information;

[0043] A first setting subunit, used to set the content information corresponding to the behavioral intention in the preset memory as information to be displayed;

[0044] and / or,

[0045] The second setting subunit is used to output the to-be-displayed information corresponding to the behavior intention through the intention-content relationship model.

[0046] In some embodiments of the present application, the device further includes a display information acquisition module, and the display information acquisition module includes:

[0047] A display information acquisition unit, used to acquire original display information;

[0048] The second acquisition module includes:

[0049] A first acquisition unit, configured to set the original display information as information to be displayed if the display state representation satisfies an information display condition;

[0050] or,

[0051] The second acquisition unit is configured to generate display content control information if the display status representation satisfies the information display condition, and filter the information to be displayed from the original display information according to the display content control information.

[0052] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps in the above-mentioned extended reality content display control method when executed by the processor.

[0053] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned extended reality content display control method are implemented.

[0054] In a fifth aspect, the embodiments of the present application further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0055] In summary, the embodiments of the present application obtain the display status information of the extended reality device, and control the acquisition of the information to be displayed and the display control of the information to be displayed through the display status indicated by the display status information, so as to realize using the display status as a condition for the extended reality device to display information, control the timing of content display of the extended reality device, avoid the uncontrolled and unconstrained random display of the content of the extended reality device, improve the effectiveness of the display of content information of the extended reality device, and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 is a scene schematic diagram of the extended reality content display control method provided in an embodiment of the present application;

[0058] Figure 2 is a flowchart of an extended reality content display control method provided in an embodiment of the present application;

[0059] Figure 3 is a schematic diagram of depth information of an extended reality device screen provided in an embodiment of the present application;

[0060] Figure 4 is a schematic diagram of a display area of ​​an extended reality device screen provided in an embodiment of the present application;

[0061] Figure 5 This is a schematic diagram of the control effect of the extended reality content display before the trip provided by the embodiment of the present application;

[0062] Figure 6 It is a schematic diagram of the display control effect of the extended reality content during travel provided by an embodiment of the present application;

[0063] Figure 7 This is a schematic diagram of the extended reality content display control effect after traveling provided by an embodiment of the present application;

[0064] Figure 8 is a flowchart of the augmented reality device provided in an embodiment of the present application executing the extended reality content display control method;

[0065] Fig. 9 is a structural diagram of an extended reality content display control device provided in an embodiment of the present application;

[0066] Fig.10 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0067] Description of Figure Numbers:

[0068] A-foreground; B-default; C-background; D-upper area; E-middle area; F-lower area; 401-processor; 402-memory; 403-power supply; 404-input unit; 10-extended reality device. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0070] The embodiments of the present application provide an extended reality content display control method, device, electronic device, and computer-readable storage medium. Specifically, the embodiments of the present application provide an extended reality content display control device suitable for an electronic device, the electronic device including an extended reality (XR) device, wherein the extended reality device includes virtual reality (VR, virtual reality), augmented reality (AR, augmented reality) and mixed reality (MR, mixed reality) and other devices. Among them, in the embodiments of the present application, the extended reality device at least includes a product in the form of glasses or a product in the form of a head-mounted display. Accordingly, the extended reality device should be at least an optically transparent product or a video-transmitting product.

[0071] See also Figure 1 , Figure 1 : is a schematic diagram of a scenario in which an extended reality device (such as optically transmittive wearable smart AR glasses) provided in an embodiment of the present application executes the extended reality content display control method, wherein the specific execution process of the extended reality device executing the extended reality content display control method is as follows:

[0072] The extended reality device 10 obtains display status information of the extended reality device 10, wherein the display status information indicates the display status of the extended reality device 10. If the display status representation satisfies the information display condition, the information to be displayed is obtained, and the optical display module of the extended reality device 10 is controlled to display the extended reality content corresponding to the information to be displayed.

[0073] Among them, the embodiment of the present application obtains the display status information of the extended reality device, and controls the acquisition of the information to be displayed and the display of the information to be displayed through the display status indicated by the display status information, so as to realize using the display status as a condition for the extended reality device to display information, control the timing of content display of the extended reality device, avoid the uncontrolled and unconstrained random display of the content of the extended reality device, improve the effectiveness of the display of content information of the extended reality device, and improve the user experience.

[0074] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0075] See also Figure 2 , Figure 2 A flowchart of an extended reality content display control method provided in an embodiment of the present application. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown in the accompanying drawings. The extended reality content display control method is applied to an extended reality device, and the specific process of the method is as follows:

[0076] 101. Obtain display status information of the extended reality device, where the display status information indicates a display status of the extended reality device.

[0077] It should be noted that the display state indicates whether the extended reality device has, or meets the display conditions, or is able to display information. For example, in the embodiment of the present application, the display state includes at least one of meeting the information display conditions or not meeting the information display conditions. Among them, meeting the information display conditions indicates that the extended reality device meets the conditions for displaying information. From the user's perspective, information should be displayed at the current moment, and from the device availability, the software and hardware modules of the extended reality device can display information at the current moment.

[0078] Among them, the display status information is the information presentation form of the display status, such as text, symbols, graphics, charts, numbers or a combination thereof, and based on the display status information, the display status of the extended reality device can be indicated or reflected.

[0079] Among them, by obtaining the display status information, it is helpful to control the information display of the extended reality device based on the display status indicated by the display status information.

[0080] 102. If the display status representation satisfies the information display condition, obtain information to be displayed.

[0081] It should be noted that the information display condition refers to the condition that needs to be met for information display. For example, the condition can be time, location, weather, user behavior, voice, device power, etc. For example, in an embodiment of the present application, a timer task is set, and when the set time is reached, the information to be displayed is automatically obtained and displayed. For example, when the user brings the extended reality device to a certain location, the acquisition of the information to be displayed is triggered. Among them, the set time of the timer task and the location that triggers the acquisition of the information to be displayed are the information display conditions in this case.

[0082] It can be understood that the information to be displayed is information used to be displayed through the extended reality device, and the information to be displayed may include various types of information such as weather information, email information, social information, entertainment information or news information.

[0083] It can be understood that the information to be displayed is obtained only when the display status representation of the extended reality device meets the information display conditions, so that the display status serves as a control condition for obtaining the information to be displayed, that is, the information to be displayed is obtained only when the conditions are met, otherwise, there is no need to obtain the information to be displayed, which can effectively avoid the acquisition of invalid information and avoid waste of resources.

[0084] Among them, in an embodiment of the present application, the information display condition can be set unchanged, for example, the information display condition can be a fixed time point or a fixed location, that is, reaching the fixed time point or the fixed location triggers the acquisition of the information to be displayed.

[0085] Correspondingly, the information display condition can also be dynamically changed. For example, the information display condition can also be dynamically changed with the date, usage time, device power, etc. For example, as the usage time increases, the information display condition is adjusted from place a to place b. For example, as the date changes, the information display condition is adjusted from nine o'clock in the evening to seven o'clock in the evening. For example, if the device is used for a long time, the information display condition is not limited to a specific place, but other places, or the real-time place where the user is located, or a single place is switched to multiple places as the information display condition, and the acquisition of the information to be displayed is triggered in the other place, or the real-time place where the user is located, or more places. For example, after using the device for a long time, not only the bedroom can obtain eye protection information, but also places such as the living room or the bathroom can trigger the acquisition of eye protection information. In another example, if the date changes from summer to autumn, or from autumn to winter, the time for obtaining the information to be displayed (such as important domestic and foreign event information of the day) can be advanced. For example, the time condition for obtaining the information to be displayed at nine o'clock in the evening is adjusted to seven o'clock in the evening, so as to obtain the information to be displayed in advance, so as to obtain the information in advance in the early bedtime season, etc.

[0086] Among them, fixed configuration information display conditions are helpful for user customization and can realize flexible configuration based on the user's single demand, while dynamically configured information display conditions are helpful for adaptive adjustment of information acquisition timing based on different environments, different considerations or the diversity of needs under different nodes, so as to meet the dynamic acquisition needs of the information to be displayed.

[0087] 103. Control the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed.

[0088] It should be noted that the optical display module is a module that displays extended reality content on an extended reality device. The optical display module can be a combination of a series of optical elements. For example, the optical display module can include a micro display screen, a sensor, a combiner, a lens, and a transmitter.

[0089] It can be understood that by obtaining the information to be displayed and controlling the extended reality device to display the extended reality content corresponding to the information to be displayed after the information display conditions are met, the control of the extended reality device displaying information can be achieved, which can avoid the extended reality device from displaying content randomly without restraint and constraints, and can accurately control the timing and content of the extended reality device displaying information, thereby reducing the presentation of invalid content.

[0090] Among them, the information display conditions can be user-defined conditions, so that the embodiment of the present application can only obtain the information to be displayed and display the extended reality content corresponding to the information to be displayed after meeting specific conditions, thereby realizing the control of the content displayed by the extended display device, improving the effectiveness of the content information display of the extended reality device, and improving the user experience.

[0091] Optionally, in the embodiment of the present application, a trigger reference condition may be set. When the trigger reference condition is met, a display state that meets the information display condition is generated, and when the trigger reference condition is not met, a display state that does not meet the information display condition is generated. For example, when it is between 7 and 9 p.m., the display state is set to meet the information display condition; when the extended reality device is used in a conference office, game, or immersive movie viewing scenario, the display state is set to not meet the information display condition.

[0092] Among them, in one embodiment of the present application, the information to be displayed is not the information currently used by the extended reality device, but information other than the currently used information, that is, information that needs to be additionally collected and generated in addition to the currently displayed content. For example, when the user is using the extended reality device based on the first application, the information to be displayed may be information collected by a second application on the extended reality device other than the first application. For example, the user is using a map application on the extended reality device, then when the user reaches a certain location, the food recommendation application is started to obtain food information near the location.

[0093] Correspondingly, the trigger reference condition can also be dynamically changed, for example, with the change of time, equipment use proficiency, frequency, holidays, weather or temperature, the corresponding trigger reference condition is dynamically generated. For example, when the use time of the extended reality device continues to increase and exceeds the set time threshold, the trigger reference condition is adjusted from G to H, and then the display state of the extended reality device is changed, so that the display state switches from not meeting the information display condition to meeting the information display condition, and then drives the extended reality device to trigger the acquisition operation of the information to be displayed, such as adjusting the trigger reference condition of two hours to one hour after continuous long-term use. After one hour, it is considered that the extended reality device at this time meets the information display condition, and the acquisition of eye protection reminder information is triggered, or because the game, office and other scenes require the user to stay focused, but because too long time is easy to cause information omission, therefore, after a long period of continuous gaming and office work, the set time of the message reminder is shortened, and when it reaches one hour, it directly triggers the acquisition of the message, etc.

[0094] Correspondingly, the trigger reference condition may also be a combination of multiple conditions, such as a comprehensive consideration of multiple conditions such as time, weather, user emotions, etc. For example, when the user is in a relatively calm mood, and the weather is good and the temperature is appropriate, it is considered that this is a suitable time to process information, and the display state is set to meet the information display conditions; when the user is in a relatively calm mood, and the weather is bad, and the temperature is high or low, it is considered that the conditions for processing messages are poor, and the display state is set to not meet the information display conditions.

[0095] Optionally, in an embodiment of the present application, whether the corresponding information display condition is met can also be analyzed and judged according to the scenario used by the extended display device, that is, optionally, in some embodiments of the present application, the step of “obtaining the display status information of the extended reality device” includes:

[0096] Determining usage scenario information of the extended reality device;

[0097] Determining a display state of the extended reality device according to the usage scenario information and a preset trigger reference condition;

[0098] Display state information is generated according to the display state.

[0099] It is understandable that the usage scenario information is information about the scenario in which the extended reality device is used, for example, the usage scenario of the extended reality device includes rest, travel, office, shopping or sports, etc. It can also be a specific location, such as a waiting room, a shopping mall, a home or a park, etc.

[0100] Among them, the preset trigger reference condition is a pre-set condition that needs to be met to obtain the information to be displayed, that is, the condition that triggers the extended reality device to obtain the information to be displayed. The condition type of the preset trigger reference condition can be time, location, device posture or device power, etc., wherein, in the embodiment of the present application, when determining the display status of the extended reality device, it is also possible to combine the preset trigger reference conditions of multiple condition types for analysis and judgment.

[0101] Among them, by combining the usage scenario information of the extended reality device, the accuracy of display state generation can be improved. For example, when the current usage scenario of the extended reality device meets a specific usage scenario, the display state is set to meet the information display condition. Specifically, for example, the rest usage scenario does not meet the information display condition, and the exercise usage scenario meets the information display condition.

[0102] Optionally, in the embodiments of the present application, corresponding trigger reference conditions may be set for different usage scenarios of the extended reality device, that is, the preset trigger reference conditions correspond to the usage scenarios one by one. For the scenario used by the extended reality device, it is determined whether the trigger reference conditions corresponding to the scenario are met, and then the display state is generated or set, that is, optionally, in some embodiments of the present application, the step of "determining the display state of the extended reality device according to the usage scenario information and the preset trigger reference conditions" includes:

[0103] Identifying a scenario type corresponding to the usage scenario information, and determining a target trigger reference condition from at least one preset trigger reference condition according to the scenario type;

[0104] Determine a display control reference factor according to the target trigger reference condition, wherein the display control reference factor includes at least one of time, location, posture, user control, power, user emotion or program running status;

[0105] Determine the real-time reference information of the display control reference factor under the usage scenario information, and if the real-time reference information meets the target trigger reference condition, set the satisfied information display condition as a display state.

[0106] For example, in an office scenario, a specific time period can be set as a trigger reference condition, and in a game scenario, a power range can be set as a trigger reference condition. Accordingly, in an office scenario, after a specific time period is reached, the display status is set to not satisfy the information display condition, and in a game scenario, after the power of the extended reality device reaches a preset power range, the display status information is set to not satisfy the information display condition.

[0107] The scene type is the type of the usage scene of the extended reality device, for example, the scene type may be home, park, waiting room, or travel, work, entertainment, etc. Each scene type may correspond to one or more target trigger reference conditions, for example, the home scene may include two target trigger reference conditions of time and power, and the work scene may correspond to three target trigger reference conditions of time, location and power.

[0108] The display control reference factor is determined based on the reference object that the target trigger reference condition focuses on. For example, if the target trigger reference condition is for time, the display control reference factor is also for time. If the target trigger reference condition is for location, the display control reference factor is also for location. The display control reference factor and the target trigger reference condition are usually in a one-to-one relationship. However, when the target trigger reference condition focuses on multiple reference objects, multiple display control reference factors correspond. For example, when the target trigger reference condition focuses on time and location, the display control reference factor includes both time and location.

[0109] Correspondingly, after determining the target trigger reference condition of concern under the scene type based on the scene type, the display control reference factor of concern for the scene type can be determined based on the target trigger reference condition, and then the corresponding real-time reference information can be extracted from the usage scene information based on the display control reference factor. For example, when the scene type is home and the target trigger reference condition is a time period, it can be determined that the display control reference factor corresponding to the target trigger reference condition is time. Therefore, the real-time time of the usage scene (that is, the real-time reference information at this time) can be obtained, and the real-time time of the usage scene is compared with the time period corresponding to the target trigger reference condition. If it is within the time period, the information display condition is met. If it is not within the time period, the information display condition is not met. The extended reality device can request and receive the current precise time from the server through the Network Time Protocol NTP (Network Time Protocol).

[0110] It should be noted that the posture refers to the posture of the user or wearer of the extended reality device, which describes the posture of the head or other parts of the body of the user or wearer of the extended reality device in space, such as head movements such as raising the head, lowering the head, and shaking the head. In the embodiment of the present application, the posture of the head or other parts of the body of the user or wearer of the extended reality device can be obtained through the parameters of the sensors built into the extended reality device. For example, the posture of the extended reality device is captured by a gyroscope, accelerometer, magnetometer, etc., and the head posture of the user or wearer is calculated by a 6DoF algorithm; for another example, the hand movements of the user or wearer are collected by a camera, and the gestures of the user or wearer are calculated by a gesture recognition algorithm.

[0111] In an embodiment of the present application, the posture change of the user or wearer of the extended reality device is used as a control condition for the display of information of the extended reality device, so that the extended reality device can perform display control based on the specific behavior of the user or the wearer, thereby improving the diversity of the control methods for the display of information of the extended reality device and improving the user experience.

[0112] Among them, battery level refers to the battery level of the extended display device; user control refers to the user's active control of the extended reality device, including voice operation, button control, gesture-based guidance control, eye viewpoint control or eye behavior control, etc.; user emotion refers to the user's emotion, including joy, tension, anxiety, anger, etc.; program running status refers to the running status of the application installed on the extended reality device, for example, running and non-running, etc. Correspondingly, the program running status may also refer to the program receiving a message or not receiving a message, etc.

[0113] In summary, the embodiments of the present application analyze and determine the information display timing of the extended reality device through multiple angles such as time, location, posture, user control, power, user emotion or program running status, so as to fully realize the control of the information display timing, avoid the uncontrolled and unconstrained random display of the content of the extended reality device, improve the effectiveness of the content information display of the extended reality device, and improve the user experience.

[0114] In the embodiment of the present application, the information to be displayed may also be collected first, and then when it is determined that the display status representation satisfies the information display condition, the information to be displayed is directly displayed.

[0115] Accordingly, the original display information may be collected first, and when the display status information indicates that the information display condition is satisfied, the original display information may be used as the information to be displayed, or a portion of the information may be selected from the original display information as the information to be displayed. That is, optionally, in some embodiments of the present application, before the step of “obtaining the display status information of the extended reality device”, the method may further include:

[0116] Get the original display information;

[0117] If the display state representation satisfies the information display condition, obtaining the information to be displayed includes:

[0118] If the display status representation satisfies the information display condition, setting the original display information as the information to be displayed;

[0119] Alternatively, if the display state representation satisfies the information display condition, display content control information is generated, and the information to be displayed is filtered from the original display information according to the display content control information.

[0120] It can be understood that the original display information is information related to the extended reality content used for display. The original display information can be collected by various sensors or received by various applications on the extended reality device. Correspondingly, the original display information can also be collected by the application on the extended reality device based on instructions to actively obtain information through the Internet or a local area network.

[0121] Accordingly, in the embodiment of the present application, the original display information can be obtained by continuous collection or periodic collection. Accordingly, when the information display condition is met, the original display information can be directly set as the information to be displayed for display on the extended reality device, or part of the information to be displayed can be selected from the original display information and set as the information to be displayed, for example, based on the user's emotions, the scene in which the user currently uses the extended reality device, or the priority of the information, the corresponding information can be selected from the original display information to obtain the information to be displayed.

[0122] In an embodiment of the present application, the display status information can also be adjusted based on the original display information. For example, after the original display information is obtained, the priority of the original display information is identified or analyzed, and the display status information is determined based on the priority. For example, when the priority of the original display information is higher, the display status information is set to meet the information display conditions; when the priority of the original display information is lower, the display status information is set to not meet the information display conditions.

[0123] Among them, the display status information is controlled by the priority of the original display information obtained, so as to realize the acquisition control of the information to be displayed, that is, when the priority of the original display information is higher, it is easier to trigger the acquisition and subsequent display of the information to be displayed, and when the priority of the original display information is lower, it is not easy to trigger the acquisition and subsequent display of the information to be displayed.

[0124] Among them, in the embodiment of the present application, the display content control information is the condition or basis for selecting the information to be displayed from the original display information, for example, the display content control information can be priority, time sequence, content volume or keyword, etc. For example, extract the information with higher priority, earlier time or larger content volume from the original display information as the information to be displayed, or extract keywords or key words from the original display information, and only use the extracted keywords or key words as the information to be displayed, so as to improve the display effect of important information or simplify the displayed information.

[0125] Among them, in the embodiment of the present application, the acquisition of original display information and the acquisition of display status information may not have a time sequence. For example, the original display information may be acquired first and then the display status of the extended reality device may be acquired or determined. Alternatively, the original display information may be acquired while or after acquiring or determining the display status of the extended reality device.

[0126] Optionally, in an embodiment of the present application, the information to be displayed may be collected based on the attributes of the user and the environmental scene information in which the user is currently located. That is, optionally, in some embodiments of the present application, the step of "obtaining the information to be displayed" includes:

[0127] Acquire attribute information of a target user holding the extended reality device and environmental scene information of the target user;

[0128] The information to be displayed is determined according to the environmental scene information and the attribute information.

[0129] It should be noted that attribute information refers to the personalized information of the user, including the user's physiological information, habit information, work plan information or life arrangement information, etc. Environmental scene information is information about the scene the user is currently in, and the environmental scene information is obtained based on the user's location or place. For example, if the user is in a waiting hall, the user is in a waiting scene; if the user is getting off a plane, the user is in a getting off a plane scene; if the user is sitting at home, the user is in a sitting at home scene, etc.

[0130] Among them, in the embodiment of the present application, the user's attribute information and scene information can be obtained through relevant sensors, for example, the physiological data of the user's body surface temperature can be obtained through a temperature sensor, and the physiological data of the user's blood pressure can be obtained through a blood pressure sensor. Correspondingly, the user's location and place can be obtained through a combination of a position locator (GPS) and a camera. For example, the user's geographical location can be obtained through a position locator, and the specific place where the user is currently located can be analyzed in combination with the picture taken by the camera. On the other hand, some attribute information of the user, such as work and rest habits, hobbies, and frequently visited destinations, can be automatically obtained by the device through artificial intelligence, big data, and other methods.

[0131] Among them, by combining the user's attribute information and the user's environmental scene information to obtain and generate the information to be displayed, the matching degree between the obtained information to be displayed and the user's expectations is improved, the accuracy of obtaining the information to be displayed is improved, and then the effectiveness of the extended reality device display information is improved. At the same time, the information to be displayed meets the environmental scene information and the user's attribute information, and the information to be displayed is also displayed in a form that is more scientific and reasonable for the device state, more correct, and more comfortable and natural for the user.

[0132] Among them, in the embodiments of the present application, in order to further improve the matching degree between the acquired information to be displayed and the user's expectations, the information to be displayed can also be determined by combining information of more dimensions. For example, the user's emotions, work and life plans in a short period of time, weather, temperature, and the city type of the target destination can be combined, and the power level of the device, time and other factors can also be combined to assist in collecting the information to be displayed.

[0133] Among them, in the embodiment of the present application, the reference information used to determine the display state of the extended reality device and the reference information used to determine the information to be displayed may involve the same information, or different reference information may be set correspondingly based on the requirements of display control and the requirements of display content accuracy. It is understandable that the embodiment of the present application does not exhaustively list the reference information required for display control and the reference information required for determining the display content. It should be understood that as long as the accuracy of the display control timing control can be improved, and the accuracy of the display content acquisition can be improved, it should be included in the scope of the embodiment of the present application.

[0134] Optionally, in the embodiments of the present application, in order to avoid the singleness of the display of the extended reality content, the display depth of the determined information to be displayed may be controlled based on the characteristics of the display depth of the extended reality device, so as to enhance the diversity of the extended reality content display control method. That is, optionally, in some embodiments of the present application, the step of "controlling the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed" includes:

[0135] Determine the display priority according to the attribute information and the information type information corresponding to the information to be displayed;

[0136] Determining display depth information of the information to be displayed according to the display priority;

[0137] The optical display module presents the extended reality content corresponding to the information to be displayed according to the display depth information.

[0138] The display priority is the priority of pushing or displaying the extended reality content, which reflects the importance of the information to be displayed, for example, high, medium and low. The higher the priority, the higher the importance of the information to be displayed. Correspondingly, the higher the importance of the information to be displayed, the higher the display priority.

[0139] The display depth information refers to the virtual three-dimensional display depth of the content unit. Different display depths present different distances to the user. For example, the display depth on the screen describes the virtuality and reality of the content when it is presented on the screen. For example, see Figure 3 As shown, Figure 3 It is a schematic diagram of the depth information of the extended reality device screen provided by the embodiment of the present application, wherein the screen includes three depths of foreground A, default B and background C (the depths of foreground-default-background increase in sequence). Accordingly, different display depths have different expressions of the importance of the information to be displayed when displayed. For example, the information presented at the foreground depth is easy for the user to observe. Therefore, some information with a high degree of importance can be presented for the foreground depth. Conversely, for the background depth that is not easy to observe, some information with a general degree of importance can be presented. Accordingly, when the information to be displayed is presented at the foreground depth, it indicates that the information to be displayed is more important. Conversely, when the information to be displayed is presented at the background depth, it indicates that the information to be displayed is not particularly important.

[0140] Among them, determining the display depth information of the information to be displayed on the screen based on the display priority helps to push or present the extended reality content corresponding to the information to be displayed to an appropriate display depth, and realizes the presentation of content based on the importance of the information, which in turn helps to attract different degrees of user attention, avoid user disgust caused by blind push, and improve user experience.

[0141] Among them, different display depths can attract the user's attention to different degrees, which helps to classify and present information based on display priority. Users can also view it selectively to improve the user experience.

[0142] Optionally, in an embodiment of the present application, when presenting the extended reality content corresponding to the information to be displayed, the display position information of the information to be displayed may be determined first, and the information to be displayed may be presented based on the display position information, thereby improving the accuracy of the extended reality content display control. That is, optionally, in some embodiments of the present application, the step of "presenting the extended reality content corresponding to the information to be displayed according to the display depth information through the optical display module" includes:

[0143] Acquiring posture information of the extended reality device;

[0144] Determine display position information of the information to be displayed according to the posture information and the display priority;

[0145] The optical display module presents the extended reality content corresponding to the information to be displayed according to the display depth information and the display position information.

[0146] The information to be displayed is presented based on the display position information, so that the information to be displayed can be presented at an appropriate position, thereby improving the accuracy of the display control of the extended reality content.

[0147] Among them, the display position information of the information to be displayed is determined by the posture information, so that the display position of the information to be displayed matches the user behavior reflected by the posture information, so that the display of the extended reality content meets the user's expectations. For example, the user can control the display position of the information through the posture, and when the corresponding user behavior is generated, the information content is displayed at the corresponding position. In addition, the display position is determined by the posture, so that the display position of the information to be displayed matches the posture information, which is also ergonomic, making the user more comfortable to watch the picture.

[0148] For example, please Figure 4 As shown, Figure 4This is a schematic diagram of the display area of ​​the extended reality device screen provided in an embodiment of the present application, wherein the screen includes an upper area D, a middle area E and a lower area F, and correspondingly, may also include a left area, a middle area and a right area (this area division method is not shown in the figure). Accordingly, when the user raises his head, the information to be displayed can be displayed in the upper area of ​​the screen, and when the user lowers his head, the information to be displayed can be displayed in the lower area or the middle area of ​​the screen, so that the user can quickly view the corresponding content in the corresponding area as the behavioral action changes, and the presentation position of the content changes with the behavioral action, which is also ergonomic and matches the user's visual angle or range when the behavioral action occurs, thereby improving the effectiveness of the presentation of the information to be displayed.

[0149] Optionally, in the embodiments of the present application, in order to attract the user's attention to the presented content information to different degrees, display style information of the information to be displayed may also be configured, and the display style of the information to be displayed may be controlled based on the display style information, thereby further improving the diversity of display methods and styles. That is, optionally, in some embodiments of the present application, the step of "presenting the extended reality content corresponding to the information to be displayed according to the display depth information and the display position information through the optical display module" includes:

[0150] Determining display style information of the information to be displayed according to the display priority;

[0151] The optical display module presents the extended reality content corresponding to the information to be displayed according to the display depth information, the display position information and the display style information.

[0152] Among them, the display style information corresponding to the information to be displayed is determined based on the display priority, so that the display style information and the corresponding display priority satisfy the correspondence relationship, emphasizing that the display style information corresponding to different display priorities is different. Based on this, different display style information corresponding to the information to be displayed of different importance can be selected to display the information to be displayed, thereby attracting different degrees of user attention, improving the diversity of display methods or styles when controlling the display of extended reality content, and improving user experience.

[0153] It is understandable that the display style includes fonts, lines, colors, positions and other styles. Based on different display priorities, different degrees of display can be achieved by thickening the lines, darkening the colors or highlighting the positions, thereby showing different appeals and attracting different degrees of user attention.

[0154] Correspondingly, in an embodiment of the present application, the optical display module can also display other content, such as film and television content, game scene content, or real-life environment content, etc. For example, the information to be displayed can be presented on the film and television content, so that the user can present additional information to be displayed on the basis of watching the film and television content.

[0155] Among them, in the embodiment of the present application, the screen can also be a light-transmitting screen, that is, not only can the screen project light to form extended reality content, but the real content of the real scene can also be viewed through the screen. For example, the screen can be the screen of an augmented reality (AR) device, such as the lenses of AR glasses. The screen is light-transmitting, and the user's eyes can view the object content in the real scene through the screen. Correspondingly, the screen can also project extended reality content. For example, the extended reality content can be projected on the screen using optical waveguide technology to achieve the superposition of extended reality content on the real scene content.

[0156] Correspondingly, for the information to be displayed superimposed on the real scene content, it is also possible to determine whether the display position coincides with the real scene content according to its corresponding display position information, and then highlight the information to be displayed or reduce the impact on the real scene content by adjusting the display style. That is, optionally, in some embodiments of the present application, the step of "presenting the extended reality content corresponding to the information to be displayed according to the display depth information and the display position information through the optical display module" includes:

[0157] Collecting a preview image previewed through the optical display module through the camera of the extended reality device;

[0158] Determining background content information corresponding to the information to be displayed from the preview screen according to the display position information;

[0159] Adjusting the initial display style information of the information to be displayed according to the content tone of the background content information to obtain target display style information;

[0160] The information to be displayed is presented through the optical display module according to the display depth information, the display position information and the target display style information.

[0161] Among them, the real scene content is observed by the user through the optical display module (such as the screen) of the extended reality device. Accordingly, a camera with the same viewing angle can be configured to shoot the real scene content that the user browses through the screen based on the camera, that is, the preview picture shot by the camera is consistent with the real scene content. Accordingly, it can be determined whether there is background content information that overlaps with the information to be displayed in the preview picture based on the display position information of the information to be displayed, and then the display style of the information to be displayed can be adjusted.

[0162] Optionally, in an embodiment of the present application, the initial display style information is the display style of the information to be displayed before it is adjusted according to the content tone of the background content information. The initial display style may be a default display style. For example, the default display style may be obtained based on the initialization configuration, such as a white font display style. Accordingly, the initial display style may also be a display style obtained by adjusting the default display style based on the display priority of the information to be displayed, such as adjusting the default white font to a red font display style. Accordingly, the target display style information obtained after adjusting the initial display style may include a blue font display style.

[0163] Correspondingly, for the purpose of reducing the impact on the real scene content, the display chromaticity or color depth of the information to be displayed can also be reduced, for example, the display style of blue fonts can be switched to white fonts or light fonts.

[0164] Among them, in the embodiments of the present application, the display position, display depth, and display style of the extended reality content can all be collectively referred to as the display form, wherein the display form can also be set to be dynamic. For example, for a certain information to be displayed, the display form of the extended reality content is adjusted according to the device power of the extended reality device, the device temperature, the ambient temperature, or the user's mood or the continuous display time. For example, when the power of the extended reality device is reduced to a power threshold, or the device temperature reaches a temperature threshold or the continuous display time reaches a time threshold, the font of the displayed content is reduced or the color brightness of the display is reduced, or the extended reality content is moved to other display positions (for example, to a position that is not easily noticed by the user), etc.

[0165] The following uses the travel application scenario as an example. For example, please refer to Figure 5-Figure 7 As shown, Figure 5-Figure 7 They are schematic diagrams of the extended reality content display control effects before, during and after a trip provided in the embodiments of the present application.

[0166] Among them, see Figure 5 As shown, before the trip, the user is about to embark on a journey through daily information analysis. When the user looks up, weather information will appear and related service information such as itinerary, planning, currency exchange, etc. will be displayed; correspondingly, the smart assistant in the extended reality device can provide customized reminders based on the user's itinerary, such as advance reminders of the best time to go out, boarding time, weather conditions, traffic conditions, etc.

[0167] Among them, see Figure 6As shown, when the user looks down at his feet while traveling, relevant information such as meetings, arrangements, local restaurant recommendations, local weather conditions, etc. can be automatically presented based on the user's location, behavior, and preferences.

[0168] Among them, see Figure 7 As shown, after the trip, it helps users organize the wonderful photos, videos and memos of the trip. For example, when the user returns home and sits on the sofa and looks up, in addition to seeing the weather information, the user can also see the wonderful photos and moments of the trip. Then, the user can choose to share and share the wonderful moments to the social platform with one click.

[0169] Optionally, in an embodiment of the present application, in order to improve the accuracy of the display control of the extended reality content, that is, the accuracy of the recommended information to be displayed, the user's intention can be first inferred based on the environmental scene information and the attribute information, and the recommended information to be displayed is determined based on the intention, that is, optionally, in some embodiments of the present application, the step of "determining the information to be displayed based on the environmental scene information and the attribute information" includes:

[0170] Determining the behavior intention of the target user according to the scenario information and the attribute information;

[0171] Setting the content information corresponding to the behavioral intention in the preset memory as information to be displayed;

[0172] and / or,

[0173] The information to be displayed corresponding to the behavior intention is output through the intention content relationship model.

[0174] Among them, behavioral intention refers to a certain purpose that the user is going to achieve, for example, the behavioral intention of wanting to view surrounding store information, hotel information, or wanting to share content.

[0175] Among them, by combining the environmental scene information and the user's attribute information, the accuracy of behavioral intention recognition can be improved. For example, the user's behavioral intention can be analyzed in combination with the user's behavioral actions in the attribute information, such as looking up while waiting for a bus (where the scene information is waiting for a bus and the behavioral action is looking up) / lowering the head when getting off the plane / looking up when sitting at home / turning the head when working in the company, etc.

[0176] Among them, in the embodiment of the present application, the preset memory stores various content information corresponding to the user's behavioral intention. By analyzing the user's behavioral intention, the corresponding information to be displayed can be matched from the preset memory. Among them, a knowledge graph can also be constructed between the behavioral intention and the information to be displayed. Through the correspondence between the behavioral intention and the entity in the knowledge graph, a mapping relationship between the information to be displayed and the behavioral intention corresponding to the entity is established. Among them, the correspondence between the behavioral intention and the information to be displayed can also be obtained through big data and machine learning methods.

[0177] Correspondingly, in an embodiment of the present application, the intent-content relationship model is a relationship model between intent and content. After obtaining the user's behavioral intention, the intent-content relationship model can be used to output the information to be displayed that meets the user's behavioral intention. Among them, the intent-content relationship model performs tasks such as prediction, classification, or regression by learning the characteristics in the data. The training of the relationship model is to capture the association between variables through a large amount of data. The information to be displayed corresponding to the user's behavioral intention is predicted by the intent-content relationship model. For example, predict the type of information required for the current behavioral intention, and then search for the information to be displayed from the local content based on the information type or obtain the information to be displayed from the cloud or server through the Internet, for example, obtain weather data and other information at the travel destination.

[0178] Optionally, in an embodiment of the present application, the posture information of the extended reality device can be obtained through sensors and cameras in the extended reality device. For example, by analyzing the shooting angle of the camera and the motion data of the sensor, the posture information of the current extended reality device is analyzed. That is, optionally, in some embodiments of the present application, the step of “obtaining the posture information of the extended reality device” includes:

[0179] Obtaining a preview image by photographing with a camera of the extended reality device;

[0180] Analyze the current shooting angle of the camera according to the preview image;

[0181] Obtaining a deflection angle of the extended reality device according to an angle detection device in the extended reality device;

[0182] The posture information of the extended reality device is determined according to the shooting angle and the deflection angle.

[0183] Among them, the angle detection device includes an angle detection sensor, which can be built into the extended reality device. For example, the angle detection device can be a sensor device such as a gyroscope, an accelerometer, or a magnetometer.

[0184] Among them, the posture information of the extended reality device is obtained by comprehensive analysis of the shooting angle and the deflection angle, which can improve the accuracy of the posture information. Among them, the shooting angle and the deflection angle can be used to analyze and judge the posture information of the extended reality device based on the corresponding weights respectively.

[0185] In summary, the embodiments of the present application obtain the display status information of the extended reality device, and control the acquisition of the information to be displayed and the display control of the information to be displayed through the display status indicated by the display status information, so as to realize using the display status as a condition for the extended reality device to display information, control the timing of content display of the extended reality device, avoid the uncontrolled and unconstrained random display of the content of the extended reality device, improve the effectiveness of the display of content information of the extended reality device, and improve the user experience.

[0186] By obtaining the posture information of the extended reality device and obtaining the user attribute information and scene information only after the posture information meets the preset trigger conditions, the information to be recommended and displayed is determined, and the information to be displayed is recommended, so as to control the timing of the extended reality content display control, improve the accuracy of the extended reality content display control timing, help to make recommendations when users need, improve the effectiveness of the extended reality content display control, and improve the user experience.

[0187] Among them, the information to be recommended and displayed is determined by the user's attribute information and the scene information in which the user is currently located, thereby improving the accuracy of the information to be displayed. When recommendations are made based on the information to be displayed, the accuracy and effectiveness of the extended reality content display control are improved.

[0188] Among them, by controlling the output of the information to be displayed at different display depths based on the characteristics of the screen of the extended reality device, while improving the diversity of display styles, corresponding recommendations can be made based on the importance of different information, avoiding user disgust and other emotions caused by indiscriminate extended reality content display control, which helps to improve the effectiveness and accuracy of content information recommendations.

[0189] In order to facilitate the understanding of the embodiments of the present application, the following will be specifically described by taking the extended reality device as an optical projection augmented reality (AR) device as an example. For details, please refer to Figure 8 As shown, Figure 8 : is a flowchart of the augmented reality device provided in an embodiment of the present application executing the extended reality content display control method, which specifically includes:

[0190] 201. Determine current posture information of the augmented reality device through a camera and an angle detection device of the augmented reality device;

[0191] 202. If the posture information satisfies a preset trigger condition, a current scene image is captured by a camera of the augmented reality device, and scene information of a target user wearing the augmented reality device is analyzed through the scene image;

[0192] The geographical location information can also be obtained in combination with GPS, and the scene information of the target user's current location can be accurately determined based on the geographical location information and the scene image;

[0193] 203. Obtain attribute information of the target user;

[0194] The attribute information includes physiological data and behavioral action data obtained through body indicator sensors, as well as user habit data read from the memory, and the user's recent work plan data and life arrangement data;

[0195] 204. Determine the behavior intention of the target user according to the scene information and the attribute information;

[0196] 205. Matching information to be displayed from a database according to the behavior intention or outputting information to be displayed corresponding to the behavior intention through an intention content relationship model;

[0197] 206. Determine the display priority according to the attribute information of the target user and the information type corresponding to the information to be displayed;

[0198] 207. Determine display depth information and initial display style information of the information to be displayed on the screen of the augmented reality device according to the display priority;

[0199] 208. Determine display position information of the information to be displayed on the screen of the augmented reality device according to the display priority and the attribute information of the target user;

[0200] 209. Determine, from the scene image according to the display position information, background content information that overlaps with the information to be displayed;

[0201] 210. Adjust the initial display style information of the information to be displayed according to the content tone of the background content information to obtain target display style information;

[0202] 211. Display the information to be displayed on a screen of the augmented reality device according to the display depth information, the display position information, and the target display style information.

[0203] Among them, the screen of the augmented reality device has a certain degree of light transmittance, through which the real scene content can be viewed, and the screen can also project extended reality content based on technologies such as diffraction optical waveguides.

[0204] Among them, the posture information of the augmented reality device reflects the behavioral action information of the user wearing the augmented reality device. Therefore, using the posture information as the basis for judging the timing of extended reality content display control is actually using the user's behavioral action as the timing judgment condition for extended reality content display control, so as to realize extended reality content display control after satisfying specific behavioral actions, avoid indiscriminate extended reality content display control or extended reality content display control based on geographic location, which is prone to inaccurate recommendation timing and user disgust, improve the accuracy of recommendation timing, improve the effectiveness of recommended content, and improve user experience.

[0205] By determining the recommended information to be displayed based on the scene information of the user and the user's attribute information, the accuracy of the information to be displayed is improved, the accuracy of the extended reality content display control is improved, and the user experience is improved.

[0206] Furthermore, the augmented reality device of the embodiment of the present application has the following advantages when performing display control of extended reality content, specifically:

[0207] Personalized information presentation: The system can present information based on the user's specific scenarios and intentions, enabling users to obtain information relevant to their current needs and improving the personalization and practicality of information.

[0208] Smarter information acquisition: With the help of comprehensive consideration of user behavioral characteristics, physiological information and user intentions, the system can intelligently present information, reduce user active intervention, and provide a more automated and convenient way to obtain information.

[0209] More intuitive user experience: By presenting information in the right scenario, timing, and user intent, it creates a more intuitive interaction and natural user experience, reducing the user's cognitive burden.

[0210] Improved work efficiency: For professional and industrial applications, it is expected to improve work efficiency and reduce error rates because it can intelligently provide users with relevant information.

[0211] Improve the quality of life: In daily life, it can provide users with a more convenient way to obtain information, thereby improving the quality of life and reducing the pressure on users.

[0212] In order to better implement the extended reality content display control method of the present application, the present application also provides an extended reality content display control device based on the above extended reality content display control method. The meanings of the terms are the same as those in the above extended reality content display control method, and the specific implementation details can refer to the description in the method embodiment.

[0213] See also Fig. 9 , Fig. 9: is a schematic diagram of the structure of an extended reality content display control device provided in an embodiment of the present application, wherein the extended reality content display control device is applied to an extended reality device, and the device can be specifically as follows:

[0214] A first acquisition module 301 is used to acquire display status information of the extended reality device, where the display status information indicates the display status of the extended reality device;

[0215] A second acquisition module 302, configured to acquire information to be displayed if the display status representation satisfies the information display condition;

[0216] The display module 303 is used to control the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed.

[0217] Optionally, in some embodiments of the present application, the first acquisition module 301 includes:

[0218] An information determining unit, used to determine usage scenario information of the extended reality device;

[0219] A state determination unit, configured to determine a display state of the extended reality device according to the usage scenario information and a preset trigger reference condition;

[0220] The state information generating unit is used to generate display state information according to the display state.

[0221] In some embodiments of the present application, the state determination unit includes:

[0222] a scene recognition subunit, configured to recognize a scene type corresponding to the usage scene information, and determine a target trigger reference condition from at least one preset trigger reference condition according to the scene type;

[0223] a factor determination subunit, configured to determine a display control reference factor according to the target trigger reference condition, wherein the display control reference factor includes at least one of time, location, posture, user manipulation, power, user emotion, or program running state;

[0224] The state setting subunit is used to determine the real-time reference information of the display control reference factor under the usage scenario information, and if the real-time reference information meets the target trigger reference condition, the satisfied information display condition is set as a display state.

[0225] In some embodiments of the present application, the second acquisition module 302 includes:

[0226] An environmental information acquisition unit, used to acquire attribute information of a target user holding the extended reality device and environmental scene information of the target user;

[0227] A display information determination unit is used to determine the information to be displayed according to the environmental scene information and the attribute information.

[0228] In some embodiments of the present application, the display module 303 includes:

[0229] a priority determination unit, configured to determine a display priority according to the attribute information and the information type corresponding to the information to be displayed;

[0230] a depth information determining unit, configured to determine display depth information of the information to be displayed according to the display priority;

[0231] A content display unit is used to present the extended reality content corresponding to the information to be displayed according to the display depth information through the optical display module.

[0232] Among them, in some embodiments of the present application, the content display unit includes:

[0233] A posture information acquisition subunit, used to acquire the posture information of the extended reality device;

[0234] a position information determining subunit, configured to determine display position information of the information to be displayed according to the posture information and the display priority;

[0235] The content display subunit is used to present the extended reality content corresponding to the information to be displayed according to the display depth information and the display position information through the optical display module.

[0236] In some embodiments of the present application, the content display subunit is specifically used for:

[0237] Determining display style information of the information to be displayed according to the display priority;

[0238] The optical display module presents the extended reality content corresponding to the information to be displayed according to the display depth information, the display position information and the display style information.

[0239] In some embodiments of the present application, the content display subunit is specifically used for:

[0240] Collecting a preview image previewed through the optical display module through the camera of the extended reality device;

[0241] Determining background content information corresponding to the information to be displayed from the preview screen according to the display position information;

[0242] Adjusting the initial display style information of the information to be displayed according to the content tone of the background content information to obtain target display style information;

[0243] The information to be displayed is presented through the optical display module according to the display depth information, the display position information and the target display style information.

[0244] In some embodiments of the present application, the display information determination unit includes:

[0245] An intention determination subunit, used to determine the behavior intention of the target user according to the environmental scene information and the attribute information;

[0246] A first setting subunit, used to set the content information corresponding to the behavioral intention in the preset memory as information to be displayed;

[0247] and / or,

[0248] The second setting subunit is used to output the to-be-displayed information corresponding to the behavior intention through the intention-content relationship model.

[0249] In some embodiments of the present application, the device further includes a display information acquisition module, and the display information acquisition module includes:

[0250] A display information acquisition unit, used to acquire original display information;

[0251] The second acquisition module 302 includes:

[0252] A first acquisition unit is configured to set the original display information as information to be displayed if the display state representation satisfies an information display condition; or

[0253] The second acquisition unit is configured to generate display content control information if the display state representation satisfies the information display condition, and filter the information to be displayed from the original display information according to the display content control information.

[0254] In the embodiment of the present application, the first acquisition module 301 first acquires the display status information of the extended reality device, and the display status information indicates the display status of the extended reality device. Then, the second acquisition module 302 acquires the information to be displayed if the display status representation satisfies the information display condition. Then, the display module 303 controls the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed.

[0255] Among them, the embodiment of the present application obtains the display status information of the extended reality device, and controls the acquisition of the information to be displayed and the display of the information to be displayed through the display status indicated by the display status information, so as to realize using the display status as a condition for the extended reality device to display information, control the timing of content display of the extended reality device, avoid the uncontrolled and unconstrained random display of the content of the extended reality device, improve the effectiveness of the display of content information of the extended reality device, and improve the user experience.

[0256] In addition, the present application also provides an electronic device, such as Fig.10 As shown, it shows a schematic diagram of the structure of the electronic device involved in this application, specifically:

[0257] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Fig.10 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0258] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0259] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0260] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0261] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0262] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby implementing the steps in any of the extended reality content display control methods provided in the embodiments of the present application.

[0263] The embodiment of the present application obtains display status information of an extended reality device, where the display status information indicates the display status of the extended reality device. If the display status representation satisfies an information display condition, information to be displayed is obtained, and an optical display module of the extended reality device is controlled to display extended reality content corresponding to the information to be displayed.

[0264] Among them, the embodiment of the present application obtains the display status information of the extended reality device, and controls the acquisition of the information to be displayed and the display of the information to be displayed through the display status indicated by the display status information, so as to realize using the display status as a condition for the extended reality device to display information, control the timing of content display of the extended reality device, avoid the uncontrolled and unconstrained random display of the content of the extended reality device, improve the effectiveness of the display of content information of the extended reality device, and improve the user experience.

[0265] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0266] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0267] To this end, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any of the extended reality content display control methods provided in the present application.

[0268] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0269] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0270] Since the instructions stored in the computer-readable storage medium can execute the steps in any of the extended reality content display control methods provided in the present application, the beneficial effects that can be achieved by any of the extended reality content display control methods provided in the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0271] The above is a detailed introduction to an extended reality content display control method, device, electronic device and computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0272] It should be noted that in the specific implementation of the present application, the real state information, posture information, captured scene images, user attribute information, user scene information, user behavior intention information, power, usage time, user emotions and other related data of the extended device are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A method for controlling display of extended reality content, characterized in that: Applied to an extended reality device, the method comprises: Acquire display status information of the extended reality device, where the display status information indicates a display status of the extended reality device; If the display status representation satisfies the information display condition, obtaining the information to be displayed; Controlling the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed; The controlling the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed includes: Acquire attribute information of a target user holding the extended reality device, and acquire posture information of the extended reality device; Determining a display priority according to the attribute information and the information type corresponding to the information to be displayed; Determining display depth information of the information to be displayed according to the display priority, and determining display position information of the information to be displayed according to the posture information and the display priority; The preview screen previewed through the optical display module is collected by the camera of the extended reality device, and the background content information corresponding to the information to be displayed is determined from the preview screen according to the display position information, and the initial display style information of the information to be displayed is adjusted according to the content tone of the background content information to obtain the target display style information; The information to be displayed is presented through the optical display module according to the display depth information, the display position information and the target display style information.

2. The method for controlling the display of extended reality content according to claim 1, characterized in that: The obtaining the display status information of the extended reality device includes: Determining usage scenario information of the extended reality device; Determining a display state of the extended reality device according to the usage scenario information and a preset trigger reference condition; Display state information is generated according to the display state.

3. The method for controlling the display of extended reality content according to claim 2, characterized in that: The determining the display state of the extended reality device according to the usage scenario information and the preset trigger reference condition includes: Identifying a scenario type corresponding to the usage scenario information, and determining a target trigger reference condition from at least one preset trigger reference condition according to the scenario type; Determine a display control reference factor according to the target trigger reference condition, wherein the display control reference factor includes at least one of time, location, posture, user control, power, user emotion or program running status; Determine the real-time reference information of the display control reference factor under the usage scenario information, and if the real-time reference information meets the target trigger reference condition, set the satisfied information display condition as a display state.

4. The method for controlling the display of extended reality content according to claim 1, wherein: The obtaining of information to be displayed includes: Acquire attribute information of a target user holding the extended reality device and environmental scene information of the target user; The information to be displayed is determined according to the environmental scene information and the attribute information.

5. The method for controlling the display of extended reality content according to claim 1, characterized in that: The controlling the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed also includes: Determining display style information of the information to be displayed according to the display priority; The optical display module presents the extended reality content corresponding to the information to be displayed according to the display depth information, the display position information and the display style information.

6. The method for controlling the display of extended reality content according to claim 4, characterized in that: The determining the information to be displayed according to the environmental scene information and the attribute information includes: Determining the behavior intention of the target user according to the environmental scene information and the attribute information; Setting the content information corresponding to the behavioral intention in the preset memory as information to be displayed; and / or, The information to be displayed corresponding to the behavior intention is output through the intention content relationship model.

7. The method for controlling the display of extended reality content according to claim 1, characterized in that: Before obtaining the display status information of the extended reality device, the method further includes: Get the original display information; If the display state representation satisfies the information display condition, obtaining the information to be displayed includes: If the display status representation satisfies the information display condition, setting the original display information as the information to be displayed; Alternatively, if the display state representation satisfies the information display condition, display content control information is generated, and the information to be displayed is filtered from the original display information according to the display content control information.

8. An extended reality content display control device, characterized in that: Applied to an extended reality device, the device comprises: A first acquisition module, configured to acquire display status information of the extended reality device, wherein the display status information indicates a display status of the extended reality device; A second acquisition module, configured to acquire information to be displayed if the display status representation satisfies an information display condition; A display module, used to control the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed; The controlling the optical display module of the extended reality device to display the extended reality content corresponding to the information to be displayed includes: Acquire attribute information of a target user holding the extended reality device, and acquire posture information of the extended reality device; Determining a display priority according to the attribute information and the information type corresponding to the information to be displayed; Determining display depth information of the information to be displayed according to the display priority, and determining display position information of the information to be displayed according to the posture information and the display priority; The preview screen previewed through the optical display module is collected by the camera of the extended reality device, and the background content information corresponding to the information to be displayed is determined from the preview screen according to the display position information, and the initial display style information of the information to be displayed is adjusted according to the content tone of the background content information to obtain the target display style information; The information to be displayed is presented through the optical display module according to the display depth information, the display position information and the target display style information.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the extended reality content display control method as described in any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps in the extended reality content display control method according to any one of claims 1 to 7 are implemented.

Images