Augmented reality display control method and device, electronic equipment and storage medium

Abstract

The invention discloses an augmented reality display control method and device, electronic equipment and a storage medium, and relates to the technical field of augmented reality, and the method comprises the steps: collecting environment state information and user state information if the augmented reality equipment detects to-be-displayed information, the method comprises the steps of obtaining environment state information and user state information, determining scene state information and attention state information according to the environment state information and the user state information, determining a display strategy according to the scene state information, the attention state information and content sensitivity information corresponding to information to be displayed, and controlling display of the information to be displayed according to the display strategy to obtain an information display result. According to the invention, the information display accords with the environment, the content sensitivity and the real-time state of the user, the privacy security of the user information can be improved, the balance between the privacy security and the user demand can be realized, and the user experience can be improved.

Description

Technical Field

[0001] This application relates to the field of extended reality technology, specifically to an extended reality display control method, device, electronic device, and storage medium. Background Technology

[0002] Extended Reality (XR) is a comprehensive term encompassing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). Extended Reality technology expands human perception and interaction, allowing interaction with the real world through digital technology to create virtual, augmented, and mixed reality experiences. Extended Reality devices are widely used in navigation, information display, and augmented reality experiences, among other applications.

[0003] With the development of extended reality, the number of users and usage scenarios for extended reality devices is constantly increasing, and the demand for extended reality capabilities is constantly emerging.

[0004] However, with the increasing number of users and usage scenarios, the issue of information privacy and security is becoming increasingly serious. How to control the display of information and protect users' privacy and security is an urgent problem to be solved. Summary of the Invention

[0005] This application provides an extended reality display control method, device, electronic device, and storage medium, which can improve user information privacy and security and enhance user experience.

[0006] In a first aspect, embodiments of this application provide an extended reality display control method, applied to an extended reality device, the method comprising: If information to be displayed is detected, environmental status information and user status information are collected. Determine the context state information and attention state information based on the environmental state information and the user state information; A display strategy is determined based on the context state information, attention state information, and content sensitivity information corresponding to the information to be displayed. The display of the information to be displayed is controlled according to the display strategy to obtain the information display result.

[0007] Secondly, embodiments of this application also provide an extended reality display control device, applied to an extended reality device, the device comprising: The data acquisition module is used to collect environmental status information and user status information if information to be displayed is detected. The information determination module is used to determine the context state information and attention state information based on the environmental state information and the user state information; The strategy determination module is used to determine a display strategy based on the context state information, attention state information, and content sensitivity information corresponding to the information to be displayed. The control module is used to control the display of the information to be displayed according to the display strategy, so as to obtain the information display result.

[0008] Optionally, in some embodiments of this application, determining the display strategy based on the context state information, attention state information, and content sensitivity information corresponding to the information to be displayed includes: The privacy risk level is determined based on the contextual state information, attention state information, and content sensitivity information. The display strategy is matched according to the privacy risk level; The display strategy includes display configuration in at least one dimension of information density, transparency, layout, or visual style.

[0009] Optionally, in some embodiments of this application, the apparatus further includes: Match external visual prompting strategies based on the aforementioned privacy risk level; While controlling the display of the information to be displayed according to the display strategy, the external visual prompts are controlled according to the external visual prompt strategy.

[0010] Optionally, in some embodiments of this application, the apparatus further includes: Media collection strategies are matched based on the privacy risk level. While controlling the display of the information to be displayed according to the display strategy, the activation of the target function is controlled according to the media acquisition strategy.

[0011] Optionally, in some embodiments of this application, determining the privacy risk level based on the contextual state information, attention state information, and content sensitivity information includes: Calculate the first privacy risk score based on the scenario status information; A second privacy risk score is calculated based on the attention state information; A third privacy risk score is calculated based on the content sensitivity information. A comprehensive privacy risk score is obtained by fusing the first privacy risk score, the second privacy risk score, and the third privacy risk score according to preset weights. The privacy risk level is matched based on the comprehensive privacy risk score.

[0012] Optionally, in some embodiments of this application, the apparatus further includes: When controlling the display of the information to be displayed based on the display strategy, if a preset gesture or voice command is detected, the display strategy is adjusted according to the preset gesture or the voice command to obtain the adjusted display strategy. The display of the information to be displayed is controlled according to the adjusted display strategy.

[0013] Optionally, in some embodiments of this application, matching the privacy risk level based on the comprehensive privacy risk score includes: The privacy risk level is determined based on the comprehensive privacy risk score and the risk score threshold. The risk score threshold is updated based on the display strategy and the adjusted display strategy.

[0014] Thirdly, embodiments of this application also provide an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps in the extended reality display control method described above.

[0015] Fourthly, embodiments of this application also provide a storage medium, including a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the extended reality display control method described above.

[0016] Fifthly, embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described in embodiments of this application.

[0017] In summary, if the extended reality device of this application detects information to be displayed, it collects environmental state information and user state information, determines scene state information and attention state information based on the environmental state information and user state information, determines a display strategy based on the scene state information, attention state information and content sensitivity information corresponding to the information to be displayed, and controls the display of the information to be displayed according to the display strategy to obtain the information display result.

[0018] In this application, the embodiment determines the context state and the user's attention state based on environmental state information and user state information, and determines the display strategy of the information to be displayed by combining the content sensitivity information of the information to be displayed. This makes the display of information conform to the environment, content sensitivity and the user's real-time state, which helps to improve the privacy and security of user information, and also helps to achieve a balance between privacy and security and user needs, thereby improving the user experience. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of a scenario in which the extended reality device provided in this application executes the extended reality display control method; Figure 2 This is a flowchart illustrating the extended reality display control method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the extended reality display control device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0021] Explanation of icon numbers: 101-Extended Reality Device; 301-Acquisition Module; 302-Information Determination Module; 303-Strategy Determination Module; 304-Control Module; 401-Processor; 402-Memory; 403-Power Supply; 404-Input Unit. Detailed Implementation

[0022] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] In the description of the embodiments of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," "third," and "fourth" may explicitly or implicitly include one or more features. In the description of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0024] In this application, the term "exemplary" is used to mean "serving as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0025] This application provides an extended reality display control method, apparatus, electronic device, and computer-readable storage medium. Specifically, this application provides an extended reality display control apparatus suitable for electronic devices, which include extended reality devices, including but not limited to head-mounted displays, wearable glasses, and may be an integrated extended reality device built into a computing processing unit, or a separate extended reality device external to the computing processing unit. The extended reality devices include, but are not limited to, airborne optical display systems (i.e., head-up display systems) used in vehicles such as aircraft, automobiles, and ships, such as AR-HUD (Augmented Reality Head-Up Display) mounted on intelligent connected vehicles, extended reality applications (such as extended reality games, virtual tourism, telemedicine, or virtual experiments) used in handheld mobile devices such as mobile phones, laptops, and tablets, and near-eye display systems used in wearable devices such as head-mounted displays and smart glasses.

[0026] For example, please see Figure 1 , Figure 1 This is a schematic diagram of a scenario in which the extended reality device, according to an embodiment of this application, executes the extended reality display control method. Specifically, the execution process of the extended reality device executing the extended reality display control method is as follows: If the extended reality device 101 detects information to be displayed, it collects environmental state information and user state information, determines context state information and attention state information based on the environmental state information and user state information, determines a display strategy based on the context state information, attention state information and content sensitivity information corresponding to the information to be displayed, and controls the display of the information to be displayed according to the display strategy to obtain the information display result.

[0027] For example, users can activate an augmented reality device in different scenarios such as subways, outdoors, or offices, and display the required information through the virtual interface of the augmented reality device. At this time, environmental status information, user status information, and the sensitivity of the content to be displayed are collected to control the display, so as to achieve a balance between the privacy and security of information display and user needs.

[0028] In summary, the embodiments of this application determine the context state and the user's attention state based on environmental state information and user state information, and determine the display strategy of the information to be displayed by combining the content sensitivity information of the information to be displayed. This makes the display of information conform to the environment, content sensitivity and the user's real-time state, which helps to improve the privacy and security of user information, and also helps to achieve a balance between privacy and security and user needs, thereby improving the user experience.

[0029] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0030] Please see Figure 2 , Figure 2 This is a flowchart illustrating an extended reality display control method provided in an embodiment of this application. Although the flowchart shows a logical order, in some cases, the steps shown or described can be performed in a different order than that shown in the flowchart. Specifically, this extended reality display control method is applied to a terminal device, and the specific flow of the extended reality display control method is as follows: S201. If information to be displayed is detected, environmental status information and user status information are collected.

[0031] The information to be displayed refers to the information to be shown on the virtual display screen of the augmented reality device. It's important to note that the virtual display screen is not the physical screen of the augmented reality device. Instead, it's a screen with a certain resolution (e.g., 1280*960) that the user can see at a distance of approximately 4 meters in front of them, determined by the direction of the optical lenses (e.g., lenses). This screen with a certain resolution is the virtual display screen. For example, taking an AR device as an example, the optical engine projects an image from a microdisplay (e.g., Micro-OLED, LCoS, DLP, etc.) onto optical lenses through an optical system. When the user wears the augmented reality device, they can see a virtual image in front of them by looking at these optical lenses; this virtual image is displayed on the virtual display screen.

[0032] Among them, environmental status information refers to the surrounding environment when the extended reality device is used, including surrounding faces, distances, and the layout and position of surrounding objects captured by the external camera. It also includes audio signals (including sound source location, timbre, etc.) captured by the microphone array.

[0033] Among them, user status information refers to the user's behavioral status. For example, the user status information includes the user's gaze focus and eye behavior such as blinking, which are collected by the internal camera, or the user's head posture and body movement status, which are collected by the inertial measurement unit.

[0034] It is understandable that when information to be displayed is detected, collecting environmental and user status information helps to control the display of the information based on these information, ensuring that the information is displayed in accordance with the expected requirements.

[0035] S202. Determine the context state information and attention state information based on the environmental state information and the user state information.

[0036] Among them, the situational status information refers to the situational mode, such as being alone, standing still in public, moving in public, one-on-one conversation, and multi-person meeting.

[0037] Among them, attention state information refers to the target where the user's attention is focused. For example, by combining eye movement, face orientation and voice data, it can be determined whether the user's attention is focused on the external real-person interactive object.

[0038] Understandably, analyzing contextual and attentional state information helps to combine the actual contextual state and user attention for display control, thereby improving the rationality of display control.

[0039] S203. Determine a display strategy based on the scenario state information, the attention state information, and the content sensitivity information corresponding to the information to be displayed.

[0040] Among them, content sensitivity information refers to the degree of sensitivity of information, which is related to the type of information. For example, weather information has low sensitivity and belongs to public information; the sensitivity of the message sender is generally low and belongs to private notification information; message text, emails, and health data are high-sensitivity information; confidential documents, development documents, etc. are classified information and belong to the highest sensitivity information.

[0041] The display strategy refers to the way, specifications, or requirements for displaying information. For example, the display strategy may include full display, filtered display, hidden or semi-transparent display, or only retaining core information while hiding all notifications.

[0042] Understandably, determining display strategies by using contextual state information, attention state information, and content sensitivity information can ensure that the display strategies align with the scenario and user needs, thereby helping to achieve a balance between privacy and security and user needs, and improving the user experience.

[0043] S204. Control the display of the information to be displayed according to the display strategy to obtain the information display result.

[0044] For example, information display can be controlled according to the display strategy configuration, such as displaying it completely, partially, or completely.

[0045] In summary, the embodiments of this application determine the context state and the user's attention state based on environmental state information and user state information, and determine the display strategy of the information to be displayed by combining the content sensitivity information of the information to be displayed. This makes the display of information conform to the environment, content sensitivity and the user's real-time state, which helps to improve the privacy and security of user information, and also helps to achieve a balance between privacy and security and user needs, thereby improving the user experience.

[0046] In this application embodiment, a privacy risk level can be determined, and a display strategy for the information to be displayed can be determined based on the privacy risk level. That is, optionally, in some embodiments of this application, the step of "determining the display strategy based on the context state information, the attention state information, and the content sensitivity information corresponding to the information to be displayed" includes: The privacy risk level is determined based on the contextual state information, the attention state information, and the content sensitivity information. The display strategy is matched according to the privacy risk level; The display strategy includes display configuration in at least one dimension of information density, transparency, layout, or visual style.

[0047] The privacy risk level is categorized as low, medium, high, or very high. Different privacy risk levels correspond to different display strategies.

[0048] Information density refers to the degree to which information is displayed, including full display, filtered display, partial hiding, or complete hiding.

[0049] Transparency refers to the transparency of information display, such as making non-critical information transparent or semi-transparent. Layout refers to the spatial design of the information's display position, such as centering or displaying it at the edge.

[0050] Visual style refers to the displayed color or contrast of information, such as dark or low-contrast display. Switching between themes can be controlled.

[0051] For example, when the privacy risk level is low, the display strategy is to display all content and notifications in full; when the privacy risk level is medium, it enters "content filtering" mode, which automatically hides the text preview of messages and emails, only showing the application icon and sender, and blurring sensitive application icons; when the privacy risk level is high, it enters "minimized interface" mode, which automatically hides or semi-transparents all unnecessary UI elements, and notifications are only indicated by a flashing neutral icon (such as a dot), without carrying any content information; when the privacy risk level is high, it enters "focus" mode, which only retains the most essential information related to the current interaction (such as the other party's name tag), and automatically hides all notifications.

[0052] In this embodiment of the application, prompts can also be output externally when information is displayed. For example, the status signal corresponding to the current display strategy can be transmitted to the surrounding area, so that people around can perceive the current display situation, avoiding the user ignoring the feelings of the other party when frequently browsing the virtual display screen, or reminding people around to move away, etc. That is, optionally, in some embodiments of this application, the method further includes: Match external visual prompting strategies based on the aforementioned privacy risk level; While controlling the display of the information to be displayed according to the display strategy, the external visual prompts are controlled according to the external visual prompt strategy.

[0053] Among them, the external visual cue strategy refers to the strategy for outputting cue messages to the outside world, and it also controls what specific cue messages are output. For example, when the privacy risk level is low, the external visual cue strategy includes turning off the LED indicator or displaying a solid green light, indicating "fully open"; when the privacy risk level is medium, the external visual cue strategy includes displaying a green breathing light on the LED indicator, indicating "basic operation, information restricted"; when the privacy risk level is high, the external visual cue strategy includes switching the LED indicator to a solid yellow light, clearly indicating to the interacting party that "the device is in a caution protection mode"; when the privacy risk level is extremely high, the external visual cue strategy includes switching the LED indicator to a solid red light, strongly indicating "recording or sensitive operation in progress", etc.

[0054] In this embodiment of the application, media collection strategies can also be matched according to the privacy risk level, so that corresponding function activation control can be adapted when display control is performed based on the privacy risk level. That is, optionally, in some embodiments of this application, the method further includes: Media collection strategies are matched based on the privacy risk level. While controlling the display of the information to be displayed according to the display strategy, the activation of the target function is controlled according to the media acquisition strategy.

[0055] The media collection strategy refers to the activation strategy for the collection function. For example, when the privacy risk level is low, the media collection strategy is that all functions are available; when the privacy risk level is medium, the media collection strategy is that the function is available, but the logging is simplified; when the privacy risk level is high, the media collection strategy is to automatically pop up a prompt asking the user whether they really need to start recording; when the privacy risk level is extremely high, the media collection strategy is to forcibly disable screenshots / screen recording when browsing confidential documents and to automatically disable recording in specific legal jurisdictions, etc.

[0056] In this embodiment of the application, the privacy risk level can be determined by a privacy risk score. The privacy risk level is determined by comprehensively considering the privacy risk scores corresponding to contextual state information, attentional state information, and content sensitivity information. Optionally, in some embodiments of this application, the step "determine the privacy risk level based on the contextual state information, the attentional state information, and the content sensitivity information" includes: Calculate the first privacy risk score based on the scenario status information; A second privacy risk score is calculated based on the attention state information; A third privacy risk score is calculated based on the content sensitivity information. A comprehensive privacy risk score is obtained by fusing the first privacy risk score, the second privacy risk score, and the third privacy risk score according to preset weights. The privacy risk level is matched based on the comprehensive privacy risk score.

[0057] For example, privacy risk scores are calculated for contextual state information, attention state information, and content sensitivity information, respectively. Then, the privacy risk scores are merged to obtain a comprehensive privacy risk score. Finally, the privacy risk level corresponding to the current comprehensive privacy risk score is determined by the correspondence between the comprehensive privacy risk score and the privacy risk level.

[0058] In this embodiment, the preset weight can be predefined according to user needs or experience values, or the preset weight can be updated and optimized according to whether the user switches the display strategy after the display is controlled according to the display strategy.

[0059] In this embodiment of the application, the contextual state information, attention state information, and content sensitivity information may each include multiple situations or multiple levels, and their respective privacy risk scores can be determined according to a one-to-one correspondence.

[0060] It is understood that when display control is performed based on a display strategy, users can switch the display strategy if they are not satisfied with the display control. That is, optionally, in some embodiments of this application, the method further includes: When controlling the display of the information to be displayed based on the display strategy, if a preset gesture or voice command is detected, the display strategy is adjusted according to the preset gesture or the voice command to obtain the adjusted display strategy. The display of the information to be displayed is controlled according to the adjusted display strategy.

[0061] For example, users can temporarily override the current display strategy through preset, subtle gestures (such as eye movements following a specific pattern or tapping the temple with a knuckle) or voice commands (such as "show details"). This allows users to change system decisions automatically, helping to meet user needs and improve convenience.

[0062] In this embodiment of the application, the privacy risk level corresponding to the comprehensive privacy risk score can be determined by judging a threshold. That is, optionally, in some embodiments of this application, the step "matching the privacy risk level according to the comprehensive privacy risk score" includes: The privacy risk level is determined based on the comprehensive privacy risk score and the risk score threshold. The risk score threshold is updated based on the display strategy and the adjusted display strategy.

[0063] For example, each privacy risk level corresponds to a risk score threshold or threshold range. The privacy risk level corresponding to the comprehensive privacy risk score is determined by comparing the comprehensive privacy risk score with the risk score threshold.

[0064] In this embodiment of the application, the risk score threshold can be updated and optimized based on whether the user adjusts the display strategy. For example, if the user adjusts the display strategy, the risk score threshold is adjusted according to the adjusted display strategy to improve the matching degree between privacy protection and user needs.

[0065] In summary, the embodiments of this application determine the context state and the user's attention state based on environmental state information and user state information, and determine the display strategy of the information to be displayed by combining the content sensitivity information of the information to be displayed. This makes the display of information conform to the environment, content sensitivity and the user's real-time state, which helps to improve the privacy and security of user information, and also helps to achieve a balance between privacy and security and user needs, thereby improving the user experience.

[0066] In particular, by outputting visual cues when controlling the display according to the display strategy, it helps to establish signal communication with the surrounding people, clarify the device status, and avoid misunderstandings due to inadequate communication. For example, it alleviates the communication anxiety of "is he recording a recording or is he looking at me?" and promotes the polite use of AR devices.

[0067] This system utilizes display strategies to control display and protect information privacy. It also employs different visual cues based on varying privacy risk levels to control the data collection process to varying degrees. This approach achieves a balance between appropriate protection and preserving the core functionality of the extended reality device as much as possible. Furthermore, controlling the data collection function helps users and device manufacturers mitigate potential data privacy legal risks.

[0068] Furthermore, users can adjust the display strategy according to their actual needs, promptly meeting their real-time requirements and improving the user experience. Additionally, the risk score threshold for determining privacy risk levels is updated based on the adjusted display strategy, optimizing the risk score threshold and improving the accuracy and rationality of subsequent display strategy analysis.

[0069] To facilitate better implementation of the extended reality display control method of this application, this application also provides an extended reality display control device based on the above-described extended reality display control method. The meanings of the terms used are the same as in the extended reality display control method described above, and specific implementation details can be found in the descriptions of the method embodiments.

[0070] Please see Figure 3 , Figure 3This is a schematic diagram of the extended reality display control device provided in an embodiment of this application. The extended reality display control device is applied to an extended reality device, and the extended reality display control device can specifically be as follows: The acquisition module 301 is used to acquire environmental status information and user status information if information to be displayed is detected. Information determination module 302 is used to determine context state information and attention state information based on the environmental state information and the user state information; The strategy determination module 303 is used to determine a display strategy based on the context state information, attention state information and content sensitivity information corresponding to the information to be displayed; The control module 304 is used to control the display of the information to be displayed according to the display strategy, so as to obtain the information display result.

[0071] Optionally, in some embodiments of this application, determining the display strategy based on the context state information, attention state information, and content sensitivity information corresponding to the information to be displayed includes: The privacy risk level is determined based on the contextual state information, attention state information, and content sensitivity information. The display strategy is matched according to the privacy risk level; The display strategy includes display configuration in at least one dimension of information density, transparency, layout, or visual style.

[0072] Optionally, in some embodiments of this application, the apparatus further includes: Match external visual prompting strategies based on the aforementioned privacy risk level; While controlling the display of the information to be displayed according to the display strategy, the external visual prompts are controlled according to the external visual prompt strategy.

[0073] Optionally, in some embodiments of this application, the apparatus further includes: Media collection strategies are matched based on the privacy risk level. While controlling the display of the information to be displayed according to the display strategy, the activation of the target function is controlled according to the media acquisition strategy.

[0074] Optionally, in some embodiments of this application, determining the privacy risk level based on the contextual state information, attention state information, and content sensitivity information includes: Calculate the first privacy risk score based on the scenario status information; A second privacy risk score is calculated based on the attention state information; A third privacy risk score is calculated based on the content sensitivity information. A comprehensive privacy risk score is obtained by fusing the first privacy risk score, the second privacy risk score, and the third privacy risk score according to preset weights. The privacy risk level is matched based on the comprehensive privacy risk score.

[0075] Optionally, in some embodiments of this application, the apparatus further includes: When controlling the display of the information to be displayed based on the display strategy, if a preset gesture or voice command is detected, the display strategy is adjusted according to the preset gesture or the voice command to obtain the adjusted display strategy. The display of the information to be displayed is controlled according to the adjusted display strategy.

[0076] Optionally, in some embodiments of this application, matching the privacy risk level based on the comprehensive privacy risk score includes: The privacy risk level is determined based on the comprehensive privacy risk score and the risk score threshold. The risk score threshold is updated based on the display strategy and the adjusted display strategy.

[0077] In this embodiment, if the acquisition module 301 detects information to be displayed, it acquires environmental state information and user state information. The information determination module 302 determines contextual state information and attentional state information based on the environmental state information and the user state information. The strategy determination module 303 determines a display strategy based on the contextual state information, attentional state information, and content sensitivity information corresponding to the information to be displayed. The control module 304 controls the display of the information to be displayed according to the display strategy to obtain the information display result.

[0078] In this application, the embodiment determines the context state and the user's attention state based on environmental state information and user state information, and determines the display strategy of the information to be displayed by combining the content sensitivity information of the information to be displayed. This makes the display of information conform to the environment, content sensitivity and the user's real-time state, which helps to improve the privacy and security of user information, and also helps to achieve a balance between privacy and security and user needs, thereby improving the user experience.

[0079] In addition, this application also provides an electronic device, such as Figure 4 As shown, it illustrates a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically: The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, thereby providing overall monitoring of the electronic device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0080] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0081] The electronic device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power equipment debugging circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0082] The electronic device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0083] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application programs stored in the memory 402, thereby implementing the steps in any of the extended reality display control methods provided in the embodiments of this application.

[0084] If the extended reality device in this application detects information to be displayed, it collects environmental state information and user state information, determines scene state information and attention state information based on the environmental state information and user state information, determines a display strategy based on the scene state information, attention state information and content sensitivity information corresponding to the information to be displayed, and controls the display of the information to be displayed according to the display strategy to obtain the information display result.

[0085] In this application, the embodiment determines the context state and the user's attention state based on environmental state information and user state information, and determines the display strategy of the information to be displayed by combining the content sensitivity information of the information to be displayed. This makes the display of information conform to the environment, content sensitivity and the user's real-time state, which helps to improve the privacy and security of user information, and also helps to achieve a balance between privacy and security and user needs, thereby improving the user experience.

[0086] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0087] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0088] To this end, this application provides a computer-readable storage medium storing a computer program that can be loaded by a processor to execute the steps of any of the extended reality display control methods provided in this application.

[0089] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0090] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0091] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the extended reality display control methods provided in this application, the beneficial effects that any of the extended reality display control methods provided in this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0092] The above provides a detailed description of an extended reality display control method, apparatus, electronic device, and computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An extended reality display control method characterized by comprising: The method applied to an extended reality device comprises: If information to be displayed is detected, collecting environmental state information and user state information; According to the environmental state information and the user state information, determining scenario state information and attention state information; According to the scenario state information, the attention state information and content sensitivity information corresponding to the information to be displayed, determining a display strategy; According to the display strategy, controlling the display of the information to be displayed to obtain an information display result.

2. The extended reality display control method of claim 1, wherein, According to the scenario state information, the attention state information and the content sensitivity information, determining a privacy risk level; According to the privacy risk level, matching the display strategy; The display strategy comprises at least one dimension of display configuration in information density, transparency, layout or visual style. The method further comprises:

3. The extended reality display control method of claim 2, wherein, According to the privacy risk level, matching an external visual prompt strategy; While controlling the display of the information to be displayed according to the display strategy, controlling the external output visual prompt according to the external visual prompt strategy. The method further comprises:

4. The extended reality display control method of claim 2, wherein, According to the privacy risk level, matching a media collection strategy; While controlling the display of the information to be displayed according to the display strategy, controlling the activation of the target function according to the media collection strategy. According to the scenario state information, the attention state information and the content sensitivity information, determining a privacy risk level, comprises:

5. The extended reality display control method of claim 2, wherein, According to the scenario state information, calculating a first privacy risk score; According to the attention state information, calculating a second privacy risk score; According to the content sensitivity information, calculating a third privacy risk score; According to a preset weight, fusing the first privacy risk score, the second privacy risk score and the third privacy risk score to obtain a comprehensive privacy risk score; According to the comprehensive privacy risk score, matching the privacy risk level. The method further comprises:

6. The extended reality display control method of claim 5, wherein, While controlling the display of the information to be displayed based on the display strategy, if a preset gesture or voice instruction is detected, adjusting the display strategy according to the preset gesture or the voice instruction to obtain an adjusted display strategy; According to the adjusted display strategy, controlling the display of the information to be displayed. According to the comprehensive privacy risk score, matching the privacy risk level, comprises:

7. The extended reality display control method of claim 6, wherein, According to the comprehensive privacy risk score and a risk score threshold, determining the privacy risk level; Wherein, the risk score threshold is updated according to the display strategy and the adjusted display strategy. The device applied to an extended reality device comprises:

8. An extended reality display control device, comprising: A collection module, configured to collect environmental state information and user state information if information to be displayed is detected; An information determination module, configured to determine scenario state information and attention state information according to the environmental state information and the user state information; A strategy determination module, configured to determine a display strategy according to the scenario state information, the attention state information and content sensitivity information corresponding to the information to be displayed; A display control module, configured to control the display of the information to be displayed according to the display strategy. A control module is configured to control display of the information to be displayed according to the display strategy, and obtain an information display result.

9. An electronic device, comprising: A computer program product comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the extended reality display control method according to any one of claims 1-7 when executing the computer program.

10. A storage medium, characterized by A storage medium has a computer program stored thereon, wherein the computer program is executable by a processor to implement the steps of the extended reality display control method according to any one of claims 1-7.

Images