Augmented reality equipment, information display method and display device thereof and storage medium

Abstract

The invention discloses an augmented reality device, an information display method and device thereof and a storage medium, and relates to the technical field of augmented reality device.The information display method of the augmented reality device comprises the steps that trigger information and multiple trigger conditions are obtained, and at least one of the multiple trigger conditions is a multi-dimensional trigger condition; when the target triggering condition matched with the triggering information exists in the multiple triggering conditions, the space capsule corresponding to the target triggering condition is displayed, according to the one or more technical schemes, at least one of the multiple triggering conditions is configured to be the multi-dimensional triggering condition, and therefore the space capsule can be displayed at the same anchor point, and the space capsule can be displayed at the same time. According to the technical scheme, different space capsules are introduced, for example, other conditions at the same time and at the same place are different, different contents are finally displayed, and compared with an existing scheme that a single anchor point is matched with a single content, the personalized requirement of a user can be better met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of extended reality devices, and particularly to an extended reality device, an information display method thereof, a display device, and a storage medium. BACKGROUND

[0002] Extended reality (XR) is a comprehensive term that covers virtual reality (VR), augmented reality (AR), and mixed reality (MR). Extended reality technology expands human perception and interaction, enabling interaction between digital technology and the real world, and creating virtual, augmented, and mixed reality experiences. Extended reality provides a more intuitive and convenient content presentation experience through virtual reality scenes, and has rapidly developed in the fields of entertainment, gaming, education, healthcare, enterprise production, and social communication, demonstrating great potential and influence.

[0003] With the development of extended reality (XR, including AR / VR / MR) technology, spatial information presentation and interaction have gradually become a key direction of intelligent human-machine interfaces. Existing XR applications are mostly based on user spatial position or gaze point to trigger information, such as popping up corresponding content when the user approaches a certain location. Although this mechanism is intuitive, it also has obvious limitations, as most systems only rely on spatial coordinates to determine whether the user has entered a certain area, resulting in single content presentation that cannot meet the user's individual needs. SUMMARY

[0004] The main purpose of the present application is to provide an extended reality device, an information display method thereof, a display device, and a storage medium, aiming to provide a flexible scheme for displaying spatial information.

[0005] To achieve the above-mentioned purpose, the present application provides an information display method of an extended reality device, which comprises: obtaining trigger information and a plurality of trigger conditions, at least one of the plurality of trigger conditions being a multi-dimensional trigger condition; when there is a target trigger condition in the plurality of trigger conditions that matches the trigger information, displaying a spatial capsule corresponding to the target trigger condition.

[0006] In an embodiment, the trigger information includes one or more of user information, scene information, and device information. The user information includes one or more of user action information, user emotion information, user physical sign information, user portrait information, and user permission information; and / or The scene information includes one or more of spatial position information, scene time information, scene environment information, and scene personnel information; and / or The device information includes one or more of state information of the device, mode information of the device.

[0007] In an embodiment, the multi-dimensional trigger condition includes a combination of scene time information, user action information, and user emotion information. The step of displaying the spatial capsule corresponding to the target trigger condition when the target trigger condition in the plurality of trigger conditions matches the trigger information includes: When the scene time information, user action information, and user emotion information contained in the trigger information all match the scene time information, user action information, and user emotion information of the multi-dimensional trigger condition, the spatial capsule corresponding to the multi-dimensional trigger condition is displayed.

[0008] In an embodiment, the step of displaying the spatial capsule corresponding to the multi-dimensional trigger condition when the scene time information, user action information, and user emotion information contained in the trigger information all match the scene time information, user action information, and user emotion information of the multi-dimensional trigger condition includes: When the time information contained in the trigger information is evening, the user action information is close to the refrigerator, and the emotion information is low emotion, the corresponding voice information is played; and / or, When the time information contained in the trigger information is morning, the user action information is close to the refrigerator, and the emotion information is normal emotion, a preset shopping interaction window is displayed.

[0009] In an embodiment, the step of displaying the spatial capsule corresponding to the target trigger condition includes: When the target trigger condition includes target object information, the user's viewing screen is obtained; The spatial capsule is displayed at the target object in the user's viewing screen.

[0010] In an embodiment, the trigger information includes a current context feature set, and the steps of obtaining the trigger information and the plurality of trigger conditions include: Real-time state data of the user is collected; The current context feature set is determined according to the real-time state data of the user.

[0011] In an embodiment, after the steps of obtaining the trigger information and the plurality of trigger conditions, the information display method includes: When the current context feature set meets a preset condition and the trigger information does not match one of the plurality of trigger conditions, a spatial capsule matching the current context feature set is automatically generated by AI, and the spatial capsule is displayed.

[0012] In an embodiment, the information display method further includes: Based on the trigger condition selected by the user and the content of the spatial capsule, the trigger condition and the content of the spatial capsule are correspondingly stored.

[0013] In an embodiment, the information display method further comprises: After receiving the spatial capsule construction instruction, a trigger condition prompt information is correspondingly output based on the historical behavior of the user; and / or, A prompt information of the construction of the spatial capsule is correspondingly output based on the historical behavior of the user; and / or, The historical behavior of the user is analyzed by an AI model, and a trigger condition and a spatial capsule prompt information are correspondingly output.

[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a display device, which comprises: The acquisition module is configured to acquire trigger information and a plurality of trigger conditions, at least one of the plurality of trigger conditions being a multi-dimensional trigger condition. The display module is configured to display a target trigger condition corresponding to a spatial capsule when there is a target trigger condition in the plurality of trigger conditions that matches the trigger information.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides an extended reality device, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the information display method of the extended reality device as described above.

[0016] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, the computer program being executed by a processor to implement the steps of the information display method of the extended reality device as described above.

[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which comprises a computer program, the computer program being executed by a processor to implement the steps of the information display method of the extended reality device as described above.

[0018] The one or more technical solutions proposed by the present application have at least the following technical effects: By configuring at least one of the plurality of trigger conditions as a multi-dimensional trigger condition, different spatial capsules can be directed to different contents at the same anchor point, such as the same time, the same place, and other conditions, ultimately exhibiting different contents. Compared with the current single anchor point matching single content solution, it can better meet the personalized needs of users. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate preferred embodiments of the present application and, together with the description, serve to explain the principles of the application.

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the field, based on these drawings, other drawings can also be obtained without creative labor.

[0021] Figure 1 The flowchart provided by the information display method of the extended reality device in Embodiment One of the present application; Figure 2 The flowchart provided by the information display method of the extended reality device in Embodiment Two of the present application; Figure 3 The flowchart provided by the information display method of the extended reality device in Embodiment Three of the present application; Figure 4 The module structure diagram of the display device in the embodiments of the present application; Figure 5 The device structure diagram of the hardware running environment involved in the information display method of the extended reality device in the embodiments of the present application; Figure 6 The display effect diagram of the first embodiment of the space capsule in the present application; Figure 7 The display effect diagram of the second embodiment of the space capsule in the present application.

[0022] The purpose implementation, functional features and advantages of the present application will be further explained with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0023] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0024] In order to better understand the technical solutions of the present application, the following will be described in detail with reference to the drawings and specific embodiments in the specification.

[0025] The main solution of the embodiments of the present application is to provide an information display method of an extended reality device, the information display method comprises obtaining trigger information and a plurality of trigger conditions, at least one of the plurality of trigger conditions is a multi-dimensional trigger condition; when there is a target trigger condition in the plurality of trigger conditions that matches the trigger information, a space capsule corresponding to the target trigger condition is displayed.

[0026] In the present embodiment, for the convenience of description, the following is described with the identification of the extended reality device as the execution subject.

[0027] Extended Reality (XR) is an integrated term that covers Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR). Extended Reality technology expands human perception and interaction, interacting with the real world through digital technology, creating virtual, augmented, and mixed reality experiences. Extended Reality provides a more intuitive and convenient content presentation experience through virtual reality scenarios, and has rapidly developed in the fields of entertainment, gaming, education, medical treatment, enterprise production and social communication, showing great potential and influence.

[0028] With the development of Extended Reality (XR, including AR / VR / MR) technology, spatial information presentation and interaction have gradually become the key direction of intelligent human-computer interface. Most existing XR applications are based on the spatial position or gaze point of the user to trigger information, such as popping up corresponding content when the user approaches a certain location. Although this mechanism is intuitive, it also has obvious limitations, and most systems only rely on spatial coordinates to determine whether the user has entered a certain area, resulting in single content presentation that cannot meet the user's individual needs.

[0029] The present application provides a solution by configuring at least one of the multiple trigger conditions as a multi-dimensional trigger condition, so that different spatial capsules can be directed to different content at the same anchor point, such as the same time, the same location, and other conditions, ultimately presenting different content. Compared with the current single-anchor matching single-content solution, it can better meet the user's individual needs.

[0030] It should be noted that the execution subject of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, an extended reality device, etc. that can realize the above functions. The following will take the extended reality device as an example to describe the present embodiment and the following embodiments.

[0031] Based on this, the present application embodiment provides an information display method of an extended reality device, referring to Figure 1 , Figure 1 is a flowchart of the first embodiment of the information display method of the extended reality device of the present application.

[0032] In the present embodiment, the information display method of the extended reality device comprises steps S10-S20: Step S10, obtain trigger information and multiple trigger conditions, at least one of the multiple trigger conditions is a multi-dimensional trigger condition; Specifically, the acquisition manner of the trigger information is not limited, which can be detecting the user and the surrounding environment information through a sensor, can be acquired from the cloud, can be acquired from the local, etc., which is not limited herein, and the trigger condition can be acquired from the cloud, can be acquired from the local, etc., which is not limited herein.

[0033] The trigger condition can be a single-dimensional trigger condition, for example, triggering a corresponding spatial capsule after the user reaches a certain position, or triggering a corresponding spatial capsule as long as the user gazes at a certain object, for example, gazing at an air conditioner to trigger a spatial capsule for operating the air conditioner, which can be a multi-dimensional trigger condition, in which case at least two conditions need to be met to trigger a corresponding capsule, for example, in the morning and gazing at a refrigerator to trigger a first capsule, in the afternoon and gazing at the refrigerator to trigger a second capsule, the display contents of the corresponding first capsule and second capsule can be consistent or inconsistent, and of course, there can be three conditions, four conditions, etc., which is not limited herein.

[0034] The plurality of trigger conditions each correspond to a spatial capsule, and when the trigger condition is met, the content of the spatial capsule can be correspondingly displayed, and the plurality of trigger conditions can be a part of single-dimensional trigger conditions, a part of multi-dimensional trigger conditions, or all multi-dimensional trigger conditions, etc., which is not limited herein.

[0035] Step S20: When there is a target trigger condition in the plurality of trigger conditions that matches the trigger information, displaying a spatial capsule corresponding to the target trigger condition.

[0036] Specifically, the trigger information can be compared with the plurality of trigger conditions one by one until a matching trigger condition is found, and then a corresponding spatial capsule is displayed.

[0037] The specific form of the spatial capsule is not limited, which can be an information board, an interactive window, for example, a window for the user to select whether to enter a certain mode, or a control panel for a certain device, and of course, the spatial capsule can also be voice information, etc., which is not limited herein. In some embodiments, the capsule is presented in the XR interface in the form of suspension, animation, superposition, etc. after being activated, and supports voice instructions, clicking, sliding, gestures, etc. interaction modes, and also supports editing, hiding, undoing, sharing, etc. operations.

[0038] Corresponding to the scheme in the embodiment of the application, by configuring at least one of the plurality of trigger conditions as a multi-dimensional trigger condition, different spatial capsules can be guided to different spaces and different content can be finally presented under the condition of the same anchor point, such as the same time, the same place, and other conditions, which can better meet the personalized needs of users compared to the current scheme of matching single content with single anchor point.

[0039] The specific implementation form of the trigger information is not limited, which can be one or more of user information, scene information, and device information, and can also be other information, which is not limited here.

[0040] In addition, it needs to be emphasized that the trigger of a certain spatial capsule can also be used as the trigger condition of the next spatial capsule, so that multiple capsules can be triggered in series according to the conditions, and the task flow or plot clue can be constructed.

[0041] Specifically, the user information can include one or more of user action information, user emotion information, user physical sign information, user portrait information, and user permission information.

[0042] More specifically, the specific implementation form of the user action information is not limited, which can be user gaze, voice command, gesture, click, stay, path, etc., which is not limited here.

[0043] For example, when the user gazes at the air conditioner for more than 5 seconds, the remote control panel of the air conditioner can be displayed, or a dialog box can appear to ask whether to turn on the air conditioner, and of course, other options are also available, such as staying for 30 seconds, saying "remind me", clicking, etc. Each action is matched with a corresponding spatial capsule to present different content, thereby realizing functions such as behavior triggering and intent confirmation. For example, when the user looks at the lamp and clicks the lamp, the content of the corresponding spatial capsule is whether to turn on / off the lamp. After receiving the user's response, the extended reality device can communicate with the housekeeper to control the corresponding lamp.

[0044] More specifically, the specific form of the user emotion information is not limited, which can be emotional anxiety, increased heart rate, excited voice, low mood, normal mood, etc., which is not limited here. Specifically, the user's emotion information can be determined by detecting the user's facial expression, voice tone, heart rate, and skin conductance.

[0045] For example, when the user's emotion is detected to be a preset emotion, such as low mood or emotional anxiety, the content of the spatial capsule is related to emotional comfort and pacification guidance, such as voice playing "You have been great today, everything will pass", etc., which is not limited here.

[0046] More specifically, the specific form of the user's vital information is not limited, which can be the user's heart rate information, blood pressure information, user's action amplitude, etc., which is not limited here. Specifically, at this time, the user's vital detection and health management function can be realized through the space capsule.

[0047] For example, when the user's heart rate is too high or too low, that is, the trigger information is that the user's heart rate does not fall within the preset range. At this time, the space capsule can correspond to an inquiry whether the user needs an emergency call. The inquiry can be in the form of voice or in the form of a dialog box. The emergency call can be to the user's emergency contact or directly to the hospital or emergency center, thereby realizing the function of user vital detection and health management.

[0048] More specifically, the specific form of the user's permission information is not limited, which can be different user permissions corresponding to different capsule trigger conditions, such as setting a private capsule only for some high-privilege users, temporary visibility, visitor prohibition, etc., which is not limited here.

[0049] Specifically, the trigger information is the user's permission information, for example, the user is the highest privilege user, at this time, the private capsule can be triggered. When the user is a low-privilege user or a visitor, only some low-privilege capsules can be triggered.

[0050] Specifically, the scene information includes one or more of spatial position information, scene time information, scene environment information, and scene personnel information.

[0051] More specifically, the specific form of the spatial position information is not limited, which can be three-dimensional coordinates, gaze objects, and close objects, etc., which is not limited here. Correspondingly, the content of the space capsule can be freely arranged by the user or generated according to actual needs. Correspondingly, the trigger information can be gazing at the desk, approaching the kitchen door, standing in front of the projection, etc., which is not limited here.

[0052] For example, when the user is in the kitchen and gazes at the food for more than a preset time, for example, 5 seconds, the space capsule is triggered to push the recipe related to the food.

[0053] When the user is in the bedroom and has been on the bed for more than a preset time, the space capsule is triggered to push the user's recent sleep analysis report.

[0054] More specifically, the specific form of the scene time information is not limited, which can be a certain period, such as morning, afternoon, evening, night, can also be date information, can also be the day of the week, whether it is a weekday, etc., which is not limited here, and the trigger information and / or the trigger condition can be Monday morning, 10 o'clock in the evening, holidays, etc., and the space capsule can realize the functions of timing reminder and period prompt.

[0055] For example, the trigger condition can remind the user to order goods in the morning every Monday, or remind the user to purchase the food materials of the week after the user gazes at the refrigerator for more than 5 seconds in the morning of Wednesday.

[0056] More specifically, the specific form of the scene environment information is not limited, which can be light intensity, noise level, temperature and humidity, space opening degree, such as dim light, high noise, and overheated temperature, etc., which is not limited here, and the space capsule can realize the functions of space adaptation reminder and environment responsive capsule.

[0057] For example, when the trigger information is the temperature and humidity in the room, and the temperature and humidity are below the threshold, the space capsule is the control panel of the air conditioner, thereby intelligently realizing the function of adjusting the temperature and humidity in the room.

[0058] More specifically, the specific form of the scene personnel information is not limited, which can be whether there is someone in the scene, user social label, team role, such as having a visitor in the scene, being a team leader, and a family shared capsule, etc., at this time, the space capsule can realize the functions of controlling content sharing and private capsule visibility.

[0059] For example, only when there is no one in the scene, the display content of part of the private space capsule is triggered, and the trigger information is that there is no other person in the scene.

[0060] Specifically, the device information includes one or more of the state information of the device and the mode information of the device.

[0061] More specifically, the specific form of the state information of the device is not limited, which can be power, networking, do not disturb state, system resource status, etc., which is not limited here, and the trigger condition can be configured as power <20%, do not disturb on, network instability, etc., and at this time, the space capsule can realize the functions of delay / silence activation and anti-disturb strategy.

[0062] For example, the trigger information obtained is that the user turns on the do not disturb, and the space capsule displays the anti-disturb strategy.

[0063] More specifically, the specific form of the mode information of the device is not limited, which can be a low power mode, a flight mode, and the like, which are not limited herein.

[0064] In some embodiments, the multi-dimensional trigger condition comprises a combination of scene time information, user action information and user emotion information. The step of displaying the spatial capsule corresponding to the target trigger condition when the target trigger condition matching the trigger information exists in the plurality of trigger conditions comprises: Step A20: When the scene time information, user action information and user emotion information contained in the trigger information match the scene time information, user action information and user emotion information of the multi-dimensional trigger condition, the spatial capsule corresponding to the multi-dimensional trigger condition is displayed correspondingly.

[0065] In the scheme of the present embodiment, the multi-dimensional trigger condition comprises a combination of scene time information, user action information and user emotion information, so that different spatial capsules can be guided for different user action information and user emotion information at the same time. Compared with the single user emotion information trigger condition, it can guide different spatial capsules by combining user action information and scene time information, and can better guide the user's emotion.

[0066] Moreover, the existing system generally lacks a perception mechanism for user psychological state and semantic intention, and cannot realize emotion-driven interactive response. In the present embodiment, the perception of user psychological state and semantic intention can realize emotion-driven interactive response.

[0067] Specifically, the step of displaying the spatial capsule corresponding to the multi-dimensional trigger condition when the scene time information, user action information and user emotion information contained in the trigger information match the scene time information, user action information and user emotion information of the multi-dimensional trigger condition comprises: Step A21: When the time information contained in the trigger information is evening, the user's action information is close to the refrigerator, and the emotion information is low, the corresponding voice information is played. Specifically, please refer to Figure 6 , Figure 6 The voice information can be information for soothing the user's emotion, such as "You have been great today, go to bed early", "You have done a good job today, try harder tomorrow", and the like, which are not limited herein.

[0068] Corresponding in the scheme of the embodiment, since the contained time information is evening, the user's action information is close to the refrigerator, and at this time it is possible that the user's work for the day causes low mood, therefore, the corresponding voice information is played, and the user's work or behavior today is recognized, which can effectively alleviate the user's mood.

[0069] Step A22: when the time information contained in the trigger information is morning, the user's action information is close to the refrigerator, and the emotion information is normal emotion, a preset shopping interaction window is displayed.

[0070] Corresponding in the scheme of the embodiment, since the contained time information is morning, the user's action information is close to the refrigerator, and the user's emotion is normal emotion, it is indicated that the user may want to purchase food materials today, therefore, a preset shopping interaction window can be displayed, so as to identify the user's intention through the scene time information, the user's action information and the user's emotion information, and corresponding recommendation is performed.

[0071] In addition, please refer to Figure 7 , Figure 7 The effect picture of the second embodiment of the spatial capsule in the application can also be that at any time, as long as the user gazes at the refrigerator for more than 5 seconds, the corresponding spatial capsule is triggered to remind the user to purchase milk.

[0072] Please refer to Figure 2 , Figure 2 The flowchart of the second embodiment of the information display method of the extended reality device in the application, in some embodiments, the step of displaying the spatial capsule corresponding to the target trigger condition comprises: Step S21: when the target trigger condition includes target object information, a viewing picture of the user is acquired. The target object information refers to that the user approaches a certain object, the user gazes at a certain object, and the user performs an action on a certain object, where the certain object refers to a target object, for example, when the user gazes at the refrigerator, the target object is the refrigerator, at this time, the viewing picture of the user is acquired, which is a picture in the virtual reality of the user, or a picture viewed by the user in reality, which can be acquired by the camera of the extended reality device.

[0073] Step S22: display the spatial capsule at the target object in the viewing picture of the user.

[0074] Corresponding in the scheme of the embodiment, the spatial capsule is displayed at the target object in the viewing picture of the user, on the one hand, the user can quickly notice the spatial capsule, and on the other hand, when the spatial capsule is a false prompt, the user turns his head away from the target object, and the spatial capsule can also correspondingly stop displaying.

[0075] Please refer to Figure 3 , Figure 3 For the flowchart of the third embodiment of the information display method of the extended reality device of the present application, in some embodiments, the trigger information includes a current context feature set, and the steps of obtaining the trigger information and the plurality of trigger conditions include: Step S11: Collecting real-time state data of the user; Specifically, the specific form of the real-time state data is not limited and can include the user's location, the user's behavior, the user's expression, the user's language, etc., which are not limited herein.

[0076] Step S12: Determining a current context feature set according to the real-time state data of the user.

[0077] Specifically, the state can be determined based on rule matching or a lightweight AI model, so as to analyze the real-time state data of the user and determine the current context feature set. For example, if the user says good with a frown, the corresponding current context feature set can include the user's slight disagreement and the user's slight disagreement.

[0078] In the scheme of the present embodiment, the current context feature set is determined according to the real-time state data of the user, and the current context feature set is used as the trigger information, so that the real-time state data of the user can be understood when the user changes the expression or changes some trigger actions, and the user's intention can be understood, so as to correspond to the trigger of the corresponding spatial capsule. Compared with ordinary mechanical matching, the introduction of the context module can more accurately analyze the user's intention and guide the user to the desired spatial capsule.

[0079] In some embodiments, after the step of obtaining the trigger information and the plurality of trigger conditions, the information display method includes: Step S30: When the current context feature set meets a preset condition and the trigger information does not match one of the plurality of trigger conditions, automatically generating a spatial capsule matched with the current context feature set by AI and displaying the spatial capsule.

[0080] It should be noted that the trigger information can only include the current context feature set, or can also include other information such as current scene information and current device information, etc., which are not limited herein.

[0081] The preset condition can be a condition preset at the factory or a condition preset by other personnel, which can be a condition representing that the current state of the user is not good, such as a current context feature set representing that the current user's mood is not good, the current user's mood is low, the current user's mood is depressed, etc., a current context feature set representing that the current user's physiological state is not good, the user speaks without strength, etc. Of course, it can also represent some special conditions.

[0082] At this time, the trigger information does not match one of the plurality of trigger conditions, that is, the spatial capsule corresponding to the current context feature set is not stored, but the user needs some emotional guidance or behavioral help at this time, therefore, the spatial capsule matching the current context feature set is automatically generated by AI, and the spatial capsule is displayed, so that the user's needs can be understood by AI, and the corresponding spatial capsule is given, thereby improving the intelligence of the spatial capsule generation, and combining the spatial capsule with the AI model, the user's needs can be better met.

[0083] More specifically, the current context feature set or trigger information can be input into AI as a whole, so that the AI automatically generates a spatial capsule matching the current context feature set, and the AI can be an AI built-in the extended reality device, or an external AI used through networking, etc., which is not limited here.

[0084] Of course, in some embodiments, when the current context feature set does not meet the preset condition, and the trigger information does not match one of the plurality of trigger conditions, the spatial capsule is not displayed.

[0085] The specific form of the spatial capsule is not limited, which can be constructed by the user himself, or the system can provide common scene templates (such as meeting reminders, kitchen memos, reading tasks, etc.), the user only needs to select one key, and the capsule can be quickly deployed, the system can also actively recommend, arrange or even pre-deploy the capsule for the user through intelligent analysis of behavior trajectory, space usage frequency, emotional trend, etc., the user can also trigger high-frequency behaviors or semantic keywords in a certain space / behavior state, and the system can pop up a real-time prompt option of "whether to create a capsule", the user can also copy the existing capsule to other spaces or scenes, modify and save it as a new capsule after quick modification, realize "space content migration", etc., which is not limited here.

[0086] In some embodiments, the information display method further comprises: Based on the trigger condition selected by the user and the content of the spatial capsule, the trigger condition and the content of the spatial capsule are stored correspondingly.

[0087] In the scheme of the present embodiment, based on the trigger condition selected by the user and the content of the spatial capsule, the trigger condition and the content of the spatial capsule are stored correspondingly, that is, the user can select the trigger condition and the content of the spatial capsule, and the corresponding spatial capsule is generated according to the user's needs, which can better meet the actual needs of the user.

[0088] Specifically, in actual application, some spatial capsules can be set for the owner of the extended reality device by family members, for example, the trigger condition can be a certain anniversary plus a user for a specific person, for example, the owner of the extended reality device, plus a specific emotion, for example, low emotion, at this time, the content of the spatial capsule can be set as a pre-set gift clue, for example, where to look and where to go, so as to better meet the needs of some users who want to surprise their families.

[0089] In some embodiments, the information display method further comprises: Step S81: After receiving the spatial capsule construction instruction, output a trigger condition prompt information based on the historical behavior of the user.

[0090] Specifically, the historical behavior of the user can be analyzed, and high-frequency historical behaviors, such as behaviors occurring more than three times in the past week, can be screened out. The trigger condition prompt information corresponding to these behaviors can be output, so as to actively and intelligently mine implicit needs, save thinking / operation burden, and improve the probability of triggering the spatial capsule.

[0091] For example, the user of the extended reality device is a child, and the parents can create a spatial capsule accordingly. When creating, the system outputs a trigger condition prompt information based on the historical behavior of the user, that is, the system outputs the behavior of the child in the recent period of time. The parents can set the content of the spatial capsule according to the behavior of the child, so as to guide the behavior of the child.

[0092] Step S82: Output a prompt information for constructing a spatial capsule based on the historical behavior of the user.

[0093] Specifically, when detecting that the user performs a certain behavior multiple times, for example, the user triggers a high-frequency behavior or a semantic keyword in a certain space / behavior state, the system pops up a real-time prompt option of "whether to create a capsule". Specifically, the user can say "don't forget" multiple times in the kitchen, and the system prompts "whether to record as a reminder capsule?". After the user confirms, the configuration is completed, so as to smoothly embed the user behavior path with high responsiveness without interrupting the scene cognition flow.

[0094] Step S83: Analyze the historical behavior of the user through an AI model, and output a trigger condition and a spatial capsule prompt information.

[0095] Specifically, the system can actively recommend, arrange or even pre-deploy capsules for the user through intelligent analysis of behavior trajectory, space usage frequency, emotion trend, etc. For example, the system detects that the user has been in a low mood on the balcony multiple times, and actively suggests creating an "emotion recording capsule". After the user confirms, it is enabled, which can actively and intelligently mine implicit needs and save thinking / operation burden.

[0096] In addition, the system can also provide a preset scene template to build a space capsule, for example, the system provides common scene templates (such as meeting reminders, kitchen memos, reading tasks, etc.), and the user only needs to select one key to quickly deploy the capsule. Specifically, the user can click "add: evening before bed template", automatically anchor the bedside, bind the evening time, and generate a voice reminder. In this way, the new user can be quickly deployed to reduce the configuration time and standardize the experience.

[0097] The capsule can also be cloned and migrated, for example, the user can copy an existing capsule to other spaces or scenes, modify it quickly and save it as a new capsule to realize "space content migration". Specifically, the user can migrate the reading summary capsule in the study to the office area and modify it to "project document recording capsule" for quick reuse. In this way, the repeated operation cost can be reduced, the space knowledge system can be formed, and the universality can be enhanced.

[0098] It should be noted that the built space capsule can be stored in the cloud or locally, which is not limited herein. In addition, the position and state between multiple terminals can be synchronized, and user grouping and shared permission control can be supported, which is not limited herein.

[0099] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the information display method of the extended reality device of the present application. Further simple transformations based on this technical concept are within the protection scope of the present application.

[0100] The present application also provides a display device, please refer to Figure 4 The display device comprises: An acquisition module 10 is configured to acquire trigger information and a plurality of trigger conditions, at least one of the plurality of trigger conditions being a multi-dimensional trigger condition. A display module 20 is configured to display a space capsule corresponding to a target trigger condition when the target trigger condition exists in the plurality of trigger conditions and matches the trigger information.

[0101] The display device provided by the present application adopts the information display method of the extended reality device in the above embodiments, which can solve the problem of single content of existing space information display. Compared with the prior art, the display device provided by the present application has the same beneficial effects as the information display method of the extended reality device provided by the above embodiments, and other technical features in the display device are the same as the features disclosed in the above embodiments. Therefore, no further description is given here.

[0102] The application provides an extended reality device, comprising: at least one processor; and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the information display method of the extended reality device in the above-mentioned embodiment one.

[0103] Reference will now be made to the following description Figure 5 , which shows a device structure diagram of a hardware operating environment of the extended reality device suitable for implementing the embodiments of the application. The extended reality device in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (for example, vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5 The extended reality device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the application.

[0104] As Figure 5As shown, the extended reality device can include a processing device 1001 (e.g., a central processor, a graphics processor, etc.) that can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the operation of the extended reality device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the extended reality device to communicate with other devices wirelessly or by wire to exchange data. Although the extended reality device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0105] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are performed.

[0106] The extended reality device provided by the present application adopts the information display method of the extended reality device in the above-mentioned embodiments, and can solve the problem of single space information display content in the prior art. Compared with the prior art, the extended reality device provided by the present application has the same beneficial effects as the information display method of the extended reality device provided by the above-mentioned embodiments, and other technical features in the extended reality device are the same as the features disclosed in the above-mentioned embodiments, which will not be repeated here.

[0107] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the embodiments above, specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0108] The above description is only specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0109] The present application provides a storage medium, which is a computer-readable storage medium, has computer-readable program instructions (i.e. computer programs) stored thereon, and the computer-readable program instructions are used to execute the information display method of the extended reality device in the above embodiments.

[0110] The computer-readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium can be transmitted in any suitable medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0111] The above computer-readable storage medium can be included in the extended reality device; or can exist separately and not be assembled into the extended reality device.

[0112] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the extended reality device, the extended reality device is caused to: acquire trigger information and a plurality of trigger conditions, at least one of the plurality of trigger conditions being a multi-dimensional trigger condition; and when there is a target trigger condition in the plurality of trigger conditions that matches the trigger information, display a spatial capsule corresponding to the target trigger condition.

[0113] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0114] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block in the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by special-purpose hardware-based systems that perform the specified functions or operations, or combinations of special-purpose hardware and computer instructions.

[0115] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0116] The readable storage medium provided by the application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the information display method of the extended reality device, and can solve the problem of single content of existing spatial information display. Compared with the prior art, the computer readable storage medium provided by the application has the same beneficial effects as the information display method of the extended reality device provided by the above-mentioned embodiments, and will not be repeated here.

[0117] The application further provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the information display method of the extended reality device as described above.

[0118] The computer program product provided by the application can solve the problem of single content of existing spatial information display. Compared with the prior art, the computer program product provided by the application has the same beneficial effects as the information display method of the extended reality device provided by the above-mentioned embodiments, and will not be repeated here.

[0119] The above-mentioned is only part of the embodiments of the application, and does not limit the patent scope of the application, and any equivalent structural transformation, direct / indirect application in other related technical fields within the technical concept of the application, and the contents of the specification and drawings are included in the patent protection scope of the application.

Claims

1. An information presentation method of an extended reality device, characterized by, The information display method comprises: obtaining trigger information and a plurality of trigger conditions, at least one of the plurality of trigger conditions being a multi-dimensional trigger condition; when there is a target trigger condition in the plurality of trigger conditions that matches the trigger information, displaying a spatial capsule corresponding to the target trigger condition.

2. The information presentation method of an extended reality device according to claim 1, wherein, The trigger information comprises one or more of user information, scene information and device information; The user information comprises one or more of user action information, user emotion information, user physical sign information, user portrait information and user permission information; and / or, The scene information comprises one or more of spatial position information, scene time information, scene environment information and scene personnel information; and / or, The device information comprises one or more of device state information and device mode information. 3.The information presentation method of the extended reality device according to claim 1, wherein, The multi-dimensional trigger condition comprises a combination of scene time information, user action information and user emotion information; The step of displaying the spatial capsule corresponding to the target trigger condition when there is a target trigger condition in the plurality of trigger conditions that matches the trigger information comprises: when the scene time information, user action information and user emotion information contained in the trigger information all match the scene time information, user action information and user emotion information of the multi-dimensional trigger condition, the spatial capsule corresponding to the multi-dimensional trigger condition is displayed correspondingly.

4. The information presentation method of an extended reality device according to claim 3, wherein, The step of displaying the spatial capsule corresponding to the multi-dimensional trigger condition when the scene time information, user action information and user emotion information contained in the trigger information all match the scene time information, user action information and user emotion information of the multi-dimensional trigger condition comprises: when the time information contained in the trigger information is evening, the user action information is close to the refrigerator and the emotion information is low emotion, corresponding voice information is played; and / or, when the time information contained in the trigger information is morning, the user action information is close to the refrigerator and the emotion information is normal emotion, a preset shopping interaction window is displayed. 5.The information presentation method of the extended reality device according to claim 1, wherein, The step of displaying the spatial capsule corresponding to the target trigger condition comprises: when the target trigger condition comprises target object information, obtaining a viewing screen of a user; displaying the spatial capsule at the target object in the viewing screen of the user.

6. The information presentation method of an extended reality device according to claim 1, wherein, The trigger information comprises a current context feature set, and the step of obtaining the trigger information and the plurality of trigger conditions comprises: collecting real-time state data of a user; determining the current context feature set according to the real-time state data of the user. 7.The information presentation method of the extended reality device according to claim 6, wherein After the step of obtaining the trigger information and the plurality of trigger conditions, the information display method comprises: when the current context feature set meets a preset condition and the trigger information does not match one of the plurality of trigger conditions, a spatial capsule matching the current context feature set is automatically generated by AI, and the spatial capsule is displayed. 8.The information presentation method of the extended reality device according to claim 1, wherein, The information display method further comprises: storing the trigger condition and the content of the spatial capsule correspondingly based on the trigger condition and the content of the spatial capsule selected by a user. 9.The information presentation method of the extended reality device according to claim 1 or 8, wherein, The information display method further comprises: After receiving the space capsule construction instruction, a corresponding output trigger condition prompt information is output based on the user's historical behavior; and / or, A corresponding output space capsule construction prompt information is output based on the user's historical behavior; and / or, The historical behavior of the user is analyzed by an AI model, and a corresponding output trigger condition and space capsule prompt information is output.

10. A display device, characterized by The display device comprises: An acquisition module is configured to acquire trigger information and a plurality of trigger conditions, at least one of the plurality of trigger conditions being a multi-dimensional trigger condition; A display module is configured to display a target trigger condition corresponding to a space capsule when there is a target trigger condition matching the trigger information in the plurality of trigger conditions.

11. An extended reality device, comprising: The extended reality device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the information display method of the extended reality device according to any one of claims 1 to 9.

12. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the information display method of the extended reality device according to any one of claims 1 to 9.

Images