Application control method and device, glasses and medium

Abstract

The invention discloses an application control method and device, glasses and a medium. The method comprises the steps that semantic text instruction information of a specified application input by a user to the XR glasses is received; judging whether the instruction information is matched with a triggering condition in a rule base or not through a rule interceptor; if yes, adjusting the instruction information according to an adjusting rule in a rule base to obtain a retrieval parameter; semantic text mixed retrieval is executed based on the retrieval parameters, and an instruction response is obtained; and controlling the specified application based on the instruction response, and displaying an execution result of the specified application on the XR glasses. According to the method, the fuzzy natural language instruction is converted into the accurate retrieval parameter, and then the intention of the user is understood through the semantic text mixed retrieval technology, so that the core problem of insufficient instruction analysis adaptation capability under multiple application scenes is effectively solved, the XR glasses can quickly and accurately understand and execute the user instruction, and the user experience is improved. And the interaction response efficiency and the operation experience fluency are obviously improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of XR glasses control, and particularly relates to an application control method and device, glasses and a medium. BACKGROUND

[0002] XR (Extended Reality) refers to combining reality and virtuality through a computer to create a virtual environment that can be interacted with a computer. It is also a general term for AR, VR, MR and other technologies. By integrating the visual interaction technologies of the three, it brings the "immersion" of seamless conversion between the virtual world and the real world to the experimenter. However, the application control method of the existing XR glasses cannot effectively analyze and adapt the semantic text instructions input by the user, resulting in low instruction execution accuracy, high response delay and poor user experience in a multi-application and multi-scene environment. SUMMARY

[0003] One or more embodiments of the present specification provide an application control method, device, glasses and medium to solve the technical problems proposed in the background.

[0004] One or more embodiments of the present specification adopt the following technical solutions: An application control method provided by one or more embodiments of the present specification, the method comprises: receiving semantic text instruction information of a specified application input by a user to XR glasses; determining whether the instruction information matches a trigger condition in a rule library through a rule interceptor; If yes, adjusting the instruction information according to the adjustment rule in the rule library to obtain a search parameter; performing semantic text mixed search based on the search parameter to obtain an instruction response; controlling the specified application based on the instruction response, and displaying the execution result of the specified application on the XR glasses.

[0005] It should be noted that the present application introduces a rule interceptor and a rule library to match and intelligently adjust the user's original instruction in real time. This method converts ambiguous natural language instructions into accurate search parameters, and then understands the user's intention through semantic text mixed search technology, thereby effectively solving the core problem of insufficient instruction analysis and adaptation capability in a multi-application scenario. The XR glasses can quickly and accurately understand and execute user instructions, significantly improving the efficiency of interactive response and the smoothness of operation experience.

[0006] Further, before determining whether the search parameter matches the trigger condition in the pre-constructed rule library through the rule interceptor, the method further comprises: According to the trigger condition and adjustment rule corresponding to each instruction keyword preset, a rule library is constructed.

[0007] It should be noted that the present application provides clear and comprehensive judgment basis for the rule interceptor by pre-systematically constructing the rule library, so that the whole system has the preprocessing ability for diversified and ambiguous instructions before receiving the instruction, thereby being able to trigger the subsequent adjustment and retrieval process more quickly and accurately in actual interaction, and finally significantly improving the understanding depth of the XR glasses for complex semantic instructions and the intelligent level of the overall interaction.

[0008] Further, the adjustment rule includes increasing the weight of the instruction keyword; The adjustment rule in the rule library adjusts the instruction information to obtain a retrieval parameter, including: If the adjustment rule is to increase the weight of the instruction keyword, the weight of the instruction keyword in the instruction information is increased based on a preset weight increase value to obtain a corresponding first retrieval parameter.

[0009] It should be noted that the present application generates an optimized retrieval parameter by increasing the weight of the instruction keyword, which strengthens the capture of the user's core intention in semantic retrieval, so that the subsequent hybrid retrieval can focus more on the essential needs of the instruction, thereby effectively reducing the ambiguity caused by ambiguous terms or diverse scenes, and significantly improving the accuracy of instruction analysis and the precision of application control.

[0010] Further, the adjustment rule includes filtering a preset irrelevant application; The adjustment rule in the rule library adjusts the instruction information to obtain a retrieval parameter, including: If the adjustment rule is to filter a preset irrelevant application, a preset specified irrelevant application of the instruction keyword in the instruction information is determined; The specified irrelevant application is filtered to obtain a corresponding second retrieval parameter.

[0011] It should be noted that the present application actively filters the irrelevant application in the instruction, which effectively purifies the environmental background of the retrieval parameter, so that the semantic retrieval can exclude interference and focus on the core application related to the user's real intention, thereby significantly improving the precision and execution efficiency of instruction analysis in a complex application environment, and finally enhancing the reliability of XR glasses interaction and the smoothness of user experience.

[0012] Further, based on the instruction response, the specified application is controlled, and after the XR glasses display the execution result of the specified application, the method further includes: Based on the execution result, the execution parameter of the specified application is monitored. updating the rule base based on the execution parameters.

[0013] It should be noted that the present application monitors application parameters after execution and updates the rule base accordingly, which establishes a closed-loop feedback mechanism for learning from actual interaction results, so that the rule base can continuously optimize and adapt to the real use habits of different users and scenarios, thereby enabling the entire system to have the ability of self-evolution, continuously improving the accuracy and control efficiency of subsequent instruction analysis, and ultimately realizing the continuous intelligent improvement of the XR glasses interaction experience.

[0014] Further, the execution parameters include state information, performance indicators, and user interaction data of the specified application, the state information includes running state and error log, the performance indicators include response time and resource usage, and the user interaction data includes operation frequency, success times, and failure times. The rule base is updated based on the execution parameters, including: analyzing the execution parameters to evaluate the effectiveness of the trigger conditions and adjustment rules in the rule base in the specified application, obtaining an analysis result; Based on the analysis result, modify or add the trigger conditions and adjustment rules in the rule base. It should be noted that the present application comprehensively utilizes multi-dimensional execution parameters such as state information, performance indicators, and user interaction data for in-depth analysis and accurate updating of the rule base, which builds an adaptive system that can continuously learn from real use scenarios, so that the trigger conditions and adjustment rules of the rule base are continuously optimized to meet actual needs, thereby significantly improving the accuracy and execution efficiency of subsequent instruction analysis, and ultimately realizing dynamic optimization and continuous enhancement of the XR glasses interaction experience.

[0015] Further, the instruction information is adjusted according to the adjustment rules in the rule base to obtain a search parameter, including: adjusting the instruction information according to the adjustment rules in the rule base to obtain an adjustment parameter; obtaining specified location information and current running application information of the user; Based on the adjustment parameter, the specified location information, and the current running application information, an adjustment parameter is obtained.

[0016] It should be noted that the present application incorporates the user's real-time location information and the current running application state into the instruction adjustment process, which enables the generation of search parameters to be deeply combined with specific environmental context and task state, thereby significantly enhancing the context awareness capability of semantic understanding, ensuring that the instruction response is more in line with the user's actual intention and operation scenario, and ultimately improving the accuracy and natural fluency of the XR glasses interaction.

[0017] An application control device provided by one or more embodiments of the present specification comprises: A receiving unit receives semantic text instruction information of a specified application input by a user to an XR glasses; A judging unit judges whether the instruction information matches a trigger condition in a rule library through a rule interceptor; An adjusting unit, if yes, adjusts the instruction information according to an adjustment rule in the rule library to obtain a retrieval parameter; A retrieving unit performs a semantic text mixed retrieval based on the retrieval parameter to obtain an instruction response; A control unit controls the specified application based on the instruction response and displays an execution result of the specified application on the XR glasses.

[0018] An XR glasses provided by one or more embodiments of the present specification comprises: At least one processor and a bus; and, A memory in communication connection with the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Receive semantic text instruction information of a specified application input by a user to an XR glasses; Judge whether the instruction information matches a trigger condition in a rule library through a rule interceptor; If yes, adjust the instruction information according to an adjustment rule in the rule library to obtain a retrieval parameter; Perform a semantic text mixed retrieval based on the retrieval parameter to obtain an instruction response; Control the specified application based on the instruction response and display an execution result of the specified application on the XR glasses.

[0019] A non-volatile computer storage medium provided by one or more embodiments of the present specification stores computer executable instructions, and the computer executable instructions are executable by a computer to implement: Receive semantic text instruction information of a specified application input by a user to an XR glasses; Judge whether the instruction information matches a trigger condition in a rule library through a rule interceptor; If yes, adjust the instruction information according to an adjustment rule in the rule library to obtain a retrieval parameter; Perform a semantic text mixed retrieval based on the retrieval parameter to obtain an instruction response; Control the specified application based on the instruction response and display an execution result of the specified application on the XR glasses.

[0020] The above at least one technical solution adopted by the embodiments of the present specification can achieve the following beneficial effects: The present application introduces a rule interceptor and a rule library to perform real-time matching and intelligent adjustment on the user's original instruction. This method converts the ambiguous natural language instruction into accurate search parameters, and then understands the user's intention through the semantic text mixed search technology, thereby effectively solving the core problem of insufficient instruction analysis and adaptation capability in multiple application scenarios. The XR glasses can quickly and accurately understand and execute user instructions, significantly improving the interaction response efficiency and the smoothness of operation experience. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the drawings needed to be used in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can obtain other drawings according to these drawings without creative labor. In the drawings: Figure 1 An application environment diagram of an application control method provided for one or more embodiments of the present specification; Figure 2 A flowchart of an application control method provided for one or more embodiments of the present specification; Figure 3 A flowchart of a rule library updating method provided for one or more embodiments of the present specification; Figure 4 An identification diagram of a search intention provided for one or more embodiments of the present specification; Figure 5 A structural diagram of an application control device provided for one or more embodiments of the present specification; Figure 6 A structural diagram of an application control glasses provided for one or more embodiments of the present specification. DETAILED DESCRIPTION

[0022] The embodiments of the present specification provide an application control method, device, glasses and medium.

[0023] In order to enable a person skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be clearly and completely described below in combination with the drawings in the specification. Obviously, the described embodiments are only part of the embodiments of the specification, not all. Based on the embodiments of the specification, all other embodiments obtained by a person skilled in the art without creative labor should be within the protection scope of the specification.

[0024] The scheme of the present application can be applied in the application control scene of the application control terminal. Figure 1 An application environment diagram of an application control method provided by an embodiment of the present application is shown in Figure 1 As shown, the terminal 102 communicates with the server 103 through the network. The data storage system 101 can store data required to be processed by the server 103. The data storage system 101 can be integrated on the server 103, or placed on the cloud or other network servers. The terminal 102 can obtain historical behavior data of a user in different operation scenes and running states of an XR glasses; extract behavior common information of the user from the historical behavior data of the user in each operation scene; extract state common information of the XR glasses from the running states of the XR glasses in each operation scene; generate glasses habit states of the XR glasses in the operation scenes in combination with the behavior common information and the state common information; and associate the operation scenes with the glasses habit states to obtain behavior habit tags. Or the above process of constructing the tags is executed in the server 103, that is, the server obtains historical behavior data of a user in different operation scenes and running states of an XR glasses; extracts behavior common information of the user from the historical behavior data of the user in each operation scene; extracts state common information of the XR glasses from the running states of the XR glasses in each operation scene; generates glasses habit states of the XR glasses in the operation scenes in combination with the behavior common information and the state common information; and associates the operation scenes with the glasses habit states to obtain behavior habit tags.

[0025] The application control terminal can specifically include a smart phone, a smart home appliance, a tablet computer, a virtual reality headset (VR headset), an augmented reality glasses (AR glasses), an electronic display screen, a mixed reality (MR) device, and the like. The MR device can be MR glasses, an MR helmet, an MR camera, and the like. The vehicle-mounted system can be a vehicle-mounted chip, a vehicle-mounted device (for example, an in-vehicle infotainment, a vehicle-mounted computer, a sensor with a voice recognition function, and the like), and the like.

[0026] Figure 2A flowchart of an application control method is provided for one or more embodiments of the present specification, which can be executed by an application control system. Some input parameters or intermediate results in the flow allow manual intervention adjustment to help improve accuracy.

[0027] The method flow steps of the embodiments of the present specification are as follows: S201, receiving semantic text instruction information of a specified application input by a user to an XR glasses.

[0028] In the embodiments of the present specification, the voice instruction of the user can be received through the voice recognition module integrated by the XR glasses and converted into text information, or the text instruction is directly received through the connected external device (such as a keyboard, gesture input). The system needs to specify the current focus or the specified application in the XR environment.

[0029] S202, judging whether the instruction information matches the trigger condition in the rule library through a rule interceptor.

[0030] In the embodiments of the present specification, a rule interceptor can be constructed, which matches the received semantic text instruction information with the trigger condition in the pre-constructed rule library in real time. The trigger condition in the rule library is usually defined as a specific keyword, phrase or instruction mode. The rule interceptor judges whether the instruction text contains a key element sufficient to trigger the subsequent adjustment flow by analyzing the instruction text.

[0031] S203, if yes, adjusting the instruction information according to the adjustment rule in the rule library to obtain a retrieval parameter.

[0032] In the embodiments of the present specification, if the instruction information matches successfully, the rule interceptor calls the adjustment rule associated with the trigger condition in the rule library. The adjustment rule defines a specific instruction optimization strategy, such as increasing the weight of a specific keyword or filtering irrelevant terms. According to these rules, the original instruction information is processed, purified or enhanced, which is converted into a set of standardized and structured retrieval parameters, providing optimized input for subsequent retrieval operations. If the instruction information does not match successfully, the default retrieval parameter set in advance can be called.

[0033] S204, performing semantic text mixed retrieval based on the retrieval parameter to obtain an instruction response.

[0034] In the embodiments of the present specification, the processed retrieval parameter can be submitted to a semantic understanding and retrieval engine. The engine adopts a mixed retrieval strategy, combining keyword matching and semantic similarity calculation, to query in a pre-defined instruction-action mapping library or knowledge base. The goal is to go beyond literal matching and deeply understand the user's intention, and finally retrieve the most matched and executable operation instruction as the instruction response.

[0035] S205, control the specified application based on the instruction response, and the XR glasses display the execution result of the specified application.

[0036] In the embodiments of the present application, the retrieved instruction response can be converted into a specific application programming interface call or control command and sent to the target specified application for execution. After the application performs the corresponding operation, the output result (such as a new virtual object, a data panel, a state change, etc.) is rendered to the display screen of the XR glasses, completing the interaction loop and providing visual feedback of the execution result to the user.

[0037] It should be noted that the present application matches and intelligently adjusts the user's original instruction in real time by introducing a rule interceptor and a rule library. This method converts ambiguous natural language instructions into accurate retrieval parameters, and then understands the user's intention through semantic text hybrid retrieval technology, thereby effectively solving the core problem of insufficient instruction analysis and adaptation capability in multiple application scenarios, enabling the XR glasses to quickly and accurately understand and execute user instructions, and significantly improving the interaction response efficiency and the smoothness of the operation experience.

[0038] Further, before determining whether the retrieval parameter matches the trigger condition in the pre-constructed rule library through the rule interceptor, the rule library can be constructed according to the pre-set trigger conditions and adjustment rules corresponding to each instruction keyword.

[0039] It should be noted that regarding the construction of the rule library, first, the system designer needs to collect common user semantic text instructions in the target application scenario, and extract the key instruction keywords from them. Second, the trigger condition for each keyword or each type of keyword is defined, which should accurately describe which instruction text will hit this rule. Then, one or more specific adjustment rules are associated with each trigger condition, which define how the system should optimize or convert the original instruction when the condition is met. Finally, the corresponding relationship between the "instruction keyword-trigger condition-adjustment rule" is stored in a machine-readable structured form (such as JSON, XML or database table) for persistent storage, forming a rule library that can be efficiently queried by the rule interceptor. This rule library is the fundamental basis for the real-time judgment and adjustment of the rule interceptor.

[0040] It should be noted that the present application provides a clear and comprehensive basis for the rule interceptor by pre-systematically constructing the rule library, so that the entire system has the preprocessing capability for diversified and ambiguous instructions before receiving the instructions, thereby enabling the subsequent adjustment and retrieval process to be faster and more accurate in actual interaction, and ultimately significantly improving the understanding depth of complex semantic instructions and the intelligent level of the overall interaction of the XR glasses.

[0041] Further, the adjustment rule can include lifting the weight of the instruction keyword; when the instruction information is adjusted according to the adjustment rule in the rule library to obtain the retrieval parameter, if the adjustment rule is lifting the weight of the instruction keyword, the weight of the instruction keyword in the instruction information is lifted based on a pre-set weight lifting value to obtain a corresponding first retrieval parameter.

[0042] It should be noted that, in the rule library construction stage, for a specific instruction keyword, its associated adjustment rule is explicitly set to "lift the weight". At the same time, a specific weight lifting value (for example, the importance of a keyword is doubled) needs to be pre-set for this rule. This configuration is stored as metadata in the rule library. When the semantic text instruction information input by the user is judged by the rule interceptor to match a certain trigger condition, the rule interceptor immediately calls the "lift the weight" adjustment rule corresponding to the condition. The system will identify the target instruction keyword contained in the instruction information, and increase the weight value of the keyword according to the pre-defined weight lifting value in the rule library. The instruction information (or its structured representation) after weight lifting is encapsulated as the first retrieval parameter. This parameter not only contains the original instruction text, but more importantly, the weight attribute of the keyword inside it has been enhanced, thereby providing an optimized query basis that can highlight the user's core intent for subsequent semantic text mixed retrieval.

[0043] It should be noted that, in the rule library construction stage, for a specific instruction keyword, its associated adjustment rule is explicitly set to "lift the weight". At the same time, a specific weight lifting value (for example, the importance of a keyword is doubled) needs to be pre-set for this rule. This configuration is stored as metadata in the rule library. When the semantic text instruction information input by the user is judged by the rule interceptor to match a certain trigger condition, the rule interceptor immediately calls the "lift the weight" adjustment rule corresponding to the condition. The system will identify the target instruction keyword contained in the instruction information, and increase the weight value of the keyword according to the pre-defined weight lifting value in the rule library. The instruction information (or its structured representation) after weight lifting is encapsulated as the first retrieval parameter. This parameter not only contains the original instruction text, but more importantly, the weight attribute of the keyword inside it has been enhanced, thereby providing an optimized query basis that can highlight the user's core intent for subsequent semantic text mixed retrieval.

[0044] Further, the adjustment rule can include filtering a pre-set irrelevant application; when the instruction information is adjusted according to the adjustment rule in the rule library to obtain the retrieval parameter, if the adjustment rule is filtering a pre-set irrelevant application, a pre-set specified irrelevant application of the instruction keyword in the instruction information is determined; the specified irrelevant application is filtered to obtain a corresponding second retrieval parameter.

[0045] It should be noted that when constructing the rule library, for a specific instruction keyword or scene, the adjustment rule is set to "filter irrelevant applications". At the same time, a set of specified irrelevant application list associated with the rule needs to be pre-configured and excluded (for example, for industrial maintenance instructions, filter out entertainment applications). When the rule interceptor determines that the semantic text instruction information input by the user hits this trigger condition, the "filter irrelevant applications" adjustment rule is called. The system then determines the current specified irrelevant application set to be filtered according to the pre-configuration in the rule library. The system identifies and filters out the keywords, context or potential association related to the specified irrelevant application in the instruction information (or its parsed intermediate representation). The instruction information after this purification process is encapsulated as the second search parameter. This parameter eliminates the interference caused by irrelevant applications, so that subsequent retrieval can focus more on the user's real intention in the current context.

[0046] It should be noted that the present application actively filters irrelevant applications in the instruction, which effectively purifies the environmental background of the search parameter, so that semantic retrieval can exclude interference and focus on the core application related to the user's real intention, thereby significantly improving the accuracy and execution efficiency of instruction analysis in a multi-application complex environment, and ultimately enhancing the reliability of XR glasses interaction and the smoothness of user experience.

[0047] Further, the rule library can be updated during execution, and for this purpose, Figure 3 A flowchart of a rule library updating method provided for one or more embodiments of the present specification is shown. Some input parameters or intermediate results in the flowchart allow manual intervention to adjust to help improve accuracy.

[0048] The method flow steps of the embodiments of the present specification are as follows: S301, receiving semantic text instruction information of a specified application input by a user to an XR glasses.

[0049] In the embodiments of the present specification, the user's voice instruction can be received through the voice recognition module integrated by the XR glasses and converted into text information, or the text instruction can be received directly through the connected external device (such as a keyboard, gesture input). The system needs to clearly indicate the current focus or specified application in the XR environment to which the instruction is directed.

[0050] S302, determining whether the instruction information matches the trigger condition in the rule library through the rule interceptor.

[0051] In the embodiments of the present specification, a rule interceptor can be constructed, which matches the received semantic text instruction information with the trigger conditions in the pre-constructed rule library in real time. The trigger conditions in the rule library are usually defined as specific keywords, phrases or instruction patterns. The rule interceptor determines whether the instruction text contains key elements sufficient to trigger the subsequent adjustment process by parsing the instruction text.

[0052] S303, if yes, adjusting the instruction information according to the adjustment rules in the rule library to obtain the retrieval parameters.

[0053] In the embodiments of the present specification, if the instruction information matches successfully, the rule interceptor calls the adjustment rules associated with the trigger conditions in the rule library. The adjustment rules define specific instruction optimization strategies, such as increasing the weight of specific keywords or filtering irrelevant terms. The original instruction information is processed, purified or enhanced according to these rules to convert it into a set of standardized and structured retrieval parameters, providing optimized input for subsequent retrieval operations. If the instruction information does not match successfully, the default retrieval parameters can be called.

[0054] S304, performing semantic text hybrid retrieval based on the retrieval parameters to obtain an instruction response.

[0055] In the embodiments of the present specification, the processed retrieval parameters can be submitted to a semantic understanding and retrieval engine. The engine uses a hybrid retrieval strategy, combining keyword matching and semantic similarity calculation, to query in a pre-defined instruction-action mapping library or knowledge base. The goal is to go beyond literal matching and deeply understand user intent, ultimately retrieving the most matching and executable operation instruction as the instruction response.

[0056] S305, controlling the specified application based on the instruction response, and displaying the execution result of the specified application on the XR glasses.

[0057] In the embodiments of the present specification, the retrieved instruction response can be converted into a specific application programming interface call or control command and sent to the target specified application for execution. After the application performs the corresponding operation, it renders the output result (such as new virtual objects, data panels, state changes, etc.) to the display screen of the XR glasses, completing the interaction loop and providing visual feedback of the execution result to the user.

[0058] S306, monitoring the execution parameters of the specified application based on the execution result.

[0059] In the embodiments of the present specification, after the designated application completes the operation in response to the instruction and outputs the execution result, the system starts a background monitoring process. The process continuously collects and records three core execution parameters by calling the system interface and the application state callback: 1) state information, including the current running state of the application (such as foreground active, background suspended) and any error logs generated; 2) performance indicators, including the response time from the instruction issuance to the result presentation of this operation, and the calculation and memory resource usage during the operation; 3) user interaction data, including the historical operation frequency of the application or specific function, and the final result label (success times or failure times) of the current instruction execution. All these parameters are aggregated and form a complete record of an execution event.

[0060] S307, updating the rule base based on the execution parameters.

[0061] In the embodiments of the present specification, the system periodically or records a certain number of execution event records as a batch, and centrally analyzes the aggregated execution parameters. The core purpose of the analysis is to evaluate the effectiveness of the rule base entry triggered by this instruction execution (i.e. its trigger condition and adjustment rule). For example, if the instruction triggered by a certain rule frequently fails or causes application response delay to rise, it is determined that its effectiveness is low. Based on the analysis result, the system will start the update operation of the rule base, and the update methods include: modifying the trigger condition of the existing rule to make it more accurate, adjusting the adjustment rule (such as changing the weight promotion value or the filtering list) to optimize the performance, or completely adding a new rule entry to cover the new instruction mode identified this time which has not been processed. Thus, the rule base can be continuously evolved and optimized.

[0062] It should be noted that, by monitoring the application parameters after execution and updating the rule base accordingly, the present application establishes a closed-loop feedback mechanism that learns from actual interaction results, so that the rule base can continuously optimize and adapt to the real use habits of different users and scenarios, thereby enabling the entire system to have the ability of self-evolution, continuously improving the accuracy and control efficiency of subsequent instruction analysis, and ultimately realizing the continuous intelligent improvement of the XR glasses interaction experience.

[0063] Further, the execution parameters include state information, performance indicators, and user interaction data of the designated application, the state information includes running state and error logs, the performance indicators include response time and resource usage, and the user interaction data includes operation frequency, success times, and failure times. When updating the rule base based on the execution parameters, the execution parameters are analyzed to evaluate the effectiveness of the trigger condition and the adjustment rule in the rule base in the designated application, and an analysis result is obtained; based on the analysis result, the trigger condition and the adjustment rule in the rule base are modified or added.

[0064] It should be noted that after the specified application completes execution, the system synchronously collects three types of execution parameters. Status information is obtained by querying the application programming interface to retrieve its running status (e.g., active, suspended, terminated) and capturing error logs; performance metrics are recorded by the system monitoring tool, showing the response time and CPU, memory, and other resource utilization rates of this operation chain; user interaction data is obtained by statistically analyzing the frequency of operations on the application from the interaction logs, and clearly marking the operation as a success or failure increment based on the execution result. All parameters are aggregated into a complete data package for subsequent analysis. The system performs batch analysis on the aggregated execution parameters. The core objective of the analysis is to evaluate the effectiveness of the rule base entries associated with this instruction execution (i.e., their triggering conditions and adjustment rules) in the specified application context. For example, if a rule triggers an instruction that results in excessively long response times, abnormal resource utilization, or a significant number of failures, its effectiveness is considered low; conversely, if the instruction execution has a high number of successes and excellent performance metrics, its effectiveness is considered high. This process ultimately produces qualitative analysis results, identifying rules that need optimization or are ineffective. Based on the analysis results, the system initiates iterative updates to the rule base. If a rule is valid but can be optimized, its triggering conditions are modified (e.g., tightening or loosening the matching pattern) or the rule is fine-tuned (e.g., optimizing the weight value or filter list). If a rule is invalid or a new efficient instruction pattern is discovered, new triggering conditions and rule entries are added. Through this closed-loop feedback mechanism, the rule base can continuously evolve, constantly improving its adaptability and accuracy in different applications and scenarios. It should be noted that this application conducts in-depth analysis and precise updates of the rule base by comprehensively utilizing multi-dimensional execution parameters such as state information, performance indicators, and user interaction data. This method constructs an adaptive system that can continuously learn from real-world usage scenarios, enabling the triggering conditions and adjustment rules of the rule base to be continuously optimized to meet actual needs. This significantly improves the accuracy and execution efficiency of subsequent instruction parsing, ultimately achieving dynamic optimization and continuous enhancement of the XR glasses' interactive experience.

[0065] Furthermore, when adjusting the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters, the instruction information is adjusted according to the adjustment rules in the rule base to obtain the adjustment parameters; the user's specified location information and currently running application information are obtained; and the adjustment parameters are obtained based on the adjustment parameters, the specified location information, and the currently running application information.

[0066] It should be noted that first, the original semantic text instruction information is preliminarily processed (such as weight promotion or filtering) according to the adjustment rules in the rule library, and a preliminary optimized intermediate product, that is, an adjustment parameter, is obtained. The system obtains the current specified position information (such as "workshop A area" and "operation room 3") of the user through the positioning sensor (such as GPS and indoor positioning beacon) of the XR glasses in parallel, and queries and obtains the current running application information (such as the identification and state of the foreground focus application) through the device operating system interface. The preliminary adjustment parameter, the specified position information and the current running application information are fused. The fusion process aims to add rich environmental context to the instruction, for example, taking the position information as a weighted factor, or taking the current running application as a screening condition. Finally, an enriched, comprehensive search parameter containing semantic instructions, environmental state and application context is generated, providing highly contextualized query basis for subsequent semantic text mixed search.

[0067] It should be noted that the present application incorporates the real-time position information of the user and the current running application state into the instruction adjustment process. This method enables the generation of search parameters to deeply combine specific environmental context and task state, thereby significantly enhancing the context awareness ability of semantic understanding, ensuring that the instruction response is more in line with the actual intention and operation scene of the user, and ultimately improving the accuracy and natural fluency of XR glasses interaction.

[0068] Further, Figure 4 An example of retrieving an intention recognition provided for one or more embodiments of the present specification is as follows: First, system preparation, configure rule library + index optimization. First, build a badcase rule library, design a rule structure containing trigger conditions (such as instruction keywords) and execution actions (such as filtering irrelevant applications), develop a visual interface for operation personnel to configure, and deploy a rule interceptor to embed the ES search link; at the same time, analyze 30-day user instruction logs to identify high-priority intentions (such as I travel guide), adjust the weight (such as increasing the weight of "application type = I travel") and segmentation strategy (such as retaining the complete keyword "3D cinema") of related fields through the ES dynamic API; in addition, configure performance optimization strategies, including designing hot instruction cache (with "instruction word + scene label" as the key, combining local cache and Redis), splitting index shards by application type, and reducing semantic calculation accuracy for non-core intentions.

[0069] The second step involves real-time processing of user commands. After a user inputs a voice / text command (converted to text), rule matching is performed. A rule interceptor first determines if the command matches the triggering conditions in the rule base. If a match is successful, the search parameters are dynamically adjusted according to the rules (e.g., increasing field weights, filtering irrelevant applications). If a match fails, default parameters are used. Subsequently, a semantic-text hybrid search is executed, incorporating hotspot caching (high-frequency commands directly return cached results), routing to the corresponding application type index shard (reducing data scanning), and enabling lightweight semantic computation for non-core intents. Finally, an optimized command response is returned (e.g., launching the target application).

[0070] The third step is performance monitoring and iteration, which involves obtaining monitoring results and iterating accordingly. Real-time monitoring focuses on three aspects: badcase repair rate (target ≥90%), first-time match rate of high-priority intents (target ≥80%), response latency (target ≤200ms), and cache hit rate (target ≥60%). Based on the monitoring results, iterative optimization is performed, updating the rule base, index configuration, and performance strategies, and feeding back to the system initialization phase to form a continuous optimization loop.

[0071] It should be noted that the following is a schematic diagram of a search intent recognition method: Step 1: Building and Deploying the Badcase Rule Base Rule base design: Rule structure: Includes "triggering conditions" (such as instruction keywords or intent words) and "execution actions". (e.g., field weight adjustment, application filtering, priority matching application), "scope of application" (e.g., all scenarios, scenic area scenarios, daily scenarios); Visual configuration interface: The developed web interface supports operators to input rules (such as selecting "Trigger condition: instruction contains '3D Cinema'", "Execute action: prioritize matching 'application name = 3D Cinema'"), and automatically generate rule scripts.

[0072] Rule activation mechanism: Add a "rule interceptor" to the ES retrieval request chain: after the instruction request arrives, the interceptor first matches the triggering conditions in the rule base; If a match is found, adjust the search parameters according to the rules (such as dynamically modifying the weight of the "Application Type" field and adding application filtering conditions), and then perform a semantic-text hybrid search. Once the rules are modified, they are synchronized to the interceptor in real time without requiring a restart of the XR glasses service, ensuring immediate effect.

[0073] Step 2: Implementation of Specific Intent Index Optimization High-priority intent recognition: Analyze the XR glasses user instruction log for the past 30 days, count the instruction frequency (e.g., "navigation start" instruction accounts for 25%), and scene importance (e.g., scenic area scene "I travel guide" affects user experience), and determine the top 5 high-priority intents.

[0074] Index adjustment configuration: Field weight: In the ES "dis_max" query (common mixed retrieval), set the boost parameter for the relevant fields of high-priority intents (e.g., "I travel guide" intent "application type = I travel" field boost=2.0, default field boost=1.0); Segmentation strategy: Add custom dictionary (e.g., "OCR translation" "3D cinema") in the "analysis" configuration of ES index to ensure complete matching of instruction keywords; Configuration dynamic effect: Update the weight and segmentation strategy in real time through the "update_settings" API of ES index without rebuilding the index.

[0075] Step 3: Performance optimization configuration Hotspot cache design: Cache object: Retrieval results of high-frequency instructions (including application ID, response priority), cache key is "instruction word + scene label" (avoid conflict of the same instruction in different scenes, such as "navigation" in scenic area and city scene with different responses); Cache strategy: Use local cache (XR glasses side) + Redis (cloud) as cache medium, set TTL (e.g., scenic area high-frequency instruction "next scenic spot" TTL=30 seconds, synchronized with the guide rhythm), and use LRU (least recently used) eviction policy when cache is full.

[0076] Index sharding configuration: Split index by application type: Create "I travel index" "navigation index" "3D cinema index" and other independent indexes, each index corresponds to an independent shard; Retrieval routing: Determine the application type according to the instruction keyword (e.g., route to "I travel index" if it contains "travel" keyword), perform retrieval only in the corresponding index, and reduce the amount of scanned data.

[0077] Lightweight semantic computation: For non-core intent instructions (e.g., low-frequency "close application" instruction), use "vector approximate matching" (e.g., "num_candidates" parameter in ES "knn" query from 1000 to 500) in semantic retrieval to reduce computational complexity.

[0078] Step 4: Effect monitoring and iteration Badcase monitoring: Real-time collection of XR glasses user feedback (such as "response error" voice feedback) and instruction logs, marking new badcases, and regularly calculating rule library repair rate (such as the proportion of successfully repaired badcases ≥ 90%).

[0079] Intention recognition rate monitoring: For high-priority intentions, sample 1000 instruction test response results and calculate the proportion of "first response matching true intention". If it is less than 80%, adjust the index weight.

[0080] Performance monitoring: Real-time monitoring of instruction response time (target ≤ 200ms) and interaction processing capacity. If there is a delay increase, check the cache hit rate (target ≥ 60%) and shard retrieval efficiency, and optimize the cache strategy or shard rules.

[0081] Figure 5 A structural schematic diagram of an application control device provided for one or more embodiments of the present specification, comprising: a receiving unit 501, a judging unit 502, an adjusting unit 503, a retrieving unit 504, and a control unit 505.

[0082] The receiving unit 501 receives semantic text instruction information of a specified application input by a user to an XR glasses; The judging unit 502 judges whether the instruction information matches the trigger condition in the rule library through a rule interceptor; The adjusting unit 503, if so, adjusts the instruction information according to the adjustment rules in the rule library to obtain retrieval parameters; The retrieving unit 504 performs semantic text mixed retrieval based on the retrieval parameters to obtain an instruction response; The control unit 505 controls the specified application based on the instruction response and displays the execution result of the specified application on the XR glasses.

[0083] Figure 6 A structural schematic diagram of an XR glasses provided for one or more embodiments of the present specification, comprising: At least one processor and a bus; and, A memory in communication connection with the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Receive semantic text instruction information of a specified application input by a user to an XR glasses; determine, by a rule interceptor, whether the instruction information matches a trigger condition in a rule library; if yes, adjust the instruction information according to an adjustment rule in the rule library to obtain a search parameter; perform semantic text mixed search based on the search parameter to obtain an instruction response; control the specified application based on the instruction response, and display an execution result of the specified application on the XR glasses.

[0084] One or more embodiments of the present specification provide a non-volatile computer storage medium storing computer executable instructions, which can implement the following when executed by a computer: receive semantic text instruction information of a specified application input by a user to an XR glasses; determine, by a rule interceptor, whether the instruction information matches a trigger condition in a rule library; if yes, adjust the instruction information according to an adjustment rule in the rule library to obtain a search parameter; perform semantic text mixed search based on the search parameter to obtain an instruction response; control the specified application based on the instruction response, and display an execution result of the specified application on the XR glasses.

[0085] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the difference from other embodiments. Especially, the device, equipment, and non-volatile computer storage medium embodiments are basically similar to the method embodiments, and the description is relatively simple. The relevant parts can be referred to the part of the method embodiment.

[0086] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the difference from other embodiments. Especially, the device, equipment, and non-volatile computer storage medium embodiments are basically similar to the method embodiments, and the description is relatively simple. The relevant parts can be referred to the part of the method embodiment.

[0087] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present specification can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0088] In the embodiments of the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other manners. For example, the described apparatus / network device embodiments are merely schematic. For example, the division of the modules or units is merely logical function division. There can be another division manner for actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0089] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0090] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit. The above-mentioned units can be realized in the form of hardware or in the form of software.

[0091] The integrated module / unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0092] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An application control method characterized by comprising: The method comprises: receiving semantic text instruction information of a specified application input by a user to an XR glasses; determining whether the instruction information matches a trigger condition in a rule library through a rule interceptor; if so, adjusting the instruction information according to an adjustment rule in the rule library to obtain a search parameter; performing semantic text mixed search based on the search parameter to obtain an instruction response; controlling the specified application based on the instruction response, and displaying an execution result of the specified application on the XR glasses.

2. The method of claim 1, wherein, Before determining whether the search parameter matches a trigger condition in a pre-constructed rule library through a rule interceptor, the method further comprises: constructing a rule library according to pre-set trigger conditions and adjustment rules corresponding to each instruction keyword.

3. The method of claim 2, wherein, The adjustment rule comprises increasing the weight of an instruction keyword; adjusting the instruction information according to the adjustment rule in the rule library to obtain a search parameter comprises: if the adjustment rule is to increase the weight of the instruction keyword, increasing the weight of the instruction keyword in the instruction information based on a pre-set weight increase value to obtain a corresponding first search parameter.

4. The method of claim 2, wherein, The adjustment rule comprises filtering pre-set irrelevant applications; adjusting the instruction information according to the adjustment rule in the rule library to obtain a search parameter comprises: if the adjustment rule is to filter pre-set irrelevant applications, determining a pre-set specified irrelevant application of an instruction keyword in the instruction information; filtering the specified irrelevant application to obtain a corresponding second search parameter.

5. The method of claim 2, wherein, After controlling the specified application based on the instruction response and displaying an execution result of the specified application on the XR glasses, the method further comprises: monitoring an execution parameter of the specified application based on the execution result; updating the rule library based on the execution parameter.

6. The method of claim 5, wherein, The execution parameter comprises state information, performance indicators, and user interaction data of the specified application, the state information comprises a running state and error logs, the performance indicators comprise response time and resource usage, and the user interaction data comprises operation frequency, success times, and failure times; updating the rule library based on the execution parameter comprises: analyzing the execution parameter to evaluate the effectiveness of the trigger condition and the adjustment rule in the rule library in the specified application to obtain an analysis result; based on the analysis result, modifying or adding the trigger condition and the adjustment rule in the rule library.

7. The method of claim 1, wherein, adjusting the instruction information according to the adjustment rule in the rule library to obtain a search parameter comprises: adjusting the instruction information according to the adjustment rule in the rule library to obtain an adjustment parameter; obtaining specified location information and current running application information of the user; obtaining an adjustment parameter based on the adjustment parameter, the specified location information, and the current running application information.

8. An application control device characterized by comprising: comprises: a receiving unit that receives semantic text instruction information of a specified application input by a user to an XR glasses; a determining unit that determines whether the instruction information matches a trigger condition in a rule library through a rule interceptor; an adjusting unit that adjusts the instruction information according to an adjustment rule in the rule library to obtain a search parameter if the instruction information matches the trigger condition; and The retrieval unit performs semantic text mixed retrieval based on the retrieval parameter to obtain an instruction response; The control unit controls the specified application based on the instruction response, and the XR glasses display the execution result of the specified application.

9. An XR eyewear, characterized by, Comprise: At least one processor and bus; And, The memory is in communication connection with the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: Receive semantic text instruction information of a specified application input by a user to the XR glasses; Determine whether the instruction information matches the trigger condition in the rule library through a rule interceptor; If yes, adjust the instruction information according to the adjustment rule in the rule library to obtain a retrieval parameter; Perform semantic text mixed retrieval based on the retrieval parameter to obtain an instruction response; Control the specified application based on the instruction response, and the XR glasses display the execution result of the specified application.

10. A non-transitory computer storage medium, comprising, The computer executable instructions are executed by the computer to enable: Receive semantic text instruction information of a specified application input by a user to the XR glasses; Determine whether the instruction information matches the trigger condition in the rule library through a rule interceptor; If yes, adjust the instruction information according to the adjustment rule in the rule library to obtain a retrieval parameter; Perform semantic text mixed retrieval based on the retrieval parameter to obtain an instruction response; Control the specified application based on the instruction response, and the XR glasses display the execution result of the specified application.

Images