Application control method and apparatus, glasses, and medium

Abstract

The application discloses an application control method and device, glasses and a medium, comprising: receiving semantic text instruction information of a specified application input by a user to XR glasses; determining whether the instruction information matches a trigger condition in a rule library through a rule interceptor; if yes, adjusting the instruction information according to an adjustment rule in the rule library to obtain a search parameter; performing semantic text mixed search based on the search parameter to obtain an instruction response; and controlling the specified application based on the instruction response and displaying the execution result of the specified application on the XR glasses. The application converts a fuzzy natural language instruction into a precise search parameter, and then understands the user's intention through a semantic text mixed search technology, thereby effectively solving the core problem of insufficient instruction analysis and adaptation capability in a multi-application scenario, enabling the XR glasses to quickly and accurately understand and execute the user's instruction, and significantly improving the interaction response efficiency and the smoothness of the operation experience.

Description

Technical Field

[0001] This application relates to the technical field of XR glasses control, and more particularly to an application control method, device, glasses, and medium. Background Technology

[0002] XR (Extended Reality) refers to the use of computers to combine the real and virtual worlds, creating a virtual environment that allows for human-computer interaction. It is also a collective term for various technologies such as AR, VR, and MR. By integrating the visual interaction technologies of these three technologies, it brings users a sense of immersion that allows for seamless transitions between the virtual and real worlds.

[0003] However, existing XR glasses application control methods struggle to effectively parse and adapt to user-input semantic text commands, resulting in low command execution accuracy, high response latency, and poor user experience in multi-application and multi-scenario environments. Summary of the Invention

[0004] This specification provides one or more embodiments of an application control method, device, glasses, and medium to solve the technical problems raised in the background art.

[0005] One or more embodiments of this specification employ the following technical solutions:

[0006] This specification provides an application control method according to one or more embodiments, the method comprising:

[0007] Receive semantic text instructions for a specified application from the user inputting them into the XR glasses;

[0008] The rule interceptor determines whether the instruction information matches the triggering conditions in the rule base.

[0009] If so, adjust the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters;

[0010] Based on the search parameters, a semantic text hybrid search is performed to obtain the command response;

[0011] The specified application is controlled based on the instruction response, and the execution result of the specified application is displayed on the XR glasses.

[0012] It should be noted that this application introduces a rule interceptor and a rule base to perform real-time matching and intelligent adjustment of the user's original instructions. This method transforms ambiguous natural language instructions into precise search parameters, and then uses semantic text hybrid retrieval technology to understand the user's intent. This effectively solves the core problem of insufficient instruction parsing and adaptation capabilities in multiple application scenarios, enabling XR glasses to quickly and accurately understand and execute user instructions, significantly improving interaction response efficiency and the smoothness of the operation experience.

[0013] Furthermore, before determining whether the retrieval parameters match the triggering conditions in the pre-built rule base using the rule interceptor, the method further includes:

[0014] A rule base is built based on the pre-defined trigger conditions and adjustment rules corresponding to each instruction keyword.

[0015] It should be noted that this application provides clear and comprehensive judgment criteria for the rule interceptor by systematically building a rule base in advance. This enables the entire system to preprocess diverse and ambiguous instructions before receiving them, thereby triggering subsequent adjustment and retrieval processes more quickly and accurately in actual interaction. Ultimately, this significantly improves the XR glasses' understanding of complex semantic instructions and the overall intelligence level of interaction.

[0016] Furthermore, the adjustment rules include increasing the weight of instruction keywords;

[0017] The step of adjusting the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters includes:

[0018] If the adjustment rule is to increase the weight of the instruction keywords, the weight of the instruction keywords in the instruction information is increased based on the preset weight increase value to obtain the corresponding first search parameters.

[0019] It should be noted that this application generates optimized search parameters by increasing the weight of instruction keywords. This method strengthens the capture of the user's core intent in semantic retrieval, enabling subsequent hybrid retrieval to focus more on the essential needs of the instruction, thereby effectively reducing ambiguity caused by vague terminology or diverse scenarios, and significantly improving the accuracy of instruction parsing and the precision of application control.

[0020] Furthermore, the adjustment rules include filtering pre-defined irrelevant applications;

[0021] The step of adjusting the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters includes:

[0022] If the adjustment rule is to filter pre-defined irrelevant applications, determine the pre-defined irrelevant applications in the instruction keywords of the instruction information;

[0023] The specified irrelevant applications are filtered to obtain the corresponding second search parameters.

[0024] It should be noted that this application effectively purifies the environmental background of the search parameters by actively filtering irrelevant applications in the instructions. This allows semantic retrieval to eliminate interference and focus on the core applications related to the user's true intent, thereby significantly improving the accuracy and execution efficiency of instruction parsing in complex multi-application environments. Ultimately, this enhances the reliability of XR glasses interaction and the smoothness of the user experience.

[0025] Furthermore, the method of controlling the designated application based on the instruction response, after the XR glasses display the execution result of the designated application, further includes:

[0026] Monitor the execution parameters of the specified application based on the execution results;

[0027] The rule base is updated based on the execution parameters.

[0028] It should be noted that this application establishes a closed-loop feedback mechanism that learns from actual interaction results by monitoring application parameters after execution and updating the rule base accordingly. This enables the rule base to continuously optimize and adapt to the real usage habits of different users and scenarios, thereby giving the entire system the ability to evolve on its own, continuously improving the accuracy of subsequent instruction parsing and control efficiency, and ultimately achieving continuous intelligent improvement of the XR glasses interaction experience.

[0029] Furthermore, the execution parameters include the status information, performance metrics, and user interaction data of the specified application. The status information includes the running status and error logs. The performance metrics include response time and resource utilization. The user interaction data includes operation frequency, number of successes, and number of failures.

[0030] Updating the rule base based on the execution parameters includes:

[0031] The execution parameters are analyzed to evaluate the effectiveness of the triggering conditions and adjustment rules in the rule base in the specified application, and the analysis results are obtained.

[0032] Based on the analysis results, modify or add triggering conditions and adjustment rules in the rule base.

[0033] It should be noted that this application conducts in-depth analysis and precise updates of the rule base by comprehensively utilizing multi-dimensional execution parameters such as state information, performance indicators, and user interaction data. This method constructs an adaptive system that can continuously learn from real-world usage scenarios, enabling the triggering conditions and adjustment rules of the rule base to be continuously optimized to meet actual needs. This significantly improves the accuracy and execution efficiency of subsequent instruction parsing, ultimately achieving dynamic optimization and continuous enhancement of the XR glasses' interactive experience.

[0034] Furthermore, adjusting the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters includes:

[0035] The instruction information is adjusted according to the adjustment rules in the rule base to obtain the adjustment parameters;

[0036] Obtain the user's specified location information and currently running application information;

[0037] The adjustment parameters are obtained based on the adjustment parameters, the specified location information, and the currently running application information.

[0038] It should be noted that by incorporating the user's real-time location information and the current running application status into the instruction adjustment process, this method enables the generation of retrieval parameters to be deeply integrated with the specific environmental context and task status, thereby significantly enhancing the contextual awareness capability of semantic understanding, ensuring that the instruction response is more in line with the user's actual intention and operation scenario, and ultimately improving the accuracy and natural fluency of XR glasses interaction.

[0039] This specification provides an application control device according to one or more embodiments, comprising:

[0040] The receiving unit receives semantic text instructions for a specified application input by the user into the XR glasses;

[0041] The judgment unit determines whether the instruction information matches the triggering conditions in the rule base through the rule interceptor;

[0042] If so, the adjustment unit adjusts the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters;

[0043] The retrieval unit performs a semantic-text hybrid retrieval based on the retrieval parameters and obtains an instruction response;

[0044] The control unit controls the designated application based on the instruction response and displays the execution result of the designated application on the XR glasses.

[0045] This specification provides one or more embodiments of an XR glasses system, comprising:

[0046] At least one processor and bus; and,

[0047] A memory communicatively connected to the at least one processor; wherein,

[0048] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0049] Receive semantic text instructions for a specified application from the user inputting them into the XR glasses;

[0050] The rule interceptor determines whether the instruction information matches the triggering conditions in the rule base.

[0051] If so, adjust the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters;

[0052] Based on the search parameters, a semantic text hybrid search is performed to obtain the command response;

[0053] The specified application is controlled based on the instruction response, and the execution result of the specified application is displayed on the XR glasses.

[0054] This specification provides one or more embodiments of a non-volatile computer storage medium storing computer-executable instructions, which, when executed by a computer, can perform the following:

[0055] Receive semantic text instructions for a specified application from the user inputting them into the XR glasses;

[0056] The rule interceptor determines whether the instruction information matches the triggering conditions in the rule base.

[0057] If so, adjust the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters;

[0058] Based on the search parameters, a semantic text hybrid search is performed to obtain the command response;

[0059] The specified application is controlled based on the instruction response, and the execution result of the specified application is displayed on the XR glasses.

[0060] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0061] This application introduces a rule interceptor and rule base to perform real-time matching and intelligent adjustment of the user's original instructions. This method transforms ambiguous natural language instructions into precise search parameters, and then uses semantic text hybrid retrieval technology to understand the user's intent. This effectively solves the core problem of insufficient instruction parsing and adaptation capabilities in multiple application scenarios, enabling XR glasses to quickly and accurately understand and execute user instructions, significantly improving interaction response efficiency and the smoothness of the operation experience. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0063] Figure 1 An application environment diagram for an application control method provided in one or more embodiments of this specification;

[0064] Figure 2 A flowchart illustrating an application control method provided for one or more embodiments of this specification;

[0065] Figure 3 A flowchart illustrating a rule base update method provided in one or more embodiments of this specification;

[0066] Figure 4 A schematic diagram illustrating a search intent recognition provided for one or more embodiments of this specification;

[0067] Figure 5 A schematic diagram of an application control device provided for one or more embodiments of this specification;

[0068] Figure 6 This is a structural schematic diagram of an application control glasses provided for one or more embodiments of this specification. Detailed Implementation

[0069] This specification provides an application control method, device, glasses, and medium through its embodiments.

[0070] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0071] The solution proposed in this application can be applied to application control scenarios in application control terminals. Figure 1 This application shows an application environment diagram of an application control method provided in an embodiment of this application, such as... Figure 1 As shown, terminal 102 communicates with server 103 via a network. Data storage system 101 can store data that server 103 needs to process. Data storage system 101 can be integrated on server 103 or placed on the cloud or other network servers. Terminal 102 can acquire historical behavior data of users in different operating scenarios and the operating status of XR glasses; extract common behavioral information of users from the historical behavior data of users in each operating scenario; extract common status information of XR glasses from the operating status of XR glasses in each operating scenario; combine the common behavioral information and the common status information to generate the glasses habit status of XR glasses in the operating scenario; associate the operating scenario with the glasses habit status to obtain behavioral habit tags. Alternatively, the process of constructing the tags described above can be executed on server 103. That is, the server obtains the user's historical behavior data in different operating scenarios and the operating status of the XR glasses; extracts common behavioral information from the user's historical behavior data in each operating scenario; extracts common status information of the XR glasses from the operating status of the XR glasses in each operating scenario; combines the common behavioral information and the common status information to generate the glasses habit status of the XR glasses in the operating scenario; and associates the operating scenario with the glasses habit status to obtain behavioral habit tags.

[0072] Specifically, the application control terminal can include smartphones, smart home appliances, tablets, virtual reality head-mounted displays (VR headsets), augmented reality glasses (AR glasses), electronic displays, and mixed reality (MR) devices, etc. MR devices can include MR glasses, MR helmets, MR cameras, etc. The in-vehicle system can include in-vehicle chips, in-vehicle devices (such as in-vehicle infotainment systems, in-vehicle computers, sensors with voice recognition capabilities, etc.).

[0073] Figure 2This diagram illustrates an application control method provided in one or more embodiments of this specification, which can be executed by an application control system. Certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0074] The method flow steps of the embodiments in this specification are as follows:

[0075] S201, Receive semantic text instruction information for a specified application input by the user into the XR glasses.

[0076] In the embodiments described in this specification, the user's voice commands can be received through the voice recognition module integrated into the XR glasses and converted into text information; or text commands can be received directly through a connected external device (such as a keyboard or gesture input). The system needs to clearly define the current focus or specified application in the XR environment to which the command is directed.

[0077] S202, the rule interceptor determines whether the instruction information matches the triggering conditions in the rule base.

[0078] In the embodiments of this specification, a rule interceptor can be constructed that matches received semantic text instruction information with triggering conditions in a pre-built rule base in real time. Triggering conditions in the rule base are typically defined as specific keywords, phrases, or instruction patterns. The rule interceptor parses the instruction text to determine whether it contains key elements sufficient to trigger subsequent adjustment processes.

[0079] S203, If so, adjust the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters.

[0080] In the embodiments described in this specification, if the instruction information matches successfully, the rule interceptor invokes the adjustment rules associated with the triggering condition in the rule base. The adjustment rules define specific instruction optimization strategies, such as increasing the weight of specific keywords or filtering irrelevant terms. Based on these rules, the original instruction information is processed, purified, or enhanced, transforming it into a set of standardized, structured search parameters to provide optimized input for subsequent search operations. If the instruction information does not match successfully, pre-set default search parameters can be invoked.

[0081] S204, Perform a semantic text hybrid search based on the search parameters to obtain an instruction response.

[0082] In the embodiments described in this specification, the processed retrieval parameters can be submitted to a semantic understanding and retrieval engine. This engine employs a hybrid retrieval strategy, combining keyword matching and semantic similarity calculation, to query a predefined instruction-action mapping library or knowledge base. Its goal is to go beyond literal matching, deeply understand user intent, and ultimately retrieve the most matching and executable operation instruction as the instruction response.

[0083] S205, based on the instruction response, control the designated application and display the execution result of the designated application on the XR glasses.

[0084] In the embodiments described in this specification, the retrieved instruction response can be converted into a specific application programming interface call or control command and sent to the target application for execution. After the application performs the corresponding operation, it renders its output (such as new virtual objects, data panels, state changes, etc.) onto the display screen of the XR glasses, completing the interaction loop and providing the user with visual feedback on the execution results.

[0085] It should be noted that this application introduces a rule interceptor and a rule base to perform real-time matching and intelligent adjustment of the user's original instructions. This method transforms ambiguous natural language instructions into precise search parameters, and then uses semantic text hybrid retrieval technology to understand the user's intent. This effectively solves the core problem of insufficient instruction parsing and adaptation capabilities in multiple application scenarios, enabling XR glasses to quickly and accurately understand and execute user instructions, significantly improving interaction response efficiency and the smoothness of the operation experience.

[0086] Furthermore, before determining whether the search parameters match the trigger conditions in the pre-built rule base through the rule interceptor, a rule base can be constructed based on the trigger conditions and adjustment rules corresponding to each pre-set instruction keyword.

[0087] It's important to note that building a rule base involves several steps. First, system designers need to extensively collect common user semantic text commands from the target application scenario and extract key command keywords. Second, for each keyword or category of keywords, their triggering conditions must be clearly defined. These conditions should precisely describe what kind of command text will trigger the rule. Next, one or more specific adjustment rules should be associated with each triggering condition. These rules define how the system should optimize or transform the original command when the condition is met. Finally, all the mappings between "command keywords - triggering conditions - adjustment rules" should be persistently stored in a machine-readable structured format (such as JSON, XML, or database tables), forming a rule base that can be efficiently queried by the rule interceptor. This rule base is the fundamental basis for subsequent real-time judgment and adjustment by the rule interceptor.

[0088] It should be noted that this application provides clear and comprehensive judgment criteria for the rule interceptor by systematically building a rule base in advance. This enables the entire system to preprocess diverse and ambiguous instructions before receiving them, thereby triggering subsequent adjustment and retrieval processes more quickly and accurately in actual interaction. Ultimately, this significantly improves the XR glasses' understanding of complex semantic instructions and the overall intelligence level of interaction.

[0089] Furthermore, the adjustment rule may include increasing the weight of the instruction keywords; when adjusting the instruction information according to the adjustment rule in the rule base to obtain the search parameters, if the adjustment rule is to increase the weight of the instruction keywords, the weight of the instruction keywords in the instruction information is increased based on a pre-set weight increase value to obtain the corresponding first search parameters.

[0090] It's important to note that during the rule base construction phase, for specific command keywords, the associated adjustment rule is explicitly set to "increase weight." Simultaneously, a specific weight increase value must be pre-defined for this rule (e.g., doubling the importance of a keyword). This configuration is stored as metadata in the rule base. When the semantic text command information entered by the user is determined by the rule interceptor to match a trigger condition, the rule interceptor immediately invokes the "increase weight" adjustment rule corresponding to that condition. The system identifies the target command keyword contained in the command information and increases the weight value of that keyword according to the predefined weight increase value in the rule base. The command information (or its structured representation) after weight increase is encapsulated as the first search parameter. This parameter not only contains the original command text, but more importantly, its internal keyword weight attribute has been enhanced, thus providing optimized query basis that better highlights the user's core intent for subsequent semantic text mixed retrieval.

[0091] It should be noted that this application generates optimized search parameters by increasing the weight of instruction keywords. This method strengthens the capture of the user's core intent in semantic retrieval, enabling subsequent hybrid retrieval to focus more on the essential needs of the instruction, thereby effectively reducing ambiguity caused by vague terminology or diverse scenarios, and significantly improving the accuracy of instruction parsing and the precision of application control.

[0092] Furthermore, the adjustment rules may include filtering pre-defined irrelevant applications; when adjusting the instruction information according to the adjustment rules in the rule base to obtain search parameters, if the adjustment rule is to filter pre-defined irrelevant applications, the specified irrelevant applications pre-defined by the instruction keywords in the instruction information are determined; the specified irrelevant applications are filtered to obtain the corresponding second search parameters.

[0093] It's important to note that when building the rule base, for specific command keywords or scenarios, the adjustment rule is set to "filter irrelevant applications." Simultaneously, a pre-defined list of specified irrelevant applications to be excluded must be established (e.g., for industrial maintenance commands, filtering out entertainment applications). When the rule interceptor determines that the user's input semantic text command information matches this trigger condition, it invokes this "filter irrelevant applications" adjustment rule. The system then determines the set of specified irrelevant applications to be filtered based on the pre-configured rule base. The system identifies and filters out keywords, context, or potential associations related to the specified irrelevant applications in the command information (or its parsed intermediate representation). The command information after this purification process is encapsulated as a second search parameter. This parameter eliminates potential interference from irrelevant applications, allowing subsequent searches to focus more on the user's true intent in the current context.

[0094] It should be noted that this application effectively purifies the environmental background of the search parameters by actively filtering irrelevant applications in the instructions. This allows semantic retrieval to eliminate interference and focus on the core applications related to the user's true intent, thereby significantly improving the accuracy and execution efficiency of instruction parsing in complex multi-application environments. Ultimately, this enhances the reliability of XR glasses interaction and the smoothness of the user experience.

[0095] Furthermore, the rule base can be updated during execution; therefore, Figure 3 This diagram illustrates a rule base update method provided in one or more embodiments of this specification. This process can be executed by a rule base update system. Certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0096] The method flow steps of the embodiments in this specification are as follows:

[0097] S301, receives semantic text instructions for a specified application input by the user into the XR glasses.

[0098] In the embodiments described in this specification, the user's voice commands can be received through the voice recognition module integrated into the XR glasses and converted into text information; or text commands can be received directly through a connected external device (such as a keyboard or gesture input). The system needs to clearly define the current focus or specified application in the XR environment to which the command is directed.

[0099] S302, the rule interceptor determines whether the instruction information matches the triggering conditions in the rule base.

[0100] In the embodiments of this specification, a rule interceptor can be constructed that matches received semantic text instruction information with triggering conditions in a pre-built rule base in real time. Triggering conditions in the rule base are typically defined as specific keywords, phrases, or instruction patterns. The rule interceptor parses the instruction text to determine whether it contains key elements sufficient to trigger subsequent adjustment processes.

[0101] S303, If so, adjust the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters.

[0102] In the embodiments described in this specification, if the instruction information matches successfully, the rule interceptor invokes the adjustment rules associated with the triggering condition in the rule base. The adjustment rules define specific instruction optimization strategies, such as increasing the weight of specific keywords or filtering irrelevant terms. Based on these rules, the original instruction information is processed, purified, or enhanced, transforming it into a set of standardized, structured search parameters to provide optimized input for subsequent search operations. If the instruction information does not match successfully, pre-set default search parameters can be invoked.

[0103] S304, Perform a semantic text hybrid search based on the search parameters to obtain an instruction response.

[0104] In the embodiments described in this specification, the processed retrieval parameters can be submitted to a semantic understanding and retrieval engine. This engine employs a hybrid retrieval strategy, combining keyword matching and semantic similarity calculation, to query a predefined instruction-action mapping library or knowledge base. Its goal is to go beyond literal matching, deeply understand user intent, and ultimately retrieve the most matching and executable operation instruction as the instruction response.

[0105] S305, based on the instruction response, control the designated application and display the execution result of the designated application on the XR glasses.

[0106] In the embodiments described in this specification, the retrieved instruction response can be converted into a specific application programming interface call or control command and sent to the target application for execution. After the application performs the corresponding operation, it renders its output (such as new virtual objects, data panels, state changes, etc.) onto the display screen of the XR glasses, completing the interaction loop and providing the user with visual feedback on the execution results.

[0107] S306, Monitor the execution parameters of the specified application based on the execution result.

[0108] In the embodiments described in this specification, after the designated application completes the operation and outputs the execution result according to the instruction response, the system starts a background monitoring process. This process continuously collects and records three core execution parameters by calling system interfaces and application status callbacks: 1) Status information, including the current running status of the application (e.g., active in the foreground, paused in the background) and any error logs generated; 2) Performance metrics, including the response time from instruction issuance to result presentation, and the computation and memory resource utilization during the operation; 3) User interaction data, including the historical operation frequency of the application or specific function, and the final result marker of this instruction execution (number of successes or failures). All these parameters are summarized to form a complete record of an execution event.

[0109] S307, Update the rule base based on the execution parameters.

[0110] In the embodiments described in this specification, the system periodically or in batches records a certain number of execution events and centrally analyzes the aggregated execution parameters. The core purpose of the analysis is to evaluate the effectiveness of the rule base entry (i.e., its triggering conditions and adjustment rules) triggered by the execution of this instruction. For example, if an instruction triggered by a certain rule frequently fails or causes increased application response latency, its effectiveness is determined to be low. Based on this analysis result, the system will initiate an update operation on the rule base. The update methods include: modifying the triggering conditions of existing rules to make them more precise, adjusting adjustment rules (such as changing weight boost values ​​or filter lists) to optimize performance, or completely adding new rule entries to cover the newly identified, unprocessed instruction patterns. This allows the rule base to continuously evolve and be optimized.

[0111] It should be noted that this application establishes a closed-loop feedback mechanism that learns from actual interaction results by monitoring application parameters after execution and updating the rule base accordingly. This enables the rule base to continuously optimize and adapt to the real usage habits of different users and scenarios, thereby giving the entire system the ability to evolve on its own, continuously improving the accuracy of subsequent instruction parsing and control efficiency, and ultimately achieving continuous intelligent improvement of the XR glasses interaction experience.

[0112] Furthermore, the execution parameters include the status information, performance metrics, and user interaction data of the specified application. The status information includes running status and error logs; the performance metrics include response time and resource utilization; and the user interaction data includes operation frequency, number of successes, and number of failures. When updating the rule base based on the execution parameters, the execution parameters are analyzed to evaluate the effectiveness of the triggering conditions and adjustment rules in the rule base in the specified application, and the analysis results are obtained. Based on the analysis results, the triggering conditions and adjustment rules in the rule base are modified or added.

[0113] It should be noted that after the specified application completes execution, the system synchronously collects three types of execution parameters. Status information is obtained by querying the application programming interface to retrieve its running status (e.g., active, suspended, terminated) and capturing error logs; performance metrics are recorded by the system monitoring tool, showing the response time and CPU, memory, and other resource utilization rates of this operation chain; user interaction data is obtained by statistically analyzing the frequency of operations on the application from the interaction logs, and clearly marking the operation as a success or failure increment based on the execution result. All parameters are aggregated into a complete data package for subsequent analysis. The system performs batch analysis on the aggregated execution parameters. The core objective of the analysis is to evaluate the effectiveness of the rule base entries associated with this instruction execution (i.e., their triggering conditions and adjustment rules) in the specified application context. For example, if a rule triggers an instruction that results in excessively long response times, abnormal resource utilization, or a significant number of failures, its effectiveness is considered low; conversely, if the instruction execution has a high number of successes and excellent performance metrics, its effectiveness is considered high. This process ultimately produces qualitative analysis results, identifying rules that need optimization or are ineffective. Based on the analysis results, the system initiates iterative updates to the rule base. If a rule is valid but can be optimized, its triggering conditions are modified (e.g., tightening or loosening the matching pattern) or the rule is fine-tuned (e.g., optimizing the weight value or filter list). If a rule is invalid or a new efficient instruction pattern is discovered, new triggering conditions and rule entries are added. Through this closed-loop feedback mechanism, the rule base can continuously evolve, constantly improving its adaptability and accuracy in different applications and scenarios.

[0114] It should be noted that this application conducts in-depth analysis and precise updates of the rule base by comprehensively utilizing multi-dimensional execution parameters such as state information, performance indicators, and user interaction data. This method constructs an adaptive system that can continuously learn from real-world usage scenarios, enabling the triggering conditions and adjustment rules of the rule base to be continuously optimized to meet actual needs. This significantly improves the accuracy and execution efficiency of subsequent instruction parsing, ultimately achieving dynamic optimization and continuous enhancement of the XR glasses' interactive experience.

[0115] Furthermore, when adjusting the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters, the instruction information is adjusted according to the adjustment rules in the rule base to obtain the adjustment parameters; the user's specified location information and currently running application information are obtained; and the adjustment parameters are obtained based on the adjustment parameters, the specified location information, and the currently running application information.

[0116] It should be noted that, firstly, the original semantic text instruction information is preliminarily processed (such as weighting or filtering) according to the adjustment rules in the rule base, resulting in a preliminary optimized intermediate product, namely the adjustment parameters. The system concurrently acquires the user's current specified location information (e.g., "Workshop A" or "Operating Room 3") through the XR glasses' positioning sensors (e.g., GPS, indoor positioning beacons), and simultaneously queries and obtains currently running application information (e.g., the identifier and status of the foreground focused application) through the device's operating system interface. The preliminary adjustment parameters, specified location information, and currently running application information are then fused. This fusion process aims to add rich environmental context to the instruction, such as using location information as a weighting factor or the currently running application as a filtering condition. Finally, an enriched (enriched with context) comprehensive retrieval parameter is generated, containing semantic instructions, environmental status, and application context, providing a highly contextualized query basis for subsequent semantic text hybrid retrieval.

[0117] It should be noted that by incorporating the user's real-time location information and the current running application status into the instruction adjustment process, this method enables the generation of retrieval parameters to be deeply integrated with the specific environmental context and task status, thereby significantly enhancing the contextual awareness capability of semantic understanding, ensuring that the instruction response is more in line with the user's actual intention and operation scenario, and ultimately improving the accuracy and natural fluency of XR glasses interaction.

[0118] Furthermore, Figure 4 The following is a schematic diagram illustrating a search intent recognition method provided for one or more embodiments of this specification:

[0119] The first step is system preparation, including configuring the rule base and optimizing the index. First, a badcase rule base is built, designing a rule structure that includes trigger conditions (such as command keywords) and execution actions (such as filtering irrelevant applications). A visual interface is developed for operations personnel to configure, and rule interceptors are deployed and embedded in the Elasticsearch retrieval chain. Simultaneously, 30 days of user command logs are analyzed to identify high-priority intents (such as "I Cultural Tourism Guide"). The weights of relevant fields are adjusted via the Elasticsearch dynamic API (e.g., increasing the weight of "Application Type = I Cultural Tourism") and word segmentation strategies (e.g., retaining the complete keyword "3D Cinema"). Furthermore, performance optimization strategies are configured, including designing a hot command cache (using "command word + scene tag" as the key, combining local caching with Redis), splitting the index into shards by application type, and reducing the semantic calculation precision for non-core intents.

[0120] The second step involves real-time processing of user commands. After a user inputs a voice / text command (converted to text), rule matching is performed. A rule interceptor first determines if the command matches the triggering conditions in the rule base. If a match is successful, the search parameters are dynamically adjusted according to the rules (e.g., increasing field weights, filtering irrelevant applications). If a match fails, default parameters are used. Subsequently, a semantic-text hybrid search is executed, incorporating hotspot caching (high-frequency commands directly return cached results), routing to the corresponding application type index shard (reducing data scanning), and enabling lightweight semantic computation for non-core intents. Finally, an optimized command response is returned (e.g., launching the target application).

[0121] The third step is performance monitoring and iteration, which involves obtaining monitoring results and iterating accordingly. Real-time monitoring focuses on three aspects: badcase repair rate (target ≥90%), first-time match rate of high-priority intents (target ≥80%), response latency (target ≤200ms), and cache hit rate (target ≥60%). Based on the monitoring results, iterative optimization is performed, updating the rule base, index configuration, and performance strategies, and feeding back to the system initialization phase to form a continuous optimization loop.

[0122] It should be noted that the following is a schematic diagram of a search intent recognition method:

[0123] Step 1: Building and Deploying the Badcase Rule Base

[0124] Rule base design:

[0125] Rule structure: Includes "triggering conditions" (such as instruction keywords or intent words) and "execution actions".

[0126] (e.g., field weight adjustment, application filtering, priority matching application), "scope of application" (e.g., all scenarios, scenic area scenarios, daily scenarios);

[0127] Visual configuration interface: The developed web interface supports operators to input rules (such as selecting "Trigger condition: instruction contains '3D Cinema'", "Execute action: prioritize matching 'application name = 3D Cinema'"), and automatically generate rule scripts.

[0128] Rule activation mechanism:

[0129] Add a "rule interceptor" to the ES retrieval request chain: after the instruction request arrives, the interceptor first matches the triggering conditions in the rule base;

[0130] If a match is found, adjust the search parameters according to the rules (such as dynamically modifying the weight of the "Application Type" field and adding application filtering conditions), and then perform a semantic-text hybrid search.

[0131] Once the rules are modified, they are synchronized to the interceptor in real time without requiring a restart of the XR glasses service, ensuring immediate effect.

[0132] Step 2: Implementation of Specific Intent Index Optimization

[0133] High-priority intent recognition:

[0134] Analyze the command logs of XR glasses users over the past 30 days, and count the frequency of commands (e.g., the "Navigation Start" command accounts for 25%) and the importance of scenarios (e.g., the "I Cultural Tourism Guide" command in scenic spots affects user experience) to determine the top 5 high-priority intents.

[0135] Index adjustment configuration:

[0136] Field weights: In Elasticsearch's "dis_max" query (commonly used in mixed searches), set the boost parameter for relevant fields of high-priority intents (e.g., for the intent "I Cultural Tourism Guide", the "Application Type = I Cultural Tourism" field boost=2.0, the default field boost=1.0).

[0137] Word segmentation strategy: Add a custom dictionary (such as "OCR translation", "3D cinema", etc.) to the "analysis" configuration of the ES index to ensure complete matching of the command keywords;

[0138] Dynamic configuration changes: Weights and word segmentation strategies are updated in real time via the ES index's "update_settings" API, without the need to rebuild the index.

[0139] Step 3: Performance Optimization Configuration

[0140] Hotspot caching design:

[0141] Cache objects: Retrieval results of high-frequency commands (including application ID and response priority), with the cache key being "command word + scene tag" (to avoid conflicts between the same command in different scenes, such as "navigation" having different responses in scenic and urban scenes);

[0142] Caching strategy: Use local cache (XR glasses) + Redis (cloud) as the caching medium, set TTL (e.g., the high-frequency command "next attraction" in the scenic area has a TTL of 30 seconds, synchronized with the tour guide rhythm), and use the LRU (Least Recently Used) eviction policy when the cache is full.

[0143] Index sharding configuration:

[0144] Split the index by application type: Create independent indexes such as "I Culture and Tourism Index", "Navigation Index", and "3D Cinema Index", with each index corresponding to an independent segment;

[0145] Search routing: The application type is determined based on the command keywords (e.g., if the keyword "culture and tourism" is included, the search is routed to "I Culture and Tourism Index"). The search is only performed on the corresponding index, reducing the amount of data scanned.

[0146] Lightweight semantic computing:

[0147] For non-core intent commands (such as the low-frequency "close app" command), use "vector approximation matching" in semantic retrieval (e.g., reduce the "num_candidates" parameter in ES's "knn" query from 1000 to 500) to reduce computational complexity.

[0148] Step 4: Performance Monitoring and Iteration

[0149] badcase monitoring:

[0150] Collect XR glasses user feedback (such as "response error" voice feedback) and command logs in real time, mark new bad cases, and regularly calculate the rule base repair rate (such as the percentage of successfully repaired bad cases ≥ 90%).

[0151] Intent recognition rate monitoring:

[0152] For high-priority intents, sample 1000 instructions to test the response results and calculate the proportion of "first response matching the true intent". If it is less than 80%, adjust the index weight.

[0153] Performance monitoring:

[0154] Monitor command response time (target ≤200ms) and interaction processing volume in real time. If latency increases, check cache hit rate (target ≥60%) and shard retrieval efficiency, and optimize caching strategies or sharding rules.

[0155] Figure 5 This specification provides a schematic diagram of the structure of an application control device according to one or more embodiments, including: a receiving unit 501, a judging unit 502, an adjusting unit 503, a searching unit 504, and a control unit 505.

[0156] The receiving unit 501 receives semantic text instruction information for a specified application input by the user into the XR glasses;

[0157] The judgment unit 502 determines whether the instruction information matches the triggering conditions in the rule base through the rule interceptor;

[0158] If so, the adjustment unit 503 adjusts the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters;

[0159] The retrieval unit 504 performs a semantic-text hybrid retrieval based on the retrieval parameters and obtains an instruction response;

[0160] The control unit 505 controls the designated application based on the instruction response and displays the execution result of the designated application on the XR glasses.

[0161] Figure 6 A schematic diagram of the structure of an XR glasses provided for one or more embodiments of this specification includes:

[0162] At least one processor and bus; and,

[0163] A memory communicatively connected to the at least one processor; wherein,

[0164] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0165] Receive semantic text instructions for a specified application from the user inputting them into the XR glasses;

[0166] The rule interceptor determines whether the instruction information matches the triggering conditions in the rule base.

[0167] If so, adjust the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters;

[0168] Based on the search parameters, a semantic text hybrid search is performed to obtain the command response;

[0169] The specified application is controlled based on the instruction response, and the execution result of the specified application is displayed on the XR glasses.

[0170] This specification provides one or more embodiments of a non-volatile computer storage medium storing computer-executable instructions, which, when executed by a computer, can perform the following:

[0171] Receive semantic text instructions for a specified application from the user inputting them into the XR glasses;

[0172] The rule interceptor determines whether the instruction information matches the triggering conditions in the rule base.

[0173] If so, adjust the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters;

[0174] Based on the search parameters, a semantic text hybrid search is performed to obtain the command response;

[0175] The specified application is controlled based on the instruction response, and the execution result of the specified application is displayed on the XR glasses.

[0176] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0177] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0180] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0181] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The aforementioned units can be implemented in hardware or software.

[0182] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0183] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An application control method, characterized in that, The method includes: Receive semantic text instructions for a specified application from the user inputting them into the XR glasses; The rule interceptor determines whether the instruction information matches the triggering conditions in the rule base. If so, adjust the instruction information according to the adjustment rules associated with the triggering condition in the rule base to obtain the retrieval parameters; Based on the search parameters, a semantic text hybrid search is performed to obtain the command response; Based on the command response, the specified application is controlled, and the execution result of the specified application is displayed on the XR glasses; The method of controlling the designated application based on the instruction response, after the XR glasses display the execution result of the designated application, further includes: Monitor the execution parameters of the specified application based on the execution results; Update the rule base based on the execution parameters; The execution parameters include the application's status information, performance metrics, and user interaction data. The status information includes the running status and error logs. The performance metrics include response time and resource utilization. The user interaction data includes operation frequency, number of successes, and number of failures. Updating the rule base based on the execution parameters includes: The execution parameters are analyzed to evaluate the effectiveness of the triggering conditions and adjustment rules in the rule base in the specified application, and the analysis results are obtained. Based on the analysis results, modify or add triggering conditions and adjustment rules in the rule base.

2. The method according to claim 1, characterized in that, Before determining whether the retrieval parameters match the triggering conditions in the pre-built rule base using the rule interceptor, the method further includes: A rule base is built based on the pre-defined trigger conditions and adjustment rules corresponding to each instruction keyword.

3. The method according to claim 2, characterized in that, The adjustment rules include increasing the weight of instruction keywords; The step of adjusting the instruction information according to the adjustment rules associated with the triggering condition in the rule base to obtain the retrieval parameters includes: If the adjustment rule is to increase the weight of the instruction keywords, the weight of the instruction keywords in the instruction information is increased based on the preset weight increase value to obtain the corresponding first search parameters.

4. The method according to claim 2, characterized in that, The adjustment rules include filtering pre-defined irrelevant applications; The step of adjusting the instruction information according to the adjustment rules associated with the triggering condition in the rule base to obtain the retrieval parameters includes: If the adjustment rule is to filter pre-defined irrelevant applications, determine the pre-defined irrelevant applications in the instruction keywords of the instruction information; The specified irrelevant applications are filtered to obtain the corresponding second search parameters.

5. The method according to claim 1, characterized in that, The step of adjusting the instruction information according to the adjustment rules in the rule base to obtain the retrieval parameters includes: The instruction information is adjusted according to the adjustment rules in the rule base to obtain the adjustment parameters; Obtain the user's specified location information and currently running application information; Based on the adjustment parameters, the specified location information, and the currently running application information, the retrieval parameters are obtained.

6. An application control device, characterized in that, include: The receiving unit receives semantic text instructions for a specified application input by the user into the XR glasses; The judgment unit determines whether the instruction information matches the triggering conditions in the rule base through the rule interceptor; If so, the adjustment unit adjusts the instruction information according to the adjustment rules associated with the triggering condition in the rule base to obtain the retrieval parameters; The retrieval unit performs a semantic-text hybrid retrieval based on the retrieval parameters and obtains an instruction response; The control unit controls the designated application based on the instruction response and displays the execution result of the designated application on the XR glasses; The control of the designated application based on the instruction response, after the XR glasses display the execution result of the designated application, further includes: Monitor the execution parameters of the specified application based on the execution results; Update the rule base based on the execution parameters; The execution parameters include the application's status information, performance metrics, and user interaction data. The status information includes the running status and error logs. The performance metrics include response time and resource utilization. The user interaction data includes operation frequency, number of successes, and number of failures. Updating the rule base based on the execution parameters includes: The execution parameters are analyzed to evaluate the effectiveness of the triggering conditions and adjustment rules in the rule base in the specified application, and the analysis results are obtained. Based on the analysis results, modify or add triggering conditions and adjustment rules in the rule base.

7. An XR pair of glasses, characterized in that, include: At least one processor and bus; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Receive semantic text instructions for a specified application from the user inputting them into the XR glasses; The rule interceptor determines whether the instruction information matches the triggering conditions in the rule base. If so, adjust the instruction information according to the adjustment rules associated with the triggering condition in the rule base to obtain the retrieval parameters; Based on the search parameters, a semantic text hybrid search is performed to obtain the command response; Based on the command response, the specified application is controlled, and the execution result of the specified application is displayed on the XR glasses; The control of the designated application based on the instruction response, after the XR glasses display the execution result of the designated application, further includes: Monitor the execution parameters of the specified application based on the execution results; Update the rule base based on the execution parameters; The execution parameters include the application's status information, performance metrics, and user interaction data. The status information includes the running status and error logs. The performance metrics include response time and resource utilization. The user interaction data includes operation frequency, number of successes, and number of failures. Updating the rule base based on the execution parameters includes: The execution parameters are analyzed to evaluate the effectiveness of the triggering conditions and adjustment rules in the rule base in the specified application, and the analysis results are obtained. Based on the analysis results, modify or add triggering conditions and adjustment rules in the rule base.

8. A non-volatile computer storage medium, characterized in that, It stores computer-executable instructions, which, when executed by a computer, can achieve the following: Receive semantic text instructions for a specified application from the user inputting them into the XR glasses; The rule interceptor determines whether the instruction information matches the triggering conditions in the rule base. If so, adjust the instruction information according to the adjustment rules associated with the triggering condition in the rule base to obtain the retrieval parameters; Based on the search parameters, a semantic text hybrid search is performed to obtain the command response; Based on the command response, the specified application is controlled, and the execution result of the specified application is displayed on the XR glasses; The control of the designated application based on the instruction response, after the XR glasses display the execution result of the designated application, further includes: Monitor the execution parameters of the specified application based on the execution results; Update the rule base based on the execution parameters; The execution parameters include the application's status information, performance metrics, and user interaction data. The status information includes the running status and error logs. The performance metrics include response time and resource utilization. The user interaction data includes operation frequency, number of successes, and number of failures. Updating the rule base based on the execution parameters includes: The execution parameters are analyzed to evaluate the effectiveness of the triggering conditions and adjustment rules in the rule base in the specified application, and the analysis results are obtained. Based on the analysis results, modify or add triggering conditions and adjustment rules in the rule base.

Images