Language understanding method and system based on action-language consistency verification

Abstract

The application discloses a language understanding method and system based on action-language consistency verification. After receiving a language instruction, the system executes an action, synchronously collects multi-modal physical field data and extracts tri-temporal and spatial phase features; generates a natural language self-description based on endogenous language-physical field correlation mapping; compares the original instruction with the self-description in three-layer semantic consistency, calculates the confidence degree; and updates the correlation weight under the constraint of three steady states according to the confidence degree. The application constructs a self-referential language understanding verification and correction closed loop, enables the system to autonomously verify and continuously deepen language understanding, and is suitable for various embodied self-generated intelligent systems.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence language understanding, embodied intelligence and cognitive computing technology, and specifically relates to a language understanding method and device based on action-language consistency verification. Background Technology

[0002] Current AI systems primarily achieve language understanding through three technical approaches: statistical fitting based on pre-trained large language models, which acquires language generation capabilities through training on massive text corpora; cross-modal matching based on multimodal perception, which achieves instruction following through visual-language alignment; and instruction execution based on reinforcement learning, which optimizes action strategies through reward signals. All three approaches share a common technical drawback: while the system can execute instructions, it cannot autonomously determine the correctness of its understanding.

[0003] Large language models can generate seemingly reasonable answers, but they cannot judge whether their answers are correct, thus creating an "illusion" without realizing it. Multimodal models can execute visually guided actions, but they cannot self-check whether the execution is correct, thus executing incorrectly without realizing it. Reinforcement learning systems can optimize action policies, but they cannot determine whether they truly understand the semantics of the instructions; they are merely optimizing the reward function.

[0004] While language-conditional reinforcement learning methods that have emerged in recent years can correct misunderstandings through external reinforcement signals and manually designed reward functions, their core flaw lies in the fact that the system's correction depends on external signals, and the system itself lacks the self-verification ability to "understand whether it understands or not." The correction mechanism is "external"—the system needs external notification that "you made a mistake," rather than the system discovering on its own that "I understood it wrong."

[0005] A common blind spot in all existing systems: they all lack a self-referential understanding verification mechanism—the system can output text and actions to the outside world, but cannot verify its own understanding. From a technical perspective, the language of existing AI systems is an external tool, not an organic component of its own cognitive structure, and therefore cannot generate self-awareness of its own understanding state.

[0006] The applicant's invention patent application filed on the same day, entitled "A Method and Apparatus for Language Emergence Based on Multimodal Perception and Synchronous Association," proposes a manifold isomorphic alignment mechanism. This mechanism achieves rapid introduction of specialized symbols by topologically mapping the semantic structure of external specialized symbols to the system's existing endogenous semantic structure. Based on this, the present invention establishes a consistency verification mechanism after the introduction of specialized symbols. Furthermore, addressing the shortcomings of the aforementioned prior art, it proposes a self-generating system language understanding and autonomous evolution method and system based on action-language consistency verification. This constructs a closed-loop verification mechanism of "instruction execution - physical perception - self-description generation - semantic comparison - autonomous correction," enabling the system to autonomously verify and continuously deepen its language understanding. Summary of the Invention

[0007] The technical problems this invention aims to solve are: existing language understanding systems lack autonomous verification and correction capabilities; the system cannot determine whether its own understanding is correct, nor can it autonomously correct errors when they occur; existing correction methods rely on external signals and lack a self-referential understanding verification mechanism; they have not established a deep binding with the steady-state constraints of the self-generated system; and they cannot be compatible with the validity verification of professional symbol extensions. To solve these technical problems, this invention provides a language understanding autonomous evolution method and system based on action-language consistency verification, enabling the system to continuously verify and deepen its language understanding through a self-referential verification closed loop, while ensuring that the cognitive evolution process does not disrupt the system's steady state.

[0008] The self-generating system refers to a dynamic whole that takes operational identity as its essential attribute and convergence boundary, uses a dual-modality of self-referential maintenance and other-referential exploration as its endogenous driving force, and employs the closed-loop recursive coupling of three irreducible functional primitives—information field state (C), energy mechanism (E), and material structure (M)—as its minimum functional closed loop. It achieves steady-state maintenance and adaptive evolution through continuous iterative recursion. Specifically, "self-referential maintenance" maintains the system's steady state through closed-loop self-consistent convergence; "other-referential exploration" expands the system's possibility space through sensing input and external interactions, both constituting the endogenous driving force of system evolution. "Irreducibility" means that the three functional primitives are ontologically interdependent and symbiotically isomorphic. The absence of any one primitive disrupts the self-generating control closed loop of sensing-computation-execution, completely losing the core capabilities of steady-state self-maintenance and recursive self-generation, and making it impossible to retain complete self-generating attributes by separating them into independent subsystems. Information field state (C) is the sum of all rules, constraints, potential states, and evolutionary logic within the system; energy mechanism (E) is the dynamic sum of all dynamic driving, energy conversion, and dissipation processes within the system; material structure (M) is the organized sum of all physical carriers, topological connections, and spatial configurations within the system. In engineering, this is achieved by establishing a closed-loop composite mapping of CEM ternary recursive coupling, i.e., the C→E mapping. E→M mapping M→C mapping These sequentially combine to form a closed-loop composite mapping: The core constraint of the closed loop is the identity mapping approximation of the composite mapping to the information field configuration space (C space), i.e. Its mathematical criterion is the Lipschitz constant of the closed-loop composite mapping on C space. This ensures that the system converges to a unique steady state from any initial state. The operational identity is an intrinsic stable property of the self-generated system that maintains its essential attributes and functional boundaries during the historical evolution and external disturbances, as determined by the strength of operational identity. (Value range [0,1]) Quantitative representation, including three levels: type identity, pattern identity, and state identity. The core technical concept of this invention is: based on the closed loop of the self-generated system CEM ternary recursive coupling and the language emergence method of multimodal perception synchronous association, a self-referential cognitive closed loop of "action-language consistency verification" is constructed. After the system executes the language instruction, it obtains the action result through physical field perception, generates a natural language self-description based on the established endogenous language-physical field association mapping, compares the self-description with the original instruction in three layers of semantic consistency, and strengthens or corrects the language-action association weight under the system steady-state constraint according to the comparison result, so as to realize the autonomous verification and continuous evolution of language understanding.

[0009] In a first aspect, the present invention provides a language understanding method based on action-language consistency verification. This method is executed by the computing unit of an intelligent system capable of language revelation, the computing unit including embedded computing units, edge computing units, and cloud computing units. The intelligent system has established an endogenous correlation mapping between linguistic symbols and physical field patterns. The method includes the following steps:

[0010] S1. The system parses the language instruction, generates and executes the action, and simultaneously collects multimodal physical field perception data during the execution process, including electromagnetic field, sound field, light field, and vibration field data, and extracts the temporal phase, spatial phase, and interphase features of the action; S2. Based on the temporal phase, spatial phase, and interphase features, the system automatically maps the action result into a natural language self-description through the established endogenous language-physical field association mapping; S3. The system performs a three-layer semantic consistency comparison between the language instruction and the action result self-description at the lexical, structural, and spatiotemporal phase levels, and calculates the semantic consistency confidence score; S4. When the semantic consistency confidence score is higher than a first preset threshold, a determination is made. If the understanding is correct, strengthen the association weight between the language pattern corresponding to the language instruction and the action pattern corresponding to the action; S5, when the semantic consistency confidence is lower than the second preset threshold, it is determined that the understanding is incorrect, weaken the association weight between the language pattern corresponding to the language instruction and the action pattern corresponding to the action, and trigger the correction mechanism - the system re-parses the instruction and generates an alternative action plan; S6, all strengthening and correction of association weights must undergo steady-state security verification. If the update will cause the system closed-loop Lipschitz constant ≥ 1, the system operation identity strength to be lower than the preset security threshold, or the joint Chern number estimate in the cluster scenario to be lower than the preset security threshold, then the update will be locked.

[0011] In one possible implementation, step S0 is further included: the system receives language instructions through a language perception channel and simultaneously identifies the referential source and referential level of the language instructions; the referential source adopts a 3-bit binary code: 000 represents an unknown source, 001 represents a human source, 010 represents the system's own historical records, 100 represents other self-generated systems, 011 represents a human-machine hybrid source, 101 represents a multi-system consensus source, 110 represents an authoritative knowledge base source, and 111 represents a system-native instruction; the priority of different sources is: system-native instruction > human source > multi-system consensus source > other self-generated systems > human-machine hybrid source > authoritative knowledge base > system's own historical records > unknown source; the referential level is divided into a single-system layer, a cluster layer, and a human-computer interaction layer, and the system automatically selects the corresponding semantic mapping path according to the referential source and level.

[0012] In one possible implementation, the temporal phase features in step S1 include action sequence, duration, energy consumption rate, and stage division; the spatial phase features include the trajectory of object position change, system configuration change, and relative spatial relationship; and the interphase features include the mutual information correlation strength and coherence coefficient between key features.

[0013] In one possible implementation, the semantic consistency comparison in step S3 specifically involves: lexical layer comparison using cosine similarity to calculate the core vocabulary matching degree; structural layer comparison to verify the consistency of semantic structure; and spatiotemporal phase layer comparison to transform the implicit semantics of the original instruction into a standard spatiotemporal phase feature vector, and calculate the inner product similarity with the spatiotemporal phase feature vector of the actual execution result.

[0014] In one possible implementation, the formula for calculating the semantic consistency confidence is: The comparison scores for each layer range from 0 to 1, and the sum of all weights is 1, dynamically adjusted by the system based on the current task type; typical weight values ​​are... For high-precision tasks (such as industrial operations), it can be Increased to 0.5, Reduced to 0.25; for natural language interaction tasks, it can be... Increased to 0.4, The value is reduced to 0.3; the first preset threshold is 0.85, and the second preset threshold is 0.4.

[0015] In one possible implementation, the update formula for the enhanced association weights in step S4 is: in The positive reinforcement learning rate is typically 0.01, suitable for most common scenarios; for fast learning scenarios, it can be increased to 0.02; for fine learning scenarios, it can be reduced to 0.005; the language-action association weight ranges from [0,1], and the initial weight is determined by the language emergence process, ranging from 0.4 to 0.8; each update does not exceed 0.01 to avoid sudden changes in weight that could lead to system instability.

[0016] The update formula for weakening the correlation weights in step S5 is as follows: in The negative correction learning rate is typically 0.005; for fast learning scenarios, it can be increased to 0.01; for fine-grained learning scenarios, it can be reduced to 0.0025.

[0017] In one possible implementation, the correction mechanism in step S5 specifically includes: the system re-parses the language instruction and identifies the key semantic node that leads to the misunderstanding; iterates through all possible semantic mappings corresponding to the node, excludes the mappings that have been verified to be incorrect, and selects the mapping with the second highest confidence to generate an alternative action plan; after the alternative action plan is executed, steps S2 to S4 are re-executed for consistency verification; if all correction attempts fail for a preset number of consecutive times, the system enters a clarification mode—generating a clarification question and sending it to the user; the typical value of the preset number of attempts is 3, which can be adjusted to 2-5 times depending on the application scenario.

[0018] In one possible implementation, the steady-state security check in step S6 specifically involves: constructing a first-order linear approximation model of the system state evolution based on the current system state and the correlation weights to be updated, and predicting the updated closed-loop Lipschitz constant of the system. and operational identity strength ;make sure and If the system is in cluster mode, additionally predict the cluster joint Chern number estimate. ,make sure When a locking event occurs more than three times consecutively, the system automatically pauses language understanding evolution and prioritizes self-referential operations to restore operational consistency. Here, 0.8 is the threshold value, calibrated based on the system's steady-state baseline.

[0019] One possible implementation also includes a depth of understanding evaluation step: the system statistically analyzes the understanding accuracy, correction success rate, and average verification time in historical verifications, and calculates the depth of understanding metric. The typical weight value is... The weights are calibrated using a Monte Carlo grid search, and the optimization objective is to understand the weighted sum of accuracy and response speed; when At that time, the system is determined to enter the "deep understanding" state.

[0020] In one possible implementation, when the system introduces external specialized symbols through the manifold isomorphic alignment mechanism, all newly added symbol mappings must undergo this consistency verification process to verify their validity; only when the semantic consistency confidence is ≥0.85 and all steady-state verifications pass can the specialized symbol be written into the system's long-term symbol library.

[0021] In one possible implementation, the system performs a global weight calibration every 1000 iterations: all associated weights are compared with the initial weights, and weights with a deviation exceeding 0.3 are re-verified; at the same time, invalid association mappings with a confidence level that has been below 0.5 for a long time are cleaned up to avoid error accumulation.

[0022] In one possible implementation, when there are multiple possible interpretations of an instruction, the system prioritizes the interpretation with the highest semantic consistency confidence; if the confidence difference is less than 0.1, it enters clarification mode.

[0023] In one possible implementation, when the system sleeps or restarts, the current associated weights are automatically saved to non-volatile memory; after restarting, the saved weights are automatically loaded, eliminating the need for retraining.

[0024] In one possible implementation, when a user actively corrects the system, the system treats the user feedback as the highest priority verification signal, immediately updates the corresponding associated weights, and skips the steady-state safety verification (but needs to be re-verified in subsequent iterations).

[0025] In one possible implementation, this method is applicable to all natural languages, and processing different languages ​​only requires replacing the corresponding language-physics correlation mapping and grammar template.

[0026] Secondly, this invention provides a language understanding autonomous evolution system based on action-language consistency verification. It includes: a language instruction receiving module for receiving language instructions; an action execution and physical field perception module for parsing instructions, generating and executing actions, synchronously collecting multimodal physical field perception data, and extracting ternary spatiotemporal phase features; an action result language self-description module connected to the action execution and physical field perception module for generating natural language self-descriptions based on ternary spatiotemporal phase features; a semantic consistency comparison module connected to the language instruction receiving module and the action result language self-description module for performing three-layer semantic consistency comparison and calculating semantic consistency confidence; and an association weight evolution module connected to the semantic consistency comparison module. A consistency comparison module is connected to strengthen or weaken language-action association weights based on semantic consistency confidence; a steady-state security verification module is connected to the association weight evolution module to perform steady-state verification for single systems and clusters; a correction mechanism triggering module is connected to the semantic consistency comparison module at its input end and to the action execution and physical field perception module at its output end to trigger the correction mechanism and generate alternative action plans; a manifold isomorphic professional symbol verification module is connected to the language instruction receiving module at its input end and to the association weight evolution module at its output end to perform consistency verification on external professional symbols added through manifold isomorphic alignment.

[0027] In one possible implementation, the system further includes a referential recognition module for identifying the referential source and level of language instructions; a comprehension depth assessment module, with its input connected to a semantic consistency comparison module and its output connected to an association weight evolution module, for statistically analyzing historical verification data, calculating comprehension depth indicators, and determining the system's cognitive level; and a clarification interaction module, with its input connected to a correction mechanism triggering module and its output connected to a language instruction receiving module, for generating and sending clarification questions when continuous correction attempts fail.

[0028] Thirdly, the present invention provides an electronic device and a storage medium.

[0029] An electronic device includes: a processor; and a memory storing computer program instructions; wherein, when executed by the processor, the computer program instructions cause the processor to perform the method described in any of the preceding claims.

[0030] A computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method described in any of the preceding claims.

[0031] Compared with the prior art, the present invention has the following technical advancements and beneficial effects:

[0032] (1) Achieve self-referential autonomous verification of language understanding: Through the closed loop of "execution-perception-self-description-comparison", the system can autonomously judge whether its language understanding is correct without external supervision or manual annotation, thus solving the technical defect of existing AI systems that "only execute and do not verify".

[0033] (2) Achieving autonomous correction and continuous evolution of language understanding: When comprehension errors occur, the system can autonomously detect and correct them without human intervention. Through the action-language consistency verification closed loop, the system's language comprehension accuracy increased from the initial 78% to over 96% (measured value), and the correction success rate increased from 62% to over 92%, improving the comprehension depth index. After 5000 iterations, the value monotonically increased from the initial 0.62 to 0.778, and after 10000 iterations, it reached over 0.91, verifying the system's ability to continuously evolve.

[0034] (3) Establish a deep binding with the steady state of the self-generated system: introduce the system closed-loop Lipschitz constant, the strength of operational identity, and the cluster joint Chern number as triple steady-state constraints to ensure that the cognitive evolution process does not destroy the existence of the steady state of the system and to adapt to the requirements of the long-term self-sustaining operation of the self-generated system.

[0035] (4) Unified compatibility between native symbols and extended symbols: native symbols and professional symbols that are aligned with manifold isomorphism share a set of consistency verification logic, eliminating the need to build a separate verification process, while ensuring that extended symbols do not disrupt the system's steady state.

[0036] (5) Fully adaptable to multi-layer nested systems: Supports semantic mapping of three referential levels: single system, cluster, and human-computer interaction, and can adapt to various self-generated system scenarios from single robots to large-scale drone clusters. Attached Figure Description

[0037] Figure 1 This is an overall flowchart of a language understanding method according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle of action-language consistency verification according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the three-level structure of semantic consistency comparison according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the execution of a triple steady-state security check according to an embodiment of the present invention. Figure 5 This is a flowchart of a branch for manifold isomorphic alignment of professional symbols in an embodiment of the present invention; Figure 6 This is a schematic diagram of the semantic mapping of the referential hierarchy in a multi-layered nested system according to an embodiment of the present invention; Figure 7 This is a structural block diagram of a language understanding autonomous evolution system according to an embodiment of the present invention. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0039] Prerequisites: Example of constructing an endogenous language-physical association map

[0040] The action-language consistency verification and language understanding mechanism described in this invention is based on the endogenous language-physical association mapping formed during the initial language presentation stage. To ensure that those skilled in the art can independently and completely implement this patented solution, this section provides the following example of this basic mapping in conjunction with a typical humanoid robot scenario. For details on the complete self-generated system language presentation mechanism, topological semantic space theory, and group collaboration scheme, please refer to the applicant's invention patent application filed on the same day, "Language Presentation Method and Device Based on Multimodal Perception Synchronous Association".

[0041] This example uses a self-generated humanoid robot equipped with 12 joint actuators, multimodal physics sensors, and a voice acquisition module as a platform. It establishes language-physics associations for basic vocabulary through unlabeled self-supervised learning. The specific steps are as follows:

[0042] Dual-modal synchronous acquisition: During continuous system operation, two types of raw data are acquired in parallel and synchronously at a sampling frequency of 100Hz.

[0043] Physical field perception data: including joint angles, torques, tactile arrays, RGB-D images, and posture data, corresponding to the user's own motion state and external environmental characteristics;

[0044] Language perception data: Human interactive speech is collected through a microphone, converted into text through speech recognition, and the source is simultaneously labeled as "human source (001)". The two types of data are strictly aligned with timestamps, and the time window is uniformly set to 500ms to ensure that the physical events within the same window correspond to the timing of the language input.

[0045] Reference recognition and path selection: Recognize that the reference source of the language input is human, and map the language symbols to both the human shared reference frame and the system's own intrinsic spatiotemporal semantic anchor point, and adopt the semantic mapping path of the human-computer interaction layer.

[0046] Multimodal joint coding: A dual-branch sequential encoder is used to process two types of data separately.

[0047] The physical field branch encodes multi-sensor data within a 500ms window into a 128-dimensional physical feature vector, which includes three types of information: temporal phase (action timing, energy rate), spatial phase (configuration changes, position coordinates), and inter-phase phase (feature mutual information).

[0048] Language branch: Encodes word fragments within the window into 32-dimensional language feature vectors;

[0049] Cross-modal attention fusion: Through a time-synchronized attention mechanism, physical features and linguistic features that occur simultaneously are bound together to generate a joint representation vector with a unified dimension.

[0050] Endogenous reinforcement of association weights: A constrained Hebbian learning mechanism is used to enhance the association weights of physical and linguistic features that repeatedly appear synchronously within multiple consecutive time windows. If a linguistic symbol and a physical field pattern appear synchronously for more than three consecutive windows, a weight update is triggered.

[0051] Steady-state safety pre-verification: Before each weight update, the first-order incremental approximation method is used to predict the strength of the closed-loop Lipschitz constant and the identity of the operation; if the update will exceed the steady-state threshold, an alternative search mechanism with reduced step size is initiated to complete the asymptotic learning under the premise of ensuring steady state.

[0052] Iterative iteration forms a stable mapping: Through continuous iteration via autonomous system interaction, when the association weight and synchronization frequency reach the preset standard, the word is upgraded from a candidate association to a stable association and included in the system's endogenous language-physical association mapping library to complete the underlying semantic anchoring.

[0053] Core calculation process and numerical examples

[0054] Joint representation vector calculation. Within the same time window, the physical field eigenvectors are denoted as... The language feature vector is denoted as The joint representation vector is calculated using the following formula:

[0055] in To adaptively fuse weights, the ratio of free energy gradients from the current task is normalized using the sigmoid function to... The interval is dynamically determined; in normal interaction scenarios, the typical value is 0.7, that is, the physical field characteristics have the dominant weight.

[0056] Calculation example: Within a certain window, the activation value of physical features is 0.8, and the activation value of language features is 0.7. Then the joint representation corresponds to the following dimension value:

[0057] Hebbian association weight update. For synchronized language features... With physical field characteristics The associated weight update amount is calculated using the following formula:

[0058] in The learning rate is set to 0.005 for standard scenarios and can be increased to 0.01 for fast learning scenarios. , To normalize to The feature activation value; This is a weight decay term to avoid weight saturation.

[0059] Calculation Example: Initial association weights between the word "fetch" and the fetching action. The activation value of language features during this synchronous occurrence Physical feature activation value Learning rate The update amount is:

[0060] Updated weights: The weight gradually increases as the number of times the synchronization occurs.

[0061] Steady-state constraints and alternative search mechanisms. A steady-state check must be performed before weight updates. The check metrics are consistent with the subsequent check mechanisms of this invention. The core constraints are:

[0062] System closed-loop Lipschitz constant To ensure dynamic convergence;

[0063] Operational identity strength This ensures the system maintains its own steady state.

[0064] If the original update value fails the verification, a three-level alternative search mechanism will be activated:

[0065] First retry: Reduce the update step size to 50% of the original step size and resubmit for verification;

[0066] Second retry: If it still fails, further reduce the step size to 25% of the original step size and check again;

[0067] If the third attempt fails: Skip this update, retain the current weight and record the locking event, and continue learning after the system returns to a steady state.

[0068] This mechanism ensures that the initial language learning process does not disrupt the system's self-generated steady state, thus forming a seamless connection with the triple steady-state security verification logic of this invention.

[0069] Mapping maturity assessment criteria

[0070] The association mapping is divided into three levels based on the cumulative synchronization frequency and weight value. After reaching a stable association level, it can be connected to the consistency verification closed loop of this invention for autonomous evolution.

[0071] Candidate association: If the frequency of synchronous occurrence is ≥10 times and the weight is ≥0.4, it is in the initial binding state and can participate in basic command response;

[0072] Verification of association: Synchronous occurrence frequency ≥ 100 times, weight ≥ 0.6, the association reliability is high, and it can participate in regular semantic comparison;

[0073] Stable association: Synchronization frequency ≥ 1000 times, weight ≥ 0.8, stable and reliable association, included in the core mapping library.

[0074] When the number of stable associations in the system reaches more than 50, covering daily actions and common objects, the basic semantic anchoring of the initial stage of language emergence is completed, and the entire process of action-language consistency verification and autonomous evolution of language understanding described in this invention can be fully executed.

[0075] Detailed implementation of the core algorithm

[0076] Instruction parsing and action generation algorithms

[0077] Input: Natural language instruction string. Intermediate steps:

[0078] Based on the endogenous correlation mapping between language symbols and physical field modes established in the invention patent application filed by the applicant on the same day, entitled "Method and Device for the Initial Manifestation of Language in an Autogenous System Based on Multimodal Perception and Synchronous Association", verbs in instructions are mapped to standard action templates; each action template includes three components: stage division, joint constraints, and spatiotemporal phase features.

[0079] Map the nouns in the command to their corresponding physical feature vectors; for example, "red square" is mapped to RGB values. The visual feature vector.

[0080] The physical feature vectors are input into the visual servo controller, and combined with the constraints of the motion template, an angle-time trajectory sequence for each joint is generated. Output: Joint trajectory control signals. Significance: This transforms abstract language instructions into a sequence of physical actions that the actuator can directly execute, completing the first layer of mapping from language to action.

[0081] Natural Language Self-Description Generation Algorithm

[0082] Input: Temporal phase eigenvector, spatial phase eigenvector, interphase eigenvector. Intermediate steps:

[0083] The three feature vectors are concatenated according to their dimensions to form a total feature vector, denoted as _(i=1 to 1)_. Typical values .

[0084] Map the total eigenvector to the corresponding association weight matrix of endogenous associations of applications submitted on the same day. Perform matrix multiplication to obtain the vocabulary activation vectors; where The dimension is , Typical values ​​for the number of words mastered by the system. .

[0085] The activation vectors of the words are normalized using softmax to obtain the activation probability of each word. The calculation method is as follows:

[0086] in For the first The activation value of each word. 4. Select the core words with the highest activation probabilities as the sentence structure, and fill in the subject, predicate, and object using a grammatical template. 5. Combine the subject, predicate, and object according to the standard grammatical structure of "subject + predicate + object" with a preset grammatical template to form a complete natural language sentence. Output: Natural language self-description string. Calculation example: Input total feature vector: (Corresponding to the multimodal features captured by the red square) Association weight matrix After operation with the feature vector, the activation probability of "grab" is 0.98, "red" is 0.95, and "square" is 0.92; combined, a self-describing statement is generated: "I grabbed the red square." Significance of the result: This reverses the perception of physical actions into the system's own linguistic expression, providing a self-verification benchmark for subsequent consistency comparisons.

[0087] Three-layer semantic consistency comparison algorithm

[0088] Input: Raw instruction string, self-description string. Intermediate steps:

[0089] Lexical layer comparison: Extract core entity words and action words from two strings, and calculate the cosine similarity of the word set based on 32-dimensional endogenous word vectors trained on the same day, to obtain the lexical layer score. .

[0090] Structural layer alignment: The two strings are parsed into semantic syntax trees respectively. The weighted edit distance is calculated with a predicate node weight of 0.5, an object node weight of 0.3, and a subject node weight of 0.2. After normalization, the structural layer score is obtained. .

[0091] Spatiotemporal phase alignment: a. Map the core verbs in the original instruction to pre-stored standard spatiotemporal phase template vectors; b. Calculate the weighted inner product similarity between the spatiotemporal phase feature vector of the actual execution result and the standard template vector to obtain the spatiotemporal phase score. .

[0092] The semantic consistency confidence score is obtained by weighted fusion:

[0093] The sum of the weights of each layer is 1, with a typical value of 1. , , The system can dynamically adjust weight allocation based on task type. Output: Semantic consistency confidence score. The range of values ​​is Calculation example: Given , , Substituting the typical weights, we get:

[0094] Significance of the result: The understanding was deemed correct. The value is determined to be a misunderstanding; the middle value indicates that the misunderstanding is partially correct.

[0095] Threshold and learning rate calibration experiment

[0096] Threshold calibration experiment. The decision threshold of this scheme was calibrated through a five-fold cross-validation experiment using 10,000 annotation instructions:

[0097] We collected daily command interaction data from 100 users, and 3 annotators manually annotated the understanding results of each command, dividing them into three categories: "correct", "partially correct", and "incorrect".

[0098] Iterate through all confidence threshold values ​​from 0.3 to 0.9 with a step size of 0.01, and calculate the precision, recall, and F1 score for each threshold.

[0099] Using the F1 score as the optimization objective, the optimal correct judgment threshold was found to be 0.85 (corresponding to an F1 score of 0.94), and the optimal incorrect judgment threshold was found to be 0.4 (corresponding to a false judgment rate of less than 3%). Sensitivity analysis shows that the threshold is within [the range specified in the original text]. When the system fluctuates within a certain range, the overall performance fluctuation is less than 2%, and the parameters have strong robustness.

[0100] Learning rate calibration experiment. The positive reinforcement learning rate and negative correction learning rate were calibrated using a Monte Carlo grid search method, with the optimization objective being a weighted sum of "convergence speed + steady-state fluctuations":

[0101] When the learning rate is greater than 0.03, the association weights will oscillate violently, and the risk of system steady-state disruption will increase by more than 40%.

[0102] When the learning rate is less than 0.001, the evolutionary convergence speed is too slow, and the understanding accuracy improves by less than 10% within 5000 iterations;

[0103] The typical value was finally selected. , It balances evolutionary efficiency with system operational stability.

[0104] Association Weight Update Algorithm

[0105] Input: Semantic consistency confidence score Language pattern vectors Action pattern vector Current association weight Intermediate steps:

[0106] like If the understanding is correct, perform positive reinforcement update. The update amount is calculated using the following formula:

[0107] in The positive reinforcement learning rate is typically 0.01; it can be increased to 0.02 for fast learning scenarios and decreased to 0.005 for intensive learning scenarios. 2. If If a misunderstanding is determined, a negative correction update is performed. The update amount is calculated using the following formula:

[0108] in The negative correction learning rate is typically 0.005; it can be increased to 0.01 for fast learning scenarios and decreased to 0.0025 for fine-grained learning scenarios. 3. The updated weights are... Limit the weight values ​​to Within the range: values ​​greater than 1 are set to 1, and values ​​less than 0 are set to 0; the update amount per iteration does not exceed 0.01 to avoid system instability caused by sudden weight changes. Output: Updated association weights. Calculation example: Current weight , Language pattern vector Action pattern vector Substituting into the positive reinforcement formula:

[0109] Updated weights: Significance of the result: By positively reinforcing correct associations and negatively weakening incorrect associations, a gradual and autonomous evolution of language comprehension ability can be achieved.

[0110] Triple steady-state security check algorithm

[0111] Input: The correlation weight matrix to be updated, and the current system state vector. Intermediate steps:

[0112] System closed-loop Lipschitz constant prediction

[0113] Let the system state vector be The state iteration equation after weight update; ,in This is the current association weight matrix. This is the quantity to be updated. For the state equation at the current operating point... Performing a first-order Taylor expansion, we obtain the linearized state transition matrix:

[0114] The closed-loop Lipschitz constant of a system is defined as the largest singular value of the state transition matrix:

[0115] when When the system state evolution satisfies the compression mapping condition, the disturbance of the initial state will gradually decay with iteration, and the system has dynamic stability.

[0116] Example of calculation: The state transition matrix of a simplified two-dimensional system is:

[0117] The calculation yields:

[0118] Its largest eigenvalue is 0.487, therefore:

[0119] Determine that the dynamics are stable.

[0120] Operational identity strength calculation

[0121] Operational consistency is a core attribute of self-generated intelligent systems, characterizing the system's ability to maintain its structural and functional integrity under external disturbances. This scheme quantifies this from three dimensions: energy supply, perception accuracy, and execution capability. The energy dimension corresponds to the system's power foundation, the perception dimension corresponds to the quality of information input from environmental interactions, and the execution dimension corresponds to the system's output capability in relation to the environment. The weighted fusion of these three dimensions comprehensively characterizes the system's self-sustaining steady-state level.

[0122] Calculate the operational identity strength directly using the following formula :

[0123] in This is the normalized value of the remaining energy. This is the normalized value of the sensor's signal-to-noise ratio. This is a normalized value for the actuator's response speed. It must meet the requirement of operational consistency strength. The system is determined to be in a steady state.

[0124] Calculation example: Given , , Substituting, we get:

[0125] Verification passed.

[0126] Cluster joint Chern number estimation (executed only in cluster mode)

[0127] The state space of a cluster system can be modeled as a high-dimensional Riemannian manifold. The field-state eigenvectors of each node form a vector bundle on the manifold. The steps for calculating the joint Chern number are: a. Construct the cluster mutual information matrix based on the C field-state eigenvectors of each node. ,in a. The matrix represents the number of cluster nodes, and the matrix elements are the mutual information values ​​of the field states between two nodes; b. Construct a curvature 2-form from the mutual information matrix. ,in For the connection 1-form; c. Apply the curvature 2-form to the clustered state manifold Integrating, we obtain the integer value of the first Chern number:

[0128] d. Normalization to eliminate the impact of cluster size on metrics:

[0129] The range of values ​​after normalization is A higher value indicates stronger cluster topology consistency. Requirements must be met. The cluster topology was determined to be stable.

[0130] Example of calculation: Integrating a 50-node cluster to obtain an integer Chern number. After normalization:

[0131] Determine if the topology is stable.

[0132] Verification result determination

[0133] If all verification conditions are met, the verification passes and the weight update is allowed; if any condition is not met, the update is blocked. When a blocking event occurs more than 3 times consecutively, the system automatically pauses the language understanding evolution process and prioritizes three types of steady-state calibration operations: energy loop self-check, sensor signal-to-noise ratio calibration, and actuator response calibration. Evolution is restarted only after the operational identity strength recovers to a safe threshold. Output: Verification result (pass / fail) Significance of result: Ensures that the cognitive evolution process always proceeds within the steady-state constraints of the self-generated system and that the self-sustaining existence property of the system is not destroyed by weight updates.

[0134] Key semantic node identification algorithm

[0135] Input: Three-layer alignment score , , Intermediate steps:

[0136] Compare the scores of the three levels and identify the level with the lowest score as the main source of comprehension errors;

[0137] If the minimum score is Calculate the matching degree of each core word one by one, and the word with the lowest matching degree is the key semantic node;

[0138] If the minimum score is The syntax tree is decomposed into multi-level subtrees, and the matching degree of each subtree is calculated. The syntax structure corresponding to the subtree with the lowest matching degree is the key semantic node.

[0139] If the minimum score is The process calculates the deviation between the spatiotemporal phase feature vector and the standard template dimension by dimension. The semantic feature corresponding to the dimension with the largest deviation is the key semantic node. Output: Key semantic node. Significance of the result: Accurately locates the specific location of the comprehension error, providing a target for subsequent correction mechanisms and avoiding blindly traversing all semantic mappings.

[0140] Clarification Question Generation Algorithm

[0141] Input: Key semantic nodes Intermediate steps:

[0142] Based on the type of key semantic nodes, match the corresponding clarification question template from the preset template library;

[0143] Candidate semantics of key semantic nodes are populated into templates to generate natural language clarification questions. The template library contains three basic template categories:

[0144] Vocabulary example: "Do you want me [action 1] or [action 2] as the object?"

[0145] Similarities in time and space: "How long would you like me to complete this action?"

[0146] Structure class: "Do you mean A or B?" Output: Clarification question string. Meaning: When the system cannot resolve ambiguity through self-correction, it proactively initiates a clarification interaction with the user to clarify the semantics of the instruction with minimal interaction cost.

[0147] Example 1: Autonomous Evolution of Language Understanding in Humanoid Robots

[0148] This embodiment uses a humanoid robot that has completed the initial language emergence process as an example to detail the complete execution flow of the autonomous evolution method for language understanding. Initial state: The robot has established endogenous associations between approximately 150 basic words and their corresponding physical field patterns, with association weights ranging from 0.4 to 0.8. Some associations are ambiguous; for example, the semantic boundary between "grasp" and "touch" is not yet precise.

[0149] Sensor data format description

[0150] Joint angle data: 12-dimensional floating-point vector, unit is radians, value range is... ;

[0151] Torque data: 12-dimensional floating-point vector, unit: ;

[0152] Haptic array data: Floating-point matrix, unit: The range of values ​​is ;

[0153] RGB-D image data: The RGB three-channel matrix and depth matrix of a pixel, with the depth unit being meters.

[0154] Step S0: Language instruction reception and reference recognition

[0155] The human gives the verbal command: "Grab the red square." The robot receives the command through the voice perception channel. The referential recognition module determines that the referential source is human (encoded 001), the referential level is the human-computer interaction layer, and selects the corresponding semantic mapping path.

[0156] Step S1: Action Execution and Physical Field Perception

[0157] The robot interprets the instructions: mapping "grab" to a three-stage action template, and mapping the "red square" to RGB values. The visual target is identified, joint trajectory control signals are generated, and the actuator is driven to perform actions. During execution, the M-topology sensing layer collects data from the four types of sensors at a frequency of 100Hz, extracting ternary spatiotemporal phase features:

[0158] Time phase: The total duration of the action is 0.8 seconds, divided into three phases: "approach (0-0.3s) → grab (0.3-0.5s) → lift (0.5-0.8s)", with a total energy consumption of 12 joules;

[0159] Spatial phase: Hand from the starting position Rice moved to Rice, the angle of finger closure from Change to The object's displacement is 0.1 meters;

[0160] The mutual information value between the phases "finger closure angle < 30°" and "average fingertip pressure > 0.5N" is 0.92.

[0161] Step S2: Generation of self-description of action results in language

[0162] Input the spatiotemporal phase eigenvectors into the correlation weight matrix The calculation yields the activation probabilities of the language symbols: "grab" 0.98, "red" 0.95, and "square" 0.92. The self-descriptive statement "I grabbed the red square" is generated by combining the grammatical templates.

[0163] Step S3: Semantic Consistency Comparison

[0164] Three-layer comparison score:

[0165] Vocabulary level: Exact match of core vocabulary. ;

[0166] Structural layer: semantic structure is completely consistent. ;

[0167] Spatiotemporal phase layer: The inner product similarity with the "grab" standard template is 1.0. .

[0168] Step S4: Consistency Determination

[0169] Substitute into the confidence formula to calculate:

[0170] The understanding in this case is correct.

[0171] Step S5: Strengthening Association Weights

[0172] Substitute into the positive reinforcement update formula to calculate the update amount:

[0173] The association weight increased from 0.75 to 0.752.

[0174] Step S6: Steady-state security verification

[0175] Predicted system status after update:

[0176] System closed-loop Lipschitz constant Dynamically stable;

[0177] Operational identity strength There exists a steady state;

[0178] The system runs independently and requires no verification of the cluster's combined data. Once all verifications pass, the weight update officially takes effect.

[0179] Error correction scenarios

[0180] When a human gives the command "touch the blue square," the robot mistakenly interprets "touch" as "grab":

[0181] Results: The robot successfully grasped the object, lifting it 0.1 meters.

[0182] Self-description: "I grabbed the blue cube";

[0183] Semantic consistency comparison: (The words “touch” and “grab” have a low word match.) , (The displacement does not match the "touch" action template);

[0184] Confidence calculation:

[0185] The judgment is based on a misunderstanding.

[0186] Correction: The key semantic node identification algorithm located the error source as the "touch" word node. It then traversed all semantic mappings corresponding to this node, excluding the verified incorrect "grab" mapping, and selected the "touch but don't grab" mapping with the second highest confidence to generate an alternative action plan. Secondary verification result: , , Substituting into the calculation, we get:

[0187] The understanding is correct; the correction is now in effect.

[0188] Understanding Deep Metrics

[0189] The formula for calculating the depth metric is as follows:

[0190] The typical weight is , , , To understand the accuracy rate, To correct the success rate, This is the normalized average verification time.

[0191] After 1000 iterations: , ,

[0192] After 3000 iterations: , ,

[0193] After 5000 iterations: , ,

[0194] After 10,000 iterations: , ,

[0195] The metric continues to rise with each iteration, eventually reaching above 0.91, indicating that the system is gradually entering a state of deep understanding.

[0196] Example 2: Collective Language Understanding Self-calibration of Unmanned Aerial Vehicle Swarms

[0197] This embodiment uses a self-generating light show cluster composed of 50 drones as an example to illustrate the autonomous evolution process of language understanding in a multi-layered nested system. Initial state: The cluster has completed the initial emergence of collective language and established collective association mappings for basic words such as "formation," "lights," "change," "takeoff," and "landing."

[0198] Step S0: Command Reception and Designation Identification

[0199] The human issues the command: "Switch to a 3x3 matrix formation, blue gradient lighting." The cluster referencing recognition module determines that the referencing source is human, the referencing level is the cluster level, and selects the cluster joint operation identity semantic mapping path.

[0200] Step S1: Collective Action Execution and Physical Field Perception

[0201] The cluster parses commands and generates a collective action plan: of the 50 drones, the core 9 form a 3×3 matrix formation, and the remaining 41 form a background array; the core drones' lights switch to a blue gradient mode. During execution, each drone's M-topology sensor collects its own position, velocity, and light wavelength data, which are shared with the cluster's C-field state through inter-phase dimensional correlation, generating a unified physical representation of the cluster.

[0202] Step S2: Generation of Collective Self-Descriptions

[0203] Extracting collective spatiotemporal phase features:

[0204] Timing: Formation change lasts 2.5 seconds, light transition lasts 10 seconds;

[0205] Spatial phase: The core UAVs form a square array of 3 rows and 3 columns, with an array spacing of 2 meters;

[0206] Phase-to-phase: UAV position error less than 0.1 meters, light wavelength error less than 5 nm, joint Chern number estimate. Generate a collective self-description statement: "The cluster has switched to a 3×3 matrix formation, with blue gradient lighting."

[0207] Step S3: Semantic Consistency Comparison

[0208] Three-layer comparison score: , , Calculate the confidence level:

[0209] The judgment is correct.

[0210] Step S4: Association Reinforcement and Steady-State Verification

[0211] Calculation of associated weight update:

[0212] Triple steady-state verification results:

[0213] Cluster joint Chen Shu Topologically stable;

[0214] Single system average Dynamically stable;

[0215] average A steady state exists. All verifications passed; this update is now effective.

[0216] manifold isomorphism professional symbol verification scenario

[0217] The cluster introduces the specialized notation "light-chasing formation" through a manifold isomorphic alignment mechanism applied for on the same day:

[0218] Execution command: "Execute beam tracking formation";

[0219] Generates a self-description: "The cluster has performed light-tracking formation";

[0220] Semantic consistency comparison: ;

[0221] Steady-state verification: All other indicators meet the requirements. Verification result: This professional symbol has passed both consistency and steady-state verification and has been officially written into the cluster's long-term symbol library.

[0222] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A language understanding method based on action-language consistency verification, characterized in that, Executed by the computing unit of an intelligent system with initial language representation capabilities, the computing unit includes embedded computing units, edge computing units, and cloud computing units. The intelligent system has established an endogenous correlation mapping between language symbols and physical field patterns. The method includes the following steps: S1, the system parses the language instruction, generates and executes the action, and simultaneously collects multimodal physical field perception data during the execution process, including electromagnetic field, sound field, light field, and vibration field data, and extracts the temporal phase, spatial phase, and interphase features of the action; S2, based on the temporal phase, spatial phase, and interphase features, the system automatically maps the action result into a natural language self-description through the established endogenous language-physical field correlation mapping; S3, the system performs three-layer linguistic analysis on the language instruction and the action result self-description: lexical layer, structural layer, and spatiotemporal phase layer. S4. When the semantic consistency confidence score is higher than the first preset threshold, the understanding is determined to be correct, and the association weight between the language pattern corresponding to the language instruction and the action pattern corresponding to the action is strengthened; S5. When the semantic consistency confidence score is lower than the second preset threshold, the understanding is determined to be incorrect, the association weight between the language pattern corresponding to the language instruction and the action pattern corresponding to the action is weakened, and a correction mechanism is triggered—the system re-parses the instruction and generates an alternative action plan; S6. All strengthening and correction of association weights must undergo steady-state security verification. If the update causes the system's closed-loop Lipschitz constant to be ≥1, the system's operational identity strength to be lower than the preset security threshold, or the joint Chern number estimate in a cluster scenario to be lower than the preset security threshold, then the update is locked.

2. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, The system also includes step S0: the system receives language instructions through a language perception channel and simultaneously identifies the referential source and referential level of the language instructions; the referential source uses a 3-bit binary code: 000 represents an unknown source, 001 represents a human source, 010 represents the system's own historical records, 100 represents other self-generated systems, 011 represents a human-machine hybrid source, 101 represents a multi-system consensus source, 110 represents an authoritative knowledge base source, and 111 represents a system-native instruction; the priority of different sources is: system-native instruction > human source > multi-system consensus source > other self-generated systems > human-machine hybrid source > authoritative knowledge base > system's own historical records > unknown source; the referential level is divided into a single-system layer, a cluster layer, and a human-computer interaction layer, and the system automatically selects the corresponding semantic mapping path according to the referential source and level.

3. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, The temporal phase features mentioned in step S1 include action sequence, duration, energy consumption rate, and stage division; the spatial phase features include the trajectory of object position change, system configuration change, and relative spatial relationship; the interphase features include the mutual information correlation strength and coherence coefficient between key features.

4. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, The semantic consistency comparison in step S3 specifically involves: lexical layer comparison using cosine similarity to calculate the core vocabulary matching degree; structural layer comparison to verify the consistency of semantic structure; and spatiotemporal phase layer comparison to transform the implicit semantics of the original instruction into a standard spatiotemporal phase feature vector, and calculate the inner product similarity with the spatiotemporal phase feature vector of the actual execution result.

5. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, The formula for calculating the semantic consistency confidence score is as follows: The comparison score for each layer ranges from 0 to 1, and the sum of all weights is 1, which is dynamically adjusted by the system according to the current task type; the first preset threshold is 0.85, and the second preset threshold is 0.

4.

6. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, The update formula for the enhanced association weights in step S4 is as follows: in The learning rate is positive; the update formula for weakening the association weights in step S5 is: in The learning rate is negative.

7. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, The correction mechanism described in step S5 specifically includes: the system re-parses the language instructions, identifies key semantic nodes that lead to misunderstandings, and generates alternative action plans; after the alternative action plans are executed, steps S2 to S4 are re-executed for consistency verification; if the correction attempts fail for a preset number of consecutive times, the system enters clarification mode—generating clarification questions and sending them to the user.

8. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, The steady-state security check mentioned in step S6 specifically involves: calculating the updated system closed-loop Lipschitz constant. ,make sure ; Calculate the updated system operational identity strength ,make sure If the system is in cluster mode, additionally calculate the cluster joint Chern number estimate. ,make sure When a locking event occurs more than three times consecutively, the system automatically pauses language understanding evolution and prioritizes self-referential operations to restore operational consistency.

9. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, It also includes a depth of understanding evaluation step: the system statistically analyzes the understanding accuracy rate, correction success rate, and average verification time in historical verifications, and calculates the understanding depth metric. when At that time, the system is determined to enter the "deep understanding" state.

10. The language understanding method based on action-language consistency verification according to claim 1, characterized in that, When the system introduces external professional symbols through the manifold isomorphic alignment mechanism, all newly added symbol mappings must be verified for validity through this consistency check process; only when the semantic consistency confidence is ≥0.85 and all steady-state checks pass can the professional symbol be written into the system's long-term symbol library.

11. A language understanding autonomous evolution system based on action-language consistency verification, characterized in that, include: The language instruction receiving module is used to receive language instructions; the action execution and physical field perception module is used to parse instructions, generate and execute actions, and simultaneously collect multimodal physical field perception data and extract ternary spatiotemporal phase features. The action result language self-description module is connected to the action execution and physical field perception module, and is used to generate natural language self-descriptions based on the ternary spatiotemporal phase features; A semantic consistency comparison module, connected to the language instruction receiving module and the action result language self-description module, is used to perform three-layer semantic consistency comparison and calculate semantic consistency confidence; an association weight evolution module, connected to the semantic consistency comparison module, is used to strengthen or weaken the language-action association weight based on the semantic consistency confidence. The steady-state security verification module, connected to the associated weight evolution module, is used to perform steady-state verification of a single system and a cluster. The correction mechanism trigger module connects to the semantic consistency comparison module at its input end and to the action execution and physical field perception module at its output end. It is used to trigger the correction mechanism and generate alternative action plans. The manifold isomorphic professional symbol verification module connects to the language instruction receiving module at its input end and to the association weight evolution module at its output end. It is used to perform consistency verification on the external professional symbols added through manifold isomorphic alignment.

12. The language understanding autonomous evolution system based on action-language consistency verification according to claim 11, characterized in that, It also includes a reference recognition module, which is used to identify the reference source and level of language instructions; and a comprehension depth evaluation module, whose input is connected to the semantic consistency comparison module and whose output is connected to the association weight evolution module, which is used to statistically analyze historical verification data, calculate the comprehension depth index and determine the system's cognitive level. It also includes a clarification interaction module, with the input end connected to the correction mechanism trigger module and the output end connected to the language instruction receiving module, used to generate and send clarification questions when continuous correction attempts fail.

13. An electronic device, characterized in that, include: processor; And a memory storing computer program instructions; when the computer program instructions are executed by the processor, the processor performs the method as described in any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, It stores computer program instructions thereon, which, when executed by a processor, implement the method as described in any one of claims 1 to 10.

Images