Semantic continuity guarantee method and device, terminal and storage medium

Abstract

The invention discloses a semantic continuity guarantee method and device, a terminal and a storage medium, and the method comprises the steps: carrying out the structural representation of a natural language demand, and determining the structural demand information; converting the structured demand information into a formalized protocol in a linear tense logic form; performing semantic representation and semantic alignment on the formalized protocol according to the first image data, and determining aligned semantic representation; decoding the aligned semantic representation, and determining executable data; determining a target mode according to the available CPU resources of the system and the edge equipment information, performing runtime guarantee based on the target mode and the executable data, and determining a runtime guarantee result; and generating a device control instruction based on the runtime guarantee result through the operating system security center. Therefore, the problems that the existing method in the prior art cannot fundamentally solve the problem of lack of full-link semantic continuity and is difficult to meet the high-reliability operation requirement of the safety key AI system in a dynamic environment can be effectively solved.

Description

Technical Field

[0001] This invention relates to the field of industrial control, and more particularly to a method, apparatus, terminal, and storage medium for ensuring semantic continuity. Background Technology

[0002] In fields such as industrial control and autonomous driving, as the requirements for autonomy and real-time performance of AI systems increase, the systems need to continuously respond to changes in demand in dynamic and complex environments. Ensuring semantic continuity has become a technical bottleneck that determines the security level of the system.

[0003] Currently, safety-critical AI systems generally employ traditional fragmented verification mechanisms. These mechanisms separate requirement analysis, formal verification, and runtime monitoring, resulting in an inherent flaw of separating requirements from implementation. Specifically, the semantic definition in the requirement phase is not effectively linked to subsequent implementation phases, causing the verification process to only perform partial checks on a single stage and failing to cover the entire semantic transmission process.

[0004] The aforementioned limitations can easily lead to serious problems in dynamic environments: In industrial scenarios, discrepancies can easily arise between demand semantics and system execution logic, causing equipment to misjudge operational defects and trigger dangerous operations in violation of regulations; in driving scenarios, breaks in semantic transmission can cause delays in the system's matching of environmental perception with demand commands, leading to safety hazards such as delayed response. Existing methods cannot fundamentally solve the problem of missing semantic continuity across the entire chain, making it difficult to meet the high reliability requirements of safety-critical AI systems in dynamic environments.

[0005] Therefore, existing technologies still need improvement and development. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a semantic continuity assurance method, device, terminal and storage medium to address the above-mentioned defects of the prior art. The aim is to solve the problem that the existing methods cannot fundamentally solve the problem of missing semantic continuity across the entire link and are difficult to meet the high reliability operation requirements of safety-critical AI systems in dynamic environments.

[0007] The technical solution adopted by this invention to solve the problem is as follows: In a first aspect, embodiments of the present invention provide a method for ensuring semantic continuity, wherein the method includes: Obtain natural language requirements, perform structured representation of the natural language requirements, and determine structured requirement information; The structured requirement information is converted into a formal specification in linear temporal logic form; Acquire first image data, and perform semantic representation and semantic alignment on the formal specification based on the first image data to determine the aligned semantic representation; The aligned semantic representation is decoded to determine executable data; Obtain information on available CPU resources and edge devices in the system; The target mode is determined based on the available CPU resources of the system and the edge device information. Runtime assurance is performed based on the target mode and the executable data, and the runtime assurance result is determined. The operating system security hub generates device control commands based on the runtime assurance results.

[0008] In one implementation method, the natural language requirement is represented in a structured manner to determine the structured requirement information, including: The natural language requirements are parsed to determine the requirements parsing information; The requirement parsing information is transformed into a multimodal form using a large language model to determine the multimodal requirement parsing information; Based on the multimodal requirement parsing information, structured information in restricted English is generated, and the structured requirement information is determined.

[0009] In one implementation method, converting the structured requirement information into a formal specification of linear temporal logic includes: The structured requirement information is verified for credibility, and the requirement verification result is determined. Obtain user confirmation requirement information for determination based on visualization-based requirement verification results, and transform the user confirmation requirement information into a formal reduction of linear temporal logic form.

[0010] In one implementation method, the structured requirement information is subjected to credibility verification, and the requirement verification result is determined, including: Obtain a domain knowledge base, filter the structured requirement information based on the domain knowledge base, and determine the filtered requirement information; Calculate the difference measure between the different filtered demand information; The requirement verification result is determined based on the filtered requirement information and the difference metric.

[0011] In one implementation method, the structured requirement information is filtered based on the domain knowledge base to determine the filtered requirement information, including: Calculate the similarity between the structured requirement information and the historical conflicting requirements in the domain knowledge base; Obtain a similarity threshold, and use the structured demand information whose demand similarity is less than the similarity threshold as the filtered demand information.

[0012] In one implementation method, semantic representation and semantic alignment are performed on the formal specification based on the first image data to determine the aligned semantic representation, including: The logical rules of the formal specification are mapped to semantic representations to determine the formal semantic representations; The visual language model aligns the formal semantic representations based on the first image data to determine the aligned semantic representation.

[0013] In one implementation, a visual language model is used to align the formal semantic representations based on the first image data to determine the aligned semantic representations, including: The formal predicates in each of the formal semantic representations are mapped to semantic concept vectors of the visual language model; The first image data is converted into a first image vector, and the cosine similarity between each semantic concept vector and the image vector is calculated. Align each formal semantic representation according to the cosine similarity to determine the aligned semantic representation.

[0014] In one implementation method, determining the target mode based on the available CPU resources of the system and the edge device information includes: Obtain the CPU threshold and determine whether the available CPU resources of the system are greater than the CPU threshold. If so, determine that the target mode is monitoring mode.

[0015] In one implementation, the method further includes: If not, obtain the edge device set, and determine whether the edge device belongs to the edge device set based on the edge device information; If so, then the target mode is determined to be the test mode; If not, then the target mode is determined to be the diagnostic mode.

[0016] In one implementation method, when the target mode is the monitoring mode, runtime assurance is performed based on the target mode and the executable data, and the runtime assurance result is determined, including: Acquire the second image data under the target mode, and calculate the similarity between the second image data and each predicate in the executable data; The system state is determined based on the similarity scores, and the current system state is determined based on the set type. The set type includes a safe type and an unsafe type. The runtime guarantee result is determined based on the current state of the system.

[0017] In one implementation method, when the target mode is the diagnostic mode, runtime assurance is performed based on the target mode and the executable data, and the runtime assurance result is determined, including: Acquire third image data in the target mode, and determine the sensitivity matrix corresponding to the executable data based on the third image data and the executable data; The current fault result is determined based on the sensitivity matrix and the executable data, and the current fault result is used as the runtime guarantee result.

[0018] In one implementation, the method further includes: Obtain abnormal executable data reported by the operating system security hub; Calculate the semantic similarity between the semantic representation and the formal reduction corresponding to the abnormal executable data; The abnormal executable data is updated based on the semantic similarity and the formal specification.

[0019] Secondly, embodiments of the present invention also provide a semantic continuity guarantee device, wherein the semantic continuity guarantee device includes: The structured representation module is used to acquire natural language requirements, perform structured representation on the natural language requirements, and determine the structured requirement information; The formal specification determination module is used to convert the structured requirement information into a formal specification in linear temporal logic form; The alignment semantic representation determination module is used to acquire first image data, perform semantic representation and semantic alignment on the formal specification based on the first image data, and determine the alignment semantic representation. The executable data determination module is used to decode the alignment semantic representation and determine the executable data; The system status acquisition module is used to acquire information on available CPU resources and edge devices in the system. The runtime assurance module is used to determine the target mode based on the available CPU resources of the system and the edge device information, perform runtime assurance based on the target mode and the executable data, and determine the runtime assurance result. The control command generation module is used to generate device control commands based on the runtime protection results through the operating system security hub.

[0020] In one implementation, the structured representation module includes: The parsing unit is used to parse the natural language requirements and determine the requirements parsing information; A multimodal conversion unit is used to perform multimodal conversion on the requirement parsing information through a large language model to determine the multimodal requirement parsing information; The structured conversion unit is used to generate structured information in restricted English based on the multimodal requirement parsing information, and to determine the structured requirement information.

[0021] In one implementation, the formal specification determination module includes: A credibility verification unit is used to verify the credibility of the structured requirement information and determine the requirement verification result. The formal specification determination unit is used to acquire user confirmation requirement information for determination based on the visualization requirement verification results, and to transform the user confirmation requirement information into a formal specification in linear temporal logic form.

[0022] In one implementation, the trustworthiness verification unit includes: The first filtering unit is used to obtain a domain knowledge base, filter the structured requirement information based on the domain knowledge base, and determine the filtered requirement information. The difference measurement calculation unit is used to calculate the difference measurement between the various filtered requirements information. The requirement verification result determination unit is used to determine the requirement verification result based on the filtered requirement information and the difference measure.

[0023] In one implementation, the first screening unit includes: A requirement similarity calculation unit is used to calculate the requirement similarity between the structured requirement information and the historical conflict requirements in the domain knowledge base; The second filtering unit is used to obtain a similarity threshold and to use the structured demand information whose demand similarity is less than the similarity threshold as the filtered demand information.

[0024] In one implementation, the alignment semantic representation determination module includes: A semantic representation unit is used to map the logical rules of the formal specification into semantic representations and determine the formal semantic representations; The semantic alignment unit is used to align each formal semantic representation according to the first image data using a visual language model, and to determine the aligned semantic representation.

[0025] In one implementation, the semantic alignment unit includes: A semantic concept vector determination unit is used to map the formal predicates in each of the formal semantic representations to semantic concept vectors of the visual language model; The cosine similarity calculation unit is used to convert the first image data into a first image vector and calculate the cosine similarity between each semantic concept vector and the image vector. A semantic alignment representation determination unit is used to align each formal semantic representation according to each cosine similarity to determine the aligned semantic representation.

[0026] In one implementation, the runtime protection module includes: The first judgment unit is used to obtain the CPU threshold and determine whether the available CPU resources of the system are greater than the CPU threshold. The monitoring mode unit is used to determine, if so, that the target mode is a monitoring mode.

[0027] In one implementation, the runtime protection module further includes: The second determination unit is used to obtain the edge device set if no, and determine whether the edge device belongs to the edge device set based on the edge device information; The test module unit is used to determine the target mode as the test mode if the condition is met. A diagnostic mode unit is used to determine the target mode as a diagnostic mode if no.

[0028] In one implementation, the monitoring mode unit includes: A predicate similarity calculation unit is used to acquire second image data under the target mode and calculate the similarity between each predicate in the second image data and the executable data; The system current state determination unit is used to determine the set type corresponding to the system state based on each similarity, and to determine the current state of the system based on the set type. The set type includes a safe type and an unsafe type. The first runtime assurance result determination unit is used to determine the runtime assurance result based on the current state of the system.

[0029] In one implementation, the diagnostic mode unit includes: A sensitivity matrix determination unit is used to acquire third image data in the target mode and determine the sensitivity matrix corresponding to the executable data based on the third image data and the executable data. The second runtime assurance result determination unit is used to determine the current fault result based on the sensitivity matrix and the executable data, and to use the current fault result as the runtime assurance result.

[0030] In one embodiment, the apparatus further includes: An abnormal executable data acquisition unit is used to acquire abnormal executable data fed back by the operating system security center; A semantic similarity calculation unit is used to calculate the semantic similarity between the semantic representation and the formal reduction corresponding to the abnormal executable data; The data update unit is used to update the abnormal executable data according to the semantic similarity and the formal specification.

[0031] Thirdly, embodiments of the present invention also provide a terminal, the terminal including a memory and one or more processors; the memory stores one or more programs; the programs include instructions for executing the semantic continuity guarantee methods as described above; the processor is used to execute the programs.

[0032] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a plurality of instructions, wherein the instructions are adapted to be loaded and executed by a processor to implement any of the semantic continuity guarantee methods described above.

[0033] The beneficial effects of this invention are as follows: This invention determines structured requirement information by structurally representing natural language requirements; converts the structured requirement information into a formal reduction of linear temporal logic; performs semantic representation and semantic alignment on the formal reduction based on first image data to determine the aligned semantic representation; decodes the aligned semantic representation to determine executable data; determines the target mode based on available CPU resources and edge device information; performs runtime assurance based on the target mode and executable data to determine the runtime assurance result; and generates device control instructions based on the runtime assurance result through the operating system security hub. Therefore, it effectively solves the problem that existing technologies and methods cannot fundamentally address the lack of semantic continuity across the entire chain, making it difficult to meet the high-reliability operation requirements of safety-critical AI systems in dynamic environments. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the semantic continuity guarantee method provided in an embodiment of the present invention.

[0036] Figure 2 This is a schematic flowchart of the structured representation provided in the embodiments of the present invention.

[0037] Figure 3 This is a schematic diagram of the formal reduction transformation process provided in the embodiments of the present invention.

[0038] Figure 4 This is a schematic diagram of the credibility verification process provided in an embodiment of the present invention.

[0039] Figure 5 This is a schematic diagram of the structured requirement information filtering process provided in an embodiment of the present invention.

[0040] Figure 6 This is a schematic diagram of the process for determining the alignment semantic representation provided in an embodiment of the present invention.

[0041] Figure 7 This is a schematic diagram of the semantic alignment process provided in an embodiment of the present invention.

[0042] Figure 8 This is a schematic diagram of the monitoring mode determination process provided in an embodiment of the present invention.

[0043] Figure 9 This is a flowchart illustrating the test mode and diagnostic mode provided in the embodiments of the present invention.

[0044] Figure 10 This is a schematic diagram of the runtime protection process under the monitoring mode provided in the embodiment of the present invention.

[0045] Figure 11 This is a schematic diagram of the runtime guarantee in the diagnostic mode provided by the embodiments of the present invention.

[0046] Figure 12 This is a flowchart illustrating the abnormal feedback provided in an embodiment of the present invention.

[0047] Figure 13 This is a schematic diagram illustrating the specific implementation process of the semantic continuity guarantee method provided in this embodiment of the invention.

[0048] Figure 14 This is a schematic diagram of the internal modules of the semantic continuity guarantee device provided in an embodiment of the present invention.

[0049] Figure 15 This is a schematic diagram of the terminal provided in an embodiment of the present invention. Detailed Implementation

[0050] This invention discloses a method, apparatus, terminal, and storage medium for ensuring semantic continuity. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention.

[0051] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0052] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0053] In fields such as industrial control and autonomous driving, as the requirements for autonomy and real-time performance of AI systems increase, the systems need to continuously respond to changes in demand in dynamic and complex environments. Ensuring semantic continuity has become a technical bottleneck that determines the security level of the system.

[0054] Currently, safety-critical AI systems generally employ traditional fragmented verification mechanisms. These mechanisms separate requirement analysis, formal verification, and runtime monitoring, resulting in an inherent flaw of separating requirements from implementation. Specifically, the semantic definition in the requirement phase is not effectively linked to subsequent implementation phases, causing the verification process to only perform partial checks on a single stage and failing to cover the entire semantic transmission process.

[0055] The aforementioned limitations can easily lead to serious problems in dynamic environments: In industrial scenarios, discrepancies can easily arise between demand semantics and system execution logic, causing equipment to misjudge operational defects and trigger dangerous operations in violation of regulations; in driving scenarios, breaks in semantic transmission can cause delays in the system's matching of environmental perception with demand commands, leading to safety hazards such as delayed response. Existing methods cannot fundamentally solve the problem of missing semantic continuity across the entire chain, making it difficult to meet the high reliability requirements of safety-critical AI systems in dynamic environments.

[0056] To address the aforementioned deficiencies in existing technologies, this invention provides a semantic continuity assurance method. This method involves: structuring natural language requirements to determine structured requirement information; converting the structured requirement information into a formal reduction of linear temporal logic; performing semantic representation and alignment of the formal reduction based on first image data to determine an aligned semantic representation; decoding the aligned semantic representation to determine executable data; determining a target pattern based on available CPU resources and edge device information; performing runtime assurance based on the target pattern and executable data to determine the runtime assurance result; and generating device control instructions based on the runtime assurance result through the operating system's security hub. Therefore, this method effectively solves the problem that existing technologies and methods cannot fundamentally address the lack of end-to-end semantic continuity, making it difficult to meet the high-reliability operation requirements of safety-critical AI systems in dynamic environments.

[0057] Exemplary method: like Figure 1 As shown, the method includes: Step S100: Obtain natural language requirements, perform structured representation of the natural language requirements, and determine the structured requirement information.

[0058] To obtain the natural language requirements corresponding to the current application scenario (such as industrial quality inspection, intelligent driving systems, energy management systems, etc.), first extract the core elements from the natural language requirements: for example, when the CPU availability of the pipeline is higher than 80%, the system enters monitoring mode. This needs to be broken down into the triggering condition (CPU availability > 80%), the subject (pipeline system), and the action (entering monitoring mode). The parsed elements are then mapped to a preset structured dimension to obtain structured requirement information.

[0059] In one implementation, such as Figure 2 As shown, the natural language requirements are represented in a structured manner to determine the structured requirement information, including: Step S101: Parse the natural language requirement to determine the requirement parsing information; Step S102: Perform multimodal transformation on the requirement parsing information using a large language model to determine the multimodal requirement parsing information; Step S103: Generate structured information for restricted English based on the multimodal requirement parsing information, and determine the structured requirement information.

[0060] Based on KaihongOS The REACT requirement parsing engine parses natural language requirements, extracting semantic, logical relationships, and constraints to prepare for subsequent transformations. In this embodiment, the requirement parsing information includes: Subject: the object the requirement targets; Action: the operation to be performed; Triggering Condition: the prerequisite for the action; Constraints: the limitations of the action; Goal: the result the requirement aims to achieve. For the requirement parsing information, which may contain multiple modalities, a Large Language Model (LLM) is used for multimodal transformation, converting it into a more structured form. This achieves a transition from unstructured to structured data, resulting in multimodal requirement parsing information. The requirement parsing information is then used to generate structured requirements in restricted English, resulting in structured requirement information. Restricted English is simplified and standardized English, avoiding ambiguity and making it easier for subsequent modules to process. This embodiment uses LLM to deconstruct natural language requirements into multiple candidate structured requirements, and restricted English grammar ensures unambiguity. DeepKaiHong has embedded a grammar validator at the KaihongOS underlying layer to automatically filter out violating candidates.

[0061] The natural language requirements are represented in a structured manner, and the mathematical representation of the structured requirement information is as follows: Let the original demand set be... Through large language models Complete mapping: , in, For a set of concepts, For concept set The Middle One element, , To satisfy grammatical constraints within a limited English-speaking space. : , in, It is a set of non-terminal symbols. For the set of terminal symbols, For the set of production rules, For the set of start symbols, Indicates will Replace with At the same time, production rules The constraint must be satisfied: the length of the left-hand side of the production rule does not exceed 2.

[0062] Step S200: Convert the structured requirement information into a formal specification in linear temporal logic form.

[0063] By mapping the demand elements (conditions, actions, and logical relationships) in structured demand information to combinations of temporal operators and predicates in LTLf, the demand is given a machine-verifiable form, thereby realizing the transformation of structured demand information into a formal specification of linear temporal logic (LTLf). At the same time, it has been extended with the integration of AI features to adapt to the needs of related systems.

[0064] In one implementation, such as Figure 3 As shown, the formal reduction of the structured requirement information into a linear temporal logical form includes: Step S201: Perform credibility verification on the structured requirement information and determine the requirement verification result; Step S202: Obtain user confirmation requirement information for determination based on the visualization requirement verification results, and transform the user confirmation requirement information into a formal reduction of linear temporal logic form.

[0065] When converting structured requirement information into a formal specification in linear temporal logic, the structured requirements are first verified for credibility. This embodiment uses the KaihongOS Trusted Verification Sandbox to verify the credibility of structured requirements, ensuring logical consistency and avoiding contradictions, while also conforming to the system's trust specifications. After verification, semantic differences discovered during the verification process (such as inconsistencies in the requirements) are presented to the user in a visual manner to facilitate troubleshooting. The user then confirms the processed requirements (including verification results and semantic differences) to ensure they align with their true intent. Finally, the user's confirmation information based on the visualized requirement verification results is obtained, and this confirmation information is converted into a formal specification in linear temporal logic.

[0066] Specifically, the formal reduction of user confirmation requirements into linear temporal logic can be expressed as: Transformation functions through formal reduction Perform the conversion: , in, Represents a set of formal specifications. For atomic propositions, Formal reduction transformation function, Represents the set of atomic propositions. This indicates that the user has confirmed the elements or concepts in the request information. This indicates that the formal reduction transformation function will be applied to the remaining part. This represents logical AND. Indicates the "next" sequential operator.

[0067] Atomic propositions satisfy: , This indicates that the system is on the target path. This indicates that a cone obstacle has been detected. It represents other atomic propositions.

[0068] In one implementation, such as Figure 4 As shown, the credibility verification of the structured requirement information is performed to determine the requirement verification result, including: Step S2011: Obtain the domain knowledge base, filter the structured requirement information based on the domain knowledge base, and determine the filtered requirement information; Step S2012: Calculate the difference measure between each filtered requirement information; Step S2013: Determine the requirement verification result based on the filtered requirement information and the difference measure.

[0069] The domain knowledge base contains historical conflict requirements. The structured requirement information is filtered through the domain knowledge base to determine whether the structured requirement information is similar to each historical conflict requirement. If it is similar, the structured requirement information is deleted; if it is not similar, the structured requirement information is used as the filtered requirement information.

[0070] In one implementation, such as Figure 5 As shown, the structured requirement information is filtered based on the domain knowledge base to determine the filtered requirement information, including: Step S20111: Calculate the similarity between the structured requirement information and the historical conflict requirements in the domain knowledge base; Step S20112: Obtain a similarity threshold, and use the structured demand information whose demand similarity is less than the similarity threshold as the filtered demand information.

[0071] For each concept in the structured requirement information and each concept in the historical conflicting requirements, the similarity between the two is calculated using a semantic similarity function to obtain the requirement similarity between the structured requirement information and the historical conflicting requirements. A preset similarity threshold is obtained, and each requirement similarity is compared with the similarity threshold. When the requirement similarity is less than the similarity threshold, it means that the structured requirement information is not a conflicting requirement, and the structured requirement information is included in the filtered requirement information. If the requirement similarity is greater than or equal to the similarity threshold, it means that the structured requirement information may be a conflicting requirement, and the structured requirement information is deleted, and the structured requirement information is fed back to the system as a historical conflicting requirement. This embodiment, by calculating the similarity between the structured requirement information and the historical conflicting requirements and performing preliminary screening of the structured requirement information based on the preset similarity threshold, can ensure the accuracy of subsequent execution operations.

[0072] For the filtered requirement information, a difference metric is calculated between each filtered requirement. If the difference metric is large (greater than a preset difference metric threshold), it indicates that the filtered requirement information may still contain anomalies. If the difference metric is small (greater than or equal to the preset difference metric threshold), the filtered requirement information does not deviate from the user-input requirement information. The calculated difference metric and the filtered requirement information are combined to form a requirement verification result, which is used to correct or generate accurate execution instructions in subsequent steps.

[0073] Based on the aforementioned domain knowledge base, the structured requirement information is filtered, and the filtered requirement information can be represented as follows: Set up structured requirements information The validation function is used to verify the similarity between structured requirement information and historical conflict requirement information. Defined as: , in, Due to the needs of historical conflicts, For domain knowledge base, The first part of the structured requirements information A concept, This indicates the number of historical conflicting requirements in the domain knowledge base. This represents the similarity calculation function. This indicates the preset similarity threshold.

[0074] Difference measurement:

[0075] in, This represents the semantic difference between the concepts, i.e., a difference measure. Representing concepts Embedded vector, The L2 norm of a vector. It is a function for maximizing the value.

[0076] This embodiment calculates semantic differences by embedding spatial distances and isolates high-risk requirements using a sandbox. KaihongOS's security isolation mechanism blocks the propagation of conflicting requirements, making it suitable for industrial control scenarios.

[0077] Step S300: Obtain first image data, perform semantic representation and semantic alignment on the formal specification based on the first image data, and determine the aligned semantic representation.

[0078] Acquire the first image data corresponding to the current application scenario (such as a single frame / sequence of images in an industrial scene). Since formal specifications are unambiguous representations of system requirements, behaviors, or attributes described using mathematical / logical language, this embodiment performs semantic representation on the formal specifications, and then aligns the semantically represented data according to the first image data to obtain an aligned semantic representation. The formal predicates are bound to VLM semantic concepts to establish a mathematical connection between machine vision and human cognition.

[0079] In one implementation, such as Figure 6 As shown, semantic representation and semantic alignment are performed on the formal specification based on the first image data to determine the aligned semantic representation, including: Step S301: Map the logical rules of the formal specification to semantic representations, and determine the formal semantic representations; Step S302: Align each formal semantic representation according to the first image data using a visual language model to determine the aligned semantic representation.

[0080] First, determine the behavioral logic corresponding to the formal specification to obtain the logical rules of the formal specification. Then, convert these logical rules into a logical description with specific semantics, forming a formal semantic representation. Process the first image data using a Visual Language Model (VAM) to identify elements (such as paths and cones) and states (such as a vehicle being on a path) in the first image data. Match the formal semantic representation with the content identified in the first image data. After matching, a semantic description that conforms to both the formal logic and the image content is obtained.

[0081] In one implementation, such as Figure 7 As shown, the visual language model aligns the formal semantic representations based on the image data to determine the aligned semantic representations, including: Step S3021: Map the formal predicates in each of the formal semantic representations to semantic concept vectors of the visual language model; Step S3022: Convert the first image data into a first image vector, and calculate the cosine similarity between each semantic concept vector and the image vector; Step S3023: Align each formal semantic representation according to each cosine similarity to determine the aligned semantic representation.

[0082] Leveraging the pre-trained knowledge of the visual language model, each predicate (e.g., `on_path`) is transformed into a vector in the model space (e.g., word vectors from a large language model or semantic vectors from a multimodal model). This vector represents the semantic meaning of the predicate, achieving a mapping from formal predicates to semantic concept vectors. The visual language model extracts features from the first image data (e.g., identifying paths, cones, vehicle positions, etc.), obtaining the first image vector. The cosine similarity between each semantic concept vector and the image vector is calculated. Based on the cosine similarity, formal semantic representations matching the image are selected, resulting in aligned semantic representations. This embodiment establishes a mathematical connection between machine vision and human cognition by binding formal predicates to VLM semantic concepts. The conversion from pixel-level to concept-level is achieved through cosine similarity.

[0083] Based on the first image data, the formal specification is semantically represented and semantically aligned. The specific semantic representation of the alignment can be expressed as follows: Define VLM embedding function Model logic mapping:

[0084] in, An embedding function representing a visual language model (mapping the input to a vector). This represents the first image data. express A 3D real vector space, that is, mapping the first image data to... The first image vector of dimension, The logical mapping function representing the linear temporal logic formula to the visual language model. The set representing linear temporal logic formulas, i.e., formal reductions. Represents the embedding space of the visual language model. Representing atomic propositions The corresponding set of semantic concepts, This represents a text embedding function.

[0085] Similarity is calculated as follows: , in, Represents the first image data With atomic propositions similarity, Represents the first image vector. The cosine similarity function is used. For the atomic proposition The corresponding similarity threshold.

[0086] Step S400: Decode the alignment semantic representation to determine executable data.

[0087] The aligned semantic representation is decoded to extract atomic propositions and timing constraints. Based on these, the system behavior requirements corresponding to each semantic are defined. The extracted information (atomic propositions, timing constraints, and system behavior requirements, etc.) is mapped to the system's predefined operation library, transforming them into system-recognizable operation units. Using LTLF timing operators, time or execution order constraints are added to the operation units. Finally, the operation units with added time or execution order constraints are converted into an executable format supported by the system.

[0088] Step S500: Obtain available CPU resources and edge device information of the system.

[0089] The system collects information on available CPU resources and edge devices. Data collection can occur before and during the execution of executable data. Available CPU resources can be obtained through the system's resource monitoring module, such as reading the current number of idle CPU cores and the current load rate of remaining computing power. Edge device information can be obtained through the device's hardware information interface, including parameters such as model, hardware version, and computing power limit.

[0090] Step S600: Determine the target mode based on the available CPU resources of the system and the edge device information, perform runtime assurance based on the target mode and the executable data, and determine the runtime assurance result.

[0091] The target mode for runtime assurance is determined based on the system's available CPU resources and edge device information (generally collected before executable data execution). Executable data is then executed within this target mode, runtime assurance is performed based on this mode, and runtime assurance results are generated. Runtime assurance results typically include verification and detection results from the runtime assurance process.

[0092] In one implementation, such as Figure 8 As shown, determining the target mode based on the available CPU resources of the system and the edge device information includes: Step S601: Obtain the CPU threshold and determine whether the available CPU resources of the system are greater than the CPU threshold; Step S602: If yes, determine that the target mode is the monitoring mode.

[0093] The system reads a preset CPU threshold from the system configuration. This threshold is pre-set based on the edge device's computing power limit, the computing power requirements of executable data, and automotive-grade safety standards. Thresholds can be static (fixed values) or dynamic (automatically adjusted based on device model and task type). The collected real-time available CPU resources are compared with the preset CPU threshold to determine if the system's available CPU resources exceed the threshold. For example, if the threshold is ≥40% CPU idle computing power, and the current idle computing power is 50%, the condition is met; if the current idle computing power is 30%, the condition is not met. If the condition is met, the target mode is set to monitoring mode. In this mode, the system initiates full-process monitoring (including timing constraint verification, semantic consistency comparison, and dynamic resource tracking) to ensure stable execution of executable data.

[0094] In one implementation, such as Figure 9 As shown, the method further includes: Step S603: If not, obtain the edge device set, and determine whether the edge device belongs to the edge device set based on the edge device information; Step S604: If yes, then determine the target mode as the test mode; Step S605: If not, then determine that the target mode is a diagnostic mode.

[0095] When the available CPU resources in the system do not meet the CPU threshold, a secondary judgment process is triggered. Based on the edge device information, the edge device corresponding to the edge device information is matched with edge devices in the edge device set to determine whether the edge device belongs to the edge device set. Alternatively, the edge device information can be matched with the edge device information in the edge device set. Matching dimensions may include: completely identical device models, hardware version compatibility, and core computing power meeting the minimum requirements of devices in the set.

[0096] If the current edge device belongs to the preset set of edge devices, it means that the device has the hardware foundation to support lightweight assurance, so the target mode is determined as the test mode. In this mode, the system will disable unnecessary full monitoring functions, and only retain core timing constraint verification and basic resource tracking (such as real-time CPU load monitoring) to ensure the basic operation of executable data with minimal computing power consumption, while avoiding system lag due to insufficient resources.

[0097] If the current edge device does not belong to the preset set of edge devices, it means that the device cannot stably support the protection logic of the test mode when CPU resources are insufficient. Therefore, the target runtime protection mode is determined to be the diagnostic mode. In this mode, the system will first trigger the fault diagnosis and degradation operation mechanism: on the one hand, it continuously collects data such as CPU resources, device status, and instruction execution anomalies to locate the root cause of insufficient resources; on the other hand, it immediately degrades executable data to the most basic safety instructions (such as maintaining a safe speed and emergency stop preparation in automotive-grade scenarios), and reports abnormal information such as device incompatibility and insufficient resources to ensure system safety.

[0098] Specifically, determining the target pattern based on available CPU resources and edge device information can be represented as follows: Routing function Defined as: , , , in, Indicates the routing function, based on time. Based on the system status (available CPU resources and edge device information), select the corresponding target mode. It is a time variable (the execution time of the executable data). This is a set of identifiers for each mode (0 corresponds to monitoring mode, 1 corresponds to testing mode, and 2 corresponds to diagnostic mode). To select the time set for the monitoring mode, For system available CPU resources, CPU resource threshold, This indicates that when the available CPU resources of the system exceed the CPU threshold at the corresponding time, belong . To select the time set for the test mode, Indicates existence Belongs to the edge device collection , Indicates when time The corresponding edge device belongs to, belong .

[0099] This embodiment automatically switches working modes based on system resource status and edge scenario characteristics. The KaihongOS real-time task scheduler ensures the priority of the monitoring mode.

[0100] In one implementation, such as Figure 10As shown, when the target mode is the monitoring mode, runtime assurance is performed based on the target mode and the executable data, and the runtime assurance result is determined, including: Step S6021: Obtain the second image data under the target mode, and calculate the similarity between the second image data and each predicate in the executable data; Step S6022: Determine the set type corresponding to the system state based on each similarity, and determine the current state of the system based on the set type. The set type includes a safe type and an insecure type. Step S6023: Determine the runtime guarantee result based on the current state of the system.

[0101] When the target mode is monitoring mode, the system activates the image acquisition module to obtain the second image data at the current moment (such as real-time road and vehicle status images captured by a camera in an automotive-grade scenario), which serves as the visual input for subsequent semantic matching. All atomic predicates contained in the executable data are extracted using a visual language model. The second image data is transformed into a second image vector using the VLM image embedding function, and each atomic predicate is transformed into a semantic concept vector. The similarity between the second image vector and each semantic concept vector is calculated. Based on the similarity scores, the set type corresponding to the current system state is determined. The set type includes a safe type and an unsafe type. A safe type indicates that the current state conforms to the semantic constraints of the executable data, while an unsafe type indicates that the current state deviates from the expected constraints of the executable data. If the set type is safe: the current system state is determined to be a normal execution state, and the runtime guarantee result is to continuously use the current executable data, maintaining full monitoring. If the set type is unsafe: the current system state is determined to be an abnormal deviation state, and the runtime guarantee result is to trigger correction instructions (such as adjusting steering or correcting speed) and increase the semantic verification frequency, while recording abnormal information for subsequent analysis.

[0102] When the target mode is the aforementioned monitoring mode, runtime assurance is performed based on the target mode and executable data. The result of the runtime assurance can be expressed as follows: Let the second image data State transition : , in, Indicates the second image data Image frames, Represents the state transition function. express Viboul vector space, For the system state set, Indicates the system state at any given time (including available CPU resources and edge device information, etc.). Indicates the system state at a given time. This represents the state transition rule (generating the next state based on the current state and the input). Indicates the first The image frame and the first Similarity of individual atomic predicates.

[0103] Output decision: , in, This is the output of the monitoring mode (runtime assurance result). This indicates a violation of the constraints. This indicates that the constraint is satisfied. Represents the set of unsafe states (unsafe types). Represents a set of security states (security types).

[0104] This embodiment transforms continuous video frames (second image data) into discrete state sequences and uses an automaton to verify timing requirements. The processing latency per frame is less than 10ms (industrial-grade latency requirement).

[0105] In one implementation, such as Figure 11 As shown, when the target mode is the diagnostic mode, runtime assurance is performed based on the target mode and the executable data to determine the runtime assurance result, including: Step S6051: Obtain the third image data in the target mode, and determine the sensitivity matrix corresponding to the executable data based on the third image data and the executable data; Step S6052: Determine the current fault result based on the sensitivity matrix and the executable data, and use the current fault result as the runtime guarantee result.

[0106] In diagnostic mode, the system first collects third-party image data adapted to this mode, extracts atomic predicates and key visual regions of the image by combining executable data, and constructs a sensitivity matrix that quantifies the sensitivity of predicates to image changes; then, it filters highly sensitive predicates through the matrix, verifies their execution status to determine the obstacle detection result, and finally uses the fault result as the runtime guarantee result.

[0107] Specifically, when the target mode is diagnostic mode, runtime guarantees are performed based on the target mode and executable data. The runtime guarantee result can be expressed as follows: Define the sensitivity matrix : , in, The sensitivity matrix represents the loss function of a DNN (Deep Neural Network), used to quantify how sensitive the model is to changes in the input. Representing the concept of atoms The corresponding sensitivity index, For the third image data Image frames in Take the expected value. This represents the loss function of a DNN. The embedding function represents the atomic concept c. Representing concepts Embedded to image frames The partial derivatives, Representing the loss function in relation to the concept Embedded partial derivatives.

[0108] Fault location function: , in, This represents the fault location function. In the concept set In the expression, take the one that maximizes the subsequent expression. , Representing loss pair concept The absolute value of the embedded partial derivative.

[0109] This embodiment calculates the concept layer gradient through backpropagation to determine the basis for model decision-making. It is applicable to defect diagnosis in the Shenzhen Kaihong Industrial Vision Quality Inspection System.

[0110] In one implementation, such as Figure 12 As shown, the method further includes: Step H100: Obtain abnormal executable data fed back by the operating system security center; Step H200: Calculate the semantic similarity between the semantic representation and the formal reduction corresponding to the abnormal executable data; Step H300: Update the abnormal executable data according to the semantic similarity and the formal specification.

[0111] Abnormal executable data is obtained from the operating system security hub. A similarity function is used to calculate the semantic similarity between the abnormal data's semantic representation and the original formal reduction. Executable data is then optimized using a requirement update rule, with the similarity multiplied by a weight as the adjustment factor. This process can be specifically represented as follows: Feedback learning function This indicates that runtime data will be used. Transform into LTLf formal specification : Suppose there is an abnormal sample (abnormal executable data): , This anomalous sample contains elements that trigger constraint violations. Image data and the corresponding serial number .

[0112] Requirements update rules: , This represents the concept space perturbation operator. Represents the original atomic proposition, This represents the updated atomic proposition. Indicates weight, Representing abnormal images With the original atomic proposition Semantic similarity.

[0113] Step S700: Generate device control instructions based on the runtime protection results through the operating system security hub.

[0114] The operating system security hub receives runtime assurance results from various modes (constraint fulfillment / violation in monitoring mode, fault results in test / diagnosis mode, etc.), first classifies the results into levels (e.g., no anomaly, partial anomaly, severe anomaly), then combines them with a preset device control rule library to map and generate corresponding device control commands (e.g., maintain the current command when there is no anomaly, trigger path correction when constraints are violated, and start emergency shutdown when a severe fault occurs), and finally sends the commands to the execution layer to achieve real-time control of edge devices.

[0115] Based on the above embodiments, the present invention also provides a semantic continuity guarantee device, such as... Figure 14 As shown, the device includes: The structured representation module 01 is used to acquire natural language requirements, perform structured representation on the natural language requirements, and determine the structured requirement information; Formal specification determination module 02 is used to convert the structured requirement information into a formal specification in linear temporal logic form; Alignment semantic representation determination module 03 is used to acquire first image data, perform semantic representation and semantic alignment on the formal specification based on the first image data, and determine the alignment semantic representation. Executable data determination module 04 is used to decode the alignment semantic representation and determine the executable data; System status acquisition module 05 is used to acquire information on available CPU resources and edge devices in the system. Runtime assurance module 06 is used to determine the target mode based on the available CPU resources of the system and the edge device information, perform runtime assurance based on the target mode and the executable data, and determine the runtime assurance result. The control instruction generation module 07 is used to generate device control instructions based on the runtime protection results through the operating system security hub.

[0116] In one implementation, the structured representation module includes: The parsing unit is used to parse the natural language requirements and determine the requirements parsing information; A multimodal conversion unit is used to perform multimodal conversion on the requirement parsing information through a large language model to determine the multimodal requirement parsing information; The structured conversion unit is used to generate structured information in restricted English based on the multimodal requirement parsing information, and to determine the structured requirement information.

[0117] In one implementation, the formal specification determination module includes: A credibility verification unit is used to verify the credibility of the structured requirement information and determine the requirement verification result. The formal specification determination unit is used to acquire user confirmation requirement information for determination based on the visualization requirement verification results, and to transform the user confirmation requirement information into a formal specification in linear temporal logic form.

[0118] In one implementation, the trustworthiness verification unit includes: The first filtering unit is used to obtain a domain knowledge base, filter the structured requirement information based on the domain knowledge base, and determine the filtered requirement information. The difference measurement calculation unit is used to calculate the difference measurement between the various filtered requirements information. The requirement verification result determination unit is used to determine the requirement verification result based on the filtered requirement information and the difference measure.

[0119] In one implementation, the first screening unit includes: A requirement similarity calculation unit is used to calculate the requirement similarity between the structured requirement information and the historical conflict requirements in the domain knowledge base; The second filtering unit is used to obtain a similarity threshold and to use the structured demand information whose demand similarity is less than the similarity threshold as the filtered demand information.

[0120] In one implementation, the alignment semantic representation determination module includes: A semantic representation unit is used to map the logical rules of the formal specification into semantic representations and determine the formal semantic representations; The semantic alignment unit is used to align each formal semantic representation according to the first image data using a visual language model, and to determine the aligned semantic representation.

[0121] In one implementation, the semantic alignment unit includes: A semantic concept vector determination unit is used to map the formal predicates in each of the formal semantic representations to semantic concept vectors of the visual language model; The cosine similarity calculation unit is used to convert the first image data into a first image vector and calculate the cosine similarity between each semantic concept vector and the image vector. A semantic alignment representation determination unit is used to align each formal semantic representation according to each cosine similarity to determine the aligned semantic representation.

[0122] In one implementation, the runtime protection module includes: The first judgment unit is used to obtain the CPU threshold and determine whether the available CPU resources of the system are greater than the CPU threshold. The monitoring mode unit is used to determine, if so, that the target mode is a monitoring mode.

[0123] In one implementation, the runtime protection module further includes: The second determination unit is used to obtain the edge device set if no, and determine whether the edge device belongs to the edge device set based on the edge device information; The test module unit is used to determine the target mode as the test mode if the condition is met. A diagnostic mode unit is used to determine the target mode as a diagnostic mode if no.

[0124] In one implementation, the monitoring mode unit includes: A predicate similarity calculation unit is used to acquire second image data under the target mode and calculate the similarity between each predicate in the second image data and the executable data; The system current state determination unit is used to determine the set type corresponding to the system state based on each similarity, and to determine the current state of the system based on the set type. The set type includes a safe type and an unsafe type. The first runtime assurance result determination unit is used to determine the runtime assurance result based on the current state of the system.

[0125] In one implementation, the diagnostic mode unit includes: A sensitivity matrix determination unit is used to acquire third image data in the target mode and determine the sensitivity matrix corresponding to the executable data based on the third image data and the executable data. The second runtime assurance result determination unit is used to determine the current fault result based on the sensitivity matrix and the executable data, and to use the current fault result as the runtime assurance result.

[0126] In one embodiment, the apparatus further includes: An abnormal executable data acquisition unit is used to acquire abnormal executable data fed back by the operating system security center; A semantic similarity calculation unit is used to calculate the semantic similarity between the semantic representation and the formal reduction corresponding to the abnormal executable data; The data update unit is used to update the abnormal executable data according to the semantic similarity and the formal specification.

[0127] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 15 As shown, the terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. The computer program is executed by the processor to implement semantic continuity assurance methods. The display screen can be a liquid crystal display (LCD) or an e-ink display.

[0128] Those skilled in the art will understand that Figure 15 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] In one implementation, the terminal's memory stores one or more programs, and these programs are configured to be executed by one or more processors, and the programs contain instructions for performing semantic continuity assurance methods.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0131] In summary, this invention discloses a method, apparatus, terminal, and storage medium for ensuring semantic continuity. The method involves: representing natural language requirements in a structured manner to determine structured requirement information; converting the structured requirement information into a formal reduction of linear temporal logic; performing semantic representation and alignment of the formal reduction based on first image data to determine an aligned semantic representation; decoding the aligned semantic representation to determine executable data; determining a target pattern based on available CPU resources and edge device information; performing runtime assurance based on the target pattern and executable data to determine the runtime assurance result; and generating device control instructions based on the runtime assurance result through the operating system security hub. Therefore, this method effectively solves the problem that existing technologies and methods cannot fundamentally address the lack of semantic continuity across the entire chain, making it difficult to meet the high-reliability operation requirements of safety-critical AI systems in dynamic environments.

[0132] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for ensuring semantic continuity, characterized in that, The method includes: Obtain natural language requirements, perform structured representation of the natural language requirements, and determine structured requirement information; The structured requirement information is converted into a formal specification in linear temporal logic form; Acquire first image data, and perform semantic representation and semantic alignment on the formal specification based on the first image data to determine the aligned semantic representation; The aligned semantic representation is decoded to determine executable data; Obtain information on available CPU resources and edge devices in the system; The target mode is determined based on the available CPU resources of the system and the edge device information. Runtime assurance is performed based on the target mode and the executable data, and the runtime assurance result is determined. The operating system security hub generates device control commands based on the runtime assurance results.

2. The semantic continuity guarantee method according to claim 1, characterized in that, The natural language requirements are represented in a structured manner to determine the structured requirement information, including: The natural language requirements are parsed to determine the requirements parsing information; The requirement parsing information is transformed into a multimodal form using a large language model to determine the multimodal requirement parsing information; Based on the multimodal requirement parsing information, structured information in restricted English is generated, and the structured requirement information is determined.

3. The semantic continuity guarantee method according to claim 1, characterized in that, The formal reduction of the structured requirement information into a linear temporal logical form includes: The structured requirement information is verified for credibility, and the requirement verification result is determined. Obtain user confirmation requirement information for determination based on visualization-based requirement verification results, and transform the user confirmation requirement information into a formal reduction of linear temporal logic form.

4. The semantic continuity guarantee method according to claim 3, characterized in that, The structured requirement information is subjected to credibility verification to determine the requirement verification result, including: Obtain a domain knowledge base, filter the structured requirement information based on the domain knowledge base, and determine the filtered requirement information; Calculate the difference measure between the different filtered demand information; The requirement verification result is determined based on the filtered requirement information and the difference metric.

5. The semantic continuity guarantee method according to claim 4, characterized in that, The structured requirement information is filtered based on the domain knowledge base to determine the filtered requirement information, including: Calculate the similarity between the structured requirement information and the historical conflicting requirements in the domain knowledge base; Obtain a similarity threshold, and use the structured demand information whose demand similarity is less than the similarity threshold as the filtered demand information.

6. The semantic continuity guarantee method according to claim 1, characterized in that, Based on the first image data, the formal specification is semantically represented and semantically aligned to determine the aligned semantic representation, including: The logical rules of the formal specification are mapped to semantic representations to determine the formal semantic representations; The visual language model aligns the formal semantic representations based on the first image data to determine the aligned semantic representation.

7. The semantic continuity guarantee method according to claim 6, characterized in that, The visual language model aligns the formal semantic representations based on the first image data to determine the aligned semantic representations, including: The formal predicates in each of the formal semantic representations are mapped to semantic concept vectors of the visual language model; The first image data is converted into a first image vector, and the cosine similarity between each semantic concept vector and the image vector is calculated. Align each formal semantic representation according to the cosine similarity to determine the aligned semantic representation.

8. The semantic continuity guarantee method according to claim 1, characterized in that, Determining the target mode based on the available CPU resources of the system and the edge device information includes: Obtain the CPU threshold and determine whether the available CPU resources of the system are greater than the CPU threshold. If so, determine that the target mode is monitoring mode.

9. The semantic continuity guarantee method according to claim 8, characterized in that, The method further includes: If not, obtain the edge device set, and determine whether the edge device belongs to the edge device set based on the edge device information; If so, then the target mode is determined to be the test mode; If not, then the target mode is determined to be the diagnostic mode.

10. The semantic continuity guarantee method according to claim 8, characterized in that, When the target mode is the monitoring mode, runtime assurance is performed based on the target mode and the executable data, and the runtime assurance result is determined, including: Acquire the second image data under the target mode, and calculate the similarity between the second image data and each predicate in the executable data; The system state is determined based on the similarity scores, and the current system state is determined based on the set type. The set type includes a safe type and an unsafe type. The runtime guarantee result is determined based on the current state of the system.

11. The semantic continuity guarantee method according to claim 9, characterized in that, When the target mode is the diagnostic mode, runtime assurance is performed based on the target mode and the executable data, and the runtime assurance result is determined, including: Acquire third image data in the target mode, and determine the sensitivity matrix corresponding to the executable data based on the third image data and the executable data; The current fault result is determined based on the sensitivity matrix and the executable data, and the current fault result is used as the runtime guarantee result.

12. The semantic continuity guarantee method according to claim 1, characterized in that, The method further includes: Obtain abnormal executable data reported by the operating system security hub; Calculate the semantic similarity between the semantic representation and the formal reduction corresponding to the abnormal executable data; The abnormal executable data is updated based on the semantic similarity and the formal specification.

13. A semantic continuity guarantee device, characterized in that, The device includes: The structured representation module is used to acquire natural language requirements, perform structured representation on the natural language requirements, and determine the structured requirement information; The formal specification determination module is used to convert the structured requirement information into a formal specification in linear temporal logic form; The alignment semantic representation determination module is used to acquire first image data, perform semantic representation and semantic alignment on the formal specification based on the first image data, and determine the alignment semantic representation. The executable data determination module is used to decode the alignment semantic representation and determine the executable data; The system status acquisition module is used to acquire information on available CPU resources and edge devices in the system. The runtime assurance module is used to determine the target mode based on the available CPU resources of the system and the edge device information, perform runtime assurance based on the target mode and the executable data, and determine the runtime assurance result. The control command generation module is used to generate device control commands based on the runtime protection results through the operating system security hub.

14. The semantic continuity guarantee device according to claim 13, characterized in that, The structured representation module includes: The parsing unit is used to parse the natural language requirements and determine the requirements parsing information; A multimodal conversion unit is used to perform multimodal conversion on the requirement parsing information through a large language model to determine the multimodal requirement parsing information; The structured conversion unit is used to generate structured information in restricted English based on the multimodal requirement parsing information, and to determine the structured requirement information.

15. The semantic continuity guarantee device according to claim 13, characterized in that, The formal specification determination module includes: A credibility verification unit is used to verify the credibility of the structured requirement information and determine the requirement verification result. The formal specification determination unit is used to acquire user confirmation requirement information for determination based on the visualization requirement verification results, and to transform the user confirmation requirement information into a formal specification in linear temporal logic form.

16. The semantic continuity guarantee device according to claim 13, characterized in that, The alignment semantic representation determination module includes: A semantic representation unit is used to map the logical rules of the formal specification into semantic representations and determine the formal semantic representations; The semantic alignment unit is used to align each formal semantic representation according to the first image data using a visual language model, and to determine the aligned semantic representation.

17. The semantic continuity guarantee device according to claim 13, characterized in that, The runtime protection module includes: The first judgment unit is used to obtain the CPU threshold and determine whether the available CPU resources of the system are greater than the CPU threshold. The monitoring mode unit is used to determine, if so, that the target mode is a monitoring mode.

18. The semantic continuity guarantee device according to claim 17, characterized in that, The runtime protection module also includes: The second determination unit is used to obtain the edge device set if no, and determine whether the edge device belongs to the edge device set based on the edge device information; The test module unit is used to determine the target mode as the test mode if the condition is met. A diagnostic mode unit is used to determine the target mode as a diagnostic mode if no.

19. A terminal, characterized in that, The terminal includes a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for executing the semantic continuity guarantee method as described in any one of claims 1-12; the processors are used to execute the programs.

20. A computer-readable storage medium storing a plurality of instructions, characterized in that, The instructions are loaded and executed by the processor to implement the steps of the semantic continuity guarantee method according to any one of claims 1-12.

Images