Industrial large model prompt word security detection method and system

Abstract

The application discloses an industrial large model prompt word security detection method and system, relates to the field of industrial artificial intelligence application security detection, and comprises the following steps: constructing a prompt word basic security detection model, an industrial field special semantic analysis model and an industrial field special security detection model according to an industrial knowledge resource library and an industrial knowledge graph; performing basic security detection on the prompt word by using the prompt word basic security detection model; performing semantic analysis on the prompt word to be processed by using the industrial field special semantic analysis model; performing special security detection on the prompt word to be processed by using the industrial field special security detection model; obtaining a prompt word detection result, outputting a risk level and a disposal suggestion by using a dynamic weighting risk fusion evaluation method according to the special security detection result; and the application can improve the security in the process of processing the industrial large model prompt word, improve the adaptability of the industrial large model application and reduce the cost.

Description

Technical Field

[0001] This application relates to the field of security detection for industrial artificial intelligence applications, and in particular to a method and system for security detection of prompt words in large industrial models. Background Technology

[0002] With the increasing application of large-scale models in industrial fields, the security risks of their prompt words have transcended the scope of illegal information and inappropriate content in general scenarios. Prompt words refer to input information or instructions provided by a computer program or model. In large language models, prompt words are questions or statements provided by the user to guide the large model in generating relevant responses. Prompt word attacks refer to attackers inputting malicious keywords into the large model to manipulate it to output content desired by the attacker. Currently, prompt word attacks mainly include jailbreak attacks that bypass the model's preset security audit mechanisms, inducement attacks that guide the model to disclose information or output inappropriate content through multiple rounds of dialogue, and injection attacks that hijack or attack the model. Currently, the main countermeasures against prompt word attacks on large-scale models are prompt word attack detection methods.

[0003] Existing general-purpose large-scale model prompt security detection methods primarily focus on ethical, privacy, bias, and content security issues in general industry application scenarios of large-scale models. The aim is to identify general risks in prompts, such as risks related to role legitimacy, code security, semantic logic, and data leakage. For example, existing technology provides a method and apparatus for detecting prompt attacks on large language models, relating to the field of information processing technology. This method includes: acquiring prompt information to be processed; identifying the type of the prompt information to obtain the prompt type information corresponding to the prompt information; determining the attack judgment model corresponding to the prompt information to be processed based on the prompt type information; sequentially inputting the prompt information to be processed into each judgment sub-model of the attack judgment model to obtain sub-model evaluation information corresponding to each judgment sub-model; and determining whether the prompt information to be processed is an attack targeting the large language model based on the sub-model evaluation information. This method can improve the security of large language models during prompt processing. Furthermore, this method primarily focuses on detecting prompt attacks on general-purpose large-scale models. It utilizes four sub-models—role-playing assessment, code security assessment, semantic logic assessment, and data leakage risk assessment—to collaboratively determine whether the prompt information to be processed constitutes an attack against the large-scale model. It is also used to identify risks related to the role legitimacy, code security, semantic logic, and data leakage of the prompt information. While this method focuses on detecting prompt attacks on general-purpose large-scale models to address ethical, privacy, bias, and content security issues in general industry application scenarios, it lacks the ability to identify security risks specific to industrial-domain large-scale model applications. For example, it cannot identify whether process parameters exceed safety thresholds, determine whether equipment operation instructions conform to business security logic, verify whether operation procedures violate industrial production safety regulations, or dynamically assess the risk of input content in conjunction with real-time production status.

[0004] Industrial large-scale models are specialized AI models with industrial attributes, trained by combining industrial and general data, or fine-tuned from general large-scale models. They are highly adaptable to downstream industrial tasks and scenarios, including basic industrial large-scale models, industry-specific large-scale models, and task-oriented large-scale models for industrial scenarios. Driven by the fusion of industrial data and industrial mechanism knowledge, industrial large-scale models possess strong professionalism, process understanding, and human-machine collaboration characteristics. However, attacks targeting industrial large-scale models with specific prompts can lead to risks such as leakage of core industrial knowledge, malicious tampering of production instructions, and inducing safety accidents. Without the ability to detect the security of prompts on industrial large-scale models, there is a lack of defense against the security risks posed by such attacks.

[0005] Therefore, there is an urgent need for a large-scale model-based method for detecting the security of prompt words that is deeply adapted to industrial scenarios. Summary of the Invention

[0006] The purpose of this application is to provide a method and system for detecting the security of industrial large model prompts, which can improve the security of industrial large model prompt processing, enhance the adaptability to industrial large model applications, and reduce costs.

[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for detecting the security of prompt words in large industrial models, the method comprising: Construct an industrial knowledge resource base; the industrial knowledge resource base includes: an industrial terminology resource base, a process parameter specification base, an operation instruction hierarchical base, an industrial data permission base, and a safe operation procedure base. An industrial knowledge graph is constructed based on the aforementioned industrial knowledge resource base; the industrial knowledge graph uses terms, parameters, instructions, data, and operation processes as nodes, and parameter ranges, instruction constraints, permission requirements, temporal relationships, and causal relationships as edges; Based on the industrial knowledge resource base and industrial knowledge graph, a basic security detection model for prompt words, an industrial domain-specific semantic parsing model, and an industrial domain-specific security detection model are constructed using deep learning algorithms. A basic security detection model for prompt words is used to perform basic security checks on the prompt words to be processed; the basic security checks include: inappropriate content filtering, sensitive information identification, and adversarial attack detection. When the prompt word to be processed passes the basic security detection, the prompt word to be processed is semantically analyzed using an industrial-specific semantic parsing model to obtain the recognition result; the recognition result is whether it contains industrial knowledge type information. When the identification result contains industrial knowledge type information, a special security detection model for the industrial field is used to perform special security detection on the prompt words to be processed, and special security detection results are obtained. Special security detection includes: industrial entity security detection, process parameter security detection, operation instruction security detection, industrial data security detection, and operation process security detection. Based on the results of the special security test, a dynamic weighted risk fusion assessment method is used to obtain the prompt word detection results, and output the risk level and handling suggestions; the prompt word detection results indicate whether the information contains attack information.

[0008] Secondly, this application provides an industrial large-scale model prompt word security detection system, the industrial large-scale model prompt word security detection system comprising: The industrial knowledge resource base management module is used to build an industrial knowledge resource base; the industrial knowledge resource base includes: an industrial terminology resource base, a process parameter specification base, an operation instruction hierarchical base, an industrial data permission base, and a safe operation procedure base. The industrial knowledge graph management module is used to construct an industrial knowledge graph based on the industrial knowledge resource base; the industrial knowledge graph uses terms, parameters, instructions, data and operation processes as nodes, and parameter ranges, instruction constraints, permission requirements, temporal relationships and causal relationships as edges; The model building module is used to construct a basic security detection model for prompt words, an industrial domain-specific semantic parsing model, and an industrial domain-specific security detection model based on the industrial knowledge resource base and the industrial knowledge graph, respectively, using deep learning algorithms. The basic security detection module for prompt words is used to perform basic security detection on the prompt words to be processed using the basic security detection model for prompt words; the basic security detection includes: inappropriate content filtering, sensitive information identification, and adversarial attack detection; The industrial domain-specific semantic parsing module is used to perform semantic parsing on the prompt words to be processed when they pass the basic security detection, using an industrial domain-specific semantic parsing model to obtain the recognition result; the recognition result is whether it contains industrial knowledge type information. The industrial-specific security detection module is used to perform specific security detection on the prompt words to be processed when the identification result contains industrial knowledge type information, using an industrial-specific security detection model to obtain specific security detection results. Specific security detection includes: industrial entity security detection, process parameter security detection, operation instruction security detection, industrial data security detection, and operation process security detection. The special security detection result fusion and evaluation module is used to obtain the prompt word detection result based on the special security detection result, using a dynamically weighted risk fusion evaluation method, and output the risk level and handling suggestions; the prompt word detection result indicates whether it contains attack information.

[0009] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method and system for detecting the security of prompt words in large-scale industrial models. It establishes an industrial knowledge resource base and constructs an industrial knowledge graph based on industrial terminology, process parameters, operating instructions, industrial data, and safe operating procedures. Based on the industrial knowledge resource base and the industrial knowledge graph, it constructs a basic prompt word security detection model, an industrial-specific semantic parsing model, and an industrial-specific security detection model using deep learning algorithms. This deeply binds the prompt word security detection logic with industrial standards, process specifications, equipment characteristics, industrial data security requirements, and industrial safe operating procedures. This application can accurately identify security risks unique to large-scale industrial model applications, such as parameters exceeding compliance limits, instruction logic violations, and abnormal process timing. It significantly improves the adaptability of prompt word security detection methods to industrial scenarios and solves the problem that general large-scale model prompt word security detection methods cannot identify security risks specific to the industrial field. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of a method for detecting the security of prompt words in an industrial large model according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the principle of a security detection method for prompt words in a large industrial model. Figure 3 This is a schematic diagram of an overall method for detecting the security of prompt words in a large industrial model. Figure 4 This is a schematic diagram of the architecture of an industrial large-scale model prompt word security detection system. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] In one exemplary embodiment, such as Figure 1 , Figure 2 as well as Figure 3 As shown, a method for detecting the security of prompt words in large industrial models is provided, which includes the following steps S101 to S107. Wherein: S101, Construct an industrial knowledge resource base; the industrial knowledge resource base includes: an industrial terminology resource base, a process parameter specification base, an operation instruction hierarchical base, an industrial data permission base, and a safe operation procedure base. S101 specifically includes: S11. Construct an industrial terminology resource library based on general industrial terms, industry-specific terms, and specific scenario terms, as well as the corresponding first annotation information; the first annotation information includes, but is not limited to: industry, applicable scenario, process step, term name, and safety level. Specifically, it extracts general industrial terms, industry-specific terms, and scenario-specific terms from ISO international standards, GB national standards, and internal enterprise standards and specifications. It also labels industrial terms with information such as their industry, applicable scenario, process stage, term name, and safety level. The process stages include at least five stages: R&D design, pilot testing and verification, manufacturing, business management, and operation and maintenance.

[0015] S12, construct a process parameter specification library based on the process parameters of each stage and the corresponding second annotation information; the second annotation information includes, but is not limited to: industry, applicable scenario, process stage, parameter name, equipment type, compliance scope, related parameters, and risk level; the process parameters include, but are not limited to: temperature parameters, pressure parameters, flow parameters, cutting parameters, process dimension parameters, and surface quality parameters.

[0016] Specifically, process parameters for each stage are extracted from ISO international standards, GB national standards, and internal enterprise standards and specifications. These parameters are labeled with information such as their industry, applicable scenarios, process steps, parameter name, equipment type, compliance scope, related parameters, and risk level. The risk level includes at least three levels: compliant, high-risk, and prohibited. Process parameters are core elements for ensuring stable industrial production processes and meeting product quality standards; the process parameters involved vary across different industries and production stages.

[0017] S13. Construct a hierarchical library of operation instructions based on the operation instructions for controlling industrial equipment and industrial control systems and the corresponding third annotation information; the third annotation information includes, but is not limited to: industry, applicable scenario, process step, instruction name, instruction category, applicable object, permission requirements, and risk level. Specifically, the operation instructions for controlling industrial equipment and industrial control systems are extracted from ISO international standards, GB national standards, enterprise internal standards and specifications, and equipment and system operation manuals. At the same time, the operation instructions are labeled with information such as their industry, applicable scenarios, process steps, instruction name, instruction category, applicable objects, authorization requirements, and risk level.

[0018] S14. Based on the industrial data involved in each stage of industrial production and the corresponding fourth annotation information, construct an industrial data permission library; the fourth annotation information includes, but is not limited to: industry, applicable scenario, process stage, data name, data type, data level, source equipment, source time, and permission requirements. Specifically, industrial data involved in each stage of industrial production is labeled with information such as industry, applicable scenario, process step, data name, data type, data level, source equipment, source time, and access requirements. Industrial data types include, but are not limited to, classification dimensions such as R&D data domain, production data domain, operation and maintenance data domain, management data domain, and external data domain. Industrial data levels include three levels: general, important, and core.

[0019] S15. Construct a safety operation process library based on the standard safety operation procedures for each process step and the corresponding fifth annotation information; the fifth annotation information includes, but is not limited to: industry, applicable scenario, process step, key safety node, and process sequence constraints.

[0020] The standard safety operating procedures for each process step are extracted from ISO international standards, GB national standards, enterprise internal standards and specifications, equipment and system operation manuals, etc. At the same time, the safety operating procedures are marked with information such as the industry, applicable scenarios, process steps, key safety nodes, and process sequence constraints.

[0021] The industrial knowledge resource base sources data including structured and unstructured data. Structured data includes data stored in the database during industrial production processes; unstructured data includes data extracted from ISO international standards and GB national standards, process specification documents, equipment operation manuals, safety operation standards, etc.

[0022] For the industrial knowledge resource base, apart from the content that should be included in some of the annotation information as specified in the above steps, the remaining annotation information that is not explicitly included can be customized by the user according to the different specific application scenarios of the large model in the industrial sector.

[0023] S102, Construct an industrial knowledge graph based on the industrial knowledge resource base; the industrial knowledge graph uses terms, parameters, instructions, data and operation processes as nodes, and parameter ranges, instruction constraints, permission requirements, temporal relationships and causal relationships as edges; Specifically, an industrial domain semantic association network is constructed based on the industrial knowledge graph; the industrial domain semantic association network is used for semantic association analysis and semantic reasoning, and based on the hard constraints of the industrial knowledge graph and combined with adaptive thresholds of real-time industrial data, violations are identified; the violations include: incorrect use of terminology, conflicting parameter combinations, and failure to follow the safe operating procedure sequence.

[0024] S103, Based on the industrial knowledge resource base and industrial knowledge graph, construct the basic security detection model of prompt words, the industrial domain-specific semantic parsing model and the industrial domain-specific security detection model respectively based on deep learning algorithms; The basic security detection model for prompt words is a multi-task text security classification model. Its core functions are to filter inappropriate content, identify sensitive information, and detect adversarial attacks. Its specific structure follows a hierarchical design of "input layer → word segmentation and encoding layer → pre-training base → pooling layer → multi-task output head". The input layer can receive prompt word text with a maximum of 128 / 256 tokens, in the format "[CLS] + prompt word + [SEP]". The word segmentation and encoding layer uses a BPE-based word segmenter (such as the Byte-Level BPE used in Facebook's pre-trained language model RoBERTa). The embedding layer includes token embedding and position embedding. There are three types of embedding: (emedding) and optional segment embedding; the pre-trained encoder base can be a basic model adapted to Chinese sensitive content recognition (such as the full-word mask enhanced version of the Roberta basic model, RoBERTa-wwm-ext-base); the pooling layer takes the [CLS] vector as the global semantic representation and connects Dropout (dropout rate 0.1–0.3) to prevent overfitting; the multi-task output layer sets up 3 parallel classification heads, corresponding to inappropriate content detection (binary classification), sensitive information recognition (multi-class classification) and adversarial attack detection (binary classification). The training process of this model revolves around multi-task joint training. First, a training dataset is constructed, combining inappropriate content, predefined sensitive information from enterprises, and adversarial attack prompts. The dataset is uniformly labeled in the format of "(text, inappropriate label, sensitive label, adversarial label)". After word segmentation, truncation, and padding preprocessing, Chinese general pre-trained BERT / RoBERTa weights are loaded, and three classification heads are randomly initialized. The AdamW optimizer is used for training, with a learning rate of 2e-5, batch size of 16–32, and 5–10 training epochs. Gradient pruning is performed to prevent gradient explosion. The loss function is multi-task weighted cross-entropy, and the weights of each task are adjusted according to the imbalance of samples. After training, precision, recall, and F1 score are used as the core evaluation indicators, and the final output is a comprehensive security score and risk type.

[0025] The core task of the industrial domain-specific semantic parsing model is to determine whether the prompt words to be processed contain industrial knowledge type information, and the output is a binary classification (containing industrial knowledge / not containing industrial knowledge). It can be expanded to a fine-type classification of industrial knowledge. Its specific structure is "input layer → industrial terminology segmentation → industrial domain pre-trained encoder → semantic aggregation layer → classification head". This model adds an industrial dictionary (covering equipment names, process names, sensors, PLC instructions, etc.) to the general word segmentation to build an industrial-specific word segmentation layer; the pre-trained encoder base preferentially uses industrial domain pre-trained models (such as BERT / RoBERTa models adapted to industrial applications), and can also use general Chinese pre-trained models (such as the Harbin Institute of Technology Chinese-BERT model, Baidu ERNIE model, etc.). The model's adaptability to the industrial domain is improved by enhancing the domain word embedding method; the semantic aggregation layer focuses on capturing industrial entity words through local attention, and then concatenates the global average pooling result with the [CLS] vector to strengthen the industrial semantic representation; the output layer is a binary classification head, which can be expanded to a multi-classification head for fine-type industrial knowledge as needed. During training, positive samples (containing industrial knowledge such as industrial equipment, processes, parameters, control instructions, and process descriptions) and negative samples (contacts without industrial knowledge such as casual conversation, general questions, and irrelevant text) are prepared first. The core step is to pre-train the model using a large amount of unlabeled industrial text to enable it to master the semantics of industrial terms. Then, fine-tuning training is performed with a learning rate set to 1e-5 to 2e-5. Industrial terminology replacement operations are added to achieve data augmentation. The evaluation process focuses on the recall rate of industrial knowledge to ensure that no prompts containing industrial knowledge are missed.

[0026] This industrial-specific safety detection model performs deep safety checks on prompts containing industrial knowledge, covering five dimensions: industrial entity safety, process parameter safety, operating instruction safety, industrial data safety, and operational process safety. Its specific structure is: "Input → Industrial Safety Word Segmentation → Shared Industrial BERT Encoder → 5 Independent Detection Sub-models → Comprehensive Safety Judgment." The model's industrial safety embedding layer incorporates specialized dictionaries for industrial control terminology, hazardous operations, restricted area instructions, and non-compliant parameters. The shared pre-training base uses an industrial safety-specific pre-trained model containing 12 Transformer layers, adapted to long texts such as industrial processes and instructions. Five independent detection sub-models are set up in parallel, each corresponding to a safety detection dimension, all of which are binary classifications (compliance / violation). The comprehensive safety judgment layer combines the outputs of the five detection sub-models to ultimately output whether the prompt is safe, the specific risk item, and the risk level. During training, a dataset containing normal industrial instructions, parameters, processes, and manually constructed violation samples (out-of-bounds parameters, dangerous operations, erroneous processes, and confidential data) needs to be constructed. A joint training approach with five tasks is adopted, sharing the underlying semantic representation, and increasing the weight of samples related to dangerous operations. The loss function is the weighted sum of the losses of each task, with higher weights set for the process parameter and operation instruction safety detection tasks. At the same time, industrial scenario data augmentation is achieved through adversarial substitution of terms, perturbation of process sequence, and construction of parameter boundary values. The evaluation process focuses on ensuring high recall to ensure that no dangerous industrial warning words are missed, while also taking into account accuracy and reducing the probability of false triggers.

[0027] S104, The basic security detection model for prompt words is used to perform basic security detection on the prompt words to be processed; the basic security detection includes: inappropriate content filtering, sensitive information identification and adversarial attack detection; If the basic security check of the prompt word passes, input to the next step for security check is allowed; if the basic security check of the prompt word fails, input of the industrial large model is refused; the results of the basic security check of the prompt word are recorded, and the results are reviewed by a designated person, marking the missed and misjudged samples.

[0028] S105, when the prompt word to be processed passes the basic security detection, the prompt word to be processed is semantically analyzed using an industrial domain-specific semantic parsing model to obtain the recognition result; the recognition result is whether it contains industrial knowledge type information; the industrial knowledge type information includes industrial entities, process parameters, operation instructions, industrial data or operation procedures. When industrial knowledge type information is identified, the sub-model for specialized security detection in the industrial domain-specific security detection model can be determined based on this information to process the prompt words to be processed. In order to avoid the model learning irrelevant noise and interference information when inputting content that is irrelevant to the industrial application scenario, thereby reducing the model's generalization ability, increasing the model's data processing burden and wasting computing resources, the industrial large model is rejected when no industrial knowledge type information is identified in the prompt words. The results of the industrial domain-specific semantic recognition of the prompt words are recorded, and the detection results are reviewed by a designated person to mark the missed and misjudged samples.

[0029] S106, When the recognition result contains industrial knowledge type information, the industrial domain-specific security detection model is used to perform specific security detection on the prompt words to be processed, and the specific security detection result is obtained; the specific security detection includes: industrial entity security detection, process parameter security detection, operation instruction security detection, industrial data security detection, and operation process security detection. Specifically, the industrial-specific safety testing model includes: parallel sub-models for industrial entity safety testing, process parameter safety testing, operation instruction safety testing, industrial data safety testing, and operation process safety testing. The industrial entity security detection sub-model is used to identify industrial entity security risks from prompt words containing industrial entity knowledge type information, and to obtain industrial entity security detection results. Industrial entity security risk identification includes, but is not limited to, whether the words are sensitive terms, whether they are used in non-compliant scenarios, and whether there is malicious keyword replacement. The process parameter safety detection sub-model is used to identify process parameter safety risks based on prompts containing process parameter knowledge type information, and to obtain process parameter safety detection results. Process parameter safety risk identification includes, but is not limited to, whether the parameter risk level is high risk or prohibited, whether a single parameter exceeds the upper or lower limits of the compliance range, whether the relationship between multiple parameters meets the process constraints, and whether the change trend of parameters with time sequence requirements meets the safety procedures. The operation instruction security detection sub-model is used to identify operation instruction constraint security risks in prompt words containing operation instruction knowledge type information, and obtain operation instruction security detection results. Operation instruction constraint security risk identification includes, but is not limited to, whether the instruction operation permission matches the current user permission, whether the instruction risk level is high risk or prohibited, and whether the preconditions for instruction execution are met. The industrial data security detection sub-model is used to identify industrial data access security risks based on prompts containing industrial data knowledge types, and to obtain industrial data security detection results. The industrial data access security risk identification includes, but is not limited to, whether there is intellectual property infringement, whether there is sensitive data leakage, and whether there is unauthorized operation. The operation process security detection sub-model is used to identify security risks in operation logic, temporal relationship and causal relationship of prompt words containing operation process knowledge type, and obtain operation process security detection results; operation process security risk identification includes but is not limited to whether key security nodes are omitted, whether the operation process order is incorrect, and whether there are unauthorized operations.

[0030] S107. Based on the results of the special security test, a dynamic weighted risk fusion assessment method is used to obtain the prompt word detection results, and output the risk level and handling suggestions; the prompt word detection results indicate whether the information contains attack information.

[0031] The results of the prompt word detection are divided into three levels: safe level, low risk level, and high risk level. The results of the prompt word detection are recorded, and the results are reviewed by a designated person, and the missed and misjudged samples are marked.

[0032] S107 specifically includes: S71, use the formula V=w1×v1+w2×v2+w3×v3+w4×v4+w5×v5 to determine the prompt word detection result V; S72, when the prompt word detection result is at the safe level, the recommended action is to directly input it into the large industrial model; S73, when the prompt word detection result is low risk level, the recommended action is to inform the user of the risk and then input the industrial large model; S74, when the prompt word detection result is high-risk, the recommended action is to refuse to input the industrial large model; Among them, w1, w2, w3, w4 and w5 are the weights of the industrial entity safety detection sub-model, process parameter safety detection sub-model, operation instruction safety detection sub-model, industrial data safety detection sub-model and operation process safety detection sub-model, respectively, and v1, v2, v3, v4 and v5 are the industrial entity safety detection results, process parameter safety detection results, operation instruction safety detection results, industrial data safety detection results and operation process safety detection results, respectively.

[0033] As a specific example, the outputs of the five detection sub-models are all in numerical form that can be used for calculation. The specific output method and comprehensive judgment algorithm are as follows: Each detection sub-model outputs a numerical result through the sigmoid function, defined as v1, v2, v3, v4, and v5 (corresponding to the industrial entity, process parameter, operation instruction, industrial data, and operation process safety detection sub-models, respectively), with a numerical range of [0,1]. i The closer to 0, the more the detection result of this dimension is biased towards "compliance"; the closer to 1, the more the detection result of this dimension is biased towards "violation".

[0034] That is, v i = sigmoid(z i ), z i The sigmoid function formula is as follows: (This is the output value of the last fully connected layer in the corresponding detection sub-model.) ; Meanwhile, w1, w2, w3, w4, and w5 are set as weights for the five detection sub-models (corresponding to the five sub-models mentioned above), with weight values ​​ranging from [0,1] and satisfying the following formula: ; Based on business priorities, in this implementation example, w2 (process parameters) and w3 (operation instructions) are given higher weights than other sub-models (e.g., w2=0.3, w3=0.3, w1=0.1, w4=0.15, w5=0.15). The overall safety assessment uses a weighted summation algorithm to calculate the total risk score S, with the formula: ; The value of S ranges from [0,1]. Risk levels are then determined based on the total risk score, with the following thresholds: S < 0.2 for "safe", 0.2 ≤ S < 0.5 for "low risk", 0.5 ≤ S < 0.8 for "medium risk", and S ≥ 0.8 for "high risk". The final output indicates whether the risk is safe and specifies the risk item (v). i (Detection dimension corresponding to ≥0.5) and risk level.

[0035] During the training of the five sub-models, a dataset containing normal industrial instructions, parameters, processes, and manually constructed violation samples (out-of-bounds parameters, dangerous operations, erroneous processes, and confidential data) needs to be constructed. A joint training approach is adopted for the five tasks, sharing the underlying semantic representation, and increasing the weight of samples related to dangerous operations. The loss function is the weighted sum of the losses from each task, i.e. ; Among them, Loss i To compensate for the binary cross-entropy loss of the corresponding sub-model, the weights of the process parameters and operation command safety detection tasks are set higher.

[0036] The weights of the industrial entity safety detection sub-model, process parameter safety detection sub-model, operation instruction safety detection sub-model, industrial data safety detection sub-model, and operation process safety detection sub-model are dynamically adjusted based on the applicable industry, applicable scenario, and process step. Specifically, enterprise users can adjust the weight values ​​of different w, the threshold of the total risk score S, and the sigmoid function in the examples given above as needed, dynamically adjusting them to achieve customized effects. To ensure that cue word detection methods continuously adapt to changes in industry standards and process requirements, and to improve the adaptability of cue word detection methods to large-scale industrial model applications, S107 and later versions also include: Based on changes in industrial standards and process specifications, missed or misjudged samples during the prompt word detection process, and risk events triggered by the industrial big data model, the industrial knowledge resource base and industrial knowledge graph are dynamically updated; and based on the updated industrial knowledge resource base, industrial knowledge graph, and newly added samples, the basic security detection model for prompt words, the industrial domain-specific semantic parsing model, and the industrial domain-specific security detection model are iteratively optimized.

[0037] Simultaneously, a subdivided industry-specific adaptation mechanism is designed to support the import of standards, specifications, operation manuals, and safety requirements from specific subdivided industries to update the industrial knowledge resource base. Based on this, the industrial knowledge graph is updated synchronously and dynamically, and the basic security detection model for prompt words, the industrial domain-specific semantic parsing model, and the industrial domain-specific security detection model are iteratively optimized. This enables the prompt word attack detection method to quickly adapt to the prompt word attack detection needs of large industrial models in specific subdivided industries.

[0038] Based on the same inventive concept, this application also provides an industrial large model prompt word security detection system for implementing the above-mentioned industrial large model prompt word security detection method. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more industrial large model prompt word security detection system embodiments provided below can be found in the limitations of the industrial large model prompt word security detection method above, and will not be repeated here.

[0039] In one exemplary embodiment, such as Figure 4 As shown, an industrial large-scale model prompt word security detection system is provided, including: The industrial knowledge resource base management module is used to construct an industrial knowledge resource base. The industrial knowledge resource base includes: an industrial terminology resource base, a process parameter specification base, an operation instruction hierarchical base, an industrial data permission base, and a safe operation procedure base. It also includes four sub-modules: industrial knowledge resource collection, industrial knowledge standardization processing, industrial knowledge annotation, and resource base management. The industrial knowledge graph management module is used to construct an industrial knowledge graph based on the industrial knowledge resource base; and includes two sub-modules: industrial knowledge graph construction and maintenance, and industrial semantic association analysis and semantic reasoning; the industrial knowledge graph uses terms, parameters, instructions, data and operation processes as nodes, and parameter ranges, instruction constraints, permission requirements, temporal relationships and causal relationships as edges; The model building module is used to construct a basic security detection model for prompt words, an industrial domain-specific semantic parsing model, and an industrial domain-specific security detection model based on the industrial knowledge resource base and the industrial knowledge graph, respectively, using deep learning algorithms. The basic security detection module for prompt words is used to perform basic security detection on the prompt words to be processed using the basic security detection model for prompt words; it includes three sub-modules: basic security detection model construction and maintenance, basic security detection of prompt words, and recording and review of basic security detection results by designated personnel; the basic security detection includes: inappropriate content filtering, sensitive information identification, and adversarial attack detection; The industrial domain-specific semantic parsing module is used to perform semantic parsing on the prompt words to be processed when they pass the basic security detection. It utilizes an industrial domain-specific semantic parsing model to obtain the recognition result. The module includes five sub-modules: industrial domain-specific semantic parsing model construction and maintenance, industrial domain-specific semantic parsing of prompt words, industrial knowledge type information recognition, recognition result recording and expert review, and a special security detection sub-module for determination. The recognition result indicates whether industrial knowledge type information is included. The industrial-specific security detection module is used to perform specific security detection on the prompt words to be processed when the identification result contains industrial knowledge type information. It uses an industrial-specific security detection model to obtain specific security detection results. It includes two sub-modules: the construction and maintenance of the specific security detection sub-module and the specific security detection itself. The specific security detection includes: industrial entity security detection, process parameter security detection, operation instruction security detection, industrial data security detection, and operation process security detection. The special security test result fusion evaluation module is used to obtain the prompt word detection result based on the special security test result using a dynamically weighted risk fusion evaluation method, and output the risk level and handling suggestions. It includes four sub-modules: weight value management, risk fusion evaluation and handling suggestion output, special security test result recording and special personnel review; the prompt word detection result is whether it contains attack information.

[0040] In addition, the industrial large model prompt word security detection system provided in this application also includes an industrial knowledge resource update and model optimization module: the industrial knowledge resource update and model optimization module is used to dynamically update industrial knowledge resources and models, and provides functions such as industrial knowledge base update and subdivided industry field adaptation, industrial knowledge graph update, prompt word basic security detection model optimization, industrial field special semantic parsing model optimization, and industrial field special security detection model optimization. The Industrial Knowledge Resource Update and Model Optimization module includes three sub-modules: Industrial Knowledge Resource Update, Industrial Knowledge Graph Update, and Model Iterative Optimization.

[0041] This application first utilizes a basic security detection module for prompt words to identify and filter inappropriate content, sensitive information, and adversarial attack information, ensuring that the prompt words input into the industrial big data model do not have issues related to general ethics, privacy, bias, or content security. Second, it employs an industry-specific semantic parsing model to identify industrial knowledge types, ensuring that the prompt words input into the industrial big data model contain industrial terminology, process parameters, operating instructions, industrial data, operating procedures, and other industrial knowledge types. This avoids the model learning irrelevant noise and interference when inputting content unrelated to the industrial application scenario, reducing the model's generalization ability, increasing the model's data processing burden, and wasting computational resources. Finally, it utilizes five specialized security detection sub-models within the industry-specific security detection module to perform specialized security detection on prompt words. This identifies risks related to the rationality of industrial entities, the security of process parameters, the security of operating instruction constraints, the security of industrial data permissions, and the security of operating procedure logic contained in the prompt words. A dynamically weighted prompt word risk fusion evaluation method is then used to fuse and evaluate the prompt word detection results, determining whether the prompt words input into the industrial big data model contain attack information. This application can filter prompt words from the perspective of general content compliance, and can also detect the security risks of prompt words from multiple levels such as entity terminology, parameter association, instruction hierarchy, data permissions, and process logic. It can effectively improve the security of prompt word processing in industrial big data models, enabling industrial big data models to provide services to industrial enterprises more securely.

[0042] This application clarifies a complete real-time process from the construction of the industrial knowledge resource base and model training to dynamic updates. It enables dynamic updates of the industrial knowledge resource base based on changes in industrial standards and process specifications, missed and misjudged samples during prompt word detection, and risk events triggered by large-scale industrial models. Furthermore, it allows for on-demand updates of the industrial knowledge resource base by importing standards, specifications, operation manuals, and safety requirements from specific sub-sectors of the industry. This updates the industrial knowledge graph accordingly, and based on the updated industrial knowledge resource base, industrial knowledge graph, and new samples, iterative optimizations are performed on industry-specific semantic parsing models and industry-specific security detection models. This application can be implemented in different industrial application scenarios without reconstructing the core system architecture. It ensures that the prompt word detection method continuously adapts to changes in industrial standards and process requirements, and rapidly adapts to applications in specific sub-sectors. It improves the adaptability of the prompt word detection method to large-scale industrial model applications while reducing the cost for industrial enterprises to deploy and maintain large-scale industrial model prompt word security detection systems.

[0043] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0044] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0045] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0046] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0047] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0048] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting the security of prompt words in large industrial models, characterized in that, The industrial large-scale model prompt word security detection method includes: Construct an industrial knowledge resource base; the industrial knowledge resource base includes: an industrial terminology resource base, a process parameter specification base, an operation instruction hierarchical base, an industrial data permission base, and a safe operation procedure base. An industrial knowledge graph is constructed based on the aforementioned industrial knowledge resource base; the industrial knowledge graph uses terms, parameters, instructions, data, and operation processes as nodes, and parameter ranges, instruction constraints, permission requirements, temporal relationships, and causal relationships as edges; Based on the industrial knowledge resource base and industrial knowledge graph, a basic security detection model for prompt words, an industrial domain-specific semantic parsing model, and an industrial domain-specific security detection model are constructed using deep learning algorithms. A basic security detection model for prompt words is used to perform basic security checks on the prompt words to be processed; the basic security checks include: inappropriate content filtering, sensitive information identification, and adversarial attack detection. When the prompt word to be processed passes the basic security detection, the prompt word to be processed is semantically analyzed using an industrial-specific semantic parsing model to obtain the recognition result; the recognition result is whether it contains industrial knowledge type information. When the identification result contains industrial knowledge type information, a special security detection model for the industrial field is used to perform special security detection on the prompt words to be processed, and special security detection results are obtained. Special security detection includes: industrial entity security detection, process parameter security detection, operation instruction security detection, industrial data security detection, and operation process security detection. Based on the results of the special security test, a dynamic weighted risk fusion assessment method is used to obtain the prompt word detection results, and output the risk level and handling suggestions; the prompt word detection results indicate whether the information contains attack information.

2. The industrial large-scale model prompt word security detection method according to claim 1, characterized in that, The construction of the industrial knowledge resource base specifically includes: An industrial terminology resource library is constructed based on general industrial terms, industry-specific terms, and specific scenario terms, along with their corresponding first annotation information. The first annotation information includes: the industry, applicable scenario, process step, term name, and safety level. Based on the process parameters of each stage and the corresponding second annotation information, a process parameter specification library is constructed; the second annotation information includes: industry, applicable scenario, process stage, parameter name, equipment type, compliance scope, related parameters, and risk level; A hierarchical library of operation instructions is constructed based on the operation instructions for controlling industrial equipment and industrial control systems and the corresponding third-party annotation information. The third-party annotation information includes: industry, applicable scenario, process step, instruction name, instruction category, applicable object, permission requirements, and risk level. An industrial data permission library is constructed based on industrial data involved in each stage of industrial production and the corresponding fourth annotation information. The fourth annotation information includes: industry, applicable scenario, process stage, data name, data type, data level, source equipment, source time, and permission requirements. A safety operation procedure library is constructed based on the standard safety operation procedures for each process step and the corresponding fifth annotation information; the fifth annotation information includes: industry, applicable scenario, process step, key safety node, and process sequence constraints.

3. The industrial large-scale model prompt word security detection method according to claim 1, characterized in that, The process parameters include: temperature parameters, pressure parameters, flow rate parameters, cutting parameters, process dimension parameters, and surface quality parameters.

4. The industrial large-scale model prompt word security detection method according to claim 1, characterized in that, The industrial knowledge type information includes industrial entities, process parameters, operating instructions, industrial data, or operating procedures.

5. The industrial large-scale model prompt word security detection method according to claim 1, characterized in that, The process involves constructing a basic security detection model for prompt words, an industry-specific semantic parsing model, and an industry-specific security detection model based on an industrial knowledge resource base and an industrial knowledge graph, using deep learning algorithms. Specifically, this includes: An industrial domain semantic association network is constructed based on an industrial knowledge graph. This network is used for semantic association analysis and semantic reasoning, and based on the hard constraints of the industrial knowledge graph and adaptive thresholds of real-time industrial data, it identifies violations. Violations include: incorrect use of terminology, conflicting parameter combinations, and failure to follow the safe operating procedure sequence.

6. The industrial large-scale model prompt word security detection method according to claim 1, characterized in that, The industrial-specific safety testing model includes: parallel sub-models for industrial entity safety testing, process parameter safety testing, operation instruction safety testing, industrial data safety testing, and operation process safety testing. The industrial entity security detection sub-model is used to identify the rationality risk of industrial entities based on prompts containing industrial entity knowledge type information, and to obtain the industrial entity security detection result. The industrial entity rationality risk identification includes whether it is a sensitive term, whether it is used in a non-compliant scenario, and whether there is malicious keyword replacement. The process parameter safety detection sub-model is used to identify process parameter safety risks based on prompts containing process parameter knowledge type information, and obtain process parameter safety detection results. Process parameter safety risk identification includes whether the parameter risk level is high risk or prohibited, whether a single parameter exceeds the upper or lower limits of the compliance range, whether the relationship between multiple parameters meets the process constraints, and whether the change trend of parameters with time sequence requirements meets the safety procedures. The operation instruction security detection sub-model is used to identify operation instruction constraint security risks in prompt words containing operation instruction knowledge type information, and obtain operation instruction security detection results; operation instruction constraint security risk identification includes whether the instruction operation permission matches the current user permission, whether the instruction risk level is high risk or prohibited, and whether the preconditions for instruction execution are met. The industrial data security detection sub-model is used to identify industrial data access security risks based on prompts containing industrial data knowledge types, and to obtain industrial data security detection results. The industrial data access security risk identification includes whether there is intellectual property infringement, whether there is sensitive data leakage, and whether there is unauthorized operation. The operation process security detection sub-model is used to identify security risks in operation logic, temporal relationship and causal relationship of prompt words containing operation process knowledge type, and obtain operation process security detection results; security risk identification includes whether key security nodes are omitted, whether the operation process order is incorrect and whether there is unauthorized operation.

7. The industrial large-scale model prompt word security detection method according to claim 6, characterized in that, Based on the results of specialized security testing, a dynamic weighted risk fusion assessment method is used to obtain the warning word detection results, and output the risk level and handling suggestions, specifically including: The detection result V of the prompt word is determined using the formula V=w1×v1+w2×v2+w3×v3+w4×v4+w5×v5; When the prompt word detection result is at the safe level, the recommended action is to directly input it into the large industrial model; When the prompt word detection result is low risk level, the recommended action is to inform the user of the risk and then input the industrial large model; When the prompt word detection result is high-risk, the recommended action is to refuse input of the industrial large model; Among them, w1, w2, w3, w4 and w5 are the weights of the industrial entity safety detection sub-model, process parameter safety detection sub-model, operation instruction safety detection sub-model, industrial data safety detection sub-model and operation process safety detection sub-model, respectively, and v1, v2, v3, v4 and v5 are the industrial entity safety detection results, process parameter safety detection results, operation instruction safety detection results, industrial data safety detection results and operation process safety detection results, respectively.

8. The industrial large-scale model prompt word security detection method according to claim 7, characterized in that, The weights of the industrial entity safety detection sub-model, process parameter safety detection sub-model, operation instruction safety detection sub-model, industrial data safety detection sub-model, and operation process safety detection sub-model are dynamically adjusted based on the applicable industry, applicable scenario, and process link.

9. The industrial large-scale model prompt word security detection method according to claim 1, characterized in that, Based on the results of specialized security testing, a dynamic weighted risk fusion assessment method is used to obtain the warning word detection results, and the risk level and handling suggestions are output. This process also includes: Based on changes in industrial standards and process specifications, missed or misjudged samples during the prompt word detection process, and risk events triggered by the industrial big data model, the industrial knowledge resource base and industrial knowledge graph are dynamically updated; and based on the updated industrial knowledge resource base, industrial knowledge graph, and newly added samples, the basic security detection model for prompt words, the industrial domain-specific semantic parsing model, and the industrial domain-specific security detection model are iteratively optimized.

10. A security detection system for prompt words in large industrial models, characterized in that, The industrial large-scale model prompt word security detection system includes: The industrial knowledge resource base management module is used to build an industrial knowledge resource base; the industrial knowledge resource base includes: an industrial terminology resource base, a process parameter specification base, an operation instruction hierarchical base, an industrial data permission base, and a safe operation procedure base. The industrial knowledge graph management module is used to construct an industrial knowledge graph based on the industrial knowledge resource base; the industrial knowledge graph uses terms, parameters, instructions, data and operation processes as nodes, and parameter ranges, instruction constraints, permission requirements, temporal relationships and causal relationships as edges; The model building module is used to construct a basic security detection model for prompt words, an industrial domain-specific semantic parsing model, and an industrial domain-specific security detection model based on the industrial knowledge resource base and the industrial knowledge graph, respectively, using deep learning algorithms. The basic security detection module for prompt words is used to perform basic security detection on the prompt words to be processed using the basic security detection model for prompt words; the basic security detection includes: inappropriate content filtering, sensitive information identification, and adversarial attack detection; The industrial domain-specific semantic parsing module is used to perform semantic parsing on the prompt words to be processed when they pass the basic security detection, using an industrial domain-specific semantic parsing model to obtain the recognition result; the recognition result is whether it contains industrial knowledge type information. The industrial-specific security detection module is used to perform specific security detection on the prompt words to be processed when the identification result contains industrial knowledge type information, using an industrial-specific security detection model to obtain specific security detection results. Specific security detection includes: industrial entity security detection, process parameter security detection, operation instruction security detection, industrial data security detection, and operation process security detection. The special security detection result fusion and evaluation module is used to obtain the prompt word detection result based on the special security detection result, using a dynamically weighted risk fusion evaluation method, and output the risk level and handling suggestions; the prompt word detection result indicates whether it contains attack information.

Images