Autonomous incremental knowledge extraction and representation method, terminal and medium

Abstract

The application provides an autonomous incremental knowledge extraction and representation method, and also provides a corresponding terminal and medium. Through the collection of system operation information data, sensor equipment state data and human subjective experience data in the system running process, a historical experience database is established. Further, the similarity data vector in the system is calculated, the initial prediction estimate value of the knowledge parameter is obtained, the new knowledge parameter prediction estimate value is inferred from the strategy-value learning model by fusing the historical experience, and the new system operation information vector and the new knowledge information vector are obtained. The strategy-value learning model is continuously optimized with the system running, and the autonomous incremental knowledge extraction and representation of the database are carried out, so as to expand the historical experience database, achieve the purpose of autonomous incremental knowledge extraction and representation, and improve the efficient utilization of knowledge in the actual production and life process.

Description

Autonomous incremental knowledge extraction and representation methods, terminals and media Technical Field

[0001] This invention relates to an information resource extraction and knowledge representation technology in the field of computer application technology. Specifically, it relates to an autonomous incremental knowledge extraction and representation method, terminal, and medium based on historical experience. Background Technology

[0002] As industrial production expands in scale and processes become increasingly complex, while operational standards exist for handling various anomalies, actual operation relies heavily on the extensive experience accumulated by specialized technicians. To preserve this subjective expert experience in a standardized format and systematically apply it to different equipment, eliminating the impact of operator skill levels, cognitive differences, and personnel turnover on performance accuracy, knowledge extraction and representation become crucial. Standardized knowledge extraction lowers the technical barriers and experience requirements for ordinary operators, enabling faster response times, enhancing software compatibility and interoperability, and facilitating unified design and management across various systems.

[0003] However, current knowledge extraction and representation in industrial production processes typically suffer from the following problems:

[0004] (1) Historical experience information was not fully utilized.

[0005] (2) Unable to engage in self-directed incremental knowledge learning.

[0006] Currently, no descriptions or reports of technologies similar to this invention have been found, and no similar information has been collected domestically or internationally. Summary of the Invention

[0007] To address the aforementioned shortcomings in existing technologies, this invention provides a method, system, terminal, and medium for autonomous incremental knowledge extraction and representation based on historical experience.

[0008] According to one aspect of the present invention, an autonomous incremental knowledge extraction and representation method is provided, comprising:

[0009] S1. Acquire system operation information data, sensor equipment status data and expert prior data during industrial production as historical experience data to establish a historical experience database.

[0010] S2, calculate the similarity between the current system operation information data and the system operation information data in the historical experience data, and obtain the initial prediction estimate of the knowledge parameters;

[0011] S3, based on the initial prediction estimate of the acquired knowledge parameters, extract the policy vector that integrates historical experience from the policy-value learning model;

[0012] S4. Based on the strategy vector that integrates historical experience, obtain the system operation information data for the next moment;

[0013] S5, store the new system operation information data, sensor device status data and policy vector at the same time as knowledge information vectors into the historical experience database;

[0014] S6, repeat S2 to S5 until the current stage of industrial production process ends, optimize the policy-value learning model, and realize the autonomous incremental knowledge extraction and representation of knowledge information vectors in the historical experience database.

[0015] Preferably, the establishment of the historical experience database includes:

[0016] The system operation information data, sensor equipment status data and corresponding expert prior data are acquired during the industrial production process to obtain raw data.

[0017] The acquired raw data is preprocessed;

[0018] For different types of preprocessed data, respectively establish a contextual knowledge dataset for storing system operation information data, a value knowledge dataset for storing sensor device status data, and a strategy knowledge dataset for storing expert prior data.

[0019] Perform association operations on the data in the three datasets: associate the data in the context knowledge dataset and the value knowledge dataset based on information at the same time, and associate the data in the value knowledge dataset and the strategy knowledge dataset based on information at the same state to obtain the association relationships;

[0020] By using the three datasets and their relationships as historical experience data, a historical experience database is formed.

[0021] Preferably, the historical experience database is represented as follows:

[0022] Φ(i)=[S(i), V(i), P(j)], i=1, 2,…,N,

[0023] Where Φ(t) represents the knowledge information vector stored in the historical experience database, S(i) represents the system operation information data in the context knowledge dataset, V(i) represents the sensor status data in the value knowledge dataset, and P(i) represents the expert prior data in the strategy knowledge dataset.

[0024] Preferably, in step S2, calculating the similarity between the current system operation information data and the historical experience data to obtain the initial prediction estimate of the knowledge parameters includes:

[0025] Design a similarity function to calculate the similarity between the current system operation information data and the system operation information data in the historical experience data;

[0026] In the historical experience database, based on the calculated similarity value, find several information vectors that are less than the threshold d and contain the data that are closest to the current system operation information.

[0027] The initial prediction estimate of the knowledge parameters is selected based on the weights.

[0028] Preferably, the similarity function is:

[0029] D(S(i),S(t))=κexp(-d(S(j),S(t)))+(1-κ)cos(S(i),S(t)),

[0030]

[0031] Where D(S(i), S(t)) represents the similarity calculation function between the current system operation information data S(t) and the historical system operation information data S(i) in the historical experience data, d(S(i), S(t)) is a supplement to the above similarity calculation function, and the equation on the right describes the calculation method between each element in the system operation information data, κ and τ l All of these are hyperparameters set based on expert experience, τ l Let S(i) represent the weight value of each element in the information vector of the system operation information data, S(t) represent the system operation information data in the contextual knowledge dataset, S(t) represent the system operation information data at the current moment, l represent the l-th element in the system operation information data, S(m) represent the index of the extreme value of the l-th element in the system operation information data in the historical experience data, and V(i) represent the sensor status data in the value knowledge dataset.

[0032] Preferably, the step of selecting the initial prediction estimate of the knowledge parameters according to the weights includes:

[0033]

[0034] Among them, Z old (t) represents the initial predicted estimate of the knowledge parameters, P(i) represents the expert prior data in the policy knowledge dataset, and k represents the k most relevant expert experience data vectors obtained from the current system operation information data through empirical selection using the similarity function; ωi This represents the weighting coefficient, ω. i for:

[0035]

[0036] Preferably, in step S3, extracting a policy vector incorporating historical experience from the policy-value learning model based on the obtained initial prediction estimate of the knowledge parameters includes:

[0037] Construct a policy-value learning model and use the obtained knowledge parameters as the initial prediction estimate Z. old The system operation information data S(t) and the current time step are used as inputs to the policy-value learning model. New knowledge parameter prediction estimates, i.e., policy vectors that integrate historical experience, are extracted from the policy-value learning model. The policy-value learning model is as follows:

[0038] Z new (t)=π(S(t), Z old (t)),

[0039] Where π represents the policy-value learning model, Z new (t) represents the new knowledge parameter prediction estimate, i.e., the strategy vector that incorporates historical experience;

[0040] The parameter update and optimization model of the policy-value learning model includes:

[0041]

[0042]

[0043]

[0044] Where J(π) represents the optimization objective function of the policy-value learning model, θ represents the update parameters in the network, and s t The state of the network at time t is represented by S(t), which is the system operation information data at the current time. t This represents the action chosen at time t in the network, i.e., the initial predicted estimate of the knowledge parameters Z. old (t), It indicates that at time t, s t Select to proceed with step a in the current state t The reward value obtained from the action.

[0045] Preferably, in step S5, the new system operation information data, sensor device status data, and strategy vector fused with historical experience at the same time are stored as knowledge information vectors in the historical experience database. Then:

[0046] Φ(t)=[S(t), V(t), Z(t)],

[0047] Where Φ(t) represents the knowledge information vector, S(t) represents the system operation information data in the historical experience database, V(t) represents the sensor device status data in the historical experience database, and Z(t) represents the strategy vector that integrates historical experience.

[0048] Preferably, in step S6, optimizing the policy-value learning model includes:

[0049] Constructing tuple information for optimization:

[0050] (S(t),S(t+1),Z(t),V(t));

[0051] Where S(t) represents the system operation information data at time t, Z(t) represents the strategy vector that integrates historical experience, V(t) represents the sensor status data at time t, and S(t+1) represents the system operation information data at time t+1 after applying the strategy vector.

[0052] As the industrial production process continues, the accumulated tuple information is used to continuously optimize the policy-value learning model, resulting in the optimal policy vector. This optimal policy vector is then combined with system operation information and equipment status information at the state points where this optimal policy vector is adopted to construct a knowledge information vector. This vector is then used for autonomous incremental extraction and representation of the historical experience database.

[0053]

[0054] Where I(i) represents the knowledge information vector, S(i) represents the system operation information data of state point i, and V(i) represents the sensor device state data of state point i. This represents the optimized policy vector adopted at state point i.

[0055] According to a second aspect of the present invention, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can be used to perform the method described in any of the preceding claims.

[0056] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform the methods described in any of the preceding claims.

[0057] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0058] The autonomous incremental knowledge extraction and representation method, terminal, and medium provided by this invention utilize existing historical experience information and autonomous incremental learning to continuously optimize the strategy-value learning model and expand the historical experience database as the industrial production process runs, thereby achieving the purpose of autonomous incremental knowledge extraction and representation and improving the efficient utilization of knowledge in actual production and life processes.

[0059] The autonomous incremental knowledge extraction and representation method, terminal, and medium provided by this invention, based on historical data and expert prior knowledge, can reduce the need for precise and complete rule writing, reduce a large amount of manual writing costs, and autonomously update and optimize as the system runs. Attached Figure Description

[0060] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0061] Figure 1 is a flowchart of an autonomous incremental knowledge extraction and representation method provided in an embodiment of the present invention.

[0062] Figure 2 is a flowchart of the autonomous incremental knowledge extraction and representation method provided by a preferred embodiment of the present invention. Detailed Implementation

[0063] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0064] Figure 1 is a flowchart of an autonomous incremental knowledge extraction and representation method provided in an embodiment of the present invention.

[0065] As shown in Figure 1, the autonomous incremental knowledge extraction and representation method provided in this embodiment may include the following steps:

[0066] S100: Acquire system operation information data, sensor equipment status data and expert prior data during industrial production as historical experience data, and establish a historical experience database.

[0067] S200, calculate the similarity between the current system operation information data and the system operation information data in historical experience data, and obtain the initial prediction estimate of knowledge parameters;

[0068] S300: Based on the initial prediction estimate of the acquired knowledge parameters, extract the policy vector that incorporates historical experience from the policy-value learning model;

[0069] S400 obtains system operation information data for the next moment based on the strategy vector that integrates historical experience;

[0070] S500 stores new system operation information data, sensor device status data, and strategy vectors fused with historical experience as knowledge information vectors in the historical experience database at the same time.

[0071] S600, repeat S200 to S500 until the current stage of industrial production process ends, optimize the strategy-value learning model, and realize the autonomous incremental knowledge extraction and representation of knowledge information vectors in the historical experience database.

[0072] In S100 of this embodiment, as a preferred embodiment, establishing a historical experience database includes:

[0073] S101: Acquire system operation information data, sensor equipment status data and corresponding expert prior data during industrial production to obtain raw data;

[0074] S102, preprocess the acquired raw data;

[0075] S103, for different types of preprocessed data, establish corresponding scenario knowledge datasets for storing system operation information data, value knowledge datasets for storing sensor device status data, and strategy knowledge datasets for storing expert prior data.

[0076] S104, perform association operations on the data in the three datasets: associate the data in the context knowledge dataset and the value knowledge dataset based on information at the same time, and associate the data in the value knowledge dataset and the strategy knowledge dataset based on information at the same state to obtain the association relationship;

[0077] S105, the three datasets and their relationships are used as historical experience data to form a historical experience database.

[0078] In S105 of this embodiment, as a preferred embodiment, the historical experience database is represented as follows:

[0079] Φ(i)=[S(i), V(i), P(j)], i=1, 2,…,N,

[0080] Where Φ(t) represents the knowledge information vector stored in the historical experience database, S(i) represents the system operation information data in the context knowledge dataset, V(i) represents the sensor status data in the value knowledge dataset, and P(i) represents the expert prior data in the strategy knowledge dataset.

[0081] In S200 of this embodiment, as a preferred embodiment, calculating the similarity between the current system operation information data and historical experience data to obtain the initial prediction estimate of the knowledge parameters may include the following steps:

[0082] S201, Design a similarity function to calculate the similarity between the current system operation information data and the historical system operation information data in the historical experience data;

[0083] S202, In the historical experience database, find several information vectors containing the closest current system operation information data that are less than the threshold d based on the calculated similarity value.

[0084] S203, Select the initial prediction estimate of the knowledge parameters according to the weights.

[0085] In S201 of this embodiment, as a preferred embodiment, the similarity function is:

[0086] D(S(i),S(t))=κexp(-d(S(j),S(t)))+(1-κ)cos(S(i),S(t)),

[0087]

[0088] Where D(S(i),S(t)) represents the similarity calculation function between the current system operation information data S(t) and the historical system operation information data S(i) in the historical experience data, d(S(i),S(t)) is a supplement to the above similarity calculation function, and the equation on the right describes the calculation method between each element in the system operation information data, κ and τ l All of these are hyperparameters set based on expert experience, τ l Let S(i) represent the weight value of each element in the information vector of the system operation information data, S(t) represent the system operation information data in the contextual knowledge dataset, S(t) represent the system operation information data at the current moment, l represent the l-th element in the system operation information data, S(m) represent the index of the extreme value of the l-th element in the system operation information data in the historical experience data, and V(i) represent the sensor status data in the value knowledge dataset.

[0089] In S203 of this embodiment, as a preferred embodiment, selecting the initial prediction estimate of the knowledge parameters according to the weights includes:

[0090]

[0091] Among them, Z old (t) represents the initial predicted estimate of the knowledge parameters, P(i) represents the expert prior data in the policy knowledge dataset, and k represents the k most relevant expert experience data vectors obtained from the current system operation information data through empirical selection using the similarity function; w i This represents the weighting coefficient, w. i for:

[0092]

[0093] In S300 of this embodiment, as a preferred embodiment, extracting a new policy vector incorporating historical experience from the policy-value learning model based on the obtained initial prediction estimate of the knowledge parameters may include the following steps:

[0094] Construct a policy-value learning model and use the obtained knowledge parameters as the initial prediction estimate Z. old The system operation information data S(t) and the current time step are used as inputs to the policy-value learning model. New knowledge parameter prediction estimates, i.e., policy vectors incorporating historical experience, are extracted from the policy-value learning model. The policy-value learning model is as follows:

[0095] Z new (t)=π(S(t),Z old (t)),

[0096] Where π represents the policy-value learning model, Z new (t) represents the new knowledge parameter prediction estimate, i.e., the strategy vector that incorporates historical experience;

[0097] The parameter update optimization model (update equation) of the policy-value learning model includes:

[0098]

[0099]

[0100]

[0101] Where J(π) represents the optimization objective function of the policy-value learning model, θ represents the update parameters in the network, and s t The state of the network at time t is represented by S(t), which is the system operation information data at the current time.t This represents the action chosen at time t in the network, i.e., the initial predicted estimate of the knowledge parameters Z. old (t), It indicates that at time t, s t Select to proceed with step a in the current state t The reward value obtained from the action.

[0102] In S500 of this embodiment, as a preferred embodiment, the new system operation information data, sensor device status data, and policy vector at the same time are stored as knowledge information vectors in the historical experience database. Then:

[0103] Φ(t)=[S(t),V(t),Z(t)],

[0104] Where Φ(t) represents the knowledge information vector, S(t) represents the system operation information data in the historical experience database context knowledge dataset, V(t) represents the sensor device status data in the historical experience database value knowledge dataset, and Z(t) represents the strategy vector that integrates historical experience.

[0105] In S600 of this embodiment, as a preferred embodiment, optimization of the policy-value learning network includes:

[0106] Constructing tuple information for optimization:

[0107] (S(t),S(t+1),Z(t),V(t));

[0108] Where S(t) represents the system operation information data at time t, Z(t) represents the strategy vector that integrates historical experience, V(t) represents the sensor status data at time t, and S(t+1) represents the system operation information data at time t+1 after applying the strategy vector.

[0109] As the industrial production process continues, the accumulated tuple information is used to continuously optimize the policy-value learning model based on the objective function and update equation in S300, thereby obtaining the optimal policy vector. Combined with the system operation information and equipment status information of the state point that adopts the optimal policy vector, a knowledge information vector is formed, which is used to autonomously and incrementally extract and represent the historical experience database.

[0110]

[0111] Where I(i) represents the knowledge information vector, S(i) represents the system operation information data of state point i, and V(i) represents the sensor device state data of state point i. This represents the optimized policy vector adopted at state point i.

[0112] Figure 2 is a flowchart of the autonomous incremental knowledge extraction and representation method provided by a preferred embodiment of the present invention.

[0113] As shown in Figure 2, the autonomous incremental knowledge extraction and representation method provided in this preferred embodiment may include the following steps:

[0114] Step 1: Data acquisition. Obtain system operation information, sensor equipment status data, and human subjective experience data during complex manufacturing processes, and establish a historical experience database.

[0115] Step 2: Calculate the similar data vector of the system at the current moment to obtain the initial prediction estimate of the knowledge parameters;

[0116] Step 3: Integrate historical experience to derive new knowledge parameter prediction estimates from the policy-value learning model;

[0117] Step 4: Run the system based on the estimated values ​​predicted by the new knowledge parameters to obtain the system operation information data for the next time step;

[0118] Step 5: Construct new knowledge information vectors and store them in the historical experience database;

[0119] Step 6: As the system runs, repeat steps 2 to 5 until the system stops, and then optimize the policy-value learning model.

[0120] As a preferred embodiment, the method for establishing a historical experience database in step 1 specifically includes the following steps:

[0121] Step 1.1: Collect system operation information data and sensor device status data pairs from the industrial production process from which the knowledge to be extracted, along with corresponding expert experience and strategy data;

[0122] Step 1.2 involves preprocessing and standardizing the collected raw data to remove useless and erroneous data and unify the data format.

[0123] Step 1.3: For different types of data, establish a scenario knowledge dataset for storing system operation information data, a value knowledge dataset for sensor status data pairs, and a strategy knowledge dataset for storing expert subjective strategy data.

[0124] Step 1.4: Perform association operations on the data in the three datasets. Associate the data in the context knowledge dataset and the value knowledge dataset based on information at the same time, and associate the data in the value knowledge dataset and the strategy knowledge dataset based on information at the same state. The three datasets and their associations together constitute the historical experience database.

[0125] In a preferred embodiment, in step 1.4, the basic knowledge information vector stored in the historical experience database is represented as follows:

[0126] Φ(i)=[S(i), V(i), P(j)], i=1, 2,…,N,

[0127] Where S(i) represents contextual knowledge data, i.e., system operation information vector, V(i) represents value knowledge data, i.e., sensor state vector, and P(i) represents strategy knowledge data, i.e., expert subjective strategy vector.

[0128] In a preferred embodiment, step 2, the method for obtaining the initial prediction estimate of the knowledge parameters specifically includes the following steps.

[0129] Step 2.1: Design a similarity function to calculate the similarity between the current system operation information vector and the historical experience information vector in the database;

[0130] Step 2.2: In the existing database information, find several information vectors that are closest to the current context knowledge information with a distance less than d based on the calculated similarity value.

[0131] Step 2.3: Select the initial prediction estimate of the knowledge parameters according to the weights.

[0132] In a preferred embodiment, in step 2.1, the similarity function is designed as follows:

[0133] D(S(i),S(t))=κexp(-d(S(j),S(t)))+(1-κ)cos(S(i),S(t)),

[0134]

[0135] Among them κ and τ l All of these are hyperparameters set based on experience, τ l This represents the weight value of each element in the information vector.

[0136] As a preferred embodiment, in step 2.3, the method for calculating the initial prediction estimate of the selected knowledge parameters based on the weights is as follows:

[0137]

[0138] Where k represents the k most relevant historical empirical strategy vectors obtained from the current system operation information vector through empirical selection using the similarity function. The weight coefficient w i The calculation method is as follows

[0139]

[0140] In a preferred embodiment, step 3, which involves inferring new parameter estimates from the learning network based on historical experience, specifically includes the following steps:

[0141] Step 3.1, the obtained initial prediction estimate Z of the knowledge parameters. old The system operation information vector S(t) and the current time step are input into the policy-value learning model;

[0142] Step 3.2: Infer new knowledge parameter estimates that incorporate historical experience from the policy-value learning model.

[0143] In a preferred embodiment, in step 3.1, the obtained initial prediction estimate Z of the knowledge parameters is... old The formula for the input policy-value learning model, which takes the system operation information vector S(t) and the current time-input policy-value learning model, is as follows:

[0144] Z new (t)=π(S(t), Z old (t)),

[0145] Where π represents the policy-value learning model, Z new (t) represents the estimated value of the new knowledge parameters that incorporate historical experience.

[0146] In a preferred embodiment, in step 3.2, new knowledge parameter estimates incorporating historical experience are obtained from the policy-value learning model. The performance index formula of the policy-value learning model is expressed as:

[0147]

[0148]

[0149]

[0150] Where s t The vector S(t) represents the system operation information vector at time t in the network, i.e., the current time. t Z represents the initial prediction estimate of the knowledge parameters, which is the action chosen at time t in the network. old (t), It indicates that at time t, s t Select to proceed with step a in the current state t The reward value obtained from the action.

[0151] In a preferred embodiment, in step 4, the system is run according to the estimated value predicted by the new knowledge parameters to obtain the system operation information vector S(t+1) for the next time step.

[0152] In a preferred embodiment, in step 5, the new knowledge information vector is stored in the historical experience database Φ(t)=[S(t), V(t), Z(t)], where Z(t) is the new strategy vector that combines historical experience.

[0153] As a preferred embodiment, in step 6, the database and policy-value learning model are optimized after the system stops, wherein the dataset used for optimization is tuple information (S(t), S(t+I), Z(t), V(t)) stored during system runtime.

[0154] As a preferred embodiment, as the system runs, the policy-value learning model will be continuously optimized based on the accumulated tuple information, and the knowledge information vector in the historical experience database will be autonomously incrementally extracted and represented according to steps 3.1 and 5.

[0155] The technical solution provided by the above embodiments of the present invention will be further described in detail below with reference to a specific application example.

[0156] This specific application example involves a method for extracting and representing autonomous incremental knowledge for PID parameter optimization based on historical experience.

[0157] The method may include the following steps:

[0158] Step 1: Data acquisition. Obtain data such as PID parameter values, input and output data at time t and time tn, and control output during the operation of the controller, and establish a historical experience database.

[0159] Furthermore, the specific operation of step 1 is as follows:

[0160] Step 1.1: For a known nonlinear controlled object, obtain the initial PID parameters using common engineering tuning methods. Use these PID parameters to perform simulation and collect experimental data of the system operation as the data vector in the initial database.

[0161] Step 1.2 involves preprocessing and standardizing the collected raw data to remove useless and erroneous data and unify the data format.

[0162] Step 1.3: For different types of data, establish a scenario knowledge dataset that stores input and output data, a value knowledge dataset that stores control output quantities, and a strategy knowledge dataset that stores PID parameter values.

[0163] Step 1.4: Perform association operations on the data in the three datasets. Associate the data in the context knowledge dataset and the value knowledge dataset based on information at the same time, and associate the data in the value knowledge dataset and the strategy knowledge dataset based on information at the same state. The three datasets and their associations together constitute the historical experience database.

[0164] Further, in step 1.4, the basic knowledge information vector stored in the historical experience database is represented as Φ(i)=[S(i), V(i), P(j)], i=1,2,…,N,

[0165] Where S(i) represents contextual knowledge data, i.e., the input-output information vector; V(i) represents value knowledge data, i.e., the control output state vector; and P(i) represents policy knowledge data, i.e., the PID parameter policy vector.

[0166] Step 2: Calculate the similar data vector of the system at the current moment to obtain the initial prediction estimate of the PID parameters.

[0167] Furthermore, the specific operation of step 2 is as follows:

[0168] Step 2.1: Design a similarity function to calculate the similarity between the current system's input / output information vector and the historical experience information vector in the database.

[0169] Step 2.2: In the existing database information, find several information vectors that are closest to the current context knowledge information with a distance less than d based on the calculated similarity value.

[0170] Step 2.3: Select the initial prediction estimate of the PID parameters according to the weights.

[0171] Furthermore, in step 2.1, the similarity function is designed as follows:

[0172] D(S(i),S(t))=κexp(-d(S(j),S(t)))+(1-κ)cos(S(i),S(t)),

[0173]

[0174] Among them κ and τ l All of these are hyperparameters set based on experience, τ l This represents the weight value of each element in the information vector.

[0175] Furthermore, in step 2.3, the method for calculating the initial prediction estimate of the PID parameters based on the weights is as follows:

[0176]

[0177] Where k represents the k most relevant historical empirical strategy vectors obtained from the current system operation information vector through empirical selection using the similarity function. The weight coefficient w i The calculation method is as follows

[0178]

[0179] Step 3: Integrate historical experience to derive new knowledge parameter prediction estimates from the policy-value learning model.

[0180] Furthermore, the specific operation of step 3 is as follows:

[0181] Step 3.1, obtain the initial predicted estimates of the PID parameters. The system operation information vector S(t) at the current moment is input into the policy-value learning model.

[0182] Step 3.2: Infer new PID parameter estimates that incorporate historical experience from the policy-value learning model.

[0183] Further, in step 3.1, the obtained initial predicted estimates of the PID parameters are... The system operation information vector S(t) at the current moment is input to the policy-value learning model formula as follows:

[0184]

[0185] Where π represents the policy-value learning model, This represents a new estimate of PID parameters that incorporates historical experience.

[0186] Further, in step 3.2, new PID parameter estimates incorporating historical experience are derived from the policy-value learning model. The performance index formula of the policy-value learning model is expressed as follows:

[0187]

[0188]

[0189]

[0190] Where s t The vector S(t) represents the system operation information vector at time t in the network, i.e., the current time. t This represents the action chosen at time t in the network, i.e., the initial prediction estimate of the PID parameters. It indicates that at time t, s t Select to proceed with step a in the current state t The reward value obtained from the action.

[0191] Step 4: Run the system based on the new initial predicted estimates of the PID parameters to obtain the system operation information vector S(t+1) for the next time step.

[0192] Step 5, create the new knowledge information vector Φ(t) = [S(t), V(t), θ] PID [(t)] is stored in the historical experience database, θ PID (t) is the new PID strategy vector that incorporates historical experience.

[0193] Step 6: As the system runs, repeat steps 2 through 5 until the system stops, and optimize the historical experience database and the policy-value learning model. The dataset used for optimization is the tuple information (S(t), S(t+1), θ) stored during system runtime. PID (t), V(t)).

[0194] Furthermore, as the system runs, the policy-value learning model will be continuously optimized based on the accumulated tuple information, and the knowledge information vectors in the historical experience database will be autonomously incrementally extracted and represented according to steps 3.1 and 5.

[0195] The technical solution provided by the above embodiments of the present invention will be further described in detail below with reference to another specific application example.

[0196] This specific application example involves a method for autonomous incremental knowledge extraction and representation from flight manuals based on historical experience.

[0197] The method may include the following steps:

[0198] Step 1: Data Acquisition. Obtain system operation information data, sensor equipment status data, and pilot subjective experience data from the flight manual to establish a historical flight experience database.

[0199] Furthermore, the specific operation of step 1 is as follows:

[0200] Step 1.1: Collect the system operation information data and sensor device status data pairs from the flight manuals from which the knowledge to be extracted, as well as the corresponding pilot flight experience and strategy data.

[0201] Step 1.2 involves preprocessing and standardizing the collected raw data to remove useless and erroneous data and unify the data format.

[0202] Step 1.3: For different types of data, establish a scenario knowledge dataset for storing system operation information data, a value knowledge dataset for sensor status data pairs, and a strategy knowledge dataset for storing pilot subjective strategy data.

[0203] Step 1.4: Perform association operations on the data in the three datasets. Associate the data in the context knowledge dataset and the value knowledge dataset based on information at the same time. Associate the data in the value knowledge dataset and the strategy knowledge dataset based on information at the same state. The three datasets and their associations together constitute the historical flight experience database.

[0204] Furthermore, in step 1.4, the basic knowledge information vector stored in the historical flight experience database is represented as follows:

[0205] Φ(i)=[S(i), V(i), P(j)], i=1, 2,…,N,

[0206] Where S(i) represents contextual knowledge data, i.e., system operation information vector, V(i) represents value knowledge data, i.e., sensor state vector, and P(i) represents strategy knowledge data, i.e., pilot subjective strategy vector.

[0207] Step 2: Calculate the similar data vector of the system at the current moment to obtain the initial prediction estimate of the flight knowledge parameters.

[0208] Furthermore, the specific operation of step 2 is as follows:

[0209] Step 2.1: Design a similarity function to calculate the similarity between the current system operation information vector and the historical flight experience information vector in the database.

[0210] Step 2.2: In the existing database information, find several information vectors that are closest to the current context knowledge information with a distance less than d based on the calculated similarity value.

[0211] Step 2.3: Select the initial prediction estimate of the flight knowledge parameters according to the weights.

[0212] Furthermore, in step 2.1, the similarity function is designed as follows:

[0213] D(S(i),S(t))=κexp(-d(S(j),S(t)))+(1-κ)cos(S(i),S(t)),

[0214]

[0215] Among them κ and τ l All of these are hyperparameters set based on experience, τ lThis represents the weight value of each element in the information vector.

[0216] Furthermore, in step 2.3, the method for calculating the initial prediction estimate of the selected flight knowledge parameters based on weights is as follows:

[0217]

[0218] Where k represents the k most relevant historical flight experience strategy vectors obtained from the current system operation information vector through empirical selection using the similarity function. The weight coefficient w i The calculation method is as follows

[0219]

[0220] Step 3: Integrate historical flight experience to derive initial prediction estimates of new flight knowledge parameters from the policy-value learning model.

[0221] Furthermore, the specific operation of step 3 is as follows:

[0222] Step 3.1, the obtained initial prediction estimate Z of the flight knowledge parameters. old The system operation information vector S(t) and the current time step are input into the policy-value learning model.

[0223] Step 3.2: Infer new flight knowledge parameter estimates that incorporate historical experience from the policy-value learning model.

[0224] Further, in step 3.1, the obtained initial prediction estimate Z of the flight knowledge parameters is... old The system operation information vector S(t) and the current system operation information vector S(t) are input to the policy-value learning model, which is expressed as follows:

[0225] Z new (t)=π(S(t), Z old (t)),

[0226] Where π represents the policy-value learning model, Z new (t) represents the new flight knowledge parameter estimate that incorporates historical experience.

[0227] Further, in step 3.2, new flight knowledge parameter estimates incorporating historical experience are inferred from the policy-value learning model. The performance index formula of the policy-value learning model is expressed as follows:

[0228]

[0229]

[0230]

[0231] Where s t The vector s(t) represents the system operation information vector at time t in the network, i.e., the current time. t Z represents the initial prediction estimate of the flight knowledge parameters, which is the action chosen to be performed at time t in the network. old (t), It indicates that at time t, s t Select to proceed with step a in the current state t The reward value obtained from the action.

[0232] Step 4: Run the system based on the initial prediction estimates of the new flight knowledge parameters to obtain the system operation information vector S(t+1) for the next moment.

[0233] Step 5: Store the new flight knowledge information vector Φ(t) = [S(t), V(t), Z(t)] into the historical flight experience database. Z(t) is the new strategy vector that combines historical flight experience.

[0234] Step 6: As the system runs, repeat steps 2 to 5 until the system stops, and optimize the historical flight experience database and the policy-value learning model. The dataset used for optimization is the tuple information (S(t), S(t+1), Z(t), V(t)) stored during system runtime.

[0235] Furthermore, as the system operates, the policy-value learning model will be continuously optimized based on the accumulated tuple information, and the flight knowledge information vector in the historical flight experience database will be autonomously incrementally extracted and represented according to steps 3.1 and 5.

[0236] One embodiment of the present invention provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to perform any of the methods described in the above embodiments.

[0237] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0238] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0239] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0240] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0241] One embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform the method of any of the above embodiments of the present invention.

[0242] Matters not covered in this invention are common knowledge.

[0243] The autonomous incremental knowledge extraction and representation method, terminal, and medium provided in the above embodiments of the present invention address the problem that existing technologies cannot effectively utilize existing historical experience information and autonomous incremental learning. By collecting system operation information data, sensor device status data, and human subjective experience data during system operation, a historical experience database is established. Further, similarity data vectors are calculated in the system to obtain initial predicted estimates of knowledge parameters. These estimates are then fused with historical experience and reasoned from the policy-value learning model to obtain new predicted estimates of knowledge parameters, thereby yielding new system operation information vectors and new knowledge information vectors. As the system operates, the policy-value learning model is continuously optimized, and autonomous incremental knowledge extraction and representation are performed on the database to expand the historical experience database, achieving the goal of autonomous incremental knowledge extraction and representation and improving the efficient utilization of knowledge in actual production and daily life.

[0244] Those skilled in the art will understand that, in addition to implementing the system and its various devices provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices of this invention function as logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices provided by this invention can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0245] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for autonomous incremental knowledge extraction and representation, characterized in that, include: S1. Acquire system operation information data, sensor equipment status data, and expert prior data during industrial production as historical experience data to establish a historical experience database. S2. Calculate the similarity between the current system operation information data and the system operation information data in the historical experience data to obtain initial prediction estimates of knowledge parameters. S3. Based on the obtained initial prediction estimates of knowledge parameters, extract a policy vector that integrates historical experience from the policy-value learning model. S4. Based on the policy vector that integrates historical experience, obtain the system operation information data for the next moment. S5. Store the new system operation information data, sensor equipment status data, and policy vector that integrates historical experience as knowledge information vectors in the historical experience database. S6. Repeat S2 to S5 until the current stage of the industrial production process ends, optimizing the policy-value learning model to achieve autonomous incremental knowledge extraction and representation of knowledge information vectors in the historical experience database. In step S3, based on the obtained initial prediction estimates of the knowledge parameters, a policy vector incorporating historical experience is extracted from the policy-value learning model, including: constructing a policy-value learning model and using the obtained initial prediction estimates of the knowledge parameters... and system operation information data at the current moment As input to the policy-value learning model, new knowledge parameter prediction estimates, i.e., policy vectors incorporating historical experience, are extracted from the policy-value learning model; wherein, the policy-value learning model is: ,in, This represents a policy-value learning model. This represents the predicted estimate of the new knowledge parameters, i.e., the policy vector incorporating historical experience; the parameter update optimization model of the policy-value learning model includes: ， ， Where J(π) represents the optimization objective function of the policy-value learning model, and θ represents the update parameters in the network. This represents the state of the network at time t, i.e., the system operation information data at the current time. ， This represents the action chosen at time t in the network, i.e., the initial predicted estimate of the knowledge parameters. ， This indicates at time t Select to proceed under status The reward value obtained from the action; in S6, the policy-value learning model is optimized, including: constructing tuple information for optimization: Where S(t) represents the system operation information data at time t, Z(t) represents the strategy vector fused with historical experience, V(t) represents the sensor device status data at time t, and S(t+1) represents the system operation information data at time t+1 after applying the strategy vector. Using the accumulated tuple information, the strategy-value learning model is continuously optimized to obtain the optimal strategy vector. Combined with the system operation information and device status information of the state point adopting the optimal strategy vector, a knowledge information vector is constructed, which is used for autonomous incremental extraction and representation of the historical experience database. Where I(i) represents the knowledge information vector, S(i) represents the system operation information data of state point i, and V(i) represents the sensor device state data of state point i. This represents the optimized policy vector adopted at state point i.

2. The autonomous incremental knowledge extraction and representation method according to claim 1, characterized in that, In step S1, establishing a historical experience database includes: acquiring system operation information data, sensor device status data, and corresponding expert prior data during industrial production to obtain raw data; preprocessing the acquired raw data; establishing corresponding scenario knowledge datasets for storing system operation information data, value knowledge datasets for storing sensor device status data, and strategy knowledge datasets for storing expert prior data for different preprocessed data; performing association operations on the data in the three datasets: associating data in the scenario knowledge dataset and value knowledge dataset based on information at the same time, and associating data in the value knowledge dataset and strategy knowledge dataset based on information at the same state to obtain association relationships; and using the three datasets and their association relationships as historical experience data to form the historical experience database.

3. The autonomous incremental knowledge extraction and representation method according to claim 2, characterized in that, The historical experience database is represented as follows: Where Φ(t) represents the knowledge information vector stored in the historical experience database. This represents system operation information data in the contextual knowledge dataset. This represents the status data of sensor devices in the valuable knowledge dataset. This represents the expert prior data in the policy knowledge dataset.

4. The autonomous incremental knowledge extraction and representation method according to claim 1, characterized in that, In step S2, calculating the similarity between the current system operation information data and the historical experience data, and obtaining the initial prediction estimate of the knowledge parameters, includes: designing a similarity function to calculate the similarity between the current system operation information data and the system operation information data in the historical experience data; in the historical experience database, finding several information vectors containing the closest current system operation information data based on the calculated similarity value and a distance less than a threshold d; and selecting the initial prediction estimate of the knowledge parameters based on the weights.

5. The autonomous incremental knowledge extraction and representation method according to claim 4, characterized in that, The similarity function is: , ,in, This represents the system operation information data at the current moment. Historical system operation information data in historical experience data ， As a supplement to the above similarity calculation function, the equation on the right describes the calculation method between each element in the system operation information data. and All of these hyperparameters were set based on expert experience. This represents the weight value of each element in the information vector of system operation information data. This represents system operation information data in the contextual knowledge dataset. This represents the system operation information data at the current moment. This indicates the first in the system operation information data. One element, This indicates the first data in the system operation information. The index of the extreme values ​​of an element in historical empirical data. This represents the status data of sensing devices in a dataset containing valuable knowledge.

6. The autonomous incremental knowledge extraction and representation method according to claim 4, characterized in that, The step of selecting the initial prediction estimate of the knowledge parameters based on the weights includes: Among them, Z old (t) represents the initial predicted estimate of the knowledge parameters. Let represent the expert prior data in the strategy knowledge dataset, and k represent the k most relevant expert experience data vectors obtained from the experience selection through the similarity function to obtain the current system operation information data. This represents the weighting coefficient, the weighting coefficient for: 。 7. The autonomous incremental knowledge extraction and representation method according to claim 1, characterized in that, In step S5, the new system operation information data, sensor device status data, and strategy vector fused with historical experience at the same time are stored as knowledge information vectors in the historical experience database. Then: Where Φ(t) represents the knowledge information vector, S(t) represents the system operation information data in the historical experience database, and V(t) represents the sensor status data in the historical experience database. This represents a strategy vector that incorporates historical experience.

8. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to perform the method of any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program can be used to perform the method of any one of claims 1-7.

Images