Decision support system and method based on large language model and reinforcement learning

Abstract

The invention relates to the technical field of reinforcement learning, and provides a decision support system and method based on a large language model and reinforcement learning, and the method comprises the steps: firstly obtaining a source state representation of reinforcement learning, carrying out the data enhancement of the source state representation through an external knowledge base, converting the enhanced state representation into a state representation of a preset large language model, and carrying out the recognition of the state representation. The method comprises the following steps: acquiring a state representation, inputting the state representation into a preset large language model to generate a reinforcement learning agent enhanced state representation function and an internal reward function, updating and maintaining a Lipexishz array based on the two functions to generate a state representation meeting a smooth condition, and finally inputting the state representation meeting the condition into a classifier decision model to generate a classifier decision model. And obtaining a decision result. Therefore, by introducing the external information, the source state characterization quality of reinforcement learning can be improved, so that the decision-making ability and adaptability of the intelligent agent are improved.

Description

Technical Field

[0001] The present application relates to the technical field of reinforcement learning, and more specifically, to a decision support system and method based on a large language model and reinforcement learning. Background Art

[0002] The state representation algorithm in reinforcement learning aims to obtain state representations that contain more task-related or environment-dynamic-related information, thereby enhancing the robustness of network training. With the development of large language models, using their powerful language understanding and generation capabilities, large language models can be used as prior knowledge for reinforcement learning algorithms to provide rich semantic information and contextual understanding, further improving the decision-making ability and adaptability of intelligent agents, enabling them to better understand and respond to complex and changing environments.

[0003] Patent CN118504612A proposes a state representation method and device based on a large language model and reinforcement learning. It first uses a natural language descriptor to convert the state in reinforcement learning into a state representation of a preset large language model. Then, the converted state is input into the large language model to generate an enhanced state representation function and an intrinsic reward function. The intrinsic reward helps the agent explore the environment. Finally, the Lipsitz array is updated and maintained through the generated function to ensure the smoothness of the state representation until the preset conditions are met.

[0004] In the above patent, the source state representation of reinforcement learning is directly converted into a preset large language model. Although directly converting the source state representation of reinforcement learning into the state representation of a preset large language model can simplify the process, this method also has some potential disadvantages. Specifically, the source state representation contains key feature information and may also contain redundant information. Direct conversion may lead to the loss or insufficient representation of important information, thereby affecting the quality of the subsequently generated state representation function and intrinsic reward function. In addition, direct conversion may ignore contextual information that is critical to understanding the environment, affecting the performance of the agent in specific tasks. If the source state representation itself contains noise or incomplete information, then directly inputting it into the large language model without processing may further amplify these problems, thereby reducing the reliability of the decision.

[0005] Therefore, an optimized decision support scheme based on large language models and reinforcement learning is desired. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present application provides a decision support system and method based on a large language model and reinforcement learning.

[0007] According to one aspect of the present application, a decision support method based on a large language model and reinforcement learning is provided, comprising: obtaining a source state representation of reinforcement learning; performing data enhancement on the source state representation of reinforcement learning based on an external knowledge base to obtain an enhanced source state representation; converting the enhanced source state representation into a state representation of a preset large language model based on a natural language descriptor, wherein the state representation of the preset large language model includes at least one of a task description, state details, output requirements, and feedback information; inputting the state representation of the preset large language model into the preset large language model to generate an enhanced state representation function and an intrinsic reward function of a reinforcement learning agent; updating and maintaining a corresponding Lipsitz array based on the enhanced state representation function and the intrinsic reward function to generate a state representation that meets a preset smoothness condition; and inputting the state representation that meets the preset smoothness condition into a decision model based on a classifier to obtain a decision result.

[0008] In the above-mentioned decision support method based on large language model and reinforcement learning, data enhancement is performed on the source state representation of the reinforcement learning based on an external knowledge base to obtain an enhanced source state representation, including: semantic embedding encoding is performed on the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base to obtain a set of semantic embedding encoding vectors of the source state representation of the reinforcement learning and semantic embedding encoding vectors of knowledge items; semantic association measurement is performed on each knowledge item semantic embedding encoding vector in the set of semantic embedding encoding vectors of the source state representation of the reinforcement learning and the knowledge item semantic embedding encoding vector to obtain a set of source state representation-knowledge item semantic matching coefficients; the knowledge item semantic embedding encoding vector corresponding to the maximum value of the source state representation-knowledge item semantic matching coefficient in the set of source state representation-knowledge item semantic matching coefficients is selected as a data enhancement material representation vector; and multi-feature semantic weighted interaction is performed on the source state representation semantic embedding encoding vector of the reinforcement learning and the data enhancement material representation vector to obtain the enhanced source state representation.

[0009] In the above-mentioned decision support method based on large language model and reinforcement learning, semantic embedding coding is performed on the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base to obtain a set of semantic embedding coding vectors of the source state representation of the reinforcement learning and semantic embedding coding vectors of knowledge items, including: semantic embedding coding is performed on the source state representation of the reinforcement learning to obtain the semantic embedding coding vector of the source state representation of the reinforcement learning; semantic embedding coding is performed on each piece of knowledge in the external knowledge base to obtain a set of semantic embedding coding vectors of the knowledge items.

[0010] In the above-mentioned decision support method based on large language model and reinforcement learning, the source state representation semantic embedding coding vector of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of knowledge item semantic embedding coding vectors are subjected to semantic association measurement to obtain a set of source state representation-knowledge item semantic matching coefficients, including: the source state representation semantic embedding coding vector of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of knowledge item semantic embedding coding vectors are respectively input into a semantic association measurement network based on a hash function to obtain a set of source state representation-knowledge item semantic matching coefficients.

[0011] In the above-mentioned decision support method based on large language model and reinforcement learning, multi-feature semantic weighted interaction is performed on the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector to obtain the enhanced source state representation, including: inputting the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector into a joint implicit feature capture network to obtain a reinforcement learning-data enhancement context joint implicit feature vector; performing feature activation based on Sigmoid function on the reinforcement learning-data enhancement context joint implicit feature vector to obtain a reinforcement learning-data enhancement conditional feature vector; calculating the reinforcement learning source state semantic contribution of the source state representation semantic embedding coding vector of the reinforcement learning relative to the reinforcement learning-data enhancement conditional feature vector; calculating the data enhancement semantic contribution of the data enhancement material representation vector relative to the reinforcement learning-data enhancement conditional feature vector; The source state semantic contribution and the data enhancement semantic contribution are normalized, and the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution are used to perform weighted modulation on the reinforcement learning source state representation semantic embedding coding vector and the data enhancement material representation vector to obtain the modulated reinforcement learning source state representation semantic embedding coding vector and the modulated data enhancement material representation vector; the modulated reinforcement learning source state representation semantic embedding coding vector is used as the query vector, the modulated data enhancement material representation vector is used as the key vector and the reinforcement learning-data enhancement conditional feature vector is used as the value vector, the modulated reinforcement learning source state representation semantic embedding coding vector, the modulated data enhancement material representation vector and the reinforcement learning-data enhancement conditional feature vector are input into a feature-to-feature significant guided interaction module based on a converter structure to obtain an enhanced source state representation vector as the enhanced source state representation.

[0012] In the above-mentioned decision support method based on large language model and reinforcement learning, the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector are input into the joint implicit feature capture network to obtain the reinforcement learning-data enhancement context joint implicit feature vector, including: adding the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector by position, multiplying the obtained reinforcement learning-data enhancement sum vector with the weight matrix and then adding it by position with the bias vector to obtain the reinforcement learning-data enhancement joint interaction vector; using the tanh function to process the reinforcement learning-data enhancement joint interaction vector to obtain the reinforcement learning-data enhancement context joint implicit feature vector.

[0013] In the above-mentioned decision support method based on large language model and reinforcement learning, the reinforcement learning source state semantic contribution of the source state representation semantic embedding coding vector of the reinforcement learning relative to the reinforcement learning-data enhancement conditional feature vector is calculated, including: calculating the positional division of the corresponding positions of the source state representation semantic embedding coding vector of the reinforcement learning and the reinforcement learning-data enhancement conditional feature vector to obtain the reinforcement learning source state semantic contribution vector; calculating the logarithmic function value with base two of the absolute value of each eigenvalue of the reinforcement learning source state semantic contribution vector to obtain the reinforcement learning source state semantic contribution logarithmic vector; calculating the positional dot product of the source state representation semantic embedding coding vector of the reinforcement learning and the reinforcement learning source state semantic contribution logarithmic vector, and performing positional multiplication on the obtained dot product vector to obtain the reinforcement learning source state semantic contribution value; calculating the exponential function with the natural constant e as the base and the reinforcement learning source state semantic contribution value as the exponent to obtain the reinforcement learning source state semantic contribution.

[0014] In the above-mentioned decision support method based on large language model and reinforcement learning, the reinforcement learning source state semantic contribution and the data enhancement semantic contribution are normalized, and the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution are used to weighted modulate the reinforcement learning source state representation semantic embedding coding vector and the data enhancement material representation vector to obtain the modulated reinforcement learning source state representation semantic embedding coding vector and the modulated data enhancement material representation vector, including: calculating the sum of the reinforcement learning source state semantic contribution and the data enhancement semantic contribution to obtain the reinforcement learning-data enhancement semantic contribution sum value; respectively divide the reinforcement learning source state semantic contribution and the data enhancement semantic contribution by the reinforcement learning-data enhancement semantic contribution sum value to obtain the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution; perform position point multiplication on the source state representation semantic embedding coding vector of the reinforcement learning and the normalized reinforcement learning source state semantic contribution to obtain the modulated reinforcement learning source state representation semantic embedding coding vector; perform position point multiplication on the data enhancement material representation vector and the normalized data enhancement semantic contribution to obtain the modulated data enhancement material representation vector.

[0015] In the above-mentioned decision support method based on large language model and reinforcement learning, the source state representation semantic embedding coding vector of the modulated reinforcement learning is used as the query vector, the modulated data enhancement material representation vector is used as the key vector and the reinforcement learning-data enhancement conditional feature vector is used as the value vector, the source state representation semantic embedding coding vector of the modulated reinforcement learning, the modulated data enhancement material representation vector and the reinforcement learning-data enhancement conditional feature vector are input into the feature-to-feature significant guided interaction module based on the converter structure to obtain the enhanced source state representation vector, including: vector multiplying the source state representation semantic embedding coding vector of the modulated reinforcement learning with the transposed vector of the modulated data enhancement material representation vector, dividing the obtained modulated reinforcement learning-data enhancement association matrix by the square root of the length of the modulated data enhancement material representation vector by position to obtain the modulated reinforcement learning-data enhancement association scaling matrix; using the softmax function to process the modulated reinforcement learning-data enhancement association scaling matrix, and matrix-vector multiplying the obtained modulated reinforcement learning-data enhancement association scaling activation matrix with the reinforcement learning-data enhancement conditional feature vector to obtain the enhanced source state representation vector.

[0016] According to another aspect of the present application, a decision support system based on a large language model and reinforcement learning is provided, comprising: a reinforcement learning source state representation data collection module, used to obtain a source state representation of reinforcement learning; a reinforcement learning source state representation data enhancement module, used to perform data enhancement on the source state representation of reinforcement learning based on an external knowledge base to obtain an enhanced source state representation; an enhanced source state representation conversion module, used to convert the enhanced source state representation into a state representation of a preset large language model based on a natural language descriptor, wherein the state representation of the preset large language model includes at least one of a task description, state details, output requirements and feedback information; a state representation function intrinsic reward function generation module, used to input the state representation of the preset large language model into the preset large language model to generate an enhanced state representation function and an intrinsic reward function of a reinforcement learning agent; a smooth state update module, used to update and maintain a corresponding Lipsitz array based on the enhanced state representation function and the intrinsic reward function to generate a state representation that meets a preset smooth condition; and a decision result generation module, used to input the state representation that meets the preset smooth condition into a decision model based on a classifier to obtain a decision result.

[0017] The present application has significant technical effects due to the adoption of the above technical solutions: the decision support system and method based on a large language model and reinforcement learning provided by the present application, which uses data analysis and enhancement technology based on deep learning to semantically embed the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base, then performs semantic association measurement on the semantic features of the source state representation of the reinforcement learning after embedding encoding and the semantic embedding features of each knowledge item, and then selects the data enhancement material representation from each semantic matching coefficient, so as to use the multi-feature semantic interaction representation between the semantic features of the source state representation of the reinforcement learning and the data enhancement material representation as the enhanced source state representation. In this way, the intelligent agent's understanding of the environment can be enhanced, and by introducing an external knowledge base, the knowledge items most relevant to the current state can be screened out to effectively eliminate redundant information in the source state representation and retain key feature information, thereby improving the quality of the state representation and improving the reliability of the decision. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1The present invention is a flowchart of a decision support method based on a large language model and reinforcement learning according to an embodiment of the present application.

[0020] Figure 2 The present invention is a flowchart of performing data enhancement on the source state representation of the reinforcement learning based on an external knowledge base to obtain an enhanced source state representation in a decision support method based on a large language model and reinforcement learning according to an embodiment of the present application.

[0021] Figure 3 In a decision support method based on a large language model and reinforcement learning according to an embodiment of the present application, a flowchart of performing multi-feature semantic weighted interaction on a source state representation semantic embedding encoding vector of the reinforcement learning and a data enhancement material representation vector to obtain the enhanced source state representation.

[0022] Figure 4 It is a system block diagram of a decision support system based on a large language model and reinforcement learning according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0024] In the field of reinforcement learning, the core goal of state representation algorithms is to capture task relevance and environmental dynamics to enhance the robustness of model training. With the rise of large language models, their capabilities in language understanding and generation can be used as prior knowledge for reinforcement learning algorithms, thereby providing rich semantic information and contextual understanding, thereby improving the decision-making ability and adaptability of intelligent agents, enabling them to better cope with complex and changing environmental challenges.

[0025] Patent CN118504612A proposes a state representation method and device based on a large language model and reinforcement learning. It first converts the state in reinforcement learning into a state representation preset by a preset large language model through a natural language descriptor. Subsequently, the converted state representation is input into the large language model to generate an enhanced state representation function and an intrinsic reward function, wherein the intrinsic reward helps the intelligent agent explore the environment. Finally, the Lipsitz array is updated and maintained through these generated functions to ensure the smoothness of the state representation until the established conditions are met.

[0026] Although converting the original state representation of reinforcement learning to the state representation of a large language model can simplify the operation process, this approach has some potential problems. Specifically, the original state representation may contain both key features and redundant information. Direct conversion may lead to the loss or insufficient representation of key information, which in turn affects the quality of the state representation function and the intrinsic reward function. In addition, direct conversion may ignore contextual information that is critical to understanding the environment, thereby affecting the performance of the agent in specific tasks. If the original state representation contains noise or incomplete information, directly inputting it into a large language model without proper processing may exacerbate these problems and reduce the reliability of decision-making.

[0027] Based on this, this application proposes a decision support method based on a large language model and reinforcement learning. Figure 1 FIG. 1 is a flow chart of a decision support method based on a large language model and reinforcement learning according to an embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the decision support method based on the large language model and reinforcement learning includes: S110, obtaining the source state representation of reinforcement learning; S120, performing data enhancement on the source state representation of reinforcement learning based on an external knowledge base to obtain an enhanced source state representation; S130, based on a natural language descriptor, converting the enhanced source state representation into a state representation of a preset large language model, wherein the state representation of the preset large language model includes at least one of task description, state details, output requirements and feedback information; S140, inputting the state representation of the preset large language model into the preset large language model to generate an enhanced state representation function and an intrinsic reward function of the reinforcement learning agent; S150, updating and maintaining the corresponding Lipsitz array based on the enhanced state representation function and the intrinsic reward function to generate a state representation that meets a preset smoothness condition; S160, inputting the state representation that meets the preset smoothness condition into a decision model based on a classifier to obtain a decision result.

[0028] In step S110, the source state representation of reinforcement learning is obtained. It should be understood that the source state representation of reinforcement learning refers to the original state information of the environment in which the agent is located when performing reinforcement learning tasks, which is the basis for the agent to make decisions and choose actions. Specifically, the source state representation usually includes the current position, state or condition data of the agent in the environment, and the direct information about the environment obtained by the agent through sensors or other perception mechanisms.

[0029] In step S120, data enhancement is performed on the source state representation of the reinforcement learning based on an external knowledge base to obtain an enhanced source state representation.

[0030] Accordingly, in the process of data enhancement of the source state representation of the reinforcement learning based on an external knowledge base to obtain an enhanced source state representation, the technical concept of the present application is to semantically embed the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base by adopting data analysis and enhancement technology based on deep learning, and then perform semantic association measurement on the semantic features of the source state representation of the reinforcement learning after embedding encoding and the semantic embedding features of each knowledge item, and then select the data enhancement material representation from each semantic matching coefficient, so as to use the multi-feature semantic interaction representation between the semantic features of the source state representation of the reinforcement learning and the data enhancement material representation as the enhanced source state representation. In this way, the intelligent agent's understanding of the environment can be enhanced, and by introducing an external knowledge base, the knowledge items most relevant to the current state can be screened out to effectively eliminate redundant information in the source state representation and retain key feature information, thereby improving the quality of the state representation and improving the reliability of the decision.

[0031] Specifically, Figure 2 The flowchart is a method for decision support based on a large language model and reinforcement learning according to an embodiment of the present application, in which data is enhanced on the source state representation of the reinforcement learning based on an external knowledge base to obtain an enhanced source state representation. Figure 2 As shown, data enhancement is performed on the source state representation of the reinforcement learning based on an external knowledge base to obtain an enhanced source state representation, including: S121, semantic embedding coding is performed on the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base to obtain a set of semantic embedding coding vectors of the source state representation of the reinforcement learning and semantic embedding coding vectors of knowledge items; S122, semantic association measurement is performed on the semantic embedding coding vector of the source state representation of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of semantic embedding coding vectors of the knowledge items to obtain a set of source state representation-knowledge item semantic matching coefficients; S123, the knowledge item semantic embedding coding vector corresponding to the maximum value of the source state representation-knowledge item semantic matching coefficient in the set of source state representation-knowledge item semantic matching coefficients is selected as the data enhancement material representation vector; S124, multi-feature semantic weighted interaction is performed on the semantic embedding coding vector of the source state representation of the reinforcement learning and the data enhancement material representation vector to obtain the enhanced source state representation.

[0032] In step S121, semantic embedding coding is performed on the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base to obtain a set of semantic embedding coding vectors of the source state representation of the reinforcement learning and semantic embedding coding vectors of knowledge items. Specifically, in an embodiment of the present application, semantic embedding coding is performed on the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base to obtain a set of semantic embedding coding vectors of the source state representation of the reinforcement learning and semantic embedding coding vectors of knowledge items, including: semantic embedding coding is performed on the source state representation of the reinforcement learning to obtain the semantic embedding coding vector of the source state representation of the reinforcement learning; semantic embedding coding is performed on each piece of knowledge in the external knowledge base to obtain the set of semantic embedding coding vectors of the knowledge items.

[0033] It should be understood that the original source state representation of reinforcement learning may be limited to numerical or symbolic forms, and lacks deep meaning. Therefore, in order to improve the depth and accuracy of the agent's understanding of the environment state, so as to better understand and extract the key semantic information in the original state representation, in the technical solution of the present application, the source state representation of the reinforcement learning is semantically embedded to obtain the source state representation semantic embedding encoding vector of the reinforcement learning. That is, by semantically embedding it, the source state of reinforcement learning can be converted into a meaningful vector form, which can capture the semantics between states to better understand the semantic information implied therein. Accordingly, considering that the external knowledge base usually contains a large number of knowledge items, in order to convert these knowledge items into a unified vector representation so that the computer can better understand and process this information, in the technical solution of the present application, each piece of knowledge in the external knowledge base is semantically embedded to obtain a set of knowledge item semantic embedding encoding vectors. Through semantic embedding encoding processing, the key information of the original knowledge item knowledge can be retained, and the semantic relationship and context information between the knowledge items can also be captured, thereby providing a basis for subsequent matching.

[0034] In step S122, the source state representation semantic embedding coding vector of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of the knowledge item semantic embedding coding vector are subjected to semantic association measurement to obtain a set of source state representation-knowledge item semantic matching coefficients. Specifically, in an embodiment of the present application, the source state representation semantic embedding coding vector of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of the knowledge item semantic embedding coding vector are subjected to semantic association measurement to obtain a set of source state representation-knowledge item semantic matching coefficients, including: the source state representation semantic embedding coding vector of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of the knowledge item semantic embedding coding vector are respectively input into a semantic association measurement network based on a hash function to obtain a set of source state representation-knowledge item semantic matching coefficients.

[0035] It should be understood that the source state representation semantic embedding coding vector of the reinforcement learning is a semantic state representation of the state of the environment in which the current reinforcement learning agent is located, such as position, speed, position of objects, health value, etc. Each knowledge item semantic embedding coding vector in the set of knowledge item semantic embedding coding vectors contains the specific meaning of each knowledge item, such as "turn right when encountering a wall". Based on this, in order to evaluate which knowledge items in the knowledge base are most relevant to the state of the current agent, so as to help the agent filter out the information that is most likely to help its decision-making from the massive knowledge, in the technical solution of the present application, the source state representation semantic embedding coding vector of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of knowledge item semantic embedding coding vectors are respectively input into a semantic association measurement network based on a hash function to obtain a set of source state representation-knowledge item semantic matching coefficients. In the semantic association measurement network of the present application, the hash function is usually used to construct a hash table or hash space, in which each hash value corresponds to a specific semantic space position. The semantic embedding encoding vector of the reinforcement learning source state representation and the semantic embedding encoding vector of the knowledge item are mapped into this hash space, and then the semantic matching coefficient is calculated by comparing their positions in the hash space. In this way, the knowledge item most relevant to the source state representation can be effectively identified, thereby providing support for enhancing the source state representation.

[0036] In step S123, the knowledge item semantic embedding coding vector corresponding to the maximum value of the source state representation-knowledge item semantic matching coefficient in the set of source state representation-knowledge item semantic matching coefficients is selected as the data enhancement material representation vector. Accordingly, in order to select the most relevant knowledge items while avoiding the introduction of irrelevant or misleading information, thereby reducing the noise in the decision-making process and improving the accuracy of the decision, so as to improve the performance of the reinforcement learning agent in the decision-making process, in the technical solution of the present application, the knowledge item semantic embedding coding vector corresponding to the maximum value of the source state representation-knowledge item semantic matching coefficient in the set of source state representation-knowledge item semantic matching coefficients is selected as the data enhancement material representation vector. That is, by selecting the knowledge item semantic embedding coding vector corresponding to the maximum matching coefficient, the agent can obtain the knowledge most relevant to the current state, which can ensure that the agent refers to the information that is most helpful to the current situation when making decisions.

[0037] In step S124, a multi-feature semantic weighted interaction is performed on the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector to obtain the enhanced source state representation. Accordingly, considering that there is a mutual implicit correlation semantic interaction relationship between the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector, and in order to improve the state representation quality of the intelligent agent, so that it not only contains the information of the original state, but also incorporates the most relevant knowledge selected from the external knowledge base to obtain a richer and more comprehensive state representation, thereby helping the intelligent agent to better understand the current environment, in the technical solution of the present application, a multi-feature semantic weighted interaction is performed on the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector to obtain an enhanced source state representation vector.

[0038] Specifically, first, the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector are captured by a joint implicit feature capture network to capture and extract the commonality or complementary information between the two, so as to better carry out the semantic interaction between the two, and obtain the reinforcement learning-data enhancement context joint implicit feature vector. Afterwards, in order to further optimize these implicit feature representations and give them appropriate weights, the reinforcement learning-data enhancement context joint implicit feature vector is activated based on the Sigmoid function to obtain the reinforcement learning-data enhancement conditional feature vector. Then, the semantic contribution of the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector relative to the reinforcement learning-data enhancement conditional feature vector is calculated respectively to evaluate and quantify the contribution of these two types of information to the final state representation. Subsequently, the obtained reinforcement learning source state semantic contribution and data enhancement semantic contribution are normalized, and the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution are used to weighted modulate the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector. It can be understood that through normalization and weighted modulation, it is possible to ensure that the contribution ratio of source state information and data enhancement information in the final state representation is reasonable, prevent a certain information source from over-dominantly dominating the decision-making process, and effectively emphasize those parts that are more important to the conditional feature vector, and obtain the semantic embedding encoding vector of the source state representation of the modulated reinforcement learning and the modulated data enhancement material representation vector. Finally, the semantic embedding encoding vector of the source state representation of the modulated reinforcement learning is used as the query vector, the modulated data enhancement material representation vector is used as the key vector, and the reinforcement learning-data enhancement conditional feature vector is used as the value vector. These three vectors are input into the feature-to-feature significant guided interaction module based on the converter structure for multi-feature guided interaction, and adaptively focus on the most important feature combination mode to obtain the enhanced source state representation vector. In this way, the intelligent agent's understanding of the environment can be enhanced, and by introducing an external knowledge base, the knowledge items most relevant to the current state can be screened out to effectively eliminate redundant information in the source state representation and retain key feature information, thereby improving the quality of state representation and improving the reliability of decision-making.

[0039] Specifically, Figure 3 The present invention is a flowchart of performing multi-feature semantic weighted interaction on the source state representation semantic embedding encoding vector of the reinforcement learning and the data enhancement material representation vector to obtain the enhanced source state representation in the decision support method based on the large language model and reinforcement learning according to the embodiment of the present application. Figure 3As shown, a multi-feature semantic weighted interaction is performed on the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector to obtain the enhanced source state representation, including: S1241, inputting the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector into a joint implicit feature capture network to obtain a reinforcement learning-data enhancement context joint implicit feature vector; S1242, performing feature activation based on a Sigmoid function on the reinforcement learning-data enhancement context joint implicit feature vector to obtain a reinforcement learning-data enhancement conditional feature vector; S1243, calculating the reinforcement learning source state semantic contribution of the source state representation semantic embedding coding vector of the reinforcement learning relative to the reinforcement learning-data enhancement conditional feature vector; S1244, calculating the data enhancement semantic contribution of the data enhancement material representation vector relative to the reinforcement learning-data enhancement conditional feature vector; S1245, The learning source state semantic contribution and the data enhancement semantic contribution are normalized, and the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution are used to weighted modulate the reinforcement learning source state representation semantic embedding coding vector and the data enhancement material representation vector to obtain the modulated reinforcement learning source state representation semantic embedding coding vector and the modulated data enhancement material representation vector; S1246, using the modulated reinforcement learning source state representation semantic embedding coding vector as the query vector, the modulated data enhancement material representation vector as the key vector and the reinforcement learning-data enhancement conditional feature vector as the value vector, the modulated reinforcement learning source state representation semantic embedding coding vector, the modulated data enhancement material representation vector and the reinforcement learning-data enhancement conditional feature vector are input into the feature-to-feature significant guided interaction module based on the converter structure to obtain the enhanced source state representation vector as the enhanced source state representation.

[0040] More specifically, in an embodiment of the present application, the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector are input into a joint implicit feature capture network to obtain a reinforcement learning-data enhancement context joint implicit feature vector, including: adding the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector by position, multiplying the obtained reinforcement learning-data enhancement sum vector with the weight matrix and then adding it by position with the bias vector to obtain a reinforcement learning-data enhancement joint interaction vector; using a tanh function to process the reinforcement learning-data enhancement joint interaction vector to obtain the reinforcement learning-data enhancement context joint implicit feature vector.

[0041] More specifically, in an embodiment of the present application, the reinforcement learning source state semantic contribution of the source state representation semantic embedding coding vector of the reinforcement learning relative to the reinforcement learning-data augmentation conditional feature vector is calculated, including: calculating the positional division of the corresponding positions of the source state representation semantic embedding coding vector of the reinforcement learning and the reinforcement learning-data augmentation conditional feature vector to obtain the reinforcement learning source state semantic contribution vector; calculating the logarithmic function value with base two of the absolute value of each eigenvalue of the reinforcement learning source state semantic contribution vector to obtain the reinforcement learning source state semantic contribution logarithmic vector; calculating the positional dot product of the source state representation semantic embedding coding vector of the reinforcement learning and the reinforcement learning source state semantic contribution logarithmic vector, and performing positional multiplication on the obtained dot product vector to obtain the reinforcement learning source state semantic contribution value; calculating an exponential function with the natural constant e as the base and the reinforcement learning source state semantic contribution value as the exponent to obtain the reinforcement learning source state semantic contribution.

[0042] More specifically, in an embodiment of the present application, the reinforcement learning source state semantic contribution and the data enhancement semantic contribution are normalized, and the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution are used to weighted modulate the reinforcement learning source state representation semantic embedding coding vector and the data enhancement material representation vector to obtain the modulated reinforcement learning source state representation semantic embedding coding vector and the modulated data enhancement material representation vector, including: calculating the sum of the reinforcement learning source state semantic contribution and the data enhancement semantic contribution to obtain the reinforcement learning-data enhancement semantic contribution sum; dividing The reinforcement learning source state semantic contribution and the data enhancement semantic contribution are divided by the reinforcement learning-data enhancement semantic contribution sum to obtain the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution; the reinforcement learning source state representation semantic embedding coding vector is point-multiplied by the normalized reinforcement learning source state semantic contribution to obtain the modulated reinforcement learning source state representation semantic embedding coding vector; the data enhancement material representation vector is point-multiplied by the normalized data enhancement semantic contribution to obtain the modulated data enhancement material representation vector.

[0043] More specifically, in an embodiment of the present application, the source state representation semantic embedding coding vector of the modulated reinforcement learning is used as a query vector, the modulated data enhancement material representation vector is used as a key vector, and the reinforcement learning-data enhancement conditional feature vector is used as a value vector. The source state representation semantic embedding coding vector of the modulated reinforcement learning, the modulated data enhancement material representation vector and the reinforcement learning-data enhancement conditional feature vector are input into a feature-to-feature significant guided interaction module based on a converter structure to obtain an enhanced source state representation vector, including: vector multiplying the source state representation semantic embedding coding vector of the modulated reinforcement learning with the transposed vector of the modulated data enhancement material representation vector, dividing the obtained modulated reinforcement learning-data enhancement association matrix by the square root of the length of the modulated data enhancement material representation vector by position to obtain a modulated reinforcement learning-data enhancement association scaling matrix; processing the modulated reinforcement learning-data enhancement association scaling matrix using a softmax function, and matrix-vector multiplying the obtained modulated reinforcement learning-data enhancement association scaling activation matrix with the reinforcement learning-data enhancement conditional feature vector to obtain the enhanced source state representation vector.

[0044] In the embodiment of the present application, specifically, a multi-feature semantic weighted interaction is performed on the source state representation semantic embedding encoding vector of the reinforcement learning and the data enhancement material representation vector, which can be expressed by the formula: ;in, and are respectively the source state representation semantic embedding encoding vector of the reinforcement learning and the data enhancement material representation vector, To add by location, and are the weight matrix and bias vector respectively, for function, is the reinforcement learning-data augmentation context joint implicit feature vector, for function, represents the reinforcement learning-data augmentation condition feature vector, The feature values ​​of each position in the semantic embedding encoding vector representing the source state of the reinforcement learning, is the eigenvalue of each position in the data augmentation material representation vector, is the feature value of each position in the reinforcement learning-data enhancement condition feature vector, The number of eigenvalues ​​in the semantic embedding encoding vector representing the source state of the reinforcement learning and the data augmentation material representation vector, represents the logarithmic function value with base 2, represents an exponential function with the natural constant e as the base, To reinforce the semantic contribution of source state in learning, Contribute to data enhancement semantics, and They are the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution, and They are the source state representation semantic embedding encoding vector of the modulated reinforcement learning and the data augmentation material representation vector after modulation, yes The transposed vector of yes Length, represents vector multiplication, yes function, is the enhanced source state representation vector.

[0045] In particular, when the data augmentation material representation vector and the reinforcement learning source state representation semantic embedding coding vector represent the knowledge semantic embedding coding of the external knowledge base and the reinforcement learning source state representation semantic embedding coding respectively, although the semantic association metric maximum matching is performed based on the hash function, when performing multi-feature semantic weighted interaction, the joint implicit conditional sparsity corresponding to the embedding coding differences under the data source semantics will still lead to the lack of local significant guided interaction between features, that is, it leads to simple repetition of interaction distribution in the local semantic domain, affecting the semantic logical dependency of the enhanced source state representation and reducing its semantic quality.

[0046] Therefore, preferably, the enhanced source state representation is optimized, specifically comprising: determining an enhanced source state association response matrix and an enhanced source state distance response matrix based on the square root of the association value of the eigenvalue of the enhanced source state representation vector of the enhanced source state representation and the L2 distance value: wherein the process can be expressed as: ;in, is the enhanced source state representation vector, express The number of eigenvalues ​​in , is the first The eigenvalues ​​at the positions, is the first The eigenvalues ​​at the positions, is the first The eigenvalues ​​at the positions, is the first The feature value at each position.

[0047] The enhanced source state representation vector is matrix-multiplied with the enhanced source state association response matrix to obtain an enhanced source state association response vector; the enhanced source state distance response matrix is ​​matrix-multiplied with the transposed vector of the enhanced source state representation vector to obtain an enhanced source state distance response vector; the enhanced source state association response matrix and the enhanced source state distance response matrix are dot-multiplied, and then the matrix-multiplied with the transposed vector of the enhanced source state representation vector to obtain an enhanced source state bias vector; the enhanced source state association response vector, the enhanced source state distance response vector and the enhanced source state bias vector are dot-multiplied to obtain an optimized enhanced source state representation vector.

[0048] Here, the optimized enhanced source state representation vector is expressed as: ;in, is the enhanced source state representation vector, express The number of eigenvalues ​​in , represents matrix multiplication, is the enhanced source state association response matrix, Indicates adding by position point. is the enhanced source state distance response matrix, for The transposed vector of is the point-add vector between the enhanced source state association response vector and the enhanced source state distance response vector, It means point multiplication by position. is the enhanced source state bias vector, The optimized enhanced source state representation vector.

[0049] That is, by using the enhanced source state representation vector as its statistical reference-free distribution response framework based on the self-association matrix and the self-distance matrix, and constructing the enhanced source state representation vector based on the retrieval-enhanced reverse response to avoid simple repetition of feature distribution, and by ensuring the enhanced source state representation vector's intrinsic mapping logic based on the retrieval-response context to avoid its surface combinatorial mapping, the semantic logic dependency mapping of the enhanced source state representation vector is achieved while maintaining the intuitive response relevance, thereby improving the semantic quality of the enhanced source state representation. In this way, the intelligent agent's understanding of the environment can be enhanced, and by introducing an external knowledge base, the knowledge items most relevant to the current state can be screened out to effectively eliminate redundant information in the source state representation and retain key feature information, thereby improving the quality of state representation and the reliability of decision-making.

[0050] In summary, data enhancement of the source state representation of the reinforcement learning based on the external knowledge base to obtain the enhanced source state representation is clearly explained, which uses data analysis and enhancement technology based on deep learning to semantically embed the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base, and then performs semantic association measurement on the semantic features of the source state representation of the reinforcement learning after embedding encoding and the semantic embedding features of each knowledge item, and then selects the data enhancement material representation from each semantic matching coefficient, so as to use the multi-feature semantic interaction representation between the semantic features of the source state representation of the reinforcement learning and the data enhancement material representation as the enhanced source state representation. In this way, the intelligent agent's understanding of the environment can be enhanced, and by introducing an external knowledge base, the knowledge items most relevant to the current state can be screened out to effectively eliminate redundant information in the source state representation and retain key feature information, thereby improving the quality of state representation and improving the reliability of decision-making.

[0051] In step S130, based on the natural language descriptor, the enhanced source state representation is converted into a state representation of a preset large language model, wherein the state representation of the preset large language model includes at least one of a task description, state details, output requirements, and feedback information. It should be understood that the natural language descriptor can convert the source state representation of reinforcement learning into a state representation that can be understood and processed by the large language model, so as to further enrich the information in the state representation by utilizing the semantic understanding ability of the large language model. Specifically, the task description is to be able to contain detailed information about the current environment and task, so as to generate a more accurate state representation. The state details are information that can provide information about each dimension of the source state representation, which may include, but is not limited to, the position, speed, and other related information of the agent, and the present application does not make specific restrictions on this. The output requirements are requirements for the large language model to output the state representation function and the intrinsic reward function enhanced by the reinforcement learning agent, such as code requirements, vector requirements, decision tree requirements, and other requirements, which can be set according to the actual situation, and the present application does not make specific restrictions on this. The feedback information includes historical feedback information of previous iterations, which helps the large language model to summarize historical experience and make improvements. The preset large language model described here can be set by those skilled in the art according to actual conditions, and this application does not impose any specific restrictions on this.

[0052] In step S140, the state representation of the preset large language model is input into the preset large language model to generate a state representation function and an intrinsic reward function enhanced by the reinforcement learning agent. It should be understood that the state representation is a way of encoding the state of the environment in reinforcement learning. A good state representation can effectively capture the characteristics of the environment, thereby helping the agent to make better decisions. In order to utilize the powerful language understanding and generation capabilities of the large language model and enhance the expressive power of the state representation, in the technical solution of the present application, the state representation of the preset large language model is input into the preset large language model to generate a state representation function and an intrinsic reward function enhanced by the reinforcement learning agent. The state representation function is a function that maps the state of the environment to a high-dimensional feature space. It converts the original state into a form that can be processed by the reinforcement learning algorithm. This mapping can extract more meaningful features, thereby improving learning efficiency. The intrinsic reward function is a concept in reinforcement learning, which refers to the ability of the agent to generate rewards based on its internal state or goal in the absence of an external reward signal. Intrinsic rewards can motivate the agent to explore the environment and promote the learning process, especially in the case of sparse rewards. By obtaining the state representation function and the intrinsic reward function, the richness of the state representation and the effectiveness of the intrinsic reward can be enhanced in the future, so that the intelligent agent can better understand the state in a complex environment and achieve higher decision-making quality.

[0053] In step S150, based on the enhanced state representation function and the intrinsic reward function, the corresponding Lipsitz array is updated and maintained to generate a state representation that meets the preset smoothness condition. It should be understood that in reinforcement learning, updating and maintaining the Lipsitz array is a method to ensure the smoothness of the state representation, which helps the reinforcement learning agent to learn and make decisions more stably. In particular, in an embodiment of the present application, an implementation method for updating and maintaining the corresponding Lipsitz array based on the enhanced state representation function and the intrinsic reward function to generate a state representation that meets the preset smoothness condition can be: sampling the enhanced state representation function and the intrinsic reward function to obtain the corresponding sampled enhanced state representation function and the sampled intrinsic reward function; based on the sampled enhanced state representation function and the sampled intrinsic reward function, using the historical training results as feedback, in each training process, the corresponding Lipsitz array is updated and maintained by soft update until the preset iteration stop condition is reached, and the state representation is output. Among them, each element in the Lipsitz array can be, but is not limited to, determined by the Lipsitz constant between each dimension of the sampled enhanced state representation in the large language model and the external reward of the environment. Among them, the expression of the Lipsitz array can be: ;in, Represents a trajectory of an agent interacting with the environment The Lipsitz array under Representing different trajectories The global Lipsitz array under , and is updated by soft update, is the hyperparameter for soft update.

[0054] It is understandable that the Lipsitz constant in the embodiment of the present application can be used to measure the smoothness of a mapping, and the higher the value of the constant, the less smooth the mapping. In the embodiment of the present application, the Lipsitz array is used to estimate the smoothness between each enhanced state representation in the large language model and the environmental task reward. The embodiment of the present application can generate a smoother state representation by updating and maintaining the corresponding Lipsitz array until the preset conditions are met.

[0055] In step S160, the state representation that meets the preset smoothness condition is input into a decision model based on a classifier to obtain a decision result. It should be understood that a classifier is a machine learning model that is trained by a large amount of labeled training data and can predict output results based on input features. The state representation that meets the preset smoothness condition is provided as input data to a decision model based on a classifier, and the model can output a decision result of the action that the agent should take in the current state based on the rich environmental dynamic information contained in the state representation. In the present application, this decision result can be a specific action or a probability distribution of an action, and is not limited here. According to the decision result, the agent can be helped to make better choices in a complex environment, thereby improving the completion rate and efficiency of the task.

[0056] In summary, the decision support method based on the large language model and reinforcement learning according to the embodiment of the present application is explained, which first obtains the source state representation of reinforcement learning, and uses an external knowledge base to perform data enhancement on it, and then converts the enhanced state representation into the state representation of the preset large language model, and then inputs the state representation into the preset large language model to generate the state representation function and intrinsic reward function enhanced by the reinforcement learning agent, and based on these two functions, the Lipsitz array is updated and maintained to generate a state representation that meets the smoothness condition, and finally, the state representation that meets the condition is input into the classifier decision model to obtain the decision result. In this way, by introducing external information, it is helpful to improve the quality of the source state representation of reinforcement learning, thereby improving the decision-making ability and adaptability of the agent.

[0057] Figure 4 : is a system block diagram of a decision support system based on a large language model and reinforcement learning according to an embodiment of the present application. Figure 4As shown, according to the embodiment of the present application, the decision support system 100 based on the large language model and reinforcement learning includes: a reinforcement learning source state representation data collection module 110, which is used to obtain the source state representation of reinforcement learning; a reinforcement learning source state representation data enhancement module 120, which is used to perform data enhancement on the source state representation of reinforcement learning based on an external knowledge base to obtain an enhanced source state representation; an enhanced source state representation conversion module 130, which is used to convert the enhanced source state representation into a state representation of a preset large language model based on a natural language descriptor, wherein the state representation of the preset large language model includes a task description, a state detail, and a state representation. At least one of the sections, output requirements and feedback information; a state representation function intrinsic reward function generation module 140, which is used to input the state representation of the preset large language model into the preset large language model to generate an enhanced state representation function and intrinsic reward function of the reinforcement learning agent; a smooth state update module 150, which is used to update and maintain the corresponding Lipsitz array based on the enhanced state representation function and the intrinsic reward function to generate a state representation that meets the preset smoothness condition; a decision result generation module 160, which is used to input the state representation that meets the preset smoothness condition into a decision model based on a classifier to obtain a decision result.

[0058] Here, those skilled in the art will appreciate that the specific functions and operations of the various units and modules in the decision support system 100 based on the large language model and reinforcement learning have been described in detail above. Figures 1 to 3 The description of the decision support method based on large language model and reinforcement learning has been introduced in detail, and therefore, its repeated description will be omitted.

[0059] In summary, the decision support system 100 based on the large language model and reinforcement learning according to the embodiment of the present application is explained, which first obtains the source state representation of reinforcement learning, and uses the external knowledge base to perform data enhancement on it, and then converts the enhanced state representation into the state representation of the preset large language model, and then inputs the state representation into the preset large language model to generate the state representation function and intrinsic reward function enhanced by the reinforcement learning agent, and based on these two functions, the Lipsitz array is updated and maintained to generate a state representation that meets the smoothness condition, and finally, the state representation that meets the condition is input into the classifier decision model to obtain the decision result. In this way, by introducing external information, it is helpful to improve the quality of the source state representation of reinforcement learning, thereby improving the decision-making ability and adaptability of the agent.

Claims

1. A decision support method based on a large language model and reinforcement learning, characterized in that: include: Obtain source state representation for reinforcement learning; Performing data enhancement on the source state representation of the reinforcement learning based on an external knowledge base to obtain an enhanced source state representation; Based on a natural language descriptor, the enhanced source state representation is converted into a state representation of a preset large language model, wherein the state representation of the preset large language model includes at least one of a task description, state details, output requirements, and feedback information; the state representation of the preset large language model is input into the preset large language model to generate a state representation function and an intrinsic reward function enhanced by the reinforcement learning agent; Based on the enhanced state representation function and the intrinsic reward function, the corresponding Lipsitz array is updated and maintained to generate a state representation that meets a preset smoothness condition; the state representation that meets the preset smoothness condition is input into a decision model based on a classifier to obtain a decision result.

2. The decision support method based on large language model and reinforcement learning according to claim 1, characterized in that: The source state representation of the reinforcement learning is data enhanced based on an external knowledge base to obtain an enhanced source state representation, including: semantically embedding the source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base to obtain a set of semantic embedding coding vectors of the source state representation of the reinforcement learning and semantic embedding coding vectors of knowledge items; performing semantic association measurement on the semantic embedding coding vector of the source state representation of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of semantic embedding coding vectors of the knowledge items to obtain a set of source state representation-knowledge item semantic matching coefficients; selecting the knowledge item semantic embedding coding vector corresponding to the maximum value of the source state representation-knowledge item semantic matching coefficient in the set of source state representation-knowledge item semantic matching coefficients as a data enhancement material representation vector; performing multi-feature semantic weighted interaction on the semantic embedding coding vector of the source state representation of the reinforcement learning and the data enhancement material representation vector to obtain the enhanced source state representation.

3. The decision support method based on large language model and reinforcement learning according to claim 2, characterized in that: The source state representation of the reinforcement learning and each piece of knowledge in the external knowledge base are semantically embedded to obtain a set of semantic embedding coding vectors of the source state representation of the reinforcement learning and semantic embedding coding vectors of knowledge items, including: semantically embedding the source state representation of the reinforcement learning to obtain the semantic embedding coding vector of the source state representation of the reinforcement learning; and semantically embedding each piece of knowledge in the external knowledge base to obtain a set of semantic embedding coding vectors of the knowledge items.

4. The decision support method based on large language model and reinforcement learning according to claim 3, characterized in that: The source state representation semantic embedding coding vector of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of knowledge item semantic embedding coding vectors are subjected to semantic association measurement to obtain a set of source state representation-knowledge item semantic matching coefficients, including: the source state representation semantic embedding coding vector of the reinforcement learning and each knowledge item semantic embedding coding vector in the set of knowledge item semantic embedding coding vectors are respectively input into a semantic association measurement network based on a hash function to obtain a set of source state representation-knowledge item semantic matching coefficients.

5. The decision support method based on large language model and reinforcement learning according to claim 4, characterized in that: The method comprises performing multi-feature semantic weighted interaction on the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector to obtain the enhanced source state representation, including: inputting the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector into a joint implicit feature capture network to obtain a reinforcement learning-data enhancement context joint implicit feature vector; performing feature activation based on a Sigmoid function on the reinforcement learning-data enhancement context joint implicit feature vector to obtain a reinforcement learning-data enhancement conditional feature vector; calculating the reinforcement learning source state semantic contribution of the source state representation semantic embedding coding vector of the reinforcement learning relative to the reinforcement learning-data enhancement conditional feature vector; calculating the data enhancement semantic contribution of the data enhancement material representation vector relative to the reinforcement learning-data enhancement conditional feature vector; comparing the reinforcement learning source state semantic contribution and the data enhancement conditional feature vector; and performing weighted interaction on the data enhancement conditional feature vector. The data enhancement semantic contribution is normalized, and the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution are used to perform weighted modulation on the reinforcement learning source state representation semantic embedding coding vector and the data enhancement material representation vector to obtain the modulated reinforcement learning source state representation semantic embedding coding vector and the modulated data enhancement material representation vector; the modulated reinforcement learning source state representation semantic embedding coding vector is used as the query vector, the modulated data enhancement material representation vector is used as the key vector, and the reinforcement learning-data enhancement conditional feature vector is used as the value vector, and the modulated reinforcement learning source state representation semantic embedding coding vector, the modulated data enhancement material representation vector and the reinforcement learning-data enhancement conditional feature vector are input into a feature-to-feature significant guided interaction module based on a converter structure to obtain an enhanced source state representation vector as the enhanced source state representation.

6. The decision support method based on large language model and reinforcement learning according to claim 5, characterized in that: The source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector are input into a joint implicit feature capture network to obtain a reinforcement learning-data enhancement context joint implicit feature vector, including: adding the source state representation semantic embedding coding vector of the reinforcement learning and the data enhancement material representation vector by position, multiplying the obtained reinforcement learning-data enhancement sum vector with a weight matrix, and then adding it by position with a bias vector to obtain a reinforcement learning-data enhancement joint interaction vector; using a tanh function to process the reinforcement learning-data enhancement joint interaction vector to obtain the reinforcement learning-data enhancement context joint implicit feature vector.

7. The decision support method based on large language model and reinforcement learning according to claim 6, characterized in that: Calculating the reinforcement learning source state semantic contribution of the source state representation semantic embedding coding vector of the reinforcement learning relative to the reinforcement learning-data enhancement conditional feature vector, including: calculating the positional division of the corresponding positions of the source state representation semantic embedding coding vector of the reinforcement learning and the reinforcement learning-data enhancement conditional feature vector to obtain the reinforcement learning source state semantic contribution vector; calculating the logarithmic function value with base two of the absolute value of each eigenvalue of the reinforcement learning source state semantic contribution vector to obtain the reinforcement learning source state semantic contribution logarithmic vector; calculating the positional dot product of the source state representation semantic embedding coding vector of the reinforcement learning and the reinforcement learning source state semantic contribution logarithmic vector, and performing positional multiplication on the obtained dot product vector to obtain the reinforcement learning source state semantic contribution value; calculating an exponential function with the natural constant e as the base and the reinforcement learning source state semantic contribution value as the exponent to obtain the reinforcement learning source state semantic contribution degree.

8. The decision support method based on large language model and reinforcement learning according to claim 7, characterized in that: The reinforcement learning source state semantic contribution and the data enhancement semantic contribution are normalized, and the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution are used to weighted modulate the reinforcement learning source state representation semantic embedding coding vector and the data enhancement material representation vector to obtain the modulated reinforcement learning source state representation semantic embedding coding vector and the modulated data enhancement material representation vector, including: calculating the sum of the reinforcement learning source state semantic contribution and the data enhancement semantic contribution to obtain the reinforcement learning-data enhancement semantic contribution sum; respectively The source state semantic contribution and the data enhancement semantic contribution are divided by the reinforcement learning-data enhancement semantic contribution sum to obtain the normalized reinforcement learning source state semantic contribution and the normalized data enhancement semantic contribution; the source state representation semantic embedding coding vector of the reinforcement learning is point-wise multiplied with the normalized reinforcement learning source state semantic contribution to obtain the modulated reinforcement learning source state representation semantic embedding coding vector; the data enhancement material representation vector is point-wise multiplied with the normalized data enhancement semantic contribution to obtain the modulated data enhancement material representation vector.

9. The decision support method based on large language model and reinforcement learning according to claim 8, characterized in that: The method comprises the following steps: using the semantic embedding coding vector of the source state representation of the modulated reinforcement learning as the query vector, using the modulated data enhancement material representation vector as the key vector and using the reinforcement learning-data enhancement conditional feature vector as the value vector, inputting the semantic embedding coding vector of the source state representation of the modulated reinforcement learning, the modulated data enhancement material representation vector and the reinforcement learning-data enhancement conditional feature vector into a feature-to-feature significant guided interaction module based on a converter structure to obtain an enhanced source state representation vector, including: vector multiplying the semantic embedding coding vector of the source state representation of the modulated reinforcement learning with the transposed vector of the modulated data enhancement material representation vector, dividing the obtained modulated reinforcement learning-data enhancement association matrix by the square root of the length of the modulated data enhancement material representation vector by position to obtain a modulated reinforcement learning-data enhancement association scaling matrix; processing the modulated reinforcement learning-data enhancement association scaling matrix using a softmax function, and matrix-vector multiplying the obtained modulated reinforcement learning-data enhancement association scaling activation matrix with the reinforcement learning-data enhancement conditional feature vector to obtain the enhanced source state representation vector.

10. A decision support system based on a large language model and reinforcement learning, characterized in that: include: A reinforcement learning source state representation data collection module is used to obtain the source state representation of reinforcement learning; A reinforcement learning source state representation data enhancement module, used for performing data enhancement on the reinforcement learning source state representation based on an external knowledge base to obtain an enhanced source state representation; An enhanced source state representation conversion module, used for converting the enhanced source state representation into a state representation of a preset large language model based on a natural language descriptor, wherein the state representation of the preset large language model includes at least one of a task description, state details, output requirements, and feedback information; a state representation function intrinsic reward function generation module, used for inputting the state representation of the preset large language model into the preset large language model to generate a state representation function and an intrinsic reward function enhanced by the reinforcement learning agent; A smooth state updating module is used to update and maintain the corresponding Lipsitz array based on the enhanced state representation function and the intrinsic reward function to generate a state representation that meets the preset smoothness condition; a decision result generating module is used to input the state representation that meets the preset smoothness condition into a decision model based on a classifier to obtain a decision result.

Images