Reinforced learning modeling method and device fusing multi-source heterogeneous data

Abstract

The invention discloses a reinforcement learning modeling method and device fusing multi-source heterogeneous data, and the method comprises the steps: obtaining the multi-source heterogeneous data from different data sources and having different data formats through an intelligent agent, carrying out the feature extraction through employing a corresponding feature extraction technology, obtaining different modal features, and carrying out the feature extraction through employing a corresponding feature extraction technology; effectively fusing different modal features to obtain environment state representation, and constructing an intelligent model of the intelligent agent by using a deep reinforcement learning algorithm based on the obtained environment state representation; different modal features extracted from different data sources are effectively fused, the fused features can comprehensively and accurately represent the current environment state, the intelligent model of the intelligent agent is constructed by using a deep reinforcement learning algorithm based on the fused environment state representation, and the intelligent model can obtain more comprehensive and accurate environment information, so that the intelligent model can be applied to the intelligent system of the intelligent agent. And the strategy is adjusted in real time to adapt to the change of the environment, so that the generalization ability and the accuracy of the intelligent model are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of building intelligent models, and in particular to a reinforcement learning modeling method and device that integrates multi-source heterogeneous data. Background Art

[0002] In the current intelligent modeling and optimization technology, reinforcement learning, especially deep reinforcement learning (DRL), has become a powerful tool for dealing with complex, high-dimensional state space and action space problems. Deep reinforcement learning combines the feature extraction ability of deep learning with the decision-making ability of reinforcement learning to achieve accurate modeling of complex environments and efficient strategy learning. However, in the face of the widespread existence of multi-source heterogeneous data (i.e., data from different data sources and in different data formats) in practical applications, how to effectively integrate this information to build a more comprehensive and accurate environmental model is still a major challenge facing the field of reinforcement learning.

[0003] Although existing reinforcement learning techniques can achieve good performance in a single data source or homogeneous data environment, they are often unable to cope with multi-source heterogeneous data. Traditional methods are mostly limited to modeling using a single data source, ignoring other potentially useful information, thereby limiting the generalization ability and accuracy of the model. Summary of the invention

[0004] In view of this, the present invention proposes a reinforcement learning modeling method and device that integrates multi-source heterogeneous data, which can solve the defect of the prior art that is limited to using a single data source for modeling.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A reinforcement learning modeling method integrating multi-source heterogeneous data, specifically comprising:

[0007] The agent acquires multi-source heterogeneous data from different data sources and in different data formats;

[0008] According to the type of multi-source heterogeneous data, the corresponding feature extraction technology is used to extract features and obtain different modal features;

[0009] Effectively integrate different modal features to obtain environmental state representation;

[0010] Based on the obtained environmental state representation, a deep reinforcement learning algorithm is used to build an intelligent model of the agent, so that the intelligent model can make optimal decisions based on the current environmental state and adjust the strategy in real time to adapt to environmental changes.

[0011] As a further optional solution of the reinforcement learning modeling method for fusing multi-source heterogeneous data, the corresponding feature extraction technology is used to extract features according to the type of multi-source heterogeneous data to obtain different modal features, specifically including:

[0012] Process image data using convolutional neural networks to extract visual features;

[0013] Use recurrent neural networks to process time series data to capture dynamic change characteristics;

[0014] Use natural language processing techniques to process text data to obtain semantic features.

[0015] As a further optional solution of the reinforcement learning modeling method for fusing multi-source heterogeneous data, the effective fusion of different modal features to obtain the environmental state representation specifically includes:

[0016] The different modal features extracted from various data sources are spliced ​​in the feature dimension to obtain a spliced ​​feature vector;

[0017] The concatenated feature vector is used as the input of the multi-layer perceptron model, and the input layer of the multi-layer perceptron model is nonlinearly transformed through the adjustable slope Softplus function;

[0018] The feature vector transformed by the adjustable slope Softplus function is passed to the hidden layer of the multi-layer perceptron model to obtain the relationship between different modal features, and feature fusion is performed based on the relationship between different modal features;

[0019] The output layer of the multi-layer perceptron model transforms the fused features into the final environment state representation.

[0020] As a further optional solution of the reinforcement learning modeling method for fusing multi-source heterogeneous data, the adjustable slope Softplus function is specifically:

[0021]

[0022] Among them, ASS(x, β) represents the eigenvector after nonlinear transformation, x represents the eigenvector after concatenation, β represents the adjustable factor of the slope, and e represents the base of the natural logarithm.

[0023] As a further optional solution of the reinforcement learning modeling method for fusing multi-source heterogeneous data, the intelligent model is constructed using a deep reinforcement learning algorithm based on the obtained environmental state representation, specifically including:

[0024] The environment state representation is input into the deep neural network, and the deep neural network outputs the corresponding action according to the current environment state representation;

[0025] Based on the output of the deep neural network, the agent selects an action to perform;

[0026] The environment feeds back a reward signal and a new state based on the action performed by the agent and the current state;

[0027] The reward function evaluates the quality of the action based on the agent's reward signal and current state, and gives a corresponding reward value;

[0028] Based on the reward value and the new state, a deep reinforcement learning algorithm is used to update the parameters of the agent’s deep neural network.

[0029] As a further optional solution of the reinforcement learning modeling method integrating multi-source heterogeneous data, the method further includes introducing an adaptive optimization strategy to adjust the parameters of the deep reinforcement learning algorithm and the intelligent model, specifically including:

[0030] During training and use, the changes in the environment are continuously monitored, wherein the changes in the environment include changes in the state space, action space, and reward function;

[0031] Regularly evaluate the performance of intelligent models, including task completion rate, accuracy and reward value indicators;

[0032] Based on environmental changes and model performance evaluation results, adaptive optimization strategies are used to adjust the parameters of deep reinforcement learning algorithms and intelligent models.

[0033] As a further optional solution of the reinforcement learning modeling method for integrating multi-source heterogeneous data, the method also includes introducing explainability technology in the intelligent model construction and decision-making process, specifically including:

[0034] In the intelligent model design stage, the explainability technology is integrated into the intelligent model architecture;

[0035] Integrate explainability components into the model’s decision-making process.

[0036] A reinforcement learning modeling device integrating multi-source heterogeneous data, comprising:

[0037] An acquisition module is used by the intelligent agent to acquire multi-source heterogeneous data from different data sources and in different data formats;

[0038] The feature extraction module is used to extract features according to the type of multi-source heterogeneous data using corresponding feature extraction technology to obtain different modal features;

[0039] Feature fusion module, used to effectively fuse features of different modalities to obtain environmental state representation;

[0040] The building module is used to build an intelligent model of the intelligent agent based on the obtained environmental state representation using a deep reinforcement learning algorithm, so that the intelligent model can make optimal decisions based on the current environmental state and adjust the strategy in real time to adapt to environmental changes.

[0041] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned reinforcement learning modeling method for fusing multi-source heterogeneous data are implemented.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned reinforcement learning modeling method for fusing multi-source heterogeneous data.

[0043] The beneficial effects of the present invention are: by effectively fusing different modal features extracted from different data sources, the fused features should be able to comprehensively and accurately represent the current environmental state. Based on the fused environmental state representation, an intelligent model of the intelligent agent is constructed using a deep reinforcement learning algorithm. The intelligent model can obtain more comprehensive and accurate environmental information, thereby making more precise decisions and adjusting strategies in real time to adapt to environmental changes, effectively improving the generalization ability and accuracy of the intelligent model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic diagram of the steps of a reinforcement learning modeling method that integrates multi-source heterogeneous data provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of the composition of a reinforcement learning modeling device that integrates multi-source heterogeneous data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] refer to Figure 1 to Figure 2, a reinforcement learning modeling method integrating multi-source heterogeneous data, specifically including:

[0049] The agent acquires multi-source heterogeneous data from different data sources and in different data formats;

[0050] According to the type of multi-source heterogeneous data, the corresponding feature extraction technology is used to extract features and obtain different modal features;

[0051] Effectively integrate different modal features to obtain environmental state representation;

[0052] Based on the obtained environmental state representation, a deep reinforcement learning algorithm is used to build an intelligent model of the agent, so that the intelligent model can make optimal decisions based on the current environmental state and adjust the strategy in real time to adapt to environmental changes.

[0053] In this embodiment, an intelligent agent (such as a robot, an autonomous vehicle, etc.) collects data from multiple data sources (such as sensors, text databases, image libraries, etc.) in an external environment with which it interacts. These data may include structured data (such as records in a database), unstructured data (such as text, images), and time series data, etc. The collected data is preprocessed by cleaning, denoising, normalization, and other operations to ensure the quality and consistency of the data; according to the type of multi-source heterogeneous data, a corresponding feature extraction technology is used, and each data type is subjected to feature extraction to obtain a feature representation of its own modality; different modal features extracted from different data sources are effectively fused, and the fused features should be able to fully and accurately represent the current environmental state; based on the fused environmental state representation, a deep reinforcement learning algorithm (such as a deep Q network, a policy gradient method, etc.) is used to construct an intelligent model of the intelligent agent, and the intelligent model can obtain more comprehensive and accurate environmental information, thereby making more accurate decisions, and adjust the strategy in real time to adapt to environmental changes, effectively improving the generalization ability and accuracy of the intelligent model.

[0054] Preferably, the method uses corresponding feature extraction technology to extract features according to the type of multi-source heterogeneous data to obtain different modal features, specifically including:

[0055] Process image data using convolutional neural networks to extract visual features;

[0056] Use recurrent neural networks to process time series data to capture dynamic change characteristics;

[0057] Use natural language processing techniques to process text data to obtain semantic features.

[0058] In this embodiment, a convolutional neural network (CNN) is used for feature extraction to capture visual features in the image, such as edges, textures, shapes, etc., a recurrent neural network (RNN) or its variants (such as LSTM, GRU) is used to process time series data to capture dynamic change features in the data, such as trends, periodicity, etc., and natural language processing technology (NLP) is used to process text data, and semantic features are obtained through word embedding, text classification, sentiment analysis and other methods.

[0059] Preferably, the effective fusion of different modal features to obtain the environmental state representation specifically includes:

[0060] The different modal features extracted from various data sources are spliced ​​in the feature dimension to obtain a spliced ​​feature vector;

[0061] The concatenated feature vector is used as the input of the multi-layer perceptron model, and the input layer of the multi-layer perceptron model is nonlinearly transformed through the adjustable slope Softplus function;

[0062] The feature vector transformed by the adjustable slope Softplus function is passed to the hidden layer of the multi-layer perceptron model to obtain the relationship between different modal features, and feature fusion is performed based on the relationship between different modal features;

[0063] The output layer of the multi-layer perceptron model transforms the fused features into the final environment state representation.

[0064] Preferably, the adjustable slope Softplus function is specifically:

[0065]

[0066] Among them, ASS(x, β) represents the eigenvector after nonlinear transformation, x represents the eigenvector after concatenation, β represents the adjustable factor of the slope, and e represents the base of the natural logarithm.

[0067] In this embodiment, by splicing features from different data sources (such as images, time series, and text) in the feature dimension, the unique information provided by each data source can be fully utilized. This splicing method enables the model to capture more comprehensive environmental information and enhances the complementarity of features; using an adjustable slope Softplus function to perform a nonlinear transformation on the spliced ​​feature vector helps to enhance the expressive power of the feature. The nonlinear transformation can introduce more nonlinear factors, making the model more flexible and robust when dealing with complex environments; the feature vector that has undergone nonlinear transformation is passed to the hidden layer of the multi-layer perceptron (MLP) model. Through the complex network structure of the MLP, the complex relationship between features of different modalities can be learned. The learning of these relationships makes feature fusion more accurate and can better reflect the real state of the environment; the output layer of the multi-layer perceptron model converts the fused features into the final environmental state representation, which not only contains information from multiple data sources, but is also optimized through feature fusion and nonlinear transformation, so it is more comprehensive and accurate. This environmental state representation can provide strong support for subsequent decision-making processes; since this technical solution can process multi-source heterogeneous data and effectively fuse them into a comprehensive environmental state representation, it improves the model's adaptability to different environments and tasks. At the same time, by introducing nonlinear transformation and complex network structure, the generalization ability of the model is also enhanced, and it can perform well in a wider range of application scenarios.

[0068] It should be noted that by adjusting β, the slope of the function can be controlled. When β increases, the slope of the function near x=0 also increases, making the function more sensitive near this point, which helps to capture subtle feature changes. When β decreases, the function becomes smoother, which is suitable for scenarios that need to avoid overfitting or capture global feature relationships. In addition, e represents the base of the natural logarithm, which is approximately equal to 2.71828.

[0069] Preferably, the intelligent model is constructed using a deep reinforcement learning algorithm based on the obtained environmental state representation, specifically including:

[0070] The environment state representation is input into the deep neural network, and the deep neural network outputs the corresponding action according to the current environment state representation;

[0071] Based on the output of the deep neural network, the agent selects an action to perform;

[0072] The environment feeds back a reward signal and a new state based on the action performed by the agent and the current state;

[0073] The reward function evaluates the quality of the action based on the agent's reward signal and current state, and gives a corresponding reward value;

[0074] Based on the reward value and the new state, a deep reinforcement learning algorithm is used to update the parameters of the agent’s deep neural network.

[0075] In this embodiment, the deep neural network is used as a policy network, which can process high-dimensional and complex environmental state representations and output precise actions to guide the intelligent agent to make decisions. This ability enables the intelligent agent to respond quickly and take appropriate actions when facing a complex and changing environment. The deep reinforcement learning algorithm continuously interacts with the environment and optimizes the parameters of the policy network based on reward value feedback. This process enables the intelligent agent to continuously adapt to changes in the environment and improve its performance in specific tasks. As training progresses, the intelligent agent can learn more efficient strategies, making it more handy when handling similar tasks. This technical solution makes full use of the advantages of reinforcement learning, that is, the intelligent agent optimizes its own behavior through continuous trial and error. Compared with traditional machine learning methods, deep reinforcement learning does not require a large amount of training. Instead of using labeled data, it learns through real-time interaction with the environment, which makes this technical solution more suitable for complex scenarios where it is difficult to obtain a large amount of labeled data; the reward function, as a criterion for evaluating the quality of actions, provides a clear direction for the learning of the intelligent agent. By constantly pursuing higher reward values, the intelligent agent can continuously optimize its own strategies and improve its performance in tasks. This continuous optimization ability makes the intelligent agent more competitive when facing long-term and complex tasks; this technical solution is not only suitable for the above-mentioned fields such as autonomous driving and energy management, but can also be widely used in robot navigation, game AI, financial investment and other fields. As long as an accurate representation of the environmental state can be constructed and trained using a deep reinforcement learning algorithm, an intelligent model with efficient decision-making ability can be constructed.

[0076] It should be noted that the environmental state representation is input into a deep neural network. This network (usually part of a deep reinforcement learning network, such as a policy network or a value network) receives the state vector as input and outputs some form of value or decision related to the current state. For the value network, it may output the expected return of all possible actions in a given state (i.e., Q value or value function). For the policy network, it may directly output the optimal or recommended action in a given state. After the agent performs an action, the environment provides a reward signal. This reward signal is the key feedback for evaluating the quality of the agent's decision. The reward function defines the immediate reward obtained by performing different actions in different states. It guides the agent on how to maximize the cumulative reward through learning. The deep reinforcement learning algorithm learns to optimize the strategy by maximizing the expectation of cumulative rewards, which usually involves updating the weights of the neural network using policy gradient methods, Q learning or its variants; the deep reinforcement learning algorithm learns and optimizes the strategy by constantly interacting with the environment. During the training process, the agent will try different actions according to the current strategy and reward function, and observe how the environment responds to these actions. Based on the observed results (i.e., state and reward), the algorithm will adjust the weights of the policy network to make better decisions in the future. This process is iterative until the agent can stably make the best decision based on the current state and adjust the strategy in real time to adapt to changes in the environment.

[0077] Preferably, the method further comprises introducing an adaptive optimization strategy to adjust the parameters of the deep reinforcement learning algorithm and the intelligent model, specifically comprising:

[0078] During training and use, the changes in the environment are continuously monitored, wherein the changes in the environment include changes in the state space, action space, and reward function;

[0079] Regularly evaluate the performance of intelligent models, including task completion rate, accuracy and reward value indicators;

[0080] Based on environmental changes and model performance evaluation results, adaptive optimization strategies are used to adjust the parameters of deep reinforcement learning algorithms and intelligent models.

[0081] In this embodiment, the environment changes are monitored in real time. By continuously monitoring the changes in the state space, action space and reward function, the intelligent model can quickly perceive the dynamics of the environment and provide a real-time basis for subsequent parameter adjustments. In the face of environmental uncertainty, the adaptive optimization strategy can timely adjust the parameters of the deep reinforcement learning algorithm and the intelligent model to ensure that the model can continue to adapt to new environmental characteristics and maintain efficient learning ability and decision-making accuracy; regular evaluation of the performance of the intelligent model, including key indicators such as task completion rate, accuracy rate and reward value, helps to accurately understand the performance of the model in the current environment. Based on the performance evaluation results, the adaptive optimization strategy can adjust the parameters of the algorithm and model in a targeted manner, optimize the model's learning process and decision-making strategy, thereby continuously improving the performance of the model. Improve the performance of the model; the adaptive optimization strategy can automatically adjust key parameters such as the learning rate and exploration strategy, avoiding the tedious process of manual parameter adjustment in traditional methods, improving learning efficiency and convergence speed. When faced with new tasks or environments, through mechanisms such as meta-learning, the intelligent model can use previously learned knowledge and experience to quickly initialize model parameters and find effective learning strategies, shortening the learning cycle; when the environment changes significantly (such as increased task difficulty, changes in data distribution, etc.), the adaptive optimization strategy can quickly adjust the model parameters to help the model quickly restore stable performance and reduce performance fluctuations and losses. By continuously optimizing algorithms and model parameters, the intelligent model can better cope with various complex and uncertain environmental factors and improve overall robustness and reliability.

[0082] It should be noted that the adjusted parameters include:

[0083] Learning Rate: The learning rate determines the magnitude of the change in the model's weights at each update. In adaptive optimization, the learning rate can be adjusted dynamically based on the performance during training. For example, a larger learning rate can be used in the early stages of training to quickly approach the optimal solution, and a smaller learning rate can be used in the later stages of training to make fine adjustments.

[0084] Exploration Strategy: In reinforcement learning, the exploration strategy determines the trade-off between trying new actions and exploiting known effective actions. Adaptive optimization can adjust the exploration strategy, such as increasing or decreasing the degree of exploration, to cope with changes in the environment and the progress of the task.

[0085] Model Parameters: For reinforcement learning algorithms that use complex models such as deep neural networks, model parameters (such as weights and biases of the neural network) are key to adjust. Adaptive optimization can automatically adjust these parameters through optimization algorithms (such as stochastic gradient descent, Adam, etc.) to minimize the loss function and improve model performance.

[0086] Discount Factor: In reinforcement learning, the discount factor is used to balance the importance of current rewards and future rewards. Adaptive optimization can adjust the discount factor according to the characteristics of the task and the dynamic changes of the environment to better evaluate the long-term performance of the agent.

[0087] Preferably, the method further includes introducing explainability technology in the intelligent model construction and decision-making process, specifically including:

[0088] In the intelligent model design stage, the explainability technology is integrated into the intelligent model architecture;

[0089] Integrate explainability components into the model’s decision-making process.

[0090] In this embodiment, by integrating the explainability technology into the intelligent model architecture, the decision-making process of the model can be visualized and traceable. For example, by visualizing the structure and weights of the policy network, or generating rules similar to decision trees to explain the generation process of the policy, the user can clearly understand how the model makes decisions, thereby increasing the transparency of the decision; introducing an explainability component into the decision-making process of the model can explain the basis and logic of the model's decision. In fields such as medicine and finance that have high requirements for decision-making interpretability, this technology can significantly improve the user's trust in the model's decision results. For example, in medical diagnosis, the model can explain the choice of treatment plan in detail based on the patient's symptoms, examination results and medical history, increasing the credibility of the treatment plan; in many application scenarios, the model's decision needs to be not only accurate, but also understandable and acceptable to users. By introducing explainability technology, the model can be made more in line with human cognitive habits, thereby improving the applicability of the model and user acceptance. For example, in the field of financial investment, showing investors the basis and logic of the reinforcement learning model's investment portfolio selection can help investors better understand the model's decision and increase the rationality of investment decisions.

[0091] It should be noted that the explainability technology is introduced in the model construction and decision-making process. The detailed steps are as follows:

[0092] First, clarify the specific requirements of the project or application for model interpretability. Different fields (such as medicine, finance, etc.) may have different requirements for interpretability, which need to be determined based on actual conditions.

[0093] Choose interpretability techniques that are appropriate for your project needs, such as visualization techniques (such as visualization of policy network structure and weights), feature importance analysis, decision trees, or rule extraction methods;

[0094] Incorporate interpretability techniques into the model architecture during the model design phase, which may include using specific neural network layers or modules (such as attention mechanisms, interpretable activation functions, etc.), or adopting hybrid models (such as a combination of decision trees and neural networks);

[0095] During model training, in addition to paying attention to the performance of the model (such as accuracy, reward value, etc.), it is also necessary to verify the interpretability of the model, which can be done by checking the generated interpretable output, such as visualized decision paths, feature importance rankings, etc.

[0096] Optimize and adjust the model based on the results of the interpretability verification. If the interpretability is insufficient, the model architecture, parameters, or training strategy may need to be adjusted.

[0097] Integrate explainability components into the model’s decision-making process. For example, when the model outputs a decision, it also outputs the basis and logic of the decision so that users or experts can understand how the model makes the decision.

[0098] Continuously monitor the interpretability and performance of the model. As the data and environment change, the model may need to be updated and improved regularly to maintain its interpretability and performance.

[0099] A reinforcement learning modeling device integrating multi-source heterogeneous data, comprising:

[0100] An acquisition module is used by the intelligent agent to acquire multi-source heterogeneous data from different data sources and in different data formats;

[0101] The feature extraction module is used to extract features according to the type of multi-source heterogeneous data using corresponding feature extraction technology to obtain different modal features;

[0102] Feature fusion module, used to effectively fuse features of different modalities to obtain environmental state representation;

[0103] The building module is used to build an intelligent model of the intelligent agent based on the obtained environmental state representation using a deep reinforcement learning algorithm, so that the intelligent model can make optimal decisions based on the current environmental state and adjust the strategy in real time to adapt to environmental changes.

[0104] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned reinforcement learning modeling method for fusing multi-source heterogeneous data are implemented.

[0105] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned reinforcement learning modeling method for fusing multi-source heterogeneous data.

[0106] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A reinforcement learning modeling method integrating multi-source heterogeneous data, characterized in that: Specifically include: The agent acquires multi-source heterogeneous data from different data sources and in different data formats; According to the type of multi-source heterogeneous data, the corresponding feature extraction technology is used to extract features and obtain different modal features; Effectively integrate different modal features to obtain environmental state representation; Based on the obtained environmental state representation, a deep reinforcement learning algorithm is used to build an intelligent model of the agent, so that the intelligent model can make optimal decisions based on the current environmental state and adjust the strategy in real time to adapt to environmental changes.

2. The reinforcement learning modeling method for integrating multi-source heterogeneous data according to claim 1 is characterized in that: According to the type of multi-source heterogeneous data, corresponding feature extraction technology is used to extract features to obtain different modal features, including: Process image data using convolutional neural networks to extract visual features; Use recurrent neural networks to process time series data to capture dynamic change characteristics; Use natural language processing techniques to process text data to obtain semantic features.

3. The reinforcement learning modeling method for integrating multi-source heterogeneous data according to claim 2 is characterized in that: The effective fusion of different modal features to obtain the environmental state representation specifically includes: The different modal features extracted from various data sources are spliced ​​in the feature dimension to obtain a spliced ​​feature vector; The concatenated feature vector is used as the input of the multi-layer perceptron model, and the input layer of the multi-layer perceptron model is nonlinearly transformed through the adjustable slope Softplus function; The feature vector transformed by the adjustable slope Softplus function is passed to the hidden layer of the multi-layer perceptron model to obtain the relationship between different modal features, and feature fusion is performed based on the relationship between different modal features; The output layer of the multi-layer perceptron model transforms the fused features into the final environment state representation.

4. The reinforcement learning modeling method for integrating multi-source heterogeneous data according to claim 3 is characterized in that: The adjustable slope Softplus function is specifically: Among them, ASS(x, β) represents the eigenvector after nonlinear transformation, x represents the eigenvector after concatenation, β represents the adjustable factor of the slope, and e represents the base of the natural logarithm.

5. The reinforcement learning modeling method for integrating multi-source heterogeneous data according to claim 4 is characterized in that: The intelligent model is constructed based on the obtained environmental state representation using a deep reinforcement learning algorithm, specifically including: The environment state representation is input into the deep neural network, and the deep neural network outputs the corresponding action according to the current environment state representation; Based on the output of the deep neural network, the agent selects an action to perform; The environment feeds back a reward signal and a new state based on the action performed by the agent and the current state; The reward function evaluates the quality of the action based on the agent's reward signal and current state, and gives a corresponding reward value; Based on the reward value and the new state, a deep reinforcement learning algorithm is used to update the parameters of the agent’s deep neural network.

6. A reinforcement learning modeling method for integrating multi-source heterogeneous data according to claim 5, characterized in that: The method also includes introducing an adaptive optimization strategy to adjust the parameters of the deep reinforcement learning algorithm and the intelligent model, specifically including: During training and use, the changes in the environment are continuously monitored, wherein the changes in the environment include changes in the state space, action space, and reward function; Regularly evaluate the performance of intelligent models, including task completion rate, accuracy and reward value indicators; Based on environmental changes and model performance evaluation results, adaptive optimization strategies are used to adjust the parameters of deep reinforcement learning algorithms and intelligent models.

7. The reinforcement learning modeling method for integrating multi-source heterogeneous data according to claim 6 is characterized in that: The method also includes introducing explainability technology into the intelligent model building and decision-making process, including: In the intelligent model design stage, the explainability technology is integrated into the intelligent model architecture; Integrate explainability components into the model’s decision-making process.

8. A reinforcement learning modeling device integrating multi-source heterogeneous data, characterized in that: include: An acquisition module is used by the intelligent agent to acquire multi-source heterogeneous data from different data sources and in different data formats; The feature extraction module is used to extract features according to the type of multi-source heterogeneous data using corresponding feature extraction technology to obtain different modal features; Feature fusion module, used to effectively fuse features of different modalities to obtain environmental state representation; The building module is used to build an intelligent model of the intelligent agent based on the obtained environmental state representation using a deep reinforcement learning algorithm, so that the intelligent model can make optimal decisions based on the current environmental state and adjust the strategy in real time to adapt to environmental changes.

9. A computing device, characterized in that The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the reinforcement learning modeling method for fusing multi-source heterogeneous data described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the reinforcement learning modeling method for fusing multi-source heterogeneous data as described in any one of claims 1 to 7.

Images