Multimodal power marketing data intelligent fusion system and method based on reinforcement learning

Abstract

The invention relates to the technical field of digital data processing, in particular to a multi-modal electric power marketing data intelligent fusion system and method based on reinforcement learning. The invention discloses a multi-modal electric power marketing data intelligent fusion system based on reinforcement learning. The multi-modal electric power marketing data intelligent fusion system comprises a multi-source data access layer, a ternary state sensing layer, an intelligent decision-making layer, a fusion execution layer and a feedback optimization layer. According to the method, through a special adaptive channel and feature standardization, the information entropy loss is reduced by 60%, and the standardization feature vector dimension unification degree reaches 100%; according to the exception handling mechanism, the service interruption duration is shortened to be less than 10 seconds, the fusion accuracy rate is stabilized to be more than 96%, the customer loss early warning response time is less than or equal to 3 seconds, and the precision marketing accuracy rate is greater than or equal to 98%.

Description

Technical Field

[0001] This invention relates to the field of digital data processing technology, specifically to a multimodal intelligent fusion system and method for electricity marketing data based on reinforcement learning. Background Technology

[0002] In the course of electricity marketing operations, massive amounts of multimodal data are generated, covering structured data from the marketing system, semi-structured data from the customer service system, and unstructured data from the multimedia system. The data generated by the marketing system exhibits multimodal characteristics: including structured user electricity consumption data, such as electricity consumption, electricity bills, and payment records, which are transmitted through REST API interfaces; semi-structured 95598 customer service work order data, such as text work orders and tag information, which are transmitted through Kafka subscriptions; and unstructured online business hall audio and video data, such as customer consultation recordings and business processing videos, which require decoding and processing.

[0003] Existing multimodal data fusion technologies have the following drawbacks:

[0004] (i) Insufficient handling of modal heterogeneity. There is a lack of dedicated multimodal adaptation channels, and there is overlap between structured and unstructured data information with large dimensional differences. Direct fusion leads to severe loss of information entropy and low fusion accuracy.

[0005] (ii) Low accuracy of state perception. Simply collecting single indicators such as data volume and CPU utilization without constructing a three-dimensional quantitative model of "data state-business state-resource state" results in insufficient accuracy of state vectors, leading to a low degree of matching between the decision-making layer's output strategy and business needs.

[0006] (iii) Lack of security and resource constraints. There is no dynamic resource verification mechanism (e.g., during peak electricity consumption periods, CPU utilization may exceed 80%, bandwidth may be overloaded, and the service level agreement (SLA) default rate may reach 15%-20%).

[0007] (iv) Low feedback optimization efficiency. The random experience playback mechanism is adopted. In the face of the long feedback cycle of power marketing business, such as the 24-hour feedback of customer churn warning, the reward sparsity problem is caused, the model training efficiency is low, and the workload of manual parameter tuning reaches 40 person-hours / week.

[0008] (v) Inadequate anomaly handling mechanism. There is no backup strategy switching path when the fusion accuracy does not meet the standard, and the business interruption time exceeds 3 minutes, affecting the customer service experience, such as the delay of targeted marketing push.

[0009] Therefore, there is an urgent need for an intelligent fusion technology that takes into account multimodal adaptation, accurate state perception, security resource constraints, and efficient feedback optimization, in order to meet the requirements of power marketing business for data fusion accuracy ≥95%, real-time performance ≤3s, and SLA default rate ≤0.1%. Summary of the Invention

[0010] This invention aims to address five major technical problems in existing multimodal power marketing data fusion technologies: modal heterogeneity conflicts, inaccurate state perception, lack of security resource constraints, low feedback optimization efficiency, and imperfect anomaly handling, thereby achieving efficient, accurate, and secure fusion of multimodal data. To achieve these objectives, an adaptive decision-making mechanism is constructed using reinforcement learning to dynamically optimize the multimodal data fusion strategy. Simultaneously, data quality assessment and business requirement awareness are combined to improve the accuracy and practicality of the fusion results. This invention provides a reinforcement learning-based intelligent fusion system and method for multimodal power marketing data, with the specific technical solution as follows:

[0011] A multimodal electricity marketing data intelligent fusion system based on reinforcement learning includes:

[0012] The multi-source data access layer is used to access structured / semi-structured / unstructured power marketing data and transmit it through a dedicated adaptation channel; it performs missing value imputation, outlier removal, feature extraction and dimensionality reduction on the data to generate standardized feature vectors; and it stores the feature vectors in a cache and dynamically adjusts the time-to-live (TTL).

[0013] The three-dimensional state perception layer is used to depend on the data adaptation, business parsing, and resource monitoring modules. It takes preprocessed features and configuration data as input, and generates a 47-dimensional state vector through AHP weights, normalization, and encoding to solve the problem of inaccurate state perception.

[0014] The intelligent decision-making layer, with the Deep Deterministic Policy Gradient (DDPG) algorithm as its core, includes an Actor network, a Critic network, a Shield module, and an action decoupling module. The Actor network outputs the original action vector A_r, the Shield module verifies the legality of A_r based on preset security rules, and projects illegal actions into legal actions A_f. The Critic network calculates the comprehensive reward based on the state and the final action A_f.

[0015] The fusion execution layer aligns the feature dimensions of each modality, performs feature-level / decision-level fusion according to A_f, compares it with the benchmark sample library, verifies the TOP-K accuracy, outputs the result if it meets the standard, and activates the backup strategy and records the log if it does not meet the standard.

[0016] The feedback optimization layer constructs (S, A, R, S') transfer samples, stores them in the priority experience pool, samples batch data according to TD-error, updates the Actor / Critic network parameters, saves checkpoints periodically, updates the intelligent decision-making layer model, and completes the closed loop.

[0017] Furthermore, the multi-source data access layer includes:

[0018] The data adaptation module configures dedicated channels for different modal data: structured data is accessed through the REST API interface, semi-structured data is accessed through Kafka subscription, and unstructured data is accessed through audio and video decoding.

[0019] The preprocessing module uses a domain model to extract features, reduce dimensionality, standardize, and unify vectors; transforming the raw data into computable vectors; specifically as follows:

[0020] For structured data, the Random Forest prediction algorithm was used to fill in missing values ​​with an accuracy of ≥92%, and the 3σ criterion was used to remove outliers with an anomaly detection rate of ≥95%. For semi-structured data, word segmentation and stop word extraction were used to extract features. For unstructured data, MFCC was used to extract features for speech and ResNet was used to extract features for images / videos.

[0021] Then, dimensionality reduction is performed using PCA component analysis or an autoencoder to output a standardized feature vector.

[0022] The Redis caching module dynamically adjusts the Time-to-Live (TTL) based on data popularity; it adopts a Redis 3-master 3-slave cluster architecture and dynamically adjusts the TTL based on data popularity.

[0023] Furthermore, the ternary state perception layer includes:

[0024] The data state quantification module uses the AHP weighting method to calculate the data quality score, extract statistical features, and generate the data state vector S_d;

[0025] The business state parsing module performs one-hot encoding on the business type, normalizes the real-time and accuracy requirements, generates a modal priority vector, and concatenates it into a business state vector S_b.

[0026] The resource status monitoring module collects CPU / memory / bandwidth utilization, queue length, and node health, and generates a resource status vector S_r.

[0027] It relies on data adaptation, business parsing, and resource monitoring modules, inputs preprocessed features and configuration data, and generates state vectors through AHP weighting, normalization, and encoding to solve the problem of inaccurate state perception; specifically, it includes a data state quantification module, a business state parsing module, and a resource state monitoring module.

[0028] Furthermore, the intelligent decision-making layer includes:

[0029] The core module of DDPG employs a dual Actor / Critic network; the Actor network outputs a hybrid action vector A_f+A_r, and the Critic network calculates the comprehensive reward R.

[0030] The Security Shield module is used to verify the validity of A_r, correct illegal actions by projection, and output legal actions.

[0031] The action decoupling module outputs a hybrid action vector A_f+A_r from the Actor network, calculates the comprehensive reward R from the Critic network, verifies the legality of A_r from the security Shield module, corrects illegal actions by projection, and outputs legal actions.

[0032] Furthermore, the fusion execution layer includes:

[0033] The feature alignment module unifies the semi-structured 256-dimensional and unstructured 512-dimensional features into 512-dimensional features through a fully connected layer, and concatenates them with the structured 512-dimensional features to form a 1280-dimensional vector.

[0034] The multi-strategy fusion module selects a fusion strategy based on A_f: feature-level fusion uses an 8-head attention Transformer to output a 512-dimensional fusion vector, while decision-level fusion uses three single-modal XGBoost models for inference plus Bayesian voting, with voting weight equal to model accuracy.

[0035] The business verification module compares the data with a benchmark sample library and calculates the TOP-K accuracy.

[0036] The exception handling module outputs the fusion result to the business system if the accuracy is ≥95%, and activates the backup strategy and records the failure log if the accuracy is <95%.

[0037] Furthermore, the feedback optimization layer includes:

[0038] The experience storage module is used to construct (S, A, R, S') transition samples, where S is the current state, A is the legal action, R is the comprehensive reward, and S' is the next state; these samples are stored in a priority experience pool, sorted by TD-error, where TD-error = Q_target - Q_eval;

[0039] The priority sampling module is used to sample batch data according to the TD-error weight, with batch_size=256, to improve training efficiency in reward-sparse scenarios.

[0040] The parameter update module is used to trigger updates every 5 minutes. It adopts the Adam gradient descent algorithm optimizer with a learning rate of 1e-4 to update the Actor / Critic network parameters, perform gradient clipping, and set a threshold of 5.0 to avoid gradient explosion.

[0041] The model iteration module saves the checkpoint to the distributed file system HDFS every hour, retains the 10 most recent versions, supports failure rollback, and the rollback time is ≤1 minute.

[0042] A multimodal power marketing data intelligent fusion method based on reinforcement learning includes the following process:

[0043] S100: Accesses structured data via REST API interface, semi-structured data via Kafka subscription, and unstructured data via audio and video decoding;

[0044] The data is imputed for missing values, removed for outliers, and features are extracted and reduced in dimensionality to generate standardized feature vectors. The feature vectors are then stored in a cache and the time-to-live (TTL) is dynamically adjusted.

[0045] S200: The AHP weighting method is used to calculate the data quality score, extract statistical features, and generate the data state vector S_d;

[0046] One-hot encoding is performed on the business type, and the real-time and accuracy requirements are normalized to generate a modal priority vector, which is then concatenated into a business state vector S_b.

[0047] Collect CPU / memory / bandwidth utilization, queue length, and node health. Based on CPU temperature / memory usage / network latency assessment, generate a 9-dimensional resource state vector S_r.

[0048] Output a unified state vector S = S_d⊕S_b⊕S_r, where ⊕ indicates vector concatenation;

[0049] S300: The Actor network outputs a hybrid action vector A_f+A_r, the Critic network calculates the comprehensive reward R, the security Shield module verifies the legality of A_r, illegal actions are projected and corrected, and legal actions are output.

[0050] S400: Align the feature dimensions of each modality, perform feature-level / decision-level fusion according to A_f, compare with the benchmark sample library, verify the TOP-K accuracy, output the result if it meets the standard, and activate the backup strategy and record the log if it does not meet the standard.

[0051] S500: Construct (S, A, R, S') transfer samples, store them in the priority experience pool, sample batch data according to TD-error, update the ctor / Critic network parameters, save checkpoints periodically, update the intelligent decision-making layer model, and complete the closed loop.

[0052] Furthermore, S300 is specifically as follows:

[0053] It adopts a 3-layer fully connected structure, including an input layer, a hidden layer, and an output layer; the output is a mixed action vector.

[0054] A dual-Q network structure is adopted to calculate both immediate and delayed rewards, avoiding overestimation by a single Q network;

[0055] Check if the resource allocation in A_r is less than or equal to the current available resources × 1.2. Illegal actions are projected to the nearest feasible region using Euclidean distance calculation, and then the legal actions are output after correction.

[0056] The hybrid action space is split into A_f and A_r, and transmitted independently to the fusion execution layer.

[0057] Furthermore, S400 is specifically as follows:

[0058] The semi-structured 256-dimensional and unstructured 512-dimensional features are unified into 512-dimensional features through a fully connected layer, and then concatenated with the structured 512-dimensional features to form a 1280-dimensional vector.

[0059] Based on A_f, the fusion strategy is selected as follows: feature-level fusion uses an 8-head attention Transformer to output a 512-dimensional fusion vector; decision-level fusion uses 3 single-modal XGBoost model inference + Bayesian voting, with voting weights equal to model accuracy.

[0060] Compare with a benchmark sample library to calculate TOP-K accuracy;

[0061] If the accuracy is ≥95%, the fusion result is output to the business system; if the accuracy is <95%, the backup strategy is activated, switching from feature level to decision level; a failure log is recorded, including time, action parameters, accuracy, and reason for the exception.

[0062] Furthermore, S500 is specifically as follows:

[0063] Construct (S, A, R, S') transition samples, where S is the current state, A is the legal action, R is the comprehensive reward, and S' is the next state. Store them in a priority experience pool and sort them by TD-error, where TD-error = Q_target - Q_eval.

[0064] Sampling batch data according to TD-error weights, batch_size=256, improves training efficiency in reward sparse scenarios;

[0065] An update is triggered every 5 minutes, using the Adam optimizer with gradient descent algorithm and a learning rate of 1e-4. The Actor / Critic network parameters are updated, and gradient clipping is performed with a threshold of 5.0 to avoid gradient explosion.

[0066] Every hour, a checkpoint is saved to the distributed file system HDFS, retaining the 10 most recent versions. Failure rollback is supported, with a rollback time of ≤1 minute.

[0067] The beneficial effects of this invention are mainly reflected in the following aspects:

[0068] (i) Modal heterogeneity conflict resolution: through dedicated adaptation channels and feature standardization, information entropy loss is reduced by 60%, and the dimensionality of standardized feature vectors reaches 100%.

[0069] (ii) Improved state perception accuracy, with the 47-dimensional ternary state vector accuracy ≥93%, and the matching degree between decision-making strategies and business needs improved to 98%;

[0070] (III) Security and resource optimization: The Security Shield mechanism reduced the SLA default rate to 0, the average CPU utilization rate from 58% to 35%, and the bandwidth utilization rate increased by 40%.

[0071] (iv) Improved feedback efficiency: Prioritized experience playback improves model training efficiency by 50% and reduces manual parameter tuning workload by 80% (from 40 person-hours / week to 8 person-hours / week).

[0072] (v) Business continuity assurance: the abnormal handling mechanism shortens the business interruption time to <10 seconds, the integration accuracy rate is stable at over 96%, the customer churn warning response time is ≤3 seconds, and the accuracy rate of precision marketing is ≥98%. Attached Figure Description

[0073] Figure 1 This is a schematic diagram of the structure of a multimodal power marketing data intelligent fusion system based on reinforcement learning according to the present invention. Detailed Implementation

[0074] A multimodal electricity marketing data intelligent fusion system based on reinforcement learning includes:

[0075] The multi-source data access layer is used to access structured / semi-structured / unstructured power marketing data and transmit it through a dedicated adaptation channel; it performs missing value imputation, outlier removal, feature extraction and dimensionality reduction on the data to generate standardized feature vectors; and it stores the feature vectors in a cache and dynamically adjusts the time-to-live (TTL).

[0076] The three-dimensional state perception layer is used to depend on the data adaptation, business parsing, and resource monitoring modules. It takes preprocessed features and configuration data as input, and generates a 47-dimensional state vector through AHP weights, normalization, and encoding to solve the problem of inaccurate state perception.

[0077] The intelligent decision-making layer, with the Deep Deterministic Policy Gradient (DDPG) algorithm as its core, includes an Actor network, a Critic network, a Shield module, and an action decoupling module. The Actor network outputs the original action vector A_r, the Shield module verifies the legality of A_r based on preset security rules, and projects illegal actions into legal actions A_f. The Critic network calculates the comprehensive reward based on the state and the final action A_f.

[0078] The fusion execution layer aligns the feature dimensions of each modality, performs feature-level / decision-level fusion according to A_f, compares it with the benchmark sample library, verifies the TOP-K accuracy, outputs the result if it meets the standard, and activates the backup strategy and records the log if it does not meet the standard.

[0079] The feedback optimization layer constructs (S, A, R, S') transfer samples, stores them in the priority experience pool, samples batch data according to TD-error, updates the Actor / Critic network parameters, saves checkpoints periodically, updates the intelligent decision-making layer model, and completes the closed loop.

[0080] The multi-source data access layer includes:

[0081] The data adaptation module configures dedicated channels for different modal data: structured data is accessed through the REST API interface, semi-structured data is accessed through Kafka subscription, and unstructured data is accessed through audio and video decoding.

[0082] The preprocessing module uses a domain model to extract features, reduce dimensionality, standardize, and unify vectors; transforming the raw data into computable vectors; specifically as follows:

[0083] For structured data, the Random Forest prediction algorithm was used to fill in missing values ​​with an accuracy of ≥92%, and the 3σ criterion was used to remove outliers with an anomaly detection rate of ≥95%. For semi-structured data, word segmentation and stop word extraction were used to extract features. For unstructured data, MFCC was used to extract features for speech and ResNet was used to extract features for images / videos.

[0084] Then, dimensionality reduction is performed using PCA component analysis or autoencoders to output standardized feature vectors; such as 512-dimensional structured vectors, 64-dimensional semi-structured vectors, and 128 / 256-dimensional unstructured vectors.

[0085] The Redis caching module dynamically adjusts the Time-To-Live (TTL) based on data usage frequency. It adopts a Redis 3-master 3-slave cluster architecture and dynamically adjusts the TTL based on data usage frequency. For example, the TTL is set to 7-30 days, where the frequency is calculated as the number of data calls per week. If the frequency is ≥10 times, the TTL is set to 30 days; if the frequency is <5 times, the TTL is set to 7 days.

[0086] The ternary state perception layer includes:

[0087] The data state quantification module uses the AHP weighting method to calculate the data quality score, extract statistical features, and generate the data state vector S_d;

[0088] The business state parsing module performs one-hot encoding on the business type, normalizes the real-time and accuracy requirements, generates a modal priority vector, and concatenates it into a business state vector S_b.

[0089] The resource status monitoring module collects CPU / memory / bandwidth utilization, queue length, and node health, and generates a resource status vector S_r.

[0090] It relies on data adaptation, business parsing, and resource monitoring modules, inputs preprocessed features and configuration data, and generates state vectors through AHP weighting, normalization, and encoding to solve the problem of inaccurate state perception; specifically, it includes a data state quantification module, a business state parsing module, and a resource state monitoring module.

[0091] The intelligent decision-making layer includes:

[0092] The core module of DDPG employs a dual Actor / Critic network; the Actor network outputs a hybrid action vector A_f+A_r, and the Critic network calculates the comprehensive reward R.

[0093] The Security Shield module is used to verify the validity of A_r, correct illegal actions by projection, and output legal actions.

[0094] The action decoupling module outputs a hybrid action vector A_f+A_r from the Actor network, calculates the comprehensive reward R from the Critic network, verifies the legality of A_r from the security Shield module, corrects illegal actions by projection, and outputs legal actions.

[0095] The fusion execution layer includes:

[0096] The feature alignment module unifies the semi-structured 256-dimensional and unstructured 512-dimensional features into 512-dimensional features through a fully connected layer, and concatenates them with the structured 512-dimensional features to form a 1280-dimensional vector.

[0097] The multi-strategy fusion module selects a fusion strategy based on A_f: feature-level fusion uses an 8-head attention Transformer to output a 512-dimensional fusion vector, while decision-level fusion uses three single-modal XGBoost models for inference plus Bayesian voting, with voting weight equal to model accuracy.

[0098] The business verification module compares the data with a benchmark sample library, which includes 200 labeled data entries covering 5 business scenarios, and calculates the TOP-K accuracy, where K=3.

[0099] The exception handling module outputs the fusion result to the business system if the accuracy is ≥95%, and activates the backup strategy if the accuracy is <95%, and records the failure log, which includes the time, action parameters, accuracy, and reason for the exception.

[0100] The feedback optimization layer includes:

[0101] The experience storage module is used to construct transition samples S, A, R, and S', where S is the current state, A is the legal action, R is the comprehensive reward, and S' is the next state. These samples are stored in a priority experience pool and sorted by TD-error, where TD-error = Q_target - Q_eval. Samples with large TD-error values ​​are often "rare but critical" transition samples, such as sudden disconnection or large overdue payments. Prioritizing these samples can quickly improve the strategy.

[0102] The priority sampling module is used to sample batch data according to the TD-error weight, with batch_size=256, to improve training efficiency in reward-sparse scenarios.

[0103] The parameter update module is used to trigger updates every 5 minutes. It adopts the Adam gradient descent algorithm optimizer with a learning rate of 1e-4 to update the Actor / Critic network parameters, perform gradient clipping, and set a threshold of 5.0 to avoid gradient explosion.

[0104] The model iteration module saves the checkpoint to the distributed file system HDFS every hour, retains the 10 most recent versions, supports failure rollback, and the rollback time is ≤1 minute.

[0105] A multimodal power marketing data intelligent fusion method based on reinforcement learning includes the following process:

[0106] S100: Accesses structured data via REST API interface, semi-structured data via Kafka subscription, and unstructured data via FFmpeg decoding library;

[0107] The data is imputed for missing values, removed for outliers, and features are extracted and reduced in dimensionality to generate standardized feature vectors. The feature vectors are then stored in a cache and their time-to-live (TTL) is dynamically adjusted.

[0108] S200: The AHP weighting method is used to calculate the data quality score and extract 16 statistical features such as data quality, data volume, completeness, accuracy, update frequency, consistency, mean / variance, etc., to generate a 23-dimensional data state vector S_d;

[0109] Five business types, including precision marketing and customer churn warning, are one-hot encoded in 5 dimensions. Real-time and accuracy requirements are normalized to generate a 5-dimensional modal priority vector, which is then concatenated into a 15-dimensional business state vector S_b. The real-time requirement takes a value of 0-1, such as 1 for churn warning and 0.3 for statistical reports. The accuracy requirement takes a value of 0-1, such as 1 for electricity bill calculation and 0.6 for activity recommendation.

[0110] Collect CPU / memory / bandwidth utilization, queue length, and node health. Based on CPU temperature / memory usage / network latency assessment, generate a 9-dimensional resource state vector S_r.

[0111] Output a unified state vector S = S_d⊕S_b⊕S_r, where ⊕ indicates vector concatenation;

[0112] S300: The Actor network outputs a hybrid action vector A_f+A_r, the Critic network calculates the comprehensive reward R, the security Shield module verifies the legality of A_r, illegal actions are projected and corrected, and legal actions are output.

[0113] S400: Align the feature dimensions of each modality, perform feature-level / decision-level fusion according to A_f, compare with the benchmark sample library, verify the TOP-K accuracy, output the result if it meets the standard, and activate the backup strategy and record the log if it does not meet the standard.

[0114] S500: Construct S, A, R, S' transition samples, store them in the priority experience pool, sample batch data according to TD-error, update ctor / Critic network parameters, save checkpoints periodically, update the intelligent decision-making layer model, and complete the closed loop.

[0115] The specific details of S300 are as follows:

[0116] It adopts a 3-layer fully connected structure, including an input layer, a hidden layer, and an output layer; the input layer is 47-dimensional, the hidden layer is 128-dimensional, and the output layer is 18-dimensional, outputting a hybrid action vector, which is fused with the policy subspace A_f∈R^10 + the resource scheduling subspace A_r∈R^8;

[0117] A dual-Q network structure is adopted to calculate immediate and delayed rewards. Immediate rewards refer to the resource consumption optimization rate, while delayed rewards refer to the business KPI achievement rate, thus avoiding overestimation by a single Q network.

[0118] Verify that the resource allocation in A_r is less than or equal to 1.2 of the current available resources, reserve 20% redundancy, and project illegal actions to the nearest feasible region using Euclidean distance calculation, and output legal actions after correction;

[0119] The hybrid action space is split into A_f, which contains the fusion algorithm selection and the number of attention heads, and A_r, which contains the CPU / memory / bandwidth allocation ratio, and is independently transmitted to the fusion execution layer.

[0120] The S400 is specifically as follows:

[0121] The semi-structured 256-dimensional and unstructured 512-dimensional features are unified into 512-dimensional features through a fully connected layer, and then concatenated with the structured 512-dimensional features to form a 1280-dimensional vector.

[0122] Based on A_f, the fusion strategy is selected as follows: feature-level fusion uses an 8-head attention Transformer to output a 512-dimensional fusion vector; decision-level fusion uses 3 single-modal XGBoost model inference + Bayesian voting, with voting weights equal to model accuracy.

[0123] Compared with the benchmark sample library, which includes 200 labeled data points covering 5 business scenarios, the TOP-K accuracy was calculated with K=3.

[0124] If the accuracy is ≥95%, the fusion result is output to the business system; if the accuracy is <95%, the backup strategy is activated, switching from feature level to decision level; a failure log is recorded, including time, action parameters, accuracy, and reason for the exception.

[0125] The S500 is specifically as follows:

[0126] Construct transition samples S, A, R, and S', where S is the current state, A is the legal action, R is the comprehensive reward, and S' is the next state. Store them in a priority experience pool and sort them by TD-error. TD-error = Q_target - Q_eval. Samples with large TD-error are often "rare but critical" transition samples, such as sudden disconnection or large overdue payment. Prioritizing these samples can quickly improve the strategy.

[0127] Sampling batch data according to TD-error weights, batch_size=256, improves training efficiency in reward sparse scenarios;

[0128] An update is triggered every 5 minutes, using the Adam optimizer with gradient descent algorithm and a learning rate of 1e-4. The Actor / Critic network parameters are updated, and gradient clipping is performed with a threshold of 5.0 to avoid gradient explosion.

[0129] Every hour, a checkpoint is saved to the distributed file system HDFS, retaining the 10 most recent versions. Failure rollback is supported, with a rollback time of ≤1 minute.

Claims

1. A multimodal intelligent fusion system for electricity marketing data based on reinforcement learning, characterized in that, include: A multi-source data access layer is used to transmit structured / semi-structured / unstructured electricity marketing data through a dedicated adaptation channel; The data is imputed for missing values, removed for outliers, and subjected to feature extraction and dimensionality reduction to generate standardized feature vectors. Store the feature vectors in a cache and dynamically adjust the Time-To-Live (TTL). The three-element state perception layer is used by the dependency data adaptation, business parsing, and resource monitoring modules. It takes preprocessed features and configuration data as input and generates state vectors through AHP weights, normalization, and encoding to solve the problem of inaccurate state perception. The intelligent decision-making layer, with the Deep Deterministic Policy Gradient (DDPG) algorithm as its core, includes an Actor network, a Critic network, a Shield module, and an action decoupling module. The Actor network outputs the original action vector A_r, the Shield module verifies the legality of A_r based on preset security rules, and projects illegal actions into legal actions A_f. The Critic network calculates the comprehensive reward based on the state and the final action A_f. The fusion execution layer aligns the feature dimensions of each modality, performs feature-level / decision-level fusion according to A_f, compares it with the benchmark sample library, verifies the TOP-K accuracy, outputs the result if it meets the standard, and activates the backup strategy and records the log if it does not meet the standard. The feedback optimization layer constructs S, A, R, and S' transition samples, stores them in the priority experience pool, samples batch data according to TD-error, updates the Actor / Critic network parameters, saves checkpoints periodically, updates the intelligent decision-making layer model, and completes the closed loop.

2. The intelligent fusion system for multimodal electricity marketing data based on reinforcement learning according to claim 1, characterized in that, The multi-source data access layer includes: The data adaptation module configures dedicated channels for different modal data: structured data is accessed through the REST API interface, semi-structured data is accessed through Kafka subscription, and unstructured data is accessed through audio and video decoding. The preprocessing module uses a domain model to extract features, reduce dimensionality, standardize, and unify vectors; transforming the raw data into computable vectors; specifically as follows: For structured data, the Random Forest prediction algorithm was used to fill in missing values ​​with an accuracy of ≥92%, and the 3σ criterion was used to remove outliers with an anomaly detection rate of ≥95%. For semi-structured data, word segmentation and stop word extraction were used to extract features. For unstructured data, MFCC was used to extract features for speech and ResNet was used to extract features for images / videos. Then, dimensionality reduction is performed using PCA component analysis or an autoencoder to output a standardized feature vector. The Redis caching module dynamically adjusts the Time-to-Live (TTL) based on data popularity; it adopts a Redis 3-master 3-slave cluster architecture and dynamically adjusts the TTL based on data popularity.

3. The intelligent fusion system for multimodal electricity marketing data based on reinforcement learning according to claim 1, characterized in that, The ternary state perception layer includes: The data state quantification module uses the AHP weighting method to calculate the data quality score, extract statistical features, and generate the data state vector S_d; The business state parsing module performs one-hot encoding on the business type, normalizes the real-time and accuracy requirements, generates a modal priority vector, and concatenates it into a business state vector S_b. The resource status monitoring module collects CPU / memory / bandwidth utilization, queue length, and node health, and generates a resource status vector S_r. It relies on data adaptation, business parsing, and resource monitoring modules to input preprocessed features and configuration data, and generates state vectors through AHP weighting, normalization, and encoding to solve the problem of inaccurate state perception; specifically, it includes a data state quantization module, a business state parsing module, and a resource state monitoring module.

4. The intelligent fusion system for multimodal electricity marketing data based on reinforcement learning according to claim 1, characterized in that, The intelligent decision-making layer includes: The core module of DDPG employs a dual Actor / Critic network; the Actor network outputs a hybrid action vector (A_f+A_r), and the Critic network calculates the comprehensive reward R. The Security Shield module is used to verify the validity of A_r, correct illegal actions by projection, and output legal actions. The action decoupling module outputs a hybrid action vector (A_f+A_r) from the Actor network, calculates the comprehensive reward R from the Critic network, verifies the legality of A_r from the security Shield module, corrects illegal action projections, and outputs legal actions.

5. The intelligent fusion system for multimodal electricity marketing data based on reinforcement learning according to claim 1, characterized in that, The fusion execution layer includes: The feature alignment module unifies the semi-structured 256-dimensional and unstructured 512-dimensional features into 512-dimensional features through a fully connected layer, and concatenates them with the structured 512-dimensional features to form a 1280-dimensional vector. The multi-strategy fusion module selects a fusion strategy based on A_f. Feature-level fusion uses an 8-head attention Transformer to output a 512-dimensional fusion vector. Decision-level fusion uses three single-modal XGBoost models for inference plus Bayesian voting, with voting weight equal to model accuracy. The business verification module compares the data with a benchmark sample library and calculates the TOP-K accuracy. The exception handling module outputs the fusion result to the business system if the accuracy is ≥95%, and activates the backup strategy and records the failure log if the accuracy is <95%.

6. The intelligent fusion system for multimodal electricity marketing data based on reinforcement learning according to claim 1, characterized in that, The feedback optimization layer includes: The experience storage module is used to construct (S, A, R, S') transition samples, where S is the current state, A is the legal action, R is the comprehensive reward, and S' is the next state; these samples are stored in a priority experience pool, sorted by TD-error, where TD-error = Q_target - Q_eval; The priority sampling module is used to sample batch data according to the TD-error weight, with batch_size=256, to improve training efficiency in reward-sparse scenarios. The parameter update module is used to trigger updates every 5 minutes. It adopts the Adam gradient descent algorithm optimizer with a learning rate of 1e-4 to update the Actor / Critic network parameters, perform gradient clipping, and set a threshold of 5.0 to avoid gradient explosion. The model iteration module saves the checkpoint to the distributed file system HDFS every hour, retains the 10 most recent versions, supports failure rollback, and the rollback time is ≤1 minute.

7. A multimodal intelligent fusion method for electricity marketing data based on reinforcement learning, characterized in that, The process includes the following: S100: Accesses structured data via REST API interface, semi-structured data via Kafka subscription, and unstructured data via audio and video decoding; The data is imputed for missing values, removed for outliers, and subjected to feature extraction and dimensionality reduction to generate standardized feature vectors. Store the feature vectors in a cache and dynamically adjust the Time-To-Live (TTL). S200: The AHP weighting method is used to calculate the data quality score, extract statistical features, and generate the data state vector S_d; One-hot encoding is performed on the business type, and the real-time and accuracy requirements are normalized to generate a modal priority vector, which is then concatenated into a business state vector S_b. Collect CPU / memory / bandwidth utilization, queue length, and node health. Based on CPU temperature / memory usage / network latency assessment, generate a 9-dimensional resource state vector S_r. Output a unified state vector S = S_d⊕S_b⊕S_r, where ⊕ indicates vector concatenation; S300: The Actor network outputs a hybrid action vector A_f+A_r, the Critic network calculates the comprehensive reward R, the security Shield module verifies the legality of A_r, illegal actions are projected and corrected, and legal actions are output. S400: Align the feature dimensions of each modality, perform feature-level / decision-level fusion according to A_f, compare with the benchmark sample library, verify the TOP-K accuracy, output the result if it meets the standard, and activate the backup strategy and record the log if it does not meet the standard. S5 00: Construct (S, A, R, S') transfer samples, store them in the priority experience pool, sample batch data according to TD-error, update the ctor / Critic network parameters, save checkpoints periodically, update the intelligent decision-making layer model, and complete the closed loop.

8. The intelligent fusion method for multimodal electricity marketing data based on reinforcement learning according to claim 7, characterized in that, The specific details of S300 are as follows: It adopts a 3-layer fully connected structure, including an input layer, a hidden layer, and an output layer; the output is a mixed action vector. A dual-Q network structure is adopted to calculate both immediate and delayed rewards, avoiding overestimation by a single Q network; Check if the resource allocation in A_r is less than or equal to the current available resources × 1.

2. Illegal actions are projected to the nearest feasible region using Euclidean distance calculation, and then the legal actions are output after correction. The hybrid action space is split into A_f and A_r, and transmitted independently to the fusion execution layer.

9. The intelligent fusion method for multimodal electricity marketing data based on reinforcement learning according to claim 7, characterized in that, The S400 is specifically as follows: The semi-structured 256-dimensional and unstructured 512-dimensional features are unified into 512-dimensional features through a fully connected layer, and then concatenated with the structured 512-dimensional features to form a 1280-dimensional vector. Based on A_f, the fusion strategy is selected as follows: feature-level fusion uses an 8-head attention Transformer to output a 512-dimensional fusion vector; decision-level fusion uses 3 single-modal XGBoost model inference + Bayesian voting, with voting weights equal to model accuracy. Compare with a benchmark sample library to calculate TOP-K accuracy; If the accuracy is ≥95%, the fusion result is output to the business system; if the accuracy is <95%, the backup strategy is activated, switching from feature level to decision level; a failure log is recorded, including time, action parameters, accuracy, and reason for the exception.

10. The intelligent fusion method for multimodal electricity marketing data based on reinforcement learning according to claim 7, characterized in that, The S500 is specifically as follows: Construct (S, A, R, S') transition samples, where S is the current state, A is the legal action, R is the comprehensive reward, and S' is the next state. Store them in a priority experience pool and sort them by TD-error, where TD-error = Q_target - Q_eval. Sampling batch data according to TD-error weights, batch_size=256, improves training efficiency in reward sparse scenarios; An update is triggered every 5 minutes, using the Adam optimizer with gradient descent algorithm and a learning rate of 1e-4. The Actor / Critic network parameters are updated, and gradient clipping is performed with a threshold of 5.0 to avoid gradient explosion. Every hour, a checkpoint is saved to the distributed file system HDFS, retaining the 10 most recent versions. Failure rollback is supported, with a rollback time of ≤1 minute.

Images