Electric power marketing business abnormity real-time detection method and system based on stream-oriented computing

Abstract

The invention relates to an electric power marketing business abnormity real-time detection method and system based on stream-oriented computation, and belongs to the technical field of electric power system optimizing.The method comprises the steps that data snapshots are extracted from an electric power marketing business system, difference comparison is conducted on the data snapshots and historical snapshots of an intermediate library, and standardized increment events are generated and stored; capturing an incremental event in real time through a data change capturing tool and pushing the incremental event to a message queue; a streaming computation engine consumes the event stream, sequentially performs data cleaning, association with a static dimension table and sliding window statistical feature calculation, and constructs a feature vector; and performing parallel analysis and weighted fusion on the feature vectors based on a business rule base and an online machine learning model to generate a comprehensive risk score, and outputting an abnormal event when the score exceeds a threshold value. According to the method, the problems of exception identification lagging and complex work order process in a traditional batch processing mode are solved, the crossing of the business risk from hour-level detection to minute-level real-time perception is realized, and the timeliness and accuracy of power marketing risk management and control are improved.

Description

Technical Field

[0001] This application relates to the field of power system optimization technology, and more specifically, to a method and system for real-time detection of anomalies in power marketing operations based on streaming computing. Background Technology

[0002] With the deepening of power market reform and the popularization of distributed energy, power marketing operations are characterized by a surge in data volume and complex, ever-changing scenarios. Traditional anomaly detection methods mainly rely on timed batch processing (T+1 mode) and manual rule screening, which have significant drawbacks: First, anomaly identification is severely lagging, making it difficult to cope with sudden risks such as billing errors, electricity theft, and parameter anomalies, and failing to meet the need for early warning; second, the detection accuracy is insufficient, static rules cannot cover increasingly complex anomaly patterns, and model updates are slow, unable to adapt to the "conceptual drift" of business data; finally, the system consumes a lot of resources and is inefficient, with full-data batch processing putting enormous pressure on computing and storage resources and making it difficult to scale. Although some streaming computing applications exist in existing technologies, most lack end-to-end optimization solutions tailored to the characteristics of power marketing operations, especially in areas such as real-time fusion of multi-source heterogeneous data, feature engineering, and hybrid judgment models. Therefore, the industry urgently needs a real-time detection method that is highly efficient, accurate, and low-resource-consumption to achieve intelligent and automated management and control of power marketing risks. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes a method and system for real-time detection of anomalies in power marketing operations based on streaming computing.

[0004] The technical solution of this invention is as follows: This invention proposes a real-time anomaly detection method for electricity marketing operations based on streaming computing, comprising the following steps: Data snapshots are extracted from the power marketing business system and compared with historical snapshots in the intermediate database to identify data change events and store them in the intermediate database. The data change capture tool captures incremental change events in the intermediate library in real time, converts them into standard message format, and pushes them to the message queue. The streaming computing engine continuously receives event streams from the message queue, performs data cleaning, associates them with static dimension tables to enrich contextual information, and performs statistical feature calculations within a sliding time window to generate feature vectors for anomaly detection. Based on a pre-configured business rule base and an online machine learning model, feature vectors are analyzed in parallel. The rule judgment results are weighted and fused with the model prediction probability to generate a comprehensive risk score. When the score exceeds a preset threshold, an abnormal change event is output.

[0005] Preferably, the difference comparison is implemented based on set operations, and the difference recognition formula is: ; In the formula: Let be the set of incremental differences at time t; Let t be the set of data snapshots extracted from the power marketing business system; This represents the set of historical snapshots in the intermediate database at time t. This is the difference operator.

[0006] Preferably, the data change capture tool parses the database's redo logs or binary logs to capture the atomicity and sequentiality of incremental change events; the standard message format for incremental change event conversion includes the primary key, operation type, data snapshots before and after the change, and database commit timestamp.

[0007] Preferably, the association with the static dimension table is achieved through the asynchronous I / O interface of the streaming computing engine or dimension table caching technology. The static dimension table at least includes peak and valley flags, display type, and electricity price category fields for business determination.

[0008] Preferably, the online machine learning model employs online logistic regression or an algorithm based on incremental gradient descent, and its model parameters support online updates using labeled streaming data to adapt to conceptual shifts in business data.

[0009] Preferably, the method further includes a deduplication and suppression step: using BloomFilter and / or Count-MinSketch data structure to quickly deduplicat high-frequency repetitive abnormal change events, and based on KeyedState to record the most recent trigger time and frequency, suppressing repeated triggering within a short period of time.

[0010] Preferably, the method further includes a streaming result write-back and optimization step, specifically: persisting the abnormal change event, the corresponding feature vector, and the fusion score to the business database in an idempotent writing manner; at the same time, storing historical abnormal events in cold, warm, and hot data storage areas, and providing data interfaces to support false alarm / missed alarm analysis and model threshold tuning, so as to achieve closed-loop optimization of the system.

[0011] On the other hand, the present invention also provides a real-time anomaly detection system for electricity marketing operations based on streaming computing, including: The data acquisition module extracts data snapshots from the power marketing business system, compares them with historical snapshots in the intermediate database, identifies data change events, and stores them in the intermediate database. The incremental capture and push module uses a data change capture tool to capture incremental change events in the intermediate database in real time, convert them into standard message format, and push them to the message queue. The feature vector construction module continuously receives event streams from the message queue, performs data cleaning, associates them with static dimension tables to enrich contextual information, and performs statistical feature calculations within a sliding time window to generate feature vectors for anomaly detection. The anomaly detection module, based on a pre-configured business rule base and an online machine learning model, performs parallel analysis on feature vectors, weights and fuses the rule judgment results with the model prediction probability to generate a comprehensive risk score, and outputs an abnormal change event when the score exceeds a preset threshold.

[0012] In another aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the real-time detection method for anomalies in power marketing business based on streaming computing as described in any embodiment of the present invention.

[0013] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the real-time detection method for anomalies in power marketing business based on streaming computing as described in any embodiment of the present invention.

[0014] The present invention has the following beneficial effects: By using a complete streaming pipeline of "incremental capture at the source → real-time message distribution → streaming feature construction → online fusion judgment", the risk detection latency is reduced from hours to seconds. Furthermore, by utilizing a hybrid judgment mechanism that combines rules and online learning models, the accuracy and recall rate of anomaly identification are significantly improved.

[0015] Online machine learning models can continuously and incrementally update using streaming feedback data, enabling the system to resist concept drift and maintain long-term detection sensitivity; combined with A / B testing and closed-loop optimization mechanisms, the detection model can be self-evolved and optimized.

[0016] By employing incremental data processing, Kafka partitioning parallelism, Flink state backend optimization, and multi-level deduplication suppression, elastic scheduling and simplified management of high-concurrency data streams are achieved. While ensuring low latency, the computational and storage resource overhead of the system is significantly reduced, and it has good scalability. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0020] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0022] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0023] Example 1: To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present application and with reference to the accompanying drawings.

[0024] To address the problems in existing technologies, this invention provides a real-time anomaly detection method for electricity marketing operations based on streaming computing, comprising the following steps: Data snapshots are extracted from the power marketing business system and compared with historical snapshots in the intermediate database to identify data change events and store them in the intermediate database. In this embodiment, taking the abnormal peak-valley meter reading rate verification business of low-voltage residential users as an example, this type of abnormality usually manifests as a logical inconsistency between user category, electricity price category, and meter reading structure, such as low-voltage residential users not configuring peak-valley meter readings but being charged residential peak-valley electricity prices.

[0025] For this business scenario, a Python script is used as the ETL driver to execute two types of tasks according to a preset strategy: scheduled batch extraction and incremental snapshot extraction. First, the connection management of the power marketing business systems is implemented. The Python script connects to each power marketing business system via JDBC / ODBC or API calls, collecting data snapshots from specified business tables or interfaces. The business tables include at least: a user profile basic information table, a metering point information table, an operating energy meter information table, an electricity price execution information table, and a display parameter configuration table. Then, based on the theme condition of "low-voltage residential users," the above business tables are jointly filtered, and the filtered theme data is uniformly output to a temporary business table TEMP, which is used to carry a snapshot of the business data under this abnormal theme at the current moment. Next, a snapshot comparison is performed between the intermediate database and the newly sampled data in the TEMP table, comparing it with the historical snapshots in the self-built Oracle intermediate database. The system performs structured comparisons to identify three types of changes: insertion, update, and deletion. Further, it unifies the structure and maps fields. Before writing to the intermediate database, the script performs encoding mapping, default null value filling, and basic format validation on the fields to ensure data consistency in the intermediate database. Finally, it generates change records, producing standardized change events for each discrepancy record. The change table is written to the intermediate database and configured as the source table for OGG collection.

[0026] The formula for difference recognition is as follows: ; In the formula: Let be the set of incremental differences at time t; Let t be the set of data snapshots extracted from the power marketing business system; This represents the set of historical snapshots in the intermediate database at time t. This is a difference operator that calculates the insert / update / delete set based on primary key and field comparisons.

[0027] This step aims to extract heterogeneous data scattered across multiple power marketing business systems into a self-built intermediate library in a controllable and auditable manner, laying the foundation for subsequent real-time incremental processing and streaming computing.

[0028] The data change capture tool captures incremental change events in the intermediate library in real time, converts them into standard message format, and pushes them to the message queue. In this embodiment, Oracle GoldenGate (OGG) is used to monitor the intermediate database result table in real time. When key fields such as low-voltage residential user peak-valley meter readings, electricity price categories, or metering point attributes are added or changed and written to the result table, OGG captures the corresponding transaction logs based on Oracle's redo / log mechanism and converts the incremental changes at the database level into event messages, which are then written to a Kafka topic, realizing the transformation from static topic data changes to a real-time message stream. The standard format for OGG output events is typically as follows: ; In the formula, Output events for OGG; To record the primary key; For the operation types: insert, update, delete; This is a snapshot of the fields before the change. This is a snapshot of the changed fields; Submit timestamps to the database.

[0029] The process from OGG to Kafka includes: event formatting, topic routing (topics are divided by business domain), serialization, and idempotent writing. In practice, to ensure time-series consistency and allow for the processing of late data, messages are appended with an event timestamp (ts) and the message header supports carrying the source transaction identifier.

[0030] At the QoS level, the OGG→Kafka framework needs to pay attention to: network jitter retry strategy, message retention period, topic partitioning strategy, and consumer offset commit strategy. Furthermore, if there are batch import scenarios at the source end, OGG should coordinate batch identifiers with the intermediate library so that consumers can identify batch events and trigger micro-batch processing.

[0031] The streaming computing engine continuously receives event streams from the message queue, performs data cleaning, associates them with static dimension tables to enrich contextual information, and performs statistical feature calculations within a sliding time window to generate feature vectors for anomaly detection. After Flink consumes the incremental event stream corresponding to the topic of abnormal peak and valley reading rates for low-voltage residential users from Kafka, it performs real-time ETL, dimension table enhancement, and window-based statistical feature construction within the stream processing topology to form a high-dimensional time-series feature vector for anomaly identification. This processing step consists of three main sub-stages: stream cleaning, dimension table (archive) join, and sliding window feature aggregation and concatenation. Details are as follows: Flink streaming cleansing involves standardizing event execution fields, parsing timestamps (event time vs. processing time), filling in null values, and removing invalid values. Real-time exception removal can utilize the Z-score method to determine dirty values ​​for numeric fields. ; In the formula, For the current observation value The standardized distance from the normal fluctuation range of historical data; This is the current observation value; , These are the mean and standard deviation calculated from the sliding window or historical statistics, respectively; k is the threshold, and values ​​exceeding this threshold are marked as instantaneous dirty values ​​or noise.

[0032] Static dimension table joins rely heavily on static / semi-static metadata such as meter profiles, customer levels, and meter models. Flink uses asynchronous I / O (designed to avoid streaming processing blocking caused by synchronous database queries) or dimension table caching (maintaining a copy of the dimension table in memory through periodic refresh or a TTL-based strategy to achieve higher read performance) to perform real-time joins with Oracle's profile tables. ; In the formula: For enhanced events, including the original event. All fields, and added from the static dimension table The dimension fields that are linked from the middle; The static dimension table contains key fields such as peak / valley indicators, reading type, and electricity price category; This is an associative operator.

[0033] Sliding window aggregation and feature stitching are used for time-series quantities (such as user electricity consumption sequences, voltage, instantaneous active power, etc.). Calculate the statistic: mean ,variance Maximum / Minimum, Rate of Change, etc. ; ; In the formula, n is the number of valid samples within the window; This is the window length.

[0034] Based on statistical measures, derived features such as Z-score and peak-to-valley ratio are constructed and concatenated with static fields to form a high-dimensional feature vector. ; In the formula, each This represents a specific statistical or business characteristic (such as peak / valley indicators, indicator type encoding, historical anomaly counts, real-time power factor, billing cycle differentials, etc.). Flink recommends storing these vectors in KeyedState (by meter point or user keying) and using RocksDB for local persistence to ensure fault tolerance and recovery capabilities.

[0035] Based on a pre-configured business rule base and an online machine learning model, feature vectors are analyzed in parallel. The rule judgment results are weighted and fused with the model prediction probability to generate a comprehensive risk score. When the score exceeds a preset threshold, an abnormal change event is output.

[0036] In this embodiment, by executing the "rule determination + online model prediction + fusion scoring" anomaly identification chain in parallel on the Flink real-time stream, minute-level judgment of archive-type and behavior-type anomalies is achieved. The specific steps include: Rule determination is based on the business rule base. Perform hard or soft matching. Example rule: If the user's electricity price category is residential, the peak / valley indicator is not specified, and the display type does not include peak / valley, then the rule outputs a high-confidence anomaly. The rules support weighted output. To express uncertainty.

[0037] Online model prediction, using online logistic regression or incremental gradient descent algorithms. Output anomaly probability: ; in, This refers to the real-time prediction probability of whether a current business data change event is abnormal at time t for an online machine learning model. For model mapping functions; For model weight parameters, online updates using labeled streaming data are supported to adapt to conceptual shifts in business data. The specific implementation is as follows: (1) Example of online logistic regression: using stochastic gradient descent (SGD) as the optimizer. When a new labeled sample... Upon arrival, ( (For anomaly labels), the model parameters are updated according to the following rules: ; In the formula: The learning rate; This is the transpose of the model parameter vector at time t; This is the model parameter vector at time t+1; For the Sigmoid function; (2) Example of an algorithm based on incremental gradient descent: Taking Incremental Support Vector Machine (SVM) as an example. This algorithm maintains a set of support vectors in a streaming environment. For a new sample, it first determines whether it satisfies the KKT conditions. If not, the new sample is added to the support vector set, and the Sequence Minimum Optimization (SMO) algorithm is used to incrementally update the Lagrange multipliers corresponding to the support vectors involved, thereby adjusting the classification hyperplane. This method is suitable for nonlinear anomaly patterns in electricity marketing data.

[0038] Model training uses labeled historical samples (generated by manual annotation or historical work order results) and online feedback data streams. Through the above incremental learning mechanism, continuous optimization is carried out to ensure that the model can quickly adapt to the conceptual drift of business data.

[0039] The final score is generated by fusing the scoring and threshold judgment, and combining the rules and model results according to weights. ; in, and To integrate weights, the following conditions are often met: It can also be dynamically adjusted according to business confidence level; To determine the overall risk score; set As the trigger threshold, if Then the abnormal event stream will be output.

[0040] For state management and deduplication, in high-frequency, repetitive triggering scenarios, KeyedState is used to record the most recent trigger time. With the number of triggers And a suppression strategy is employed to reduce noise. If necessary, Count-MinSketch or Bloom filter can be used for fast deduplication; where: Suppression strategy based on KeyedState: Using the user or meter as the primary key, maintain a ValueState in Flink's KeyedState. <Tuple2<Long, Integer> >Status, used to record the most recent trigger time Compared to the recent trigger count c. When a new abnormal event occurs, check the current time. and The difference Δt. If Δt < If a time window is set to suppress, such as 5 minutes, then the output of the current event is suppressed and the count c is incremented by 1; otherwise, the output is normal and updated. Reset the counter c to the current time.

[0041] Fast deduplication based on Bloom Filter: For identical anomalies within a short period (e.g., the same change event being captured multiple times due to network jitter), a global Bloom Filter is initialized in the Flink job. <string>When an event flows in, a unique key is generated based on its primary key and time window identifier. First, the Bloom Filter is queried. If the return value is false, it indicates that the event must not have occurred recently, so it is allowed to proceed with subsequent processing, and the key is added to the Bloom Filter.

[0042] If the return value is true, a second judgment is made based on the KeyedState: if Δt is extremely small (e.g., less than 1 second), it is determined to be a duplicate event and is filtered out.

[0043] This two-stage deduplication mechanism of "BloomFilter coarse screening + KeyedState fine judgment" can efficiently filter duplicate events in massive amounts of data while ensuring an extremely low false positive rate.

[0044] As a preferred embodiment of this example, the method further includes a streaming result write-back and optimization step, specifically: persisting the abnormal change event, the corresponding feature vector, and the fusion score to the business database in an idempotent writing manner; at the same time, storing historical abnormal events in cold, warm, and hot data storage areas, and providing data interfaces to support false alarm / missed alarm analysis and model threshold tuning, so as to achieve closed-loop optimization of the system.

[0045] Flink writes the identified results, along with relevant features and evidence, into the system's business database, and also writes them to cold / warm storage as needed to support offline analysis. A structured example of the written records is as follows: ; In the formula, Used as a unique identifier for the event; Identify users or metering points; A slice of the feature vector at the time of triggering; For integration scoring; Risk level; To trigger evidence; This is the event timestamp.

[0046] To ensure consistency, an idempotent write strategy is used, and asynchronous retries and dead-letter queue handling are implemented for write failures. This step also provides ad-hoc query and batch export interfaces, supporting model false positive / false negative analysis, threshold sensitivity testing, and A / B testing. Closed-loop optimization of the system is achieved through periodic or triggered micro-batch training and rule adjustments.

[0047] Example 2: This embodiment provides a real-time anomaly detection system for electricity marketing operations based on streaming computing, including: The data acquisition module extracts data snapshots from the power marketing business system, compares them with historical snapshots in the intermediate database, identifies data change events, and stores them in the intermediate database. The incremental capture and push module uses a data change capture tool to capture incremental change events in the intermediate database in real time, convert them into standard message format, and push them to the message queue. The feature vector construction module continuously receives event streams from the message queue, performs data cleaning, associates them with static dimension tables to enrich contextual information, and performs statistical feature calculations within a sliding time window to generate feature vectors for anomaly detection. The anomaly detection module, based on a pre-configured business rule base and an online machine learning model, performs parallel analysis on feature vectors, weights and fuses the rule judgment results with the model prediction probability to generate a comprehensive risk score, and outputs an abnormal change event when the score exceeds a preset threshold.

[0048] Example 3: This embodiment proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the real-time detection method for anomalies in power marketing business based on streaming computing as described in any embodiment of the present invention.

[0049] Example 4: This embodiment proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the real-time detection method for anomalies in power marketing business based on streaming computing as described in any embodiment of the present invention.

[0050] In this embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0051] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0052] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0053] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0054] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.< / string>

Claims

1. A method for real-time anomaly detection in electricity marketing operations based on streaming computing, characterized in that, Includes the following steps: Data snapshots are extracted from the power marketing business system and compared with historical snapshots in the intermediate database to identify data change events and store them in the intermediate database. The data change capture tool captures incremental change events in the intermediate library in real time, converts them into standard message format, and pushes them to the message queue. The streaming computing engine continuously receives event streams from the message queue, performs data cleaning, associates them with static dimension tables to enrich contextual information, and performs statistical feature calculations within a sliding time window to generate feature vectors for anomaly detection. Based on a pre-configured business rule base and an online machine learning model, feature vectors are analyzed in parallel. The rule judgment results are weighted and fused with the model prediction probability to generate a comprehensive risk score. When the score exceeds a preset threshold, an abnormal change event is output.

2. The real-time anomaly detection method for electricity marketing operations based on streaming computing as described in claim 1, characterized in that: The difference comparison is implemented based on set operations, and the difference recognition formula is as follows: ； In the formula: Let be the set of incremental differences at time t; Let t be the set of data snapshots extracted from the power marketing business system; This represents the set of historical snapshots in the intermediate database at time t. This is the difference operator.

3. The real-time anomaly detection method for electricity marketing operations based on streaming computing as described in claim 1, characterized in that: The data change capture tool parses the database's redo logs or binary logs to capture the atomicity and sequentiality of incremental change events; The standard message format for incremental change event transformation includes the primary key, operation type, data snapshots before and after the change, and database commit timestamp.

4. The real-time anomaly detection method for electricity marketing operations based on streaming computing as described in claim 1, characterized in that: The association with the static dimension table is achieved through the asynchronous I / O interface of the streaming computing engine or dimension table caching technology. The static dimension table at least includes peak and valley flags, display type, and electricity price category fields for business determination.

5. The real-time anomaly detection method for electricity marketing operations based on streaming computing according to claim 1, characterized in that: The online machine learning model employs online logistic regression or incremental gradient descent algorithms, and its model parameters can be updated online using labeled streaming data to adapt to the conceptual drift of business data.

6. The real-time anomaly detection method for electricity marketing operations based on streaming computing according to claim 1, characterized in that: The method also includes deduplication and suppression steps: using BloomFilter and / or Count-Min Sketch data structures to quickly deduplicat high-frequency repeated abnormal change events, and using KeyedState to record the most recent trigger time and frequency to suppress repeated triggering within a short period of time.

7. The real-time anomaly detection method for electricity marketing business based on streaming computing according to claim 1, characterized in that: The method also includes a streaming result write-back and optimization step, specifically: persisting abnormal change events, corresponding feature vectors, and fusion scores to the business database in an idempotent write manner; at the same time, storing historical abnormal events in cold, warm, and hot data storage areas, and providing data interfaces to support false alarm / missed alarm analysis and model threshold tuning, so as to achieve closed-loop optimization of the system.

8. A real-time anomaly detection system for electricity marketing operations based on streaming computing, characterized in that, include: The data acquisition module extracts data snapshots from the power marketing business system, compares them with historical snapshots in the intermediate database, identifies data change events, and stores them in the intermediate database. The incremental capture and push module uses a data change capture tool to capture incremental change events in the intermediate database in real time, convert them into standard message format, and push them to the message queue. The feature vector construction module continuously receives event streams from the message queue, performs data cleaning, associates them with static dimension tables to enrich contextual information, and performs statistical feature calculations within a sliding time window to generate feature vectors for anomaly detection. The anomaly detection module, based on a pre-configured business rule base and an online machine learning model, performs parallel analysis on feature vectors, weights and fuses the rule judgment results with the model prediction probability to generate a comprehensive risk score, and outputs an abnormal change event when the score exceeds a preset threshold.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the real-time detection method for anomalies in power marketing business based on streaming computing as described in claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the real-time detection method for anomalies in power marketing business based on streaming computing as described in claims 1-7.

Images