Intelligent closed loop based power transaction data collection and re-collection system and method

Abstract

This disclosure relates to the field of power automation technology, specifically to a power trading data acquisition and supplementation system and method based on intelligent closed-loop. The system includes: a multi-terminal collaborative acquisition module for maintaining multiple independent user sessions and assigning an independent browser execution context to each user session; a data integrity check module for performing dual checks to identify missing data and generate abnormal records; and an intelligent supplementation control module for implementing a hierarchical supplementation strategy for abnormal records. By setting up the multi-terminal collaborative acquisition module, this disclosure enables the system to assign an independent browser execution context to each user session, achieving isolated execution of multi-user acquisition tasks at the resource and state levels. This not only eliminates the strong dependence on dedicated security hardware plug-and-play and intranet environments but also removes the performance bottleneck caused by single-point sequential acquisition, thereby significantly improving the system's deployment flexibility and concurrent acquisition efficiency.

Description

Technical Field

[0001] This invention relates to the field of power automation technology, specifically to a power trading data acquisition and supplementation system and method based on intelligent closed-loop. Background Technology

[0002] With the increasing development of the electricity spot market and trading mechanisms, the real-time collection and integrity assurance of electricity trading data are becoming increasingly important for market monitoring.

[0003] However, in relevant data acquisition solutions, the contradiction between rigid acquisition architecture and lack of data quality assurance has long existed, making it difficult for the system to ensure both high timeliness of power trading data acquisition and data integrity, as well as the security and compliance of the target platform. Therefore, how to build a multi-terminal collaborative acquisition system with intelligent closed-loop capabilities, which can reduce strong hardware dependence while achieving fine-grained completion of heterogeneous data, has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the present invention provides a power trading data acquisition and supplementation system and method based on intelligent closed loop, in order to solve how to build an acquisition system with multi-terminal collaboration and intelligent closed loop capability, so as to achieve fine-grained supplementation of heterogeneous data while eliminating strong hardware dependence.

[0005] This disclosure provides a power trading data acquisition and supplementation system based on intelligent closed-loop, the system comprising:

[0006] The multi-terminal collaborative data acquisition module is used to maintain multiple independent user sessions and allocate an independent browser execution context to each user session, and concurrently execute the data acquisition task of the target transaction data through the independent browser execution context; The data integrity check module is used to make a dual judgment based on the response status code of the acquisition task and the non-empty status of the business data content, identify missing data and generate anomaly records; The intelligent data acquisition control module is used to implement a tiered data acquisition strategy for abnormal records. The tiered data acquisition strategy is configured to: push the acquisition task into the data acquisition queue without immediately retrying in response to acquisition failure or missing data detected during inspection; scan the data acquisition queue through a timed scheduling mechanism and perform differentiated data acquisition based on data attributes, including: automatically calling the acquisition interface to perform data acquisition for public data; and simulating user operations to perform data acquisition for private data after verifying the security authentication medium status.

[0007] This disclosure also provides a method for collecting and supplementing electricity trading data based on a smart closed loop, the method including: Maintain multiple independent user sessions and assign an independent browser execution context to each user session, and concurrently execute the target transaction data collection task through the independent browser execution context; The system uses a dual approach, combining the response status code of the data collection task with the non-empty status of the business data content, to identify missing data and generate anomaly records. A tiered supplementary data collection strategy is implemented for abnormal records. The tiered supplementary data collection strategy includes: in response to data collection failure or missing data detected during inspection, pushing the data collection task into the supplementary data collection queue without immediately retrying; scanning the supplementary data collection queue through a timed scheduling mechanism and performing differentiated supplementary data collection based on data attributes, including: for public data, automatically calling the data collection interface to perform supplementary data collection; for private data, simulating user operations to perform supplementary data collection after verifying the status of the security authentication medium.

[0008] This disclosure also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the above-described intelligent closed-loop-based power trading data acquisition and supplementary acquisition method.

[0009] This disclosure also provides a computer-readable storage medium storing computer instructions for enabling a computer to implement the aforementioned intelligent closed-loop-based power trading data acquisition and supplementary acquisition method.

[0010] This disclosure also provides a computer program product, including computer instructions for causing a computer to execute the above-described intelligent closed-loop-based power trading data acquisition and supplementary acquisition method.

[0011] The intelligent closed-loop-based power trading data acquisition and supplementary acquisition system and method of the above embodiments of this disclosure, by setting up a multi-terminal collaborative acquisition module, enables the system to allocate an independent browser execution context for each user session, realizing the isolated execution of multi-user acquisition tasks at the resource and state levels. This not only eliminates the strong dependence on the plugging and unplugging of dedicated security hardware and the intranet environment, but also removes the performance blocking caused by single-point sequential acquisition, thereby significantly improving the system's deployment flexibility and concurrent acquisition efficiency. By setting up an intelligent supplementary acquisition control module and adopting an asynchronous scheduling strategy of "failure enqueueing," the system avoids instantaneous high-frequency retries caused by network fluctuations, effectively reducing the access pressure on the target trading platform. Combined with a layered supplementary acquisition strategy for public / private data, the system ensures both automation and compliance of business operations, effectively avoiding the risk of triggering anti-crawler mechanisms and improving the stability and survival rate of acquisition operations.

[0012] In addition, by setting up a data integrity check module, the system implements a dual judgment on the collection results based on the response status code and whether the business content is not empty. This can accurately identify hidden deficiencies where the network is connected but the data is empty, ensuring the integrity of the final data stored in the database. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 An exemplary schematic diagram of the architecture of a power trading data acquisition and supplementary acquisition system based on an intelligent closed loop according to an embodiment of this disclosure is shown; Figure 2 An exemplary schematic diagram of another architecture of the power trading data acquisition and supplementary acquisition system based on intelligent closed loop according to an embodiment of the present disclosure is shown; Figure 3 An exemplary schematic diagram of another architecture of the power trading data acquisition and supplementary acquisition system based on intelligent closed loop according to an embodiment of the present disclosure is shown; Figure 4 A flowchart illustrating the power trading data acquisition and supplementation method based on intelligent closed-loop provided in this embodiment of the present disclosure is shown. Figure 5 The flowchart of the real-time data acquisition process based on the intelligent closed-loop power trading data acquisition and supplementary acquisition method provided in this embodiment is shown. Figure 6 A flowchart illustrating the intelligent supplementary data collection mechanism of the power trading data collection and supplementary data collection method based on intelligent closed loop provided in this disclosure embodiment is shown. Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0015] Against the backdrop of the deepening of the electricity spot market and medium- and long-term trading mechanisms, the real-time performance, completeness, and accuracy of electricity trading data have become core elements for ensuring market monitoring, historical review, and decision support. With the increasing frequency of transactions and the expansion of data dimensions, the system places more stringent demands on the timeliness and coverage of data collection. To acquire data scattered across various trading platforms, traditional solutions mainly rely on manual operation or single-point automated programs. Manual solutions depend on operators manually logging into the platform to query and record data; while single-point acquisition solutions typically use server-side programs to periodically access interfaces or simulate requests to obtain data, attempting to reduce labor costs and improve data acquisition efficiency through automation.

[0016] However, the aforementioned related technologies often have the following problems: 1. The data collection architecture of related technologies is rigid and highly dependent on the operating environment. Existing automated data collection solutions mostly adopt a centralized architecture, with all requests issued through a single service node. This not only easily creates performance bottlenecks but also makes it difficult to flexibly cope with the complex security authentication mechanisms of power trading platforms. For example, when dealing with private data that requires dedicated hardware (such as UKey) or access to specific intranet environments, general-purpose crawler programs are often incompatible, requiring manual operation. In addition, the lack of differentiated data collection frequency control strategies makes the system highly susceptible to triggering the target platform's anti-crawler mechanisms due to excessive access pressure when facing high-frequency data requests, resulting in IP blocking or service interruption.

[0017] 2. The relevant technologies lack data integrity verification and closed-loop recovery mechanisms, resulting in high operation and maintenance costs. Existing solutions typically determine the success of data collection solely based on network connectivity, failing to identify "hidden missing" phenomena where the network is connected but business data is empty. When data omissions or anomalies occur, the lack of automated missing data identification and supplementary data queue management prevents the system from automatically initiating retries or remediation, leading to irreversible gaps in historical data. This forces operations and maintenance personnel to frequently intervene manually to troubleshoot and supplement data, failing to achieve an automated closed loop from fault discovery to fault recovery, severely limiting the system's business continuity and data quality assurance capabilities.

[0018] To address the aforementioned issues, various embodiments of this disclosure provide a power transaction data acquisition and supplementary acquisition system based on an intelligent closed loop. The system includes: a multi-terminal collaborative acquisition module, used to maintain multiple independent user sessions and allocate an independent browser execution context to each user session, concurrently executing the acquisition task of the target transaction data through the independent browser execution context; a data integrity check module, used to perform dual judgment based on the response status code of the acquisition task and the non-empty status of the business data content, identifying missing data and generating abnormal records; and an intelligent supplementary acquisition control module, used to execute a hierarchical supplementary acquisition strategy for abnormal records. The hierarchical supplementary acquisition strategy is configured to: in response to acquisition failure or the discovery of missing data, push the acquisition task into a supplementary acquisition queue without immediately retrying; scan the supplementary acquisition queue through a timed scheduling mechanism and perform differentiated supplementary acquisition based on data attributes, including: for public data, automatically calling the acquisition interface to perform supplementary acquisition; for private data, simulating user operations to perform supplementary acquisition after verifying the security authentication medium status.

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0020] Please refer to Figure 1 , Figure 1 An exemplary schematic diagram of the architecture of a power trading data acquisition and supplementary acquisition system based on an intelligent closed loop, according to an embodiment of this disclosure, is shown. Figure 1 As shown, the intelligent closed-loop power trading data acquisition and supplementary acquisition system 100 (hereinafter referred to as system 100) includes: The multi-terminal collaborative acquisition module 101 is used to maintain multiple independent user sessions and assign an independent browser execution context to each user session, and concurrently execute the acquisition task of the target transaction data through the independent browser execution context.

[0021] In this embodiment, the multi-terminal collaborative acquisition module 101 can be understood as a logical unit or processing engine responsible for managing the allocation of acquisition terminal resources and the distribution of tasks. This module can adopt a lightweight server-side design, with the server only responsible for task scheduling and result aggregation, thereby avoiding the pressure bottleneck of centralized acquisition.

[0022] A user session refers to a continuous interactive connection established between a specific trading entity and a trading platform in an electricity trading scenario. This connection may include, but is not limited to, attributes such as identity authentication information, operating permissions, and session lifecycle. Trading entities may include, but are not limited to, electricity sales companies and power generation enterprises.

[0023] The multi-terminal collaborative acquisition module 101 allocates an independent browser execution context to each user. Here, the browser execution context can refer to an independent operating environment with web page rendering capabilities, script execution capabilities, and network request capabilities.

[0024] In a specific implementation, the browser execution context can be, but is not limited to, a headless browser instance, a browser window with a graphical user interface, or a packaged standalone web container process.

[0025] Specifically, the multi-terminal collaborative acquisition module 101 establishes a correspondence between system user identifiers and execution environments through a preset mapping configuration mechanism. When the system starts or receives an acquisition command, the multi-terminal collaborative acquisition module 101 does not reuse resources within a single process space, but instead initializes a completely isolated browser execution context for each transaction account to be acquired based on the mapping relationship. This isolation can include process-level isolation and resource state isolation. This architecture supports cross-platform deployment, thus eliminating reliance on a specific intranet environment and significantly reducing hardware and maintenance costs.

[0026] Furthermore, concurrent execution can refer to the multi-terminal collaborative acquisition module 101 being able to simultaneously drive multiple independent browser execution contexts to work without blocking each other.

[0027] For example, when the system needs to collect contract data from account A and bidding data from account B at the same time, the multi-terminal collaborative collection module 101 can simulate user operations or call interfaces in the first execution context corresponding to account A, while processing the corresponding collection logic in parallel in the second execution context corresponding to account B.

[0028] Understandably, this design overcomes the risk of session crosstalk that may arise in traditional single-threaded sequential data collection or single browser multi-tab mode, while also eliminating strong binding to specific intranet physical terminals, enabling data collection tasks to run efficiently in the cloud or on general servers in a multi-instance manner.

[0029] The data integrity check module 102 is used to make a dual judgment based on the response status code of the acquisition task and the non-empty status of the business data content, identify missing data and generate anomaly records.

[0030] In this embodiment, the data integrity check module 102 can be configured as a data instruction verification engine or verification algorithm component running in the business logic layer.

[0031] Here, the main responsibility of the data integrity check module 102 is to ensure that the collected data is not only successful at the network transmission level, but also valid at the business semantic level.

[0032] Specifically, dual determination can refer to performing logical checks in two dimensions simultaneously when verifying the collected results.

[0033] The first dimension could be a status check at the network protocol level, i.e., the response status code; this typically involves determining whether the Hypertext Transfer Protocol (HTTP) status code returned by the server belongs to the success category (such as the 2xx series), or whether the application programming interface (API) specific return code indicates that the request has been processed correctly.

[0034] Here, the 2xx series can refer to all HTTP status codes between 200 and 299. A "2" at the beginning can indicate success, meaning that the server has received the request and completed its processing.

[0035] The second dimension could be application-level content checks, namely the non-empty state of business data content. These checks can be designed to address "false success" scenarios, such as when the target server returns a status code indicating success, but the response body only contains an empty list or a preset placeholder indicating "no data available".

[0036] Furthermore, during the specific execution process, the data integrity check module 102 receives the original response returned by the acquisition task; if the response status code indicates an error, it is directly determined that the data is missing; if the response status code is successful, the key business fields in the response body are further parsed; only when the key business field is empty, has a length of zero, or contains an invalid value is it considered that the business-level data is missing.

[0037] In addition, the data integrity check module 102 can also execute automatic inspection logic; the module automatically calculates the time range to be inspected according to the granularity type defined in the collection rules, and performs a full-coverage check at a fixed time every day (e.g., 1 a.m.) to identify gaps in historical data.

[0038] Furthermore, once data loss is identified, the data integrity check module 102 can generate a structured exception record. This exception record can contain the complete context required for subsequent remediation processes, such as including but not limited to: the name of the target object of the missing data, the originally planned time range for data collection, the specific basis for determining that the data is missing, and the timestamp of the exception.

[0039] Here, the contents of the anomaly record can be persistently stored in the database as input information for subsequent triggering of intelligent supplementary sampling logic.

[0040] The intelligent supplementary sampling control module 103 is used to execute a tiered supplementary sampling strategy for abnormal records.

[0041] In this embodiment, the tiered supplementary data acquisition strategy is configured as follows: in response to acquisition failure or missing data detected during inspection, the acquisition task is pushed into the supplementary data acquisition queue without immediate retry; the supplementary data acquisition queue is scanned through a timed scheduling mechanism, and differentiated supplementary data acquisition is performed according to data attributes, including: for public data, automatically calling the acquisition interface to perform supplementary data acquisition; for private data, simulating user operations to perform supplementary data acquisition after verifying the security authentication medium status.

[0042] The intelligent supplementary acquisition control module 103 can be regarded as the asynchronous scheduling engine in the system. The core design concept of this module is to decouple the execution logic of the acquisition task from the retry logic in time, thereby building a non-intrusive and platform-friendly automated recovery mechanism.

[0043] Specifically, the aforementioned response to acquisition failure or missing data, pushing the acquisition task into the waiting queue instead of immediately retrying, can refer to the adoption of a negative feedback error handling mode.

[0044] Understandably, in such solutions, programs often retry multiple times in a tight loop immediately after a failure, which can easily cause a sudden surge in request traffic when the network is unstable, thereby triggering the denial-of-service protection or IP blocking of the target trading platform.

[0045] Unlike related solutions, the intelligent supplementary sampling control module 103 in this embodiment only encapsulates the abnormal record as a task object to be processed after receiving it, and stores the task object in a persistent supplementary sampling queue, and then immediately releases the current thread resources; this "failure-suspend" mechanism can ensure that the system will not launch continuous and high-frequency attack access to the target platform.

[0046] Furthermore, scanning the queue of tasks to be supplemented through a timed scheduling mechanism can refer to the system maintaining a daemon process or time trigger independent of the main acquisition process. This mechanism can be woken up according to a preset time window (e.g., every hour, or during off-peak hours at night, such as 2 a.m.) to read backlogged tasks in the queue in batches.

[0047] Here, the timed scheduling mechanism can employ a blocking scheduler and is configured with strict scheduling parameters to ensure task robustness. For example, `max_instances=1` can be set to prevent task concurrency conflicts; `coalesce=True` can be set to merge missed execution opportunities; and `misfire_grace_time` can be set to allow tasks to be executed after a certain delay. These configurations ensure that tasks can still execute reliably in abnormal scenarios such as system restarts and clock drift.

[0048] Furthermore, regarding the differentiated supplementary collection of data attributes, the system divides data into public data and private data based on the ease of data acquisition and security requirements.

[0049] Public data can refer to publicly available market information that requires only a basic account and password login or can be accessed without any barriers, such as listing information and announcement data. For public data, the intelligent data collection control module 103 can automatically call the underlying data interface or launch a headless browser to silently collect data when triggered on a timer, without manual intervention. After successful data collection, the system performs a uniqueness check. If the check passes, the data is written to the normal data table, and the abnormal status is updated to "resolved".

[0050] Private data can refer to data that involves the privacy of specific trading entities, fund transactions, or data that requires strong identity authentication to access, such as settlement statements and position details; access to private data is usually restricted by physical security devices.

[0051] Security authentication media can refer to hardware devices used to store digital certificates, private keys, or perform encryption operations, such as USB keys, smart cards, or biometric identification devices.

[0052] Specifically, for the supplementary collection of private data, the intelligent supplementary collection control module 103 can first detect whether the host machine or server has been correctly connected to the corresponding security authentication medium and is in a ready state; only when the detection is passed will the intelligent supplementary collection control module 103 start the process of simulating user operation (such as simulating mouse clicks, simulating keyboard input of password) to complete the data acquisition. If the detection fails, for example, if the U-shield is not inserted, the intelligent supplementary sampling control module 103 will choose to skip this execution or suspend the alarm, instead of forcibly attempting to lock the account.

[0053] Understandably, this layered design achieves automation while strictly adhering to the security and compliance requirements of the electricity trading scenario.

[0054] In addition, the system may also include a unified data pass-through and statistics module ( Figure 1 (Not shown in the image), all acquisition and supplementary acquisition results can be submitted through the unified data transmission and statistics module's unified transmission interface.

[0055] The unified data pass-through and statistics module can support classification by "normal / abnormal" and provide multi-dimensional statistical capabilities, including aggregated analysis by time, target name, data type, and abnormality category. It also supports rapid anomaly location by recording pass-through logs in real time.

[0056] The intelligent closed-loop-based power trading data acquisition and supplementary acquisition system and method disclosed above, through the setting of a multi-terminal collaborative acquisition module, enables the system to allocate an independent browser execution context for each user session, achieving isolated execution of multi-user acquisition tasks at the resource and state levels. This not only eliminates the strong dependence on dedicated security hardware plug-and-play and intranet environment, but also removes the performance blocking caused by single-point sequential acquisition, thereby significantly improving the system's deployment flexibility and concurrent acquisition efficiency. By setting an intelligent supplementary acquisition control module and adopting an asynchronous scheduling strategy of "failure enqueueing," the system avoids instantaneous high-frequency retries caused by network fluctuations, effectively reducing the access pressure on the target trading platform. Combined with a layered supplementary acquisition strategy for public / private data, the system ensures both automation and compliance of business operations, effectively avoiding the risk of triggering anti-crawler mechanisms and improving the stability and survival rate of acquisition operations. By setting a data integrity check module, the system implements dual judgment of response status code and non-empty business content for the acquisition results, which can accurately identify implicit missing data where the network is connected but the data is empty, ensuring the integrity of the final data stored.

[0057] In one possible implementation of the above embodiments, the multi-terminal collaborative acquisition module 101 is further configured to execute a attention-based hierarchical acquisition strategy; wherein, the attention-based hierarchical acquisition strategy includes: Based on business importance, target transaction data is divided into three preset levels: real-time data, important data, and general data; The three preset levels are configured with incrementally increasing data collection time intervals, where the data collection time interval for real-time data is shorter than that for important data, and the data collection time interval for important data is shorter than that for general data. When executing a data collection task, the collection requests are dynamically scheduled according to the preset level to which the target transaction data belongs and the corresponding collection time interval.

[0058] In this embodiment, the attention-based hierarchical collection strategy can be a dynamic resource scheduling mechanism based on business semantic priority, which aims to solve the resource waste and target platform overload problems caused by uniform polling collection in a one-size-fits-all manner.

[0059] Specifically, the system establishes a three-level classification model based on the frequency of data changes, their impact on transaction decisions, and timeliness requirements.

[0060] Real-time data can refer to business data that is updated very frequently and is crucial for capturing trading opportunities, such as real-time order book data during continuous bidding and spot order data; the value of this type of data decreases exponentially over time.

[0061] Important data can refer to business data with a moderate update frequency, mainly used for intraday settlement or position analysis, such as contract position information, settlement curve statistics, etc.; this type of data needs to maintain a high level of freshness, but allows for delays of up to a second.

[0062] General data can refer to business data that is relatively static, has a low update frequency, or is only used for archiving and retrospection, such as historical transaction announcements, daily market operation reports, or historical contract data.

[0063] Furthermore, configuring incrementally increasing data acquisition time intervals for the three preset levels can refer to establishing a negative correlation between acquisition frequency and data level.

[0064] As an example configuration, the system can set the collection interval for real-time data to the second level (e.g., 3 seconds) to ensure that no price fluctuations are missed; set the collection interval for important data to the half-minute level (e.g., 30 seconds) to balance timeliness and load; and set the collection interval for general data to the minute level or even the hour level (e.g., 5 minutes) to avoid invalid requests.

[0065] When performing the data collection task, the multi-terminal collaborative data collection module 101 does not use a fixed timer. Instead, it can identify the level of the target transaction data by reading the metadata tag and dynamically load the corresponding scheduling parameters (such as Cron expressions or Interval parameters).

[0066] Understandably, this mechanism ensures that system computing and network bandwidth resources are allocated to high-value data, enabling refined control over data collection activities.

[0067] The intelligent closed-loop-based power trading data acquisition and supplementary acquisition system and method of the above embodiments of this disclosure effectively reduces invalid high-frequency access to the power trading platform by significantly reducing the acquisition frequency of general and important data. This differentiated control strategy can significantly reduce the average request density of acquisition behavior, thereby simulating the normal human access rhythm and greatly reducing the risk of triggering the target platform's security protection mechanism. In addition, under limited hardware resources and network bandwidth, this strategy can also ensure that core real-time bidding data can obtain the highest acquisition priority and response speed, meeting the stringent requirements of second-level response in power spot trading; at the same time, it avoids wasting computing power on static historical data, achieving optimal configuration of overall system performance.

[0068] In one possible implementation of the above embodiments, the intelligent supplementary acquisition control module 103 further includes a state closed-loop management unit 1031, which manages the lifecycle of the acquisition task based on a state machine model; the state machine model includes four states: pending, processing, resolved, and failed. The status closed-loop management unit 1031 is configured such that when the acquisition task fails to execute in the processing state, it does not retry immediately, but updates the status of the acquisition task to failure, and the timed scheduling mechanism automatically rolls the failure status back to the pending state in the next scheduling cycle to trigger a new round of supplementary acquisition attempts.

[0069] In this embodiment, the state closed-loop management unit 1031 can be understood as a task flow controller built based on the principle of Finite State Machine (FSM).

[0070] The state closed-loop management unit 1031 can treat each acquisition or supplementary acquisition task as an independent state object, and constrain the behavior of the system through strictly defined state change rules to prevent the task from falling into logical deadlock or uncontrollable concurrent state.

[0071] Specifically, the four states in the state machine model can be defined as follows: "PENDING" can indicate that a task has been generated and entered into the queue, is in a static waiting state, has not yet been allocated computing resources, and is waiting to be awakened by the timed scheduling mechanism; "PENDING" can be the initial state or the reset state of all supplementary data collection tasks.

[0072] The "PROCESSING" state indicates that the task has been selected by the scheduler and the multi-terminal collaborative acquisition module 101 is allocating a browser context to execute the specific acquisition logic. The "PROCESSING" state can act as a "lock" to prevent the same task from being executed repeatedly by multiple scheduling instances.

[0073] "RESOLVED" indicates that the data has been successfully collected and passed the integrity verification, and the data has been securely stored in the database; the "RESOLVED" status can also be the final state of the task's lifecycle.

[0074] Failure (FAILED) can indicate that an error such as network anomaly, authentication timeout, or null data occurred during the "Processing" stage mentioned above.

[0075] Furthermore, the aforementioned implementation method's "when the data acquisition task fails while in the processing state, instead of immediately retrying, the status of the data acquisition task is updated to 'failed'" embodies a design philosophy combining "fast failure and delayed recovery." That is, in traditional exception handling logic, the program often performs multiple retry cycles immediately after catching an exception; however, in this embodiment, once an exception occurs, the state closed-loop management unit 1031 immediately terminates the current execution thread, records the error log, marks the task status in the database as "failed," and then releases all occupied browser and network resources.

[0076] Furthermore, the aforementioned "automatic rollback of the failed state to the pending state by the timed scheduling mechanism in the next scheduling cycle" can mean that the system does not maintain a long-term "retrying" state; when the timed scheduler (e.g., the supplementary data collection task running at 2 AM every day) restarts, it scans all tasks marked as "pending" as well as those in the "failed" state that have not reached the maximum attempt threshold. The process from "failed" to "pending" can be an automatic rollback path, which can be implicitly driven by the timer mechanism.

[0077] The scheduler implicitly treats or explicitly resets the status of these "failed" tasks to "pending," and then reinstates them into the execution queue in a new time window. This mechanism utilizes time intervals as a natural buffer, thereby enabling automatic rollback and re-triggering of task status.

[0078] The intelligent closed-loop power trading data acquisition and supplementation system and method disclosed above, by abandoning the aggressive strategy of "immediate retry upon failure," allows the system to proactively avoid network fluctuations or temporary platform restrictions. This delayed retry mechanism based on a longer scheduling cycle provides the target trading platform with sufficient recovery time, fundamentally preventing the possibility of being identified as a malicious attack due to consecutive erroneous requests within a short period, thus achieving platform-friendly data acquisition. Furthermore, it prevents task backlog and logical deadlocks, improving system robustness. The system does not need to maintain complex retry counters or complex exception recovery logic in memory; it only needs to rely on persistent states in the database and periodic scans by the timer to maintain closed-loop operation. This stateless or weakly stateful design significantly reduces system coupling and memory overhead, thereby simplifying operation and maintenance logic and reducing state management costs.

[0079] In one possible implementation of the above embodiments, the multi-terminal collaborative acquisition module 101 further includes a pressure monitoring and circuit breaker unit 1011, which is used to count the access frequency to the target platform in real time and calculate the pressure multiple of the current access frequency based on the preset normal manual operation frequency. When the pressure multiple exceeds the preset multiple threshold, the pressure monitoring and circuit breaker unit 1011 automatically extends the collection time interval of the current level in the attention level collection strategy until the pressure multiple falls back to the preset range of simulated human access; wherein, the actual access frequency corresponding to the preset multiple threshold is strictly less than the anti-crawler security baseline of the target platform.

[0080] In this embodiment, in order to enhance the survivability of the system in complex network environments, the multi-terminal collaborative acquisition module further integrates a pressure monitoring and circuit breaker unit 1011; the function of the pressure monitoring and circuit breaker unit 1011 is not simply passive recording, but can be an active traffic shaping controller, thereby aiming to balance acquisition efficiency and the security and compliance of the target platform in real time.

[0081] Specifically, the stress monitoring and circuit breaker unit 1011 can be independent of offline log analysis, but can use online calculation to count the frequency of access to the target platform in real time. The stress monitoring and circuit breaker unit 1011 maintains a sliding time window in memory, for example with a granularity of 1 minute or 5 minutes, and accumulates the total number of HTTP requests issued by all browser execution contexts within the window in real time.

[0082] Based on this, the system can introduce a quantitative indicator called the pressure multiplier, which is calculated based on the ratio of the current real-time request rate to the normal manual operation frequency of traders. The system aims to control the pressure multiplier so that the system's maximum request frequency is always far below the anti-crawler security baseline of the target power trading platform.

[0083] For example, suppose a trader's normal manual query frequency is 15 requests per minute, and the system's warning pressure multiplier is set at 4.0 times (i.e., 60 requests per minute). Meanwhile, the target platform's anti-scraping security baseline, which it absolutely cannot allow to be exceeded, is 5 requests per second (i.e., 300 requests per minute). When the system detects that the aggregated request rate reaches 60 requests per minute (pressure multiplier reaching 4.0), although this is still far below the platform's anti-scraping red line (300 requests), the system will still proactively trigger a circuit breaker for security and compliance reasons.

[0084] Understandably, the stress ratio can intuitively reflect the load intensity of the current data collection activity on the target server.

[0085] Furthermore, the preset multiple threshold and the corresponding frequency upper limit are not static fixed values. The system can support dynamic evolution based on the anti-crawler rules of the target power trading platform or analysis of historical operation logs to continuously optimize and adaptively lower the frequency upper limit. Thus, while ensuring the basic data collection needs, the system's request density is continuously compressed, thereby approaching the normal manual operation rhythm of real traders.

[0086] Furthermore, the pressure monitoring and circuit breaker unit 1011 can have a built-in dynamic soft circuit breaker mechanism. For example, when the calculated pressure multiple exceeds a preset warning threshold, the system will not adopt an aggressive hard shutdown strategy, but can automatically trigger degradation protection. Specifically, the pressure monitoring and circuit breaker unit 1011 can temporarily take over the scheduling of the attention-level data collection strategy, forcibly extending the collection time interval for all current levels (including real-time data, important data, and general data).

[0087] For example, the system can apply a penalty coefficient positively correlated with the pressure multiple to dynamically adjust the real-time data acquisition task from once every 3 seconds to once every 12 seconds. This state will continue until the pressure multiple in subsequent statistical periods falls back to the preset safe range, at which point the system will automatically remove the restriction and restore the original acquisition frequency.

[0088] The intelligent closed-loop power trading data acquisition and supplementation system and method of the above embodiments of this disclosure can construct a closed-loop active defense system by introducing a dynamic adjustment mechanism based on feedback control. Under the premise of ensuring business continuity, it can avoid the risk of IP being blocked or locked to the greatest extent, thereby greatly improving the system's survival rate.

[0089] In one possible implementation of the above embodiments, the system further includes a data persistence storage module 104, which adopts a hierarchical storage architecture that separates the main table and the sub-table; wherein, the main table is used to store the metadata of the acquisition task, and the metadata includes at least the task ID, target name and execution status; the sub-table is used to store unstructured power trading business data in JSONB format; The persistent data storage module 104 ensures idempotent data writing by using a combination constraint of task ID and business unique key.

[0090] In this embodiment, the data persistence storage module 104 can be designed as a hierarchical storage engine that combines the transaction characteristics of relational databases with the flexibility of document databases; the data persistence storage module 104 is intended to resolve the contradiction between the stability of metadata structure and the variability of business data structure in the actual collection process of power transaction data.

[0091] Specifically, a hierarchical storage architecture that separates master tables and child tables can refer to a system that physically or logically splits the storage dimension of data into management views and data views. The system may include rule configuration tables, normal data tables, and abnormal data tables.

[0092] Here, the rule configuration table can be used to store the supplementary collection rules for each target object, including granularity type, time range strategy, collection method (e.g., API or page simulation), maximum number of retries, enabled status, etc., and supports JSONB format configuration items.

[0093] The main table is part of the normal data table and can serve as the system's scheduling and indexing hub. It stores only task metadata that is highly structured, occupies little space, and has a very high query frequency. In addition to the task ID that must be included as a foreign key, the main table can also maintain the target name for aggregation statistics by trading instrument, as well as the execution status to support the state machine's flow logic.

[0094] Understandably, this design allows the system to scan only the lightweight main table when performing large-scale task filtering, status polling, or log auditing, thus avoiding I / O blocking caused by loading large business messages.

[0095] Sub-tables can focus on carrying high-dimensional business loads. Considering that the frequent iteration of electricity market rules leads to frequent changes in the format of transaction reports, this embodiment uses the JSONB (BinaryJSON) format in the sub-table to store unstructured electricity transaction business data.

[0096] Here, unlike text JSON, JSONB can perform binary encoding and pre-parsing at the time of writing, thereby supporting efficient key-value indexes and nested queries; this means that when the power trading center adjusts the data format, the system can directly store the newly added heterogeneous fields without performing high-risk database table structure changes.

[0097] The exception data table can also adopt a hierarchical design. Its main table identifies the exception summary, such as including but not limited to the target, type, and resolution status; its sub-tables can record the details of each re-collection behavior, such as including but not limited to the number of retries, execution time, and error information, and can support the tracking of multiple retries of the same exception.

[0098] Furthermore, to address the common problem of repeated fetching in distributed data collection, the data persistence storage module 104 can build an idempotent write mechanism at the database level.

[0099] Here, the system utilizes the database engine's unique indexing capabilities to establish a combined constraint consisting of a "task ID" and a "business unique key." The business unique key can be a unique identifier extracted based on a combination of core business attributes of the target transaction data (such as transaction time, transaction node, or data identifier). During write operations, the database can enforce verification of the uniqueness of this combined key; if an attempt is made to write an existing record, the database will throw a constraint conflict exception. The module then catches this exception and marks it as "duplicate ignored" rather than "write failed," thus ensuring strict data uniqueness at the physical storage level.

[0100] The intelligent closed-loop-based power trading data acquisition and supplementation system and method disclosed above, by setting up a persistent data storage module and adopting a hierarchical storage architecture that separates the main table and sub-tables, utilizes the main table to store structured metadata and the sub-tables to store unstructured business data in JSONB format. This allows the system to adapt to frequent changes in power trading data formats without modifying the database table structure, reducing system maintenance costs and development risks. Furthermore, by separating the main and sub-tables, frequently accessed task status queries and large-volume business data read / write operations are physically or logically isolated, thereby reducing disk I / O overhead during task scheduling and improving the system's query response speed and concurrent processing capabilities. Moreover, by establishing a combined constraint mechanism based on task ID and unique business keys, idempotency checks are enforced at the database level, effectively preventing duplicate data writes caused by network retries or multi-threaded concurrency, ensuring the uniqueness and accuracy of historical power trading data.

[0101] In one possible implementation of the above embodiments, the system further includes an exception handling and alarm module 105, which is used to trigger an alarm when the supplementary sampling strategy fails to execute or when manual intervention is required. The exception handling and alarm module 105 is equipped with a deduplication mechanism: before triggering an alarm, it checks whether there is already an exception record of the same type within a preset time window; if so, it intercepts the alarm notification; if not, it allows the alarm notification and records the timestamp of the exception.

[0102] In this embodiment, the anomaly handling and alarm module 105 can be configured as an operation and maintenance monitoring component with intelligent filtering capabilities, which can be used to solve the problem of excessive invalid alarms that may occur during long-term unattended operation of automated systems.

[0103] Specifically, triggering an alarm when the supplementary data acquisition strategy fails or requires manual intervention can refer to the module's real-time monitoring of critical system failure events. Supplementary data acquisition strategy failure can mean that a data acquisition task remains in a failed state after multiple retries (e.g., reaching the maximum number of retries) and cannot be recovered automatically. The need for manual intervention typically involves scenarios involving the acquisition of private data, such as the system detecting blocking errors like the security authentication medium not being inserted, driver malfunction, or expired account password.

[0104] When such an event occurs, the system generates a raw alarm request containing the error code, the target object, and stack information.

[0105] Furthermore, the inclusion of a deduplication mechanism means that the exception handling and alarm module 105 adds a time-based sliding window verification step before sending notifications to the outside.

[0106] The system can maintain an exception status table in memory or a cache database to record recently sent alarm fingerprints.

[0107] Specifically, when a new original alarm request is received, the anomaly handling and alarm module 105 can first extract the characteristic information of the request (such as "error type + target transaction account") to generate a unique anomaly fingerprint; Subsequently, the anomaly handling and alarm module 105 can query whether the fingerprint has been sent within a preset time window before the current moment (e.g., the last 30 minutes).

[0108] If the query result is yes, it indicates that the fault belongs to a known and notified recurring event. The exception handling and alarm module 105 can directly block this alarm notification, only add the error count in the background log, and no longer send real-time messages to the operation and maintenance personnel. If the query result is negative, it indicates that this is a new fault or that the previous alarm has exceeded the suppression period. The anomaly handling and alarm module 105 can allow the alarm notification to reach the operation and maintenance personnel via email, SMS or instant messaging tools, and at the same time update the most recent sending timestamp of the anomaly fingerprint, thereby starting a new deduplication period.

[0109] The intelligent closed-loop-based power trading data acquisition and supplementation system and method disclosed above effectively filters redundant and duplicate alarms in network fluctuation or persistent fault scenarios by setting up an anomaly handling and alarm module and configuring a deduplication mechanism, and using a preset time window to detect and intercept anomaly records of the same type. This avoids the waste of operation and maintenance resources caused by the proliferation of invalid notifications. Furthermore, by only allowing alarm notifications that occur for the first time or exceed the suppression period, it ensures that operation and maintenance personnel can focus on real and urgent fault events, improving the response speed and operation and maintenance efficiency of critical system faults.

[0110] In one embodiment, please refer to Figure 2 , Figure 2 An exemplary schematic diagram of another architecture of the intelligent closed-loop-based power trading data acquisition and supplementary acquisition system according to an embodiment of this disclosure is shown. Figure 2 As shown, in the intelligent closed-loop power trading data acquisition and supplementary acquisition system, the multi-terminal collaborative acquisition module 101 includes a pressure monitoring and circuit breaker unit 1011, and the intelligent supplementary acquisition control module 103 includes a status closed-loop management unit 1031. Furthermore, the intelligent closed-loop power trading data acquisition and supplementary acquisition system also includes a data persistent storage module 104 and an anomaly handling and alarm module 105.

[0111] In one embodiment, please refer to Figure 3 , Figure 3 This illustration shows an exemplary schematic diagram of yet another architecture of the intelligent closed-loop-based power trading data acquisition and supplementary acquisition system according to an embodiment of this disclosure, such as... Figure 3 As shown, the system can also adopt a layered architecture design, which includes, from top to bottom: a scheduling layer, a business layer, a service layer, a data layer, and a presentation layer. This layered design can achieve high cohesion and low coupling of the system by decoupling the responsibilities of each layer.

[0112] Specifically, the scheduling layer can serve as the closed-loop control hub of the system, primarily used to carry the timed scheduling mechanism described in the aforementioned embodiments. The core of this layer may include a timed task scheduler, which is configured with flexible scheduling logic supporting Cron expressions to trigger the aforementioned data inspection tasks (e.g., 1 AM daily) and supplementary data collection execution tasks (e.g., 2 AM daily).

[0113] In addition, the scheduling layer includes a concurrency control mechanism, which can be used to implement the aforementioned strict scheduling parameter configuration. Specifically, this means limiting the number of concurrent execution instances of the same task (e.g., the max_instances=1 configuration mentioned earlier) to implement a strategy to prevent task concurrency conflicts. Meanwhile, the task execution monitoring component is responsible for listening to the task status and driving the state machine transition when a task fails, thus implementing the aforementioned automatic rollback logic.

[0114] Furthermore, the business layer can serve as the core logic engine of the system, and can be the specific residing layer for the aforementioned functional modules. Specifically, the business layer can include the aforementioned multi-terminal collaborative acquisition module 101 (corresponding to...). Figure 2The data acquisition module, intelligent data acquisition control module 103 (corresponding to the intelligent data acquisition module in the diagram), data integrity check module 102 (corresponding to the data inspection module in the diagram), and anomaly handling and alarm module 105 (corresponding to the anomaly handling module in the diagram) are all located in this layer. These modules work collaboratively to execute core business logic, including priority-based data acquisition, dual-judgment identification of missing data, execution of layered data acquisition strategies, and closed-loop status management.

[0115] Furthermore, the service layer, acting as the system's external interface gateway, provides standardized interaction channels. This layer primarily includes REST API interfaces and data pass-through services. The REST API interfaces can respond to manually triggered commands (such as the aforementioned manually triggered supplementary data collection procedure for private data). The data pass-through service corresponds to the aforementioned unified data pass-through and statistics module, responsible for receiving the raw data returned by the collection end, standardizing it, and forwarding it to the data layer to ensure data flow consistency.

[0116] Furthermore, the data layer can be used to carry the aforementioned persistent data storage module 104; this layer can be deployed with a relational database that supports a hierarchical storage architecture, specifically implementing the aforementioned storage model that separates the main table and the sub-table.

[0117] Among them, the structured and semi-structured storage components can correspond to the aforementioned JSONB format storage logic and are used to store unstructured power transaction business data in the sub-table; the relational database component is used to store task metadata (such as task ID, target name and execution status) and rule configuration table in the main table.

[0118] In addition, this layer also ensures idempotent data writing at the physical storage level through the aforementioned combined constraint mechanism (task ID + business unique key).

[0119] Furthermore, the presentation layer can correspond to the set of browser execution contexts maintained by the aforementioned multi-terminal collaborative acquisition module 101. This layer is not a traditional user graphical interface, but rather refers to a pool of runtime environments used to execute specific acquisition tasks. This layer achieves the aforementioned resource and state isolation through multi-user browser instances and independent execution context technology.

[0120] Through a multi-port mapping mechanism, the system can precisely control the concurrent execution of each independent browser execution context, thereby supporting the implementation of multi-terminal collaborative data collection at the underlying level.

[0121] In summary, as Figure 2As shown, the system constructs a closed-loop data flow of "presentation layer acquisition → data layer storage → business layer verification → scheduling layer feedback": the presentation layer executes specific acquisition actions; the data layer persistently stores the acquisition results; the business layer (data integrity check module) performs dual judgment based on the stored data; the scheduling layer determines whether to trigger supplementary acquisition in the next cycle based on the abnormal record status generated by the judgment, thereby realizing the aforementioned intelligent closed loop.

[0122] Further reference Figure 4 , Figure 4 This diagram illustrates a flowchart of a power trading data acquisition and supplementation method based on an intelligent closed-loop system, as provided in an embodiment of this disclosure, applied to the above-mentioned... Figure 1-3 In any of the following power trading data acquisition and supplementation systems based on intelligent closed-loop systems, the process of this method may include the following steps: Step S401: Maintain multiple independent user sessions and assign an independent browser execution context to each user session, and concurrently execute the target transaction data collection task through the independent browser execution context; Step S402: Based on the response status code of the acquisition task and the non-empty status of the business data content, a dual judgment is made to identify missing data and generate an anomaly record. Step S403: Implement a tiered supplementary data collection strategy for abnormal records. The tiered supplementary data collection strategy includes: in response to data collection failure or missing data detected during inspection, pushing the data collection task into the supplementary data collection queue without immediately retrying; scanning the supplementary data collection queue through a timed scheduling mechanism and performing differentiated supplementary data collection based on data attributes, including: for public data, automatically calling the data collection interface to perform supplementary data collection; for private data, simulating user operations to perform supplementary data collection after verifying the security authentication medium status.

[0123] In one possible implementation of the above embodiments, the collection task of target transaction data is executed concurrently through an independent browser execution context, and the collection strategy based on the level of attention is further included. The level of attention collection strategy includes: dividing the target transaction data into three preset levels: real-time data, important data, and general data according to business importance; configuring incremental collection time intervals for the three preset levels respectively, wherein the collection time interval for real-time data is less than the collection time interval for important data, and the collection time interval for important data is less than the collection time interval for general data; when executing the collection task, the collection request is dynamically scheduled according to the preset level to which the target transaction data belongs and the corresponding collection time interval.

[0124] In one possible implementation of the above embodiments, the hierarchical supplementary acquisition strategy for abnormal records also includes managing the lifecycle of acquisition tasks based on a state machine model. The state machine model includes four states: pending, processing, resolved, and failed. Managing the lifecycle of acquisition tasks based on the state machine model includes: when an acquisition task fails to execute in the processing state, it is not retried immediately, but the state of the acquisition task is updated to failed, and the timed scheduling mechanism automatically rolls the failed state back to the pending state in the next scheduling cycle to trigger a new round of supplementary acquisition attempts.

[0125] In one possible implementation of the above embodiments, the method further includes: real-time statistics of the access frequency to the target platform and calculation of the pressure multiple; when the pressure multiple exceeds a preset threshold, automatically extending the collection time interval of the current level in the attention level collection strategy until the pressure multiple falls back to a preset safe range.

[0126] In one possible implementation of the above embodiments, the method further includes persistent storage of the collected data through a hierarchical storage architecture that separates the main table and the sub-table; wherein, the main table is used to store the metadata of the collection task, and the metadata includes at least the task ID, target name and execution status; the sub-table is used to store unstructured power transaction business data in JSONB format; persistent storage includes ensuring idempotent writing of data through a combination constraint of task ID and business unique key.

[0127] In one possible implementation of the above embodiments, the method further includes: triggering an alarm when the supplementary sampling strategy fails or requires manual intervention; and executing a deduplication mechanism before triggering the alarm, the deduplication mechanism including: detecting whether there is already an abnormal record of the same type within a preset time window; if so, intercepting the alarm notification; if not, allowing the alarm notification and recording the timestamp of the abnormality.

[0128] In one embodiment, please refer to Figure 5 , Figure 5 This illustration shows a flowchart of the real-time data acquisition process for the power trading data acquisition and supplementary acquisition method based on intelligent closed-loop provided in this embodiment of the present disclosure, such as... Figure 5 As shown, the process includes the following steps: Step S501: The data collection task is started.

[0129] Here, the multi-terminal collaborative acquisition module responds to the trigger command of the timer scheduler or external manual request, initializes the acquisition task and allocates the corresponding resources.

[0130] Step S502: Identify the importance of the data business; if the business importance of the target transaction data is Level 1, proceed to step S503; if the business importance of the target transaction data is Level 2, proceed to step S504; if the business importance of the target transaction data is Level 3, proceed to step S505.

[0131] Here, the system reads the metadata tags or preset configuration table of the target transaction data and maps them to a three-level classification model to determine the subsequent scheduling strategy.

[0132] Step S503, Level 1: Real-time data.

[0133] Here, this type of data corresponds to high-frequency scenarios such as continuous bidding or spot market declaration. The system loads a second-level (e.g., 3-second) collection time interval parameter for it to ensure extremely high data timeliness.

[0134] Step S504, Level 2: Important Data.

[0135] Here, this type of data corresponds to analysis scenarios such as contract holdings or settlement curve statistics. The system loads a collection time interval parameter of half a minute (e.g., 30 seconds) to achieve a balance between timeliness and load.

[0136] Step S505, Level 3: General Data.

[0137] Here, this type of data corresponds to static scenarios such as historical announcements or daily market operation reports. The system loads a long collection interval parameter at the minute level (such as 5 minutes) to avoid invalid access.

[0138] Step S506: Real-time pressure monitoring.

[0139] Here, the stress monitoring and circuit breaker unit uses a sliding time window to count the total number of requests issued by all browser execution contexts in real time and calculates the stress multiple relative to the security baseline.

[0140] Step S507: Does the pressure multiplier exceed the preset threshold? If yes, proceed to step S508; if no, proceed to step S509.

[0141] Here, the system compares the calculated real-time pressure multiple with a preset warning threshold (e.g., 4 times) to determine whether traffic shaping or degradation protection needs to be triggered.

[0142] Step S508, pressure too high.

[0143] Here, the system triggers a dynamic soft circuit breaker mechanism, automatically applying a penalty coefficient to extend the current level of data collection interval and proactively reducing the request density until the pressure drops.

[0144] Step S509, pressure is normal.

[0145] Here, the system maintains the original data acquisition time interval determined by steps S503 to S505 unchanged, and records the execution log of this scheduling normally.

[0146] Step S510: Perform data acquisition.

[0147] Here, the assigned independent browser execution context initiates actual network requests to the target power trading platform and parses the data according to the finally determined time interval parameters.

[0148] In one embodiment, please refer to Figure 6 , Figure 6 The flowchart of the intelligent supplementary data collection mechanism of the power trading data collection and supplementary data collection method based on intelligent closed loop provided in this disclosure embodiment is shown, as follows: Figure 6 As shown, the process includes the following steps: Step S601: Intelligent identification of missing data.

[0149] Here, the data integrity check module automatically scans historical data within a target time range according to a preset inspection granularity (such as 1 a.m. every day) to discover potential data gaps.

[0150] Step S602, dual determination mechanism.

[0151] Here, the system performs dual verification of the collected results at both the network layer and the business layer. Data is only confirmed as missing when the response status code is not successful (e.g., not 2xx) or the business data content is empty.

[0152] Step S603: Determine the data type. If it is public, proceed to step S604; if it is private, proceed to step S605.

[0153] Here, the intelligent data acquisition control module diverts missing data into public or private data processing paths based on the ease of data acquisition and security requirements.

[0154] Step S604, Public Data Supplement Collection, includes the following sub-steps: Step S604a: If it fails, it will be added to the queue.

[0155] Here, the system adopts a negative feedback mode. When the acquisition fails, it prohibits immediate execution of loop retry and instead directly terminates the current thread to release resources.

[0156] Step S604b: Store the queue of samples to be collected.

[0157] Here, the system encapsulates failed tasks as pending objects and persists them to the pending data collection queue in the database, marking their status as "pending".

[0158] Step S604c: Timer scan triggered.

[0159] Here, the timed scheduling mechanism wakes up the queue in a preset time window and reads backlogged tasks in batches to prepare for execution.

[0160] Step S605, Private Data Supplementation, includes the following sub-steps: Step S605a: Manually trigger startup.

[0161] Here, for data requiring physical insertion and removal of security authentication media, the system needs to wait for the maintenance personnel to confirm that the device is ready before manually triggering the signal.

[0162] Step S605b: If it fails, enter the private queue.

[0163] Here, if the simulation operation fails, the task is also pushed into a private data queue for supplementary data collection, the number of retries is recorded and it awaits further processing.

[0164] Step S605c: Timer scan triggered.

[0165] Here, the same timed scheduling mechanism also scans the private queue. If the maximum retry threshold is not reached, the scheduling will be attempted again; if the threshold is exceeded, it will be marked as requiring manual intervention.

[0166] Step S606: Perform supplementary sampling.

[0167] Here, the intelligent data acquisition control module automatically selects to call the underlying API interface (for public data) or launch a browser to simulate user operations (for private data) to perform specific data acquisition actions based on the data attributes.

[0168] Step S607, result processing.

[0169] Here, the data persistence storage module performs idempotency checks on the re-collected data based on the business unique key to prevent duplicate data from being entered into the database.

[0170] Step S608: If the processing result is failure (status: pending), re-enqueue and wait for the next trigger.

[0171] Here, the status closed-loop management unit updates the task status to FAILED, which will automatically roll back to PENDING in the next scheduling cycle, thus forming a closed loop waiting for the next timed trigger.

[0172] Step S609: If the processing result is successful (status: resolved), the process ends.

[0173] Here, the data is successfully written to the normal data table, the task status is updated to resolved (RESOLVED), and the lifecycle of this data collection task is terminated.

[0174] It should be noted that the above-described intelligent closed-loop-based power trading data acquisition and supplementation system, when implementing the corresponding intelligent closed-loop-based power trading data acquisition and supplementation method, is only illustrated by the division of the above-described program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the above system can be divided into different program modules to complete all or part of the processing described above. Furthermore, the system provided in the above embodiments and the corresponding... Figure 3 The embodiments of the methods shown belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0175] This disclosure also provides an electronic device having the above-described features. Figure 1-3 The system shown is a power trading data acquisition and supplementary acquisition system based on intelligent closed loop.

[0176] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown.

[0177] The following is a detailed reference. Figure 7 The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present disclosure. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0178] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0179] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the intelligent closed-loop-based power trading data acquisition and supplementary acquisition method of this disclosure.

[0180] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0181] This disclosure also provides a computer-readable storage medium. The methods described above according to this disclosure can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the intelligent closed-loop-based power trading data acquisition and supplementary acquisition method shown in the above embodiments is implemented.

[0182] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0183] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A power trading data acquisition and supplementation system based on intelligent closed-loop, characterized in that, The system includes: The multi-terminal collaborative acquisition module is used to maintain multiple independent user sessions and allocate an independent browser execution context to each user session, and concurrently execute the acquisition task of the target transaction data through the independent browser execution context; The data integrity check module is used to make a dual judgment based on the response status code of the acquisition task and the non-empty status of the business data content, identify missing data and generate anomaly records; The intelligent data acquisition control module is used to execute a hierarchical data acquisition strategy for the abnormal records. The hierarchical data acquisition strategy is configured to: push the acquisition task into the data acquisition queue without immediately retrying in response to acquisition failure or missing data detected during inspection; scan the data acquisition queue through a timed scheduling mechanism and perform differentiated data acquisition based on data attributes, including: automatically calling the acquisition interface to perform data acquisition for public data; and simulating user operations to perform data acquisition for private data after verifying the security authentication medium status.

2. The system according to claim 1, characterized in that, The multi-terminal collaborative data collection module is also configured to execute a tiered data collection strategy based on attention levels; wherein, the tiered data collection strategy includes: Based on business importance, target transaction data is divided into three preset levels: real-time data, important data, and general data; The three preset levels are configured with incrementally increasing collection time intervals, wherein the collection time interval for real-time data is less than the collection time interval for important data, and the collection time interval for important data is less than the collection time interval for general data. When performing the data collection task, the data collection requests are dynamically scheduled according to the preset level to which the target transaction data belongs and the corresponding data collection time interval.

3. The system according to claim 1, characterized in that, The intelligent data acquisition control module also includes a state closed-loop management unit, which manages the lifecycle of the acquisition task based on a state machine model; the state machine model includes four states: pending, processing, resolved, and failed. The state closed-loop management unit is configured such that when a data acquisition task fails to execute in the processing state, it does not immediately retry, but updates the state of the data acquisition task to failure, and the timed scheduling mechanism automatically rolls the failure state back to the pending state in the next scheduling cycle to trigger a new round of supplementary data acquisition attempts.

4. The system according to claim 2, characterized in that, The multi-terminal collaborative acquisition module also includes a pressure monitoring and circuit breaker unit, which is used to count the access frequency to the target platform in real time and calculate the pressure multiple of the current access frequency based on the preset normal manual operation frequency. When the pressure multiple exceeds a preset multiple threshold, the pressure monitoring and circuit breaker unit automatically extends the collection time interval of the current level in the attention level collection strategy until the pressure multiple falls back to the preset range of simulated human access; wherein, the actual access frequency corresponding to the preset multiple threshold is strictly less than the anti-crawler security baseline of the target platform.

5. The system according to claim 1, characterized in that, It also includes a data persistence storage module, which adopts a hierarchical storage architecture that separates the main table and sub-tables; wherein, the main table is used to store the metadata of the acquisition task, and the metadata includes at least the task ID, target name and execution status; the sub-table is used to store unstructured power trading business data in JSONB format; The persistent data storage module ensures idempotent data writing by using a combination constraint of task ID and business unique key.

6. The system according to claim 1, characterized in that, It also includes an anomaly handling and alarm module, which is used to trigger alarms when the supplementary sampling strategy fails to execute or when manual intervention is required; The exception handling and alarm module is configured with a deduplication mechanism, which includes: before triggering an alarm, checking whether there is already an exception record of the same type within a preset time window; if so, then blocking the alarm notification; if not, then allowing the alarm notification and recording the timestamp of the exception.

7. A method for collecting and supplementing power trading data based on intelligent closed-loop, characterized in that, The method includes: Maintain multiple independent user sessions and assign an independent browser execution context to each user session, and concurrently execute the target transaction data collection task through the independent browser execution context; The system uses a dual approach, combining the response status code of the data collection task with the non-empty status of the business data content, to identify missing data and generate anomaly records. A tiered supplementary data collection strategy is implemented for the abnormal records. The tiered supplementary data collection strategy includes: in response to data collection failure or missing data detected during inspection, pushing the data collection task into the supplementary data collection queue without immediately retrying; scanning the supplementary data collection queue through a timed scheduling mechanism and performing differentiated supplementary data collection based on data attributes, including: for public data, automatically calling the data collection interface to perform supplementary data collection; for private data, simulating user operations to perform supplementary data collection after verifying the security authentication medium status.

8. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes these computer instructions to perform the power trading data acquisition and supplementary acquisition method based on intelligent closed loop as described in claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, which are used to cause the computer to execute the power trading data acquisition and supplementary acquisition method based on intelligent closed loop as described in claim 7.

10. A computer program product, characterized in that, It includes computer instructions, which are used to cause the computer to execute the power trading data acquisition and supplementary acquisition method based on intelligent closed loop as described in claim 7.

Images