A method, device, medium and equipment for transmission and distribution of energy storage power station data

Abstract

A method, device, medium and equipment for transmission and distribution of energy storage power station data are disclosed. The method comprises: receiving each target time sequence file encrypted and sent by each local edge computing device of the energy storage power station through a transmission protocol; storing each target time sequence file in the same topic by a message queue cluster, and storing each target time sequence file in the same topic as a partition key to the site identification, so that the target time sequence files of the same energy storage power station are located in the same partition; and consuming each target time sequence file in the same topic in parallel through a plurality of consumer groups; wherein at least one consumer group writes the target time sequence file obtained by consumption to a time sequence database after parsing, and at least another consumer group archives the target time sequence file obtained by consumption to cold data storage. The disclosure improves the distribution efficiency of the energy storage power station time sequence data in the cloud under the premise of ensuring the correctness of the data time sequence.

Description

Technical Field

[0001] This disclosure relates to the field of energy storage technology, specifically to a method, apparatus, medium, and equipment for transmitting and distributing data from an energy storage power station. Background Technology

[0002] Electrochemical energy storage power stations continuously generate a large amount of time-series data during operation, such as battery voltage, current, temperature, and state of charge. As the scale of energy storage power stations expands, the amount of data generated by a single station can reach 10GB per day. In practical applications, multiple energy storage power stations typically operate simultaneously, and the time-series data generated by each station needs to be uploaded to the cloud via a network for unified reception and storage, serving various application scenarios such as operation monitoring, algorithm analysis, and long-term archiving.

[0003] In the above scenario, the cloud needs to simultaneously receive massive amounts of time-series data uploaded by numerous energy storage power stations and distribute the received data to different application scenarios, while ensuring the correctness of the data's order during distribution. How to improve the distribution efficiency of energy storage power station time-series data in the cloud while ensuring the correct data order has become an urgent technical problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical issues, this disclosure provides a method, apparatus, medium, and equipment for transmitting and distributing data from an energy storage power station, thereby improving the distribution efficiency of time-series data from the energy storage power station in the cloud while ensuring the correct data timing.

[0005] The first aspect of this disclosure provides a method for transmitting and distributing data from an energy storage power station, comprising: receiving encrypted target time-series files sent by local edge computing devices of each energy storage power station via a transmission protocol; wherein each target time-series file carries a site identifier corresponding to the energy storage power station; a message queue cluster storing each target time-series file in the same topic, and partitioning the target time-series files in the same topic using the site identifier as the partition key, so that the target time-series files of the same energy storage power station are located in the same partition; and consuming each target time-series file in the same topic in parallel by multiple consumer groups; wherein at least one consumer group parses the consumed target time-series files and writes them into a time-series database, and at least another consumer group archives the consumed target time-series files to cold data storage.

[0006] Furthermore, before receiving the encrypted target time series files sent by the local edge computing devices of each energy storage power station via the transmission protocol, the method further includes: each local edge computing device of each energy storage power station parses the original time series files generated by each energy storage power station to obtain original time series data; the original time series files are data files generated by each energy storage power station after sampling the operating data of the equipment within the station; preprocessing the original time series data to obtain target time series data; the preprocessing includes extreme value calculation, invalid data filtering, and fusion of data sequences from multiple individual cells; modeling the target time series data according to the equipment level of each energy storage power station to generate columnar compressed target time series files, and sending the target time series files via the transmission protocol.

[0007] Furthermore, the preprocessing includes: statistically analyzing the extreme values ​​of the sampled data within a preset period according to four levels: battery box, battery cluster, battery stack, and battery compartment, to obtain an extreme value data sequence; removing invalid data points from the original time series data according to preset validity rules, and replacing data points with preset special values ​​with null values; and fusing the data sequences of multiple individual cells according to the battery cluster to which the individual cells belong to generate a fused data sequence corresponding to the battery cluster level.

[0008] Furthermore, each target timing file is a compressed file compatible with the TSFile format; within the file body of each target timing file, the timing data is divided into multiple data block groups according to the device type, and each data block group is further divided into multiple data blocks according to the device.

[0009] Furthermore, a verification header is attached to each target time series file. The verification header includes a flag field, a version field, a time field, and a checksum field. The checksum field stores the CRC32 checksum of the corresponding target time series file. After receiving each target time series file, the cloud performs integrity verification on the received target time series file according to the checksum in the verification header. In response to a failed verification, the corresponding local edge computing device of the energy storage power station is triggered to retransmit the target time series file that failed the verification.

[0010] Furthermore, the transmission protocol is MQTT; each target time series file is published with a preset topic, which includes the site identifier of the corresponding energy storage power station; encrypted transmission is achieved by overlaying a TLS / SSL encryption layer on top of the TCP connection; in response to transmission failure or interruption, the local edge computing device of the corresponding energy storage power station performs breakpoint resumption or failure retry on the target time series file that failed to be transmitted or was interrupted.

[0011] Furthermore, the multiple consumer groups include: a producer consumer group, used to parse the target time-series files obtained from consumption in real time and write them into a production time-series database, where the data retention time in the production time-series database is not less than a first retention time; an analysis consumer group, used to parse the target time-series files obtained from consumption in full and write them into an analysis time-series database, where the data retention time in the analysis time-series database is a second retention time; and an archive consumer group, used to directly archive the target time-series files obtained from consumption to a cold data warehouse; wherein, the first retention time is greater than the second retention time.

[0012] A second aspect of this disclosure provides a data transmission and distribution apparatus for an energy storage power station, comprising: a receiving module configured to receive encrypted target time-series files sent by local edge computing devices of each energy storage power station via a transmission protocol; wherein each target time-series file carries a site identifier of the corresponding energy storage power station; a storage module configured to store each target time-series file in the same topic using a message queue cluster, and to partition the target time-series files in the same topic using the site identifier as a partition key, so that the target time-series files of the same energy storage power station are located in the same partition; and a parsing module configured to consume each target time-series file in the same topic in parallel by multiple consumer groups; wherein at least one consumer group parses the consumed target time-series file and writes it into a time-series database, and at least another consumer group archives the consumed target time-series file to cold data storage.

[0013] A third aspect of this disclosure provides a computer-readable storage medium storing a computer program for executing a method for transmitting and distributing data from an energy storage power station according to any of the embodiments of the first aspect described above.

[0014] A fourth aspect of this disclosure provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the method for transmitting and distributing energy storage power station data according to any embodiment of the first aspect described above.

[0015] The data transmission and distribution method for energy storage power stations provided in this disclosure uses a message queue cluster to store the target time-series files of each energy storage power station in the same topic and uses the site identifier as the partition key for partitioned storage. This ensures that the target time-series files of the same energy storage power station are located in the same partition. By utilizing the ordered storage characteristic of data within the same partition, the correct order of data distribution for the same energy storage power station is guaranteed. Multiple consumer groups consume the target time-series files in the same topic in parallel. At least one consumer group parses the target time-series file and writes it into the time-series database, while at least another consumer group archives the target time-series file to cold data storage. This allows the cloud to receive and store the same data only once and distribute it to different application scenarios simultaneously. Thus, while ensuring the correct data time-series, the distribution efficiency of energy storage power station time-series data in the cloud is improved. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the architecture of a data transmission and distribution system for an energy storage power station provided in this disclosure.

[0017] Figure 2 This is a flowchart illustrating a method for transmitting and distributing data from an energy storage power station, as disclosed in this publication.

[0018] Figure 3 This is a schematic diagram of the structure of a target timing file provided in this disclosure.

[0019] Figure 4 This is a schematic diagram of the structure of a data transmission and distribution device for an energy storage power station provided in this disclosure.

[0020] Figure 5 This is a structural diagram of an electronic device disclosed herein. Detailed Implementation

[0021] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0022] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0023] Application Overview The cloud needs to simultaneously receive massive amounts of time-series data uploaded by numerous energy storage power stations and distribute the received data to different application scenarios such as operation monitoring, algorithm analysis, and long-term archiving, while ensuring the correctness of the data order during distribution. To address these technical issues, in this embodiment, the cloud receives encrypted target time-series files, each carrying a site identifier, sent by the local edge computing devices of each energy storage power station via a transmission protocol. A message queue cluster stores the target time-series files of each energy storage power station in the same topic, and partitions the target time-series files within the same topic using the site identifier as the partition key. This ensures that the target time-series files of the same energy storage power station are located in the same partition, leveraging the ordered storage characteristic of data within the same partition to guarantee the correct time-series data of the same energy storage power station. Multiple consumer groups consume the target time-series files in the same topic in parallel. At least one consumer group parses the target time-series files and writes them into the time-series database, while at least another consumer group archives the target time-series files to cold data storage. This achieves one-time reception and parallel distribution across multiple scenarios, thereby improving the distribution efficiency of energy storage power station time-series data in the cloud while ensuring the correct data time-series.

[0024] Exemplary System Figure 1 This is a schematic diagram of the architecture of a data transmission and distribution system for an energy storage power station provided in this disclosure.

[0025] like Figure 1 As shown, the system comprises multiple energy storage power stations and a cloud platform, and can be broadly structured into a four-layer architecture: a station-side edge layer, a cloud access layer, a cloud distribution layer, and a cloud storage layer. Each energy storage power station constitutes the station-side edge layer, including on-site equipment, an on-site energy management system (EMS), and local edge computing devices. On-site equipment refers to various devices within the energy storage power station that generate operational data, including energy storage units such as battery compartments, battery stacks, battery clusters, battery boxes, and individual battery cells, as well as auxiliary equipment such as converters and temperature control devices. The on-site EMS samples the operational data of the on-site equipment (such as battery voltage, current, temperature, and state of charge) to generate raw time-series files. The local edge computing devices obtain the raw time-series files from the on-site EMS, parse, preprocess, and model and compress them to generate target time-series files, and then encrypt and send the target time-series files to the cloud via the network.

[0026] In some embodiments, the local edge computing device can also selectively write the generated target time series file into a local database for local access, so that the station can query and use it locally; the energy storage power station can also deploy a local security system for managing and analyzing the locally stored data, and the local security system can be deployed as a standalone unit.

[0027] The cloud layer comprises a cloud access layer, a cloud distribution layer, and a cloud storage layer. The cloud access layer deploys a message access cluster, which receives encrypted target time-series files from the local edge computing devices of each energy storage power station via a transmission protocol and bridges the received target time-series files to a message queue cluster. The cloud distribution layer deploys a message queue cluster, which stores the target time-series files from each energy storage power station in the same topic and partitions the target time-series files within the same topic using the site identifier as the partition key. The cloud storage layer deploys multiple consumer groups and corresponding storage systems. Multiple consumer groups consume target time-series files from the same topic in parallel. Specifically, the producer consumer group parses the consumed target time-series files in real time and writes them to the production time-series database; the analysis consumer group parses all consumed target time-series files and writes them to the analysis time-series database; and the archiving consumer group directly archives the consumed target time-series files to the cold data warehouse.

[0028] In some embodiments, the cloud may further include application servers and algorithm servers. The application server connects to the production time-series database to provide application services such as operation monitoring, production management, and decision support based on the data in the production time-series database. The algorithm server connects to the analysis time-series database to execute various algorithm analyses by calling data from the analysis time-series database. The analysis time-series database can return the data required by the algorithm to the algorithm server. The cold data warehouse can perform warehouse management operations on archived target time-series files, including warehouse information management, data inbound, data outbound, content acquisition, deduplication, compression, and comprehensive management, to support long-term, low-cost storage of massive amounts of data.

[0029] It should be noted that, Figure 1 Each component can use conventional names used in product implementation: the local edge computing device can be a station server or edge computing device in the implementation; the original time series file can be a station data file generated in accordance with the power industry's E-format data exchange standard in the implementation; the target time series file can be a columnar compressed file compatible with the TSFile format in the implementation; the message access cluster can be an EMQX cluster in the implementation; the message queue cluster can be a Kafka cluster in the implementation; the production time series database and the analysis time series database can be IoTDB databases in the implementation, and deployed in the form of database clusters respectively; the cold data storage can be a cold data warehouse or object storage in the implementation.

[0030] In some embodiments, the local edge computing device may be an industrial control computer or server deployed locally at the energy storage power station; the cloud may be deployed on a server cluster of a public cloud or a private cloud; the message access cluster, message queue cluster, each consumer group and each storage system may be deployed on different physical servers or virtual machines, or they may be partially merged and deployed, and this disclosure does not limit this.

[0031] Exemplary methods Figure 2 This is a flowchart illustrating a method for transmitting and distributing data from an energy storage power station according to an exemplary embodiment of this disclosure. This embodiment can be applied in the cloud, such as... Figure 2 As shown, it includes the following steps: Step 210: Receive each target time series file encrypted by the local edge computing device of each energy storage power station through the transmission protocol; wherein each target time series file carries the site identifier of the corresponding energy storage power station.

[0032] For example, a communication connection based on a transmission protocol is pre-established between the cloud-based message access cluster and the local edge computing devices of each energy storage power station. After generating a target time-series file, each local edge computing device of the energy storage power station sends the target time-series file as a transmission message to the cloud through an encrypted channel corresponding to the transmission protocol. After receiving the target time-series file, the message access cluster bridges it to the message queue cluster. Since each target time-series file carries the site identifier of the corresponding energy storage power station, the cloud can determine which energy storage power station the target time-series file originates from upon receiving it.

[0033] The transmission protocol refers to the communication protocol used to transmit data between the local edge computing device of the energy storage power station and the cloud. This disclosure does not limit the specific type of transmission protocol; for example, it can be the Message Queuing Telemetry Transport (MQTT) protocol or other protocols that support message transmission.

[0034] Local edge computing devices for energy storage power stations refer to computing devices deployed locally at the energy storage power station, directly connected to the station's energy management system, and processing data files generated by the energy storage power station. These devices may be station-side servers or edge computing devices.

[0035] The target time-series file refers to the time-series data file generated by the local edge computing device of the energy storage power station after processing the original time-series file generated by the energy storage power station. It is a data unit received, stored, and distributed by the cloud. The generation process of the target time-series file can be referred to the embodiments corresponding to steps 310 to 330 below. The site identifier refers to the identification information used to uniquely identify the energy storage power station. For example, it can be the number or code of the energy storage power station. Each energy storage power station corresponds to a site identifier, and different energy storage power stations have different site identifiers. The site identifier can be carried in the file content of the target time-series file or in the topic used to transmit the target time-series file.

[0036] Encrypted transmission refers to the transmission method of sending target time-series files through an encrypted channel. For example, an encryption layer can be superimposed on the connection corresponding to the transmission protocol to ensure the security of the target time-series files during transmission.

[0037] Step 220: The message queue cluster stores each target time series file in the same topic, and uses the site identifier as the partition key to partition and store each target time series file in the same topic, so that the target time series files of the same energy storage power station are located in the same partition.

[0038] For example, the message queue cluster receives target time-series files from the message access cluster bridge and stores them uniformly in the same topic. During storage, the message queue cluster extracts the site identifier carried in the target time-series file, uses the site identifier as the partition key, and routes the target time-series file to the corresponding partition under the same topic for storage based on the partition key. Since target time-series files with the same partition key are routed to the same partition, the target time-series files of the same energy storage power station are all located in the same partition; and the message queue cluster can ensure that the data in the same partition is stored in the order of being written, so the target time-series files of the same energy storage power station maintain time-order order within the same partition.

[0039] Among them, a message queue cluster refers to a distributed message middleware cluster composed of multiple message queue service nodes, used to receive, store and distribute messages, such as a Kafka cluster.

[0040] A topic is a channel in a message queue cluster used for logically classifying messages. The same topic is used to aggregate messages of the same type. In this embodiment, the target time-series files of each energy storage power station are stored in the same topic, meaning that data from all energy storage power stations are aggregated on the same logical channel, allowing the cloud to manage and distribute data from multiple stations in a unified manner.

[0041] A partition is a physical sharding of a topic. A topic can be divided into multiple partitions, which can be distributed across different nodes in the message queue cluster to support parallel read and write operations. The partition key is the basis for the message queue cluster to determine the partition to which a message belongs. The message queue cluster routes messages to the corresponding partitions based on the partition key; messages with the same partition key will be routed to the same partition. In this embodiment, the site identifier is used as the partition key, ensuring that target time-series files from the same energy storage power station are routed to the same partition. This leverages the ordered storage characteristic of data within the same partition to guarantee the correct time-series of data from the same energy storage power station. The number of partitions can be configured according to the number of energy storage power stations and the data scale. Messages within the same topic can be saved for a preset retention period.

[0042] In one implementation, the message queue cluster can adopt a cluster deployment mode consisting of multiple broker nodes, for example, a cluster consisting of 3 broker nodes; the topic name of the same topic can be, for example, FTM_TS, the number of partitions can be set to, for example, 12, and the retention period of messages in the same topic can be set to, for example, 7 days. The specific configuration can be based on the number of energy storage power stations, data scale and business needs.

[0043] Step 230: Multiple consumer groups consume target time-series files in the same topic in parallel; wherein at least one consumer group parses the target time-series files obtained from consumption and writes them into the time-series database, and at least another consumer group archives the target time-series files obtained from consumption to cold data storage.

[0044] For example, multiple consumer groups are deployed in the cloud, each subscribing to the same topic and consuming target time-series files within that topic independently and in parallel. Each consumer group reads target time-series files from different partitions of the same topic according to its own consumption schedule and processes the consumed target time-series files based on its corresponding data storage scenario: at least one consumer group parses the consumed target time-series files and writes them into a time-series database for querying and analysis; at least another consumer group does not parse the consumed target time-series files but instead archives them to cold data storage to achieve long-term retention of the original data.

[0045] In this context, a consumer group refers to a collection of consumer instances in a message queue cluster that collaboratively consume messages. Consumer instances within the same consumer group share the same consumption progress, and the same partition is consumed by only one consumer instance from the same consumer group at any given time. Different consumer groups operate independently, each maintaining its own consumption progress. Data on the same topic can be consumed repeatedly by multiple consumer groups without affecting each other.

[0046] Parallel consumption refers to a consumption method in which multiple consumer groups simultaneously read data from the same topic and process it individually. Since each consumer group is independent of the others, the cloud only needs to receive and store the target time-series file uploaded by each energy storage power station once, and then distribute the same data to different application scenarios simultaneously through multiple consumer groups.

[0047] Time-series databases are databases optimized for writing, storing, and querying time-series data with timestamps. For example, an IoTDB database is suitable for scenarios such as operation monitoring and algorithm analysis that require querying data by time dimension. Cold data storage refers to low-cost storage for long-term preservation of data with low access frequency. For example, a cold data warehouse is suitable for long-term archiving scenarios such as data backup and audit traceability.

[0048] The data transmission and distribution method for energy storage power stations provided in this disclosure receives encrypted target time-series files, each carrying a site identifier, sent by the local edge computing devices of each energy storage power station through a transmission protocol. This allows the cloud to distinguish the data source while receiving the data. The method uses a message queue cluster to store the target time-series files in the same topic and partitions them using the site identifier as the partition key. This ensures that target time-series files from the same energy storage power station reside in the same partition, leveraging the ordered storage characteristic of data within the same partition to guarantee the correct order of data distribution from the same energy storage power station. Furthermore, multiple consumer groups consume the target time-series files in the same topic in parallel. At least one consumer group parses the target time-series file and writes it to the time-series database, while at least another consumer group archives the target time-series file to cold data storage. This allows the cloud to receive and store the same data only once, simultaneously meeting the data needs of different application scenarios. Each consumer group is independent and does not affect the others, thus improving the distribution efficiency of energy storage power station time-series data in the cloud while ensuring the correct data timing.

[0049] In some embodiments, steps 310 to 330 are included before step 210. Specifically: Step 310: The local edge computing devices of each energy storage power station parse the original time series files generated by each energy storage power station to obtain the original time series data. The original time series files are data files generated by each energy storage power station after sampling the operating data of the equipment in the station.

[0050] For example, the energy management system of each energy storage power station samples the operating data of the equipment within the station according to a preset sampling period, and organizes the sampled data into a raw time-series file according to a preset file format. After obtaining the raw time-series file, the local edge computing device parses the raw time-series file according to the parsing rules corresponding to the file format, reads each data record in the file, and obtains the raw time-series data.

[0051] The original time-series file refers to the data file generated by an energy storage power station after sampling the operating data of its equipment. In the power industry, the station-side data files of energy storage power stations are usually generated in accordance with the E-format data exchange standard, so the original time-series file can also be called an E-file. The original time-series data refers to the time-ordered data sequence parsed from the original time-series file. Each data record can include a timestamp, measurement point identifier, and data value.

[0052] In one implementation, the original time series file may include types such as telemetry data files, teleindication data files, alarm data files, and model data files. For telemetry and teleindication data, the content of each sample is converted and stored in the target time series file corresponding to the sampling section. For alarm data, when generating the target time series file each time, it is checked whether an alarm file exists. If it exists, the alarm content is stored in the target time series file generated this time.

[0053] In one implementation, for model data, after the model file generated at the station (e.g., a model file with the .DL extension) is reviewed and modified to a preset extension (e.g., the .DMV extension), the local edge computing device recognizes the model file, updates the local power station configuration table, and encapsulates the model file into a transmission message to send to the cloud model topic (e.g., the FTM_DM topic). The cloud forwards the model file in the model topic to the power station model topic (e.g., the substation_model topic) of the message queue cluster to trigger the power station model update.

[0054] Step 320: Preprocess each original time series data to obtain each target time series data; the preprocessing includes extreme value calculation, invalid data filtering, and fusion of data sequences from multiple individual cells.

[0055] For example, after parsing the raw time-series data, the local edge computing device performs preprocessing on the raw time-series data: Through extreme value calculation, it statistically analyzes the extreme values ​​of the sampled data within a preset period, replacing the full sampled data with extreme value data sequences to compress the data size; through invalid data filtering, it removes invalid data points from the raw time-series data to improve data quality; and by fusing data sequences from multiple individual battery cells, it merges the data sequences at the individual battery cell level into data sequences at the upper-layer device level, reducing the number of data sequences that need to be transmitted and stored. After preprocessing, the target time-series data is obtained.

[0056] In this context, a single battery cell refers to the smallest unit of a battery in an energy storage power station. Since there are numerous single battery cells in an energy storage power station, uploading the data sequence of each single cell separately would generate a massive amount of data. For specific implementation methods of extreme value calculation, invalid data filtering, and data sequence fusion, please refer to the detailed implementation methods of the preprocessing steps below. The target time-series data refers to the time-series data obtained after preprocessing the original time-series data, used to generate the target time-series file.

[0057] Step 330: Model the target time series data according to the equipment level of each energy storage power station, generate columnar compressed target time series files, and send the target time series files through the transmission protocol.

[0058] For example, the local edge computing device establishes a correspondence model between the device level and the target time series data according to the device level of the energy storage power station, and organizes the target time series data under the corresponding device level; during the modeling process, the row-organized sampling data is converted into column-stored data by row-column transposition, and the column-stored data is compressed and encoded to generate a column-compressed target time series file; then, the target time series file is encrypted and sent to the cloud through the transmission protocol connection with the cloud.

[0059] In this context, "equipment hierarchy" refers to the hierarchical inclusion relationship between equipment within an energy storage power station, which may include four levels: battery box, battery cluster, battery stack, and battery compartment. Modeling refers to establishing a mapping relationship between time-series data and energy storage power station equipment, ensuring that each data sequence corresponds to a specific device and equipment hierarchy. Columnar compression refers to a data organization method that organizes time-series data by columns and performs compressed encoding. Data in the same column has the same data type, resulting in higher compression efficiency and easier reading by measurement point. The specific file structure of the target time-series file can be found in the following text. Figure 3 The described embodiments.

[0060] In this embodiment, the original time-series file is parsed, preprocessed, and modeled and compressed at the station via a local edge computing device, significantly reducing the amount of data uploaded to the cloud and alleviating network transmission and cloud reception pressures. Through extreme value calculation and cell data sequence fusion, the data size is compressed while ensuring no loss of key data features. Invalid data filtering improves the quality of uploaded data. By completing data modeling and columnar compression at the station, the target time-series file received by the cloud can be directly used for storage and distribution without further format conversion in the cloud, reducing the cloud's processing burden and thus improving the overall efficiency of data upload and distribution.

[0061] In some embodiments, the preprocessing in step 320 may include: statistically analyzing the extreme values ​​of sampled data within a preset period according to four levels: battery box, battery cluster, battery stack, and battery compartment, to obtain an extreme value data sequence; removing invalid data points from the original time series data according to preset validity rules, and replacing data points with preset special values ​​with null values; and fusing the data sequences of multiple individual cells according to the battery cluster to which the individual cells belong to generate a fused data sequence corresponding to the battery cluster level.

[0062] For example, taking the extreme value calculation at the battery box level as an example, the local edge computing device compares the sampled data points belonging to the same battery box within a preset period (e.g., 15 minutes), takes the maximum and minimum values, and combines them with the corresponding timestamps to form the extreme value data of the battery box within that preset period; arranging the extreme value data of each preset period in chronological order yields the extreme value data sequence for the battery box level. The extreme value calculation at the battery cluster, battery stack, and battery compartment levels is similar to that at the battery box level, with the statistical objects being sampled data belonging to the same battery cluster, the same battery stack, and the same battery compartment, respectively.

[0063] Here, a battery box refers to a battery device composed of multiple individual battery cells connected in series and parallel; a battery cluster refers to a battery unit composed of multiple battery boxes connected in series; a battery stack refers to a battery unit composed of multiple battery clusters connected in parallel; and a battery compartment refers to a compartment-level unit that houses the battery stack and related equipment. The preset period refers to the time window for extreme value statistics, which can be set according to the data sampling frequency and application requirements. Extreme values ​​refer to the maximum and minimum values ​​of the statistical object in the sampled data within the preset period. The maximum and minimum values ​​reflect the operating boundaries of the equipment within that preset period, retaining the characteristic information required for operational monitoring while compressing the amount of data.

[0064] Preset validity rules refer to pre-defined rules used to determine whether a data point is valid, such as data values ​​exceeding the physical range or abnormal timestamps. Invalid data points are those that do not meet the validity rules. Preset special values ​​are specific values ​​in the sampled data that are conventionally designated to represent invalid or abnormal values, such as 65535. Data points with preset special values ​​are replaced with null values ​​to avoid these values ​​from being included in subsequent extreme value statistics and other calculations, which could lead to incorrect results. A fused data sequence is a battery cluster-level data sequence generated by merging the data sequences of multiple individual cells belonging to the same battery cluster. For example, the data values ​​of each individual cell within the same battery cluster at the same time can be summarized and statistically analyzed to generate a data sequence at the battery cluster level.

[0065] In one implementation, preprocessing may also include data cleaning: converting the battery box voltage according to a preset ratio, for example, dividing the original voltage value of the battery box by 1000; and replacing data points with preset special values ​​such as individual cell voltage and temperature with null values.

[0066] In this embodiment, by statistically analyzing extreme values ​​at four levels—battery box, battery cluster, battery stack, and battery compartment—the compressed extreme value data sequence can cover all management levels from the battery box to the battery compartment, meeting the monitoring needs of different levels. By removing invalid data points and replacing data points with preset special values ​​with null values, interference from abnormal data on the statistical results is avoided, improving the accuracy of uploaded data. By fusing the data sequences of individual cells at the battery cluster level, the massive sequences at the cell level are compressed into sequences at the battery cluster level, significantly reducing the number of uploaded and stored data sequences, further reducing the pressure on data transmission and storage.

[0067] In some embodiments, each target timing file is a compressed file compatible with the TSFile format; within the file body of each target timing file, the timing data is divided into multiple data block groups according to device type, and each data block group is further divided into multiple data blocks according to device. To better understand the structure of the target timing file, the following describes... Figure 3 To explain, Figure 3 This is a schematic diagram of the structure of a target timing file provided in this disclosure.

[0068] like Figure 3 As shown, the target timing file includes a check header and a file body. The specific content of the check header can be found in the following embodiment regarding the check header structure. Within the file body, the timing data is first divided into multiple data block groups according to device type. Device types may include, for example, AC / DC converters, battery stacks, battery clusters, power stations, etc. Each data block group includes a data block group header and multiple data blocks. Each data block corresponds to one device, and each data block includes a data block header and at least one data page. For example, a data block group whose device type is a battery cluster may include data blocks corresponding to each battery cluster in an energy storage power station. The data blocks corresponding to battery clusters may include data pages for measurement points such as cell voltage. Similarly, a data block group whose device type is a power station may include data pages for measurement points such as total active power and state of charge.

[0069] TSFile is a columnar storage file format for time-series data, employing a design that separates indexing and data storage. Target time-series files compatible with TSFile can be directly read and written by time-series databases that support this format. A data block group is a storage unit within the file body that organizes data according to device type; a data block is a storage unit within a data block group that organizes data according to device; and a data page is the smallest storage unit within a data block. In one implementation, the tail of the target time-series file may also include a block index, a sequence index, and a filter (e.g., a Bloom filter) to quickly locate the target data block and determine if the data sequence exists when reading the file.

[0070] In one implementation, when mapping the original time series file to the target time series file, a tree-structured time series model can be used to organize the devices and measurement points. The device identifier can be in the format of root.sg.{tableName}. Aligned time series storage can be used to store multiple measurement points of the same device to improve query and write efficiency. Data type mapping can include: mapping time type to 64-bit integer and using TS_2DIFF encoding, mapping string type to text type and using dictionary encoding and GZIP compression, and using encoding methods such as run-length encoding (RLE) for continuous values ​​such as temperature.

[0071] In one implementation, columnar compression can employ a time-series data compression algorithm based on outlier separation: by setting upper and lower outlier separation thresholds, outliers in the sampled data are separated from normal values, and the separated data are then compressed in bit width, thereby improving the compression ratio.

[0072] In this embodiment, by setting the target time series file as a compressed file compatible with the TSFile format, the cloud can directly reuse the parsing tool of this format when parsing and storing the target time series file in the database. The target time series file can be directly written to a time series database that supports this format or directly archived without format conversion. By dividing the data block group by device type and the data block by device, the data organization in the file corresponds to the equipment structure of the energy storage power station. During parsing and querying, the data of the target device can be quickly located, which improves the efficiency of data reading and distribution.

[0073] In some embodiments, a verification header is attached to each target time series file. The verification header includes a flag field, a version field, a time field, and a checksum field. The checksum field stores the Cyclic Redundancy Check 32 (CRC32) checksum of the corresponding target time series file. After receiving each target time series file, the cloud performs integrity verification on the received target time series file according to the checksum in the verification header. In response to a failed verification, the corresponding local edge computing device of the energy storage power station is triggered to retransmit the target time series file that failed the verification.

[0074] For example, after generating the target time series file, the local edge computing device calculates the CRC32 checksum of the target time series file and organizes the flag information, version information, file generation time, and CRC32 checksum into a checksum header according to a preset field order. This header is then appended to the file header of the target time series file and sent together. Upon receiving the target time series file, the cloud recalculates the CRC32 checksum of the received file and compares the calculated result with the checksum stored in the checksum field of the checksum header. If they match, the checksum passes; if they do not match, it indicates that the target time series file has been corrupted during transmission. The cloud determines that the checksum has failed and sends a retransmission request to the corresponding local edge computing device in the energy storage power station, triggering the device to retransmit the failed target time series file.

[0075] The checksum header is a data area appended to the target time-series file for identification and verification. The flag field stores information identifying the file type or protocol; the version field stores the file format version information so the cloud can parse the file according to the corresponding version's parsing rules; the time field stores the generation time of the target time-series file; and the checksum field stores the checksum of the target time-series file. The CRC32 checksum is a 32-bit checksum calculated using the CRC32 algorithm on the target time-series file. Integrity verification refers to the process of verifying whether the received file is consistent with the sent file.

[0076] In one implementation, the length of the check header can be 32 bytes, of which the flag field is 2 bytes, the version field is 2 bytes, the time field is 14 bytes, the checksum field is 8 bytes and stores the hexadecimal representation of the CRC32 checksum of the target time series file, and the remaining bytes are reserved fields that can be used for subsequent expansion.

[0077] In this embodiment, by attaching a check header including a flag field, a version field, a time field, and a checksum field to the target time series file, the cloud can quickly identify the file type, version, and generation time when receiving the file; integrity verification using a CRC32 checksum can detect data corruption that occurred during the transmission of the target time series file; by triggering the corresponding energy storage power station to retransmit when the verification fails, the integrity and reliability of the data received by the cloud are ensured, and corrupted data is prevented from entering the storage and distribution stages.

[0078] In some embodiments, the transmission protocol is the MQTT protocol; each target time series file is published with a preset topic, which includes the site identifier of the corresponding energy storage power station; encrypted transmission is achieved by superimposing a Transport Layer Security / Secure Sockets Layer (TLS / SSL) encryption layer on top of the TCP connection; in response to transmission failure or transmission interruption, the local edge computing device of the corresponding energy storage power station performs breakpoint resumption or failure retry on the target time series file that has failed to be transmitted or has been interrupted.

[0079] For example, the local edge computing device acts as an MQTT client, establishing an MQTT connection with the cloud-based message access cluster. It then publishes the target time-series file as a message to a preset topic. The preset topic is composed of a topic prefix and the corresponding energy storage power station's site identifier; for example, the preset topic could be "FTM / Site Identifier." The cloud can identify the source power station of the target time-series file based on the preset topic. Regarding transmission security, the MQTT connection between the local edge computing device and the cloud is established on top of a TCP connection, with a TLS / SSL encryption layer superimposed on top of the TCP connection to encrypt the transmitted data. In terms of transmission reliability, if transmission fails or is interrupted, the local edge computing device caches the target time-series file locally and performs resume transmission or retry upon connection restoration.

[0080] MQTT is a lightweight messaging protocol based on a publish / subscribe model, suitable for data transmission in scenarios with limited network bandwidth and unstable connections. Publishing refers to the action of an MQTT client sending a message to a specified topic. In one implementation, the Quality of Service (QoS) level for message transmission can be set to QoS1 to ensure that the message is delivered at least once. TLS / SSL encryption is a security encryption layer superimposed between the TCP connection and the application layer protocol, used for encrypting and authenticating transmitted data. Resuming interrupted transmission refers to continuing the transmission of the unfinished portion from the point of interruption; retrying on failure refers to retrying the transmission action after a failure.

[0081] In this embodiment, by employing the MQTT protocol and publishing the target time series file with a preset topic containing a site identifier, the cloud can quickly identify the data source based on the topic and uniformly access data from multiple sites. Furthermore, the lightweight nature of the MQTT protocol is suitable for scenarios with a large number of energy storage power stations and continuous data uploads from a single station. By overlaying a TLS / SSL encryption layer on top of the TCP connection, the security of data transmission is ensured. Through breakpoint resume or failure retry mechanism, reliable uploading of the target time series file is ensured in unstable network scenarios, avoiding data loss.

[0082] In one implementation, the TLS / SSL encryption layer can use the TLS 1.3 protocol. Dual authentication can also be used between the cloud and local edge computing devices, including username and password-based password authentication (e.g., verification via the built-in database of the message access cluster) and certificate-based two-way authentication (the client and server mutually verify certificates, which can be issued by an authoritative certificate authority or a self-signed certificate). The server of the message access cluster can be configured with a secure socket type listener to enable two-way authentication and enforce verification of the peer's certificate. In another implementation, the storage layer can use the Advanced Encryption Standard (AES)-256 algorithm combined with the Chinese national cryptographic algorithm SM4 to achieve dual-channel encryption for transmission and storage.

[0083] In some embodiments, the multiple consumer groups include: a producer consumer group, used to parse the target time-series file obtained from consumption in real time and write it into a production time-series database, wherein the data retention time in the production time-series database is not less than a first retention time; an analysis consumer group, used to parse the target time-series file obtained from consumption in its entirety and write it into an analysis time-series database, wherein the data retention time in the analysis time-series database is a second retention time; and an archive consumer group, used to directly archive the target time-series file obtained from consumption to a cold data warehouse; wherein the first retention time is greater than the second retention time.

[0084] For example, the producer-consumer group corresponds to the operation monitoring scenario. After consuming the target time-series file from the same topic, the producer-consumer group parses the time-series data in the file in real time and writes it to the production time-series database for real-time querying and display by the operation monitoring system. The data in the production time-series database needs to be stored long-term, with a retention period of no less than the first retention period, for example, the first retention period can be 3 years. The analysis-consumer group corresponds to the algorithm analysis scenario. After consuming the target time-series file from the same topic, the analysis-consumer group performs full parsing on the file and writes all time-series data to the analysis-time-series database for use in algorithm model training and analysis. The data retention period in the analysis-time-series database is the second retention period, for example, the second retention period can be 14 days. The archiving-consumer group corresponds to the long-term archiving scenario. After consuming the target time-series file from the same topic, the archiving-consumer group does not parse the file, but directly archives the target time-series file to the cold data warehouse in its original form, achieving long-term retention of the original data.

[0085] Among them, the production time-series database refers to a time-series database oriented towards the production operation environment, serving operational monitoring scenarios with high requirements for data real-time performance; the analysis time-series database refers to a time-series database oriented towards the data analysis environment, serving algorithm analysis scenarios; and the cold data warehouse refers to a warehouse-type storage used for long-term archiving of low-frequency access data. The first retention period and the second retention period can be set according to business needs. A longer first retention period than the second ensures that production data with high real-time requirements is available online for a long time, while analysis data is stored short-term as needed, balancing business needs and storage costs. Real-time parsing refers to the processing method of parsing and writing to the target time-series file immediately after it is obtained; full parsing refers to the processing method of parsing all time-series data in the target time-series file.

[0086] In this embodiment, by assigning different data storage scenarios to the producer consumer group, the analysis consumer group, and the archive consumer group, the same target time-series file can simultaneously meet the needs of three application scenarios: operation monitoring, algorithm analysis, and long-term archiving. By setting different data retention periods for the production time-series database and the analysis time-series database, with the first retention period being longer than the second retention period, the storage scale of the analysis data is controlled while ensuring the long-term availability of production data, thus reducing storage costs. The archive consumer group archives the files directly without parsing them, avoiding modifications to the original files during parsing and ensuring the originality and integrity of the archived data.

[0087] To better understand the overall process of data transmission and distribution in energy storage power stations, the following example illustrates the process.

[0088] First, the local edge computing device of the energy storage power station encrypts and sends the generated target time-series file to the cloud's message access cluster via a transmission protocol. The target time-series file carries the site identifier of the corresponding energy storage power station. Then, the message access cluster bridges the received target time-series file to the message queue cluster. The message queue cluster stores the target time-series files of each energy storage power station in the same topic and partitions them using the site identifier as the partition key, ensuring that the target time-series files of the same energy storage power station are located in the same partition. Next, multiple consumer groups consume the target time-series files in the same topic in parallel. At least one consumer group parses the consumed target time-series file and writes it to the time-series database, while at least another consumer group archives the consumed target time-series file to cold data storage. In addition, after receiving the target time-series file, the cloud can also perform integrity verification on the target time-series file based on the checksum in the verification header. If the verification fails, the corresponding local edge computing device of the energy storage power station is triggered to retransmit the target time-series file.

[0089] In other embodiments, the transport protocol can be replaced with encrypted transport protocols such as Hypertext Transfer Protocol Secure (HTTPS) or gRPC, while the core logic remains unchanged; the time series database can also be adapted to other types of time series databases through a unified adaptation layer encapsulation; cold data storage can also be replaced with object storage to archive target time series files.

[0090] Exemplary device Figure 4 This is a schematic diagram of the structure of a data transmission and distribution device for an energy storage power station provided in this disclosure. For example... Figure 4 As shown, the data transmission and distribution device 400 for an energy storage power station includes a receiving module 410, a storage module 420, and a parsing module 430. Wherein: The receiving module 410 is configured to receive encrypted target time series files sent by the local edge computing devices of each energy storage power station through a transmission protocol; wherein each target time series file carries the site identifier of the corresponding energy storage power station. Storage module 420 is configured as a message queue cluster to store each target time series file in the same topic, and uses the site identifier as the partition key to partition and store each target time series file in the same topic, so that the target time series files of the same energy storage power station are located in the same partition. The parsing module 430 is configured to consume target time-series files in the same topic in parallel by multiple consumer groups; wherein at least one consumer group parses the consumed target time-series files and writes them into the time-series database, and at least another consumer group archives the consumed target time-series files to cold data storage.

[0091] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section above, and will not be repeated here.

[0092] Exemplary electronic devices Figure 5 A structural diagram of an electronic device provided in an embodiment of this disclosure includes at least one processor 111 and a memory 112.

[0093] The processor 111 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 11 to perform desired functions.

[0094] The memory 112 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 111 may execute one or more computer program instructions to implement the data transmission and distribution methods and / or other desired functions of the energy storage power station described in the various embodiments of this disclosure above.

[0095] In one example, the electronic device 11 may also include an input device 113 and an output device 114, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0096] The input device 113 may also include, for example, a keyboard, a mouse, etc.

[0097] The output device 114 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0098] Of course, for the sake of simplicity, Figure 5 Only some of the components of the electronic device 11 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 11 may include any other suitable components depending on the specific application.

[0099] Exemplary computer program products and computer-readable storage media In addition to the methods and devices described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the methods for transmitting and distributing energy storage power station data according to the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0100] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0101] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods for transmitting and distributing energy storage power station data according to the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0102] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires; portable disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM) or flash memory; optical fiber; compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof.

[0103] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0104] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for transmitting and distributing data in an energy storage power station, characterized in that, include: The system receives encrypted target time-series files sent by the local edge computing devices of each energy storage power station via a transmission protocol; wherein each target time-series file carries the site identifier of the corresponding energy storage power station. The message queue cluster stores the target time-series files in the same topic, and uses the site identifier as the partition key to partition the target time-series files in the same topic, so that the target time-series files of the same energy storage power station are located in the same partition. Multiple consumer groups consume the target time-series files in the same topic in parallel; wherein at least one consumer group parses the consumed target time-series files and writes them into the time-series database, and at least another consumer group archives the consumed target time-series files to cold data storage.

2. The method according to claim 1, characterized in that, Before receiving the encrypted target time-series files sent by the local edge computing devices of each energy storage power station via the transmission protocol, the method further includes: The local edge computing devices of each energy storage power station parse the original time-series files generated by each energy storage power station to obtain original time-series data; the original time-series files are data files generated by each energy storage power station after sampling the operating data of the equipment within the station. The original time-series data are preprocessed to obtain target time-series data; the preprocessing includes extreme value calculation, invalid data filtering, and fusion of data sequences from multiple individual battery cells; The target time series data are modeled according to the equipment level of each energy storage power station, and columnar compressed target time series files are generated. The target time series files are then sent through the transmission protocol.

3. The method according to claim 2, characterized in that, The preprocessing includes: statistically analyzing the extreme values ​​of sampled data within a preset period according to four levels: battery box, battery cluster, battery stack, and battery compartment, to obtain an extreme value data sequence; removing invalid data points from the original time series data according to preset validity rules, and replacing data points with preset special values ​​with null values; and fusing the data sequences of multiple individual cells according to the battery cluster to which the individual cell belongs to generate a fused data sequence corresponding to the battery cluster level.

4. The method according to claim 2, characterized in that, Each target timing file is a compressed file compatible with the TSFile format; Within each target timing file, the timing data is divided into multiple data block groups according to the device type, and each data block group is further divided into multiple data blocks according to the device.

5. The method according to claim 2, characterized in that, A verification header is attached to each target time series file. The verification header includes a flag field, a version field, a time field, and a checksum field. The checksum field stores the CRC32 checksum of the corresponding target time series file. After receiving the target time series files, the cloud performs integrity verification on the received target time series files according to the checksum in the check header; In response to a failed verification, the corresponding local edge computing device of the energy storage power station is triggered to retransmit the target timing file that failed the verification.

6. The method according to claim 2, characterized in that, The transmission protocol is MQTT; each target time series file is published with a preset topic, and the preset topic contains the site identifier of the corresponding energy storage power station; the encrypted transmission is achieved by superimposing a TLS / SSL encryption layer on top of the TCP connection; in response to transmission failure or transmission interruption, the local edge computing device of the corresponding energy storage power station performs breakpoint resumption or failure retry on the target time series file that has failed to be transmitted or has been interrupted.

7. The method according to claim 1, characterized in that, The plurality of consumer groups include: a producer consumer group, used to parse the target time-series files obtained from consumption in real time and write them into a production time-series database, wherein the data retention period in the production time-series database is not less than a first retention period; an analysis consumer group, used to parse the target time-series files obtained from consumption in full and write them into an analysis time-series database, wherein the data retention period in the analysis time-series database is a second retention period; and an archive consumer group, used to directly archive the target time-series files obtained from consumption to a cold data warehouse; wherein the first retention period is greater than the second retention period.

8. A data transmission and distribution device for an energy storage power station, characterized in that, include: The receiving module is configured to receive encrypted target time-series files sent by the local edge computing devices of each energy storage power station via a transmission protocol; wherein each target time-series file carries the site identifier of the corresponding energy storage power station. The storage module is configured to store the target time-series files in the same topic as a message queue cluster, and to partition the target time-series files in the same topic using the site identifier as the partition key, so that the target time-series files of the same energy storage power station are located in the same partition. The parsing module is configured to consume the target time-series files in the same topic in parallel by multiple consumer groups; wherein at least one consumer group parses the consumed target time-series files and writes them into the time-series database, and at least another consumer group archives the consumed target time-series files to cold data storage.

9. A computer-readable storage medium storing a computer program for executing the data transmission and distribution method for an energy storage power station as described in any one of claims 1-7.

10. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the data transmission and distribution method for energy storage power stations as described in any one of claims 1-7.

Images