Historical data writing method and device, electronic equipment and storage medium
By calculating the optimal number of working threads and the thread mapping dictionary in lithium battery production, and using a hash function to allocate battery data to the thread queue, the problem of data storage stability and reliability in large-scale capacity testing scenarios is solved, achieving efficient and reliable data writing and storage.
Patent Information
- Application Number
- CN202511066267.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-11-21
AI Technical Summary
In the lithium battery production process, existing technologies struggle to efficiently and reliably write historical data for each battery in large-scale capacity-building scenarios, leading to system crashes, data backlogs, delays, and data loss, failing to meet the stability and reliability requirements of data storage.
By calculating the optimal number of worker threads, a thread pool and thread mapping dictionary are created. Battery data is allocated to the thread queue using a hash function, and file handles are opened as needed for data writing, ensuring that each battery corresponds to a CSV file. Multi-core CPUs are used to process data writing tasks in parallel.
It achieves system stability and data integrity in large-scale battery processing scenarios, avoids file handle limitations, ensures data order and timing, reduces the risk of data backlog and loss, and improves the efficiency and reliability of data storage.
Smart Images
Figure CN120994334A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data storage technology for lithium battery production equipment, specifically relating to a method, apparatus, electronic device, and storage medium for writing historical data. Background Technology
[0002] In the lithium battery production process, formation and capacity testing are key technological stages that determine battery performance and quality. The core task of this stage is to perform the first charge-discharge activation of the batteries and to accurately sort them according to their capacity. The various data generated during this process, such as timestamps, voltage, current, capacity, and temperature, are important bases for evaluating battery performance and ensuring production quality.
[0003] To ensure the traceability and ease of management of production data, there are strict requirements for the storage of this data: each battery generates a set of historical data every second during the formation and capacity testing process, including parameters such as timestamp, voltage, current, capacity, and temperature. This data must be appended to the corresponding CSV file in chronological order, and the principle of "one battery corresponds to one CSV file" must be strictly followed to achieve effective isolation of data from different batteries.
[0004] However, in large-scale capacity-building scenarios, when the system processes tens of thousands of batteries simultaneously, data storage faces severe challenges. Specifically, each battery generates a set of data per second, which needs to be written to an independent CSV file. This requires the system to efficiently and reliably handle massive and frequent file write operations.
[0005] Currently, existing technologies have significant bottlenecks and shortcomings in addressing the aforementioned data storage needs, making it difficult to meet the data storage requirements of large-scale, segmented storage scenarios, as follows:
[0006] (1) Create and maintain a long-term open file handle for each battery so that data can be written to the corresponding CSV file in real time.
[0007] However, in large-scale capacity testing scenarios, when the number of batteries being processed simultaneously exceeds the maximum number of file handles allowed by the operating system, data from newly connected batteries cannot be written because the corresponding file handles cannot be opened, leading to system crashes. Clearly, this solution is unsuitable for large-scale capacity testing scenarios.
[0008] (2) The entire system uses only one global file handle. All battery data is serially written to the same file through this handle, or complex internal logic is needed to distinguish the data of different batteries and write them to the corresponding files.
[0009] However, this solution suffers from extremely low write efficiency: in high-concurrency data writing scenarios, a large number of write requests will create a serious bottleneck on a single channel, failing to meet the high frequency requirement of writing per second per battery, which can easily cause data backlog, delay, or even data loss, seriously affecting the real-time performance and integrity of production data.
[0010] It does not meet storage requirements: it is difficult to achieve the strict isolation requirement of "one file per battery". To achieve this requirement, extremely complex internal logic needs to be designed to distinguish the data of different batteries and write it to the corresponding file, which not only increases the difficulty of system development, but is also prone to errors in the logic processing.
[0011] High risk of data chaos: Since all data is written to the same file, it takes a lot of time and effort to filter out the data of specific batteries from the massive amount of data when analyzing, tracing and locating faults in the later stages. This is not only inefficient, but may also lead to bias in the analysis results due to data chaos, affecting the accurate assessment of battery performance.
[0012] Therefore, a new technical solution is urgently needed to solve the above problems in order to ensure the stability, efficiency and reliability of data storage during the large-scale batch production process. Summary of the Invention
[0013] To overcome the shortcomings of the prior art, the present invention provides a method, apparatus, electronic device and storage medium for writing historical data, aiming to ensure the stability, efficiency and reliability of data storage during large-scale batch production.
[0014] The technical means adopted by this invention to solve its technical problem is: a method for writing historical data, the improvement of which includes:
[0015] Step S1: Obtain system resource parameters and calculate the optimal number of worker threads T based on the current number of physical CPU cores P of the computer and the maximum number of file handles H allowed by the operating system;
[0016] Step S2: Create TH worker threads according to the optimal number of worker threads, each thread having a capacity of TH. i Create a dedicated first-in-first-out memory data queue Q for each thread. i ,i∈(1,…,T);
[0017] Step S3: Create a global thread mapping dictionary; the identifier key of the thread mapping dictionary is an integer, with a value range from 0 to T-1; the unique associated data value of the thread mapping dictionary is the corresponding thread object TH. i and its queue Q i ;
[0018] Step S4: Receive battery data packets from the capacity formation system in real time, combine the key identification information in the battery data packets into a unique string, calculate the assignment key value R of the unique string, and query the thread mapping dictionary based on the assignment key value R.
[0019] Step S5: Based on the fact that the dispatch key R exists in the thread mapping dictionary, directly obtain the associated thread TH. i and queue Q i Write all the data from the battery data packet into Q. i At the end, start all TH worker threads, each thread TH i Execute the loop and listen to the dedicated queue Q. i Based on the dedicated queue Q i If not empty, write the data in the battery data package to the unique CSV file corresponding to the battery data package and close immediately.
[0020] Step S1 in the above technical solution includes:
[0021] Call the system API to obtain the current number of physical CPU cores P and the maximum number of file handles H allowed by the operating system;
[0022] The optimal number of worker threads T is calculated based on P and H, where T = min(P,H)-1.
[0023] Step S4 in the above technical solution includes:
[0024] Step S401: Receive a battery data packet from the formation and capacity testing system in real time; the data packet includes: timestamp, voltage, current, capacity, temperature, and key identification information: device ID and battery channel ID;
[0025] Step S402: Combine the identification information into a unique string KeyStr = DeviceID + "_" + ChannelID, and use a 32-bit hash function to calculate the hash value HA of KeyStr;
[0026] Step S403: Calculate the assigned key value R = HA%T based on the hash value HA of the unique string KeyStr, where % represents the modulo operation, and the assigned key value R is in the range [0, T-1].
[0027] Step S404: Using the dispatch key value R as a clue and querying the thread mapping dictionary, an existence result is obtained; the existence result is used to indicate whether the dispatch key value R exists in the thread mapping dictionary.
[0028] In the above technical solution, step S5 is based on the dedicated queue Q.i If not empty, the data in the battery data package will be written to the unique CSV file corresponding to the battery data package, including:
[0029] Step S501: From the dedicated queue Q i A battery data packet is extracted from the header and parsed to obtain the DeviceID, ChannelID, and measurement data from the data packet.
[0030] Step S502: Construct a unique CSV file corresponding to the battery based on the DeviceID and ChannelID;
[0031] Step S503: In response to opening the file handle, open the CSV file in append mode, append the timestamp, voltage, current, capacity, and temperature data in the data packet to the opened CSV file in CSV format, and immediately close the opened file handle.
[0032] The hash functions mentioned in the above technical solutions include, but are not limited to, MurmurHash3,FNV-1a.
[0033] In step S5 of the above technical solution, before writing the battery data packet to the CSV file, the data packet is checked for integrity, including checking the rationality of the timestamp and the validity of the voltage, current and temperature value ranges. If the check fails, the data packet is marked as abnormal and transferred to the error queue, and an alarm mechanism is triggered to notify the operation and maintenance personnel.
[0034] The technical means adopted by this invention to solve its technical problem is: a historical data writing device, the device comprising:
[0035] The thread calculation module is used to obtain system resource parameters and calculate the optimal number of worker threads T based on the current number of physical CPU cores P of the computer and the maximum number of file handles H allowed by the operating system.
[0036] The thread pool creation module is used to create TH worker threads according to the optimal number of worker threads, with each thread having a value of TH. i Create a dedicated first-in-first-out memory data queue Q for each thread. i ,i∈(1,…,T);
[0037] The thread mapping creation module is used to create a global thread mapping dictionary; the identifier key of the thread mapping dictionary is an integer, with a value range from 0 to T-1; the unique associated data value of the thread mapping dictionary is the corresponding thread object TH. i and its queue Q i ;
[0038] The mapping query module is used to receive battery data packets from the capacity formation system in real time, combine the key identification information in the battery data packets into a unique string, calculate the assignment key value R of the unique string, and query the thread mapping dictionary based on the assignment key value R.
[0039] The data writing module is used to directly retrieve the associated thread TH based on the fact that the dispatch key R exists in the thread mapping dictionary. i and queue Q i Write all the data from the battery data packet into Q. i At the end, start all TH worker threads, each thread TH i Execute the loop and listen to the dedicated queue Q. i Based on the dedicated queue Q i If not empty, write the data in the battery data package to the unique CSV file corresponding to the battery data package and close immediately.
[0040] The technical means adopted by this invention to solve its technical problem is: an electronic device, comprising: at least one processor and at least one memory, wherein,
[0041] The memory stores program instructions or code;
[0042] The program instructions or code are loaded and executed by the processor, enabling the electronic device to implement the historical data writing method as described in any of the preceding claims.
[0043] The technical means adopted by the present invention to solve its technical problem is: a storage medium storing program instructions or code thereon, wherein the program instructions or code are loaded and executed by a processor to implement the historical data writing method as described in any of the above claims.
[0044] The beneficial effects of this invention are:
[0045] ① By adopting a strategy of opening file handles on demand and closing them immediately after writing, and limiting the number of threads working at the same time (T is strictly controlled based on the number of CPU cores P and the maximum number of file handles H), the system ensures that the number of file handles opened at any time does not exceed T, and T is much smaller than H (T is usually a few to tens, while H is thousands or even tens of thousands). This completely avoids the operating system's file handle limit and ensures system stability in large-scale battery processing scenarios.
[0046] ② The data writing task is processed in parallel by using T (usually close to the number of CPU cores) worker threads, which makes full use of the computing power of multi-core CPUs; the data of different batteries is relatively evenly distributed to each thread queue by hash modulo (HA%T), avoiding single-thread bottleneck.
[0047] ③ By constructing filenames using a unique identifier (DeviceID+ChannelID), the requirement of "one CSV file for one battery" is strictly implemented. The data within each file is written in the order of reception (i.e., enqueueing order) through a queue mechanism, satisfying the data ordering requirement. Hash distribution is based on a fixed identifier, ensuring that all data from the same battery is always allocated to the same thread queue, thus writing to the same file, which also implicitly guarantees the ordering of data within that file. Attached Figure Description
[0048] Figure 1 This is a flowchart illustrating a historical data writing method according to an embodiment of the present invention;
[0049] Figure 2 This is a flowchart illustrating step S4 of an embodiment of the present invention;
[0050] Figure 3 This is a flowchart illustrating another method for step S4 in an embodiment of the present invention;
[0051] Figure 4 This is a flowchart illustrating step S5 of a method according to an embodiment of the present invention.
[0052] Figure 5 This is a structural block diagram of a historical data writing device according to an embodiment of the present invention;
[0053] Figure 6 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0054] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0055] The following will clearly and completely describe the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this invention can be combined interactively without contradicting each other.
[0056] like Figure 1 As shown, the present invention provides a method for writing historical data, including:
[0057] Step S1: Obtain system resource parameters and calculate the optimal number of worker threads T based on the current number of physical CPU cores P of the computer and the maximum number of file handles H allowed by the operating system.
[0058] Specifically, the system API is called to obtain the current number of physical CPU cores P of the computer and the maximum number of file handles H allowed by the operating system. A file handle (or file descriptor) is a unique identifier (usually an integer) assigned by the operating system kernel to each file or resource opened by a process. Processes use file handles to perform read and write operations on files, and the operating system has an upper limit (H) on the number of file handles that a single process can hold simultaneously.
[0059] The optimal number of worker threads T is calculated based on P and H, where T = min(P,H)-1.
[0060] Where min(P,H) represents the minimum value between the number of physical CPU cores P and the maximum number of file handles H, the optimal number of worker threads T is the minimum value minus 1.
[0061] The above steps ensure that the number of file handles opened simultaneously at any given time does not exceed T by limiting the number of threads working at the same time (T is strictly controlled based on the number of CPU cores P and the maximum number of file handles H). Furthermore, T is much smaller than H (T is usually in the range of a few to tens, while H is in the range of thousands or even tens of thousands). This completely avoids the operating system's file handle limit and ensures system stability in large-scale battery processing scenarios.
[0062] Step S2: Create TH worker threads according to the optimal number of worker threads to form a thread pool; each thread TH i Create a dedicated first-in-first-out memory data queue Q for each thread. i ,i∈(1,…,T).
[0063] A thread is the smallest unit of computation that an operating system can schedule. It is contained within a process and is the actual unit of operation within a process. A process can contain multiple threads that share memory space. A thread pool is a form of multithreaded processing that pre-creates a group of threads and assigns them to idle threads when tasks arrive, avoiding the overhead of frequently creating and destroying threads.
[0064] Step S3: Create a global thread mapping dictionary Dict; the identifier key of the thread mapping dictionary Dict is an integer, with a value range from 0 to T-1; the unique associated data value of the thread mapping dictionary is the corresponding thread object TH. i and its queue Q iDuring initialization, the thread mapping dictionary Dict is empty.
[0065] Queue: A first-in-first-out (FIFO) data structure. Elements are added from the tail and removed from the head. In-Memory Queue: A queue data structure stored in memory (RAM).
[0066] Step S4: Receive battery data packets from the capacity formation system in real time, combine the key identification information in the battery data packets into a unique string, calculate the assignment key value R of the unique string, and query the thread mapping dictionary based on the assignment key value R.
[0067] In one possible implementation, such as Figure 2 As shown, step S4 includes the following steps:
[0068] Step S401: Receive a battery data packet from the formation and capacity testing system in real time; the data packet includes: timestamp, voltage, current, capacity, temperature, and key identification information: device ID and battery channel ID;
[0069] Step S402: Combine the identification information into a unique string KeyStr = DeviceID + "_" + ChannelID, and calculate the hash value HA of KeyStr using a 32-bit hash function. The hash function maps input data of arbitrary length to a fixed-length (e.g., 32-bit) unique or near-unique output value.
[0070] Optionally, the 32-bit hash function includes, but is not limited to, MurmurHash3,FNV-1a.
[0071] Step S403: Calculate the assigned key value R = HA%T based on the hash value HA of the unique string KeyStr, where % represents the modulo operation. In this embodiment, it means taking the remainder when the integer HA is divided by the integer T. The assigned key value R is in the range of [0, T-1].
[0072] Step S404: Using the dispatch key value R as a clue and querying the thread mapping dictionary, an existence result is obtained; the existence result is used to indicate whether the dispatch key value R exists in the thread mapping dictionary.
[0073] In the above embodiments, the data of different batteries is distributed relatively evenly to each thread queue by hash modulo (HA%T), thus avoiding the single-thread bottleneck.
[0074] In one possible implementation, such as Figure 3As shown, step S4 further includes:
[0075] Step S405: If the existence result indicates that the dispatch key value R does not exist in the thread mapping dictionary, it means that this is the first time the dispatch key value R has been encountered. Even so, the dispatch key value R does not need to create a new thread, because the thread pool has already been created in step S3.
[0076] Based on this, the assignment key R is added as an identifier Key to the thread mapping dictionary, and its unique associated data value is associated with the thread TH pre-assigned for the assignment key R in step S2. i and its queue Q i It's worth noting that the threads and queues were already created in steps S2 and S3 during initialization; this step simply establishes the mapping relationship.
[0077] Step S5: Based on the fact that the dispatch key R exists in the thread mapping dictionary, directly obtain the associated thread TH. i and queue Q i Write all the data from the battery data packet into Q. i At the end, start all TH worker threads, each thread TH i Execute the loop and listen to the dedicated queue Q. i Based on the dedicated queue Q i If not empty, write the data in the battery data package to the unique CSV file corresponding to the battery data package and close immediately.
[0078] Data can be quickly placed into the memory queue and returned without waiting for disk I / O to complete, ensuring high efficiency in receiving data. Worker threads focus on disk writing, and this producer-consumer model effectively decouples data receiving and storage.
[0079] The battery data package contains identification information and measurement data.
[0080] In one possible implementation, such as Figure 4 As shown, the dedicated queue Q is used as a basis for... i If not empty, the data in the battery data package will be written to the unique CSV file corresponding to the battery data package, including:
[0081] Step S501: From the dedicated queue Q i A battery data packet is extracted from the header and parsed to obtain the DeviceID, ChannelID, and measurement data from the data packet.
[0082] Step S502: Construct a unique CSV file corresponding to the battery based on the DeviceID and ChannelID;
[0083] Step S503: In response to opening the file handle, open the CSV file in append mode; if the file does not exist, create a new one; append the timestamp, voltage, current, capacity, and temperature data from the data packet to the opened CSV file in CSV format and immediately close the opened file handle.
[0084] The above embodiments construct filenames using a unique identifier (DeviceID + ChannelID), strictly implementing the requirement of "one CSV file per battery." Data within each file is written in the order of reception (i.e., enqueue order) through a queue mechanism, satisfying data ordering requirements. Hash dispatch based on a fixed identifier ensures that all data from the same battery is always allocated to the same thread queue, thus writing to the same file. This also implicitly guarantees the order of data within that file, avoiding system-level errors caused by handle exhaustion and reducing the risk of data backlog and loss due to write bottlenecks.
[0085] In one possible implementation, in step S5, before writing the battery data packet to the CSV file, the data packet is checked for integrity, including checking the validity of the timestamp, voltage, current, and temperature value ranges. If the check fails, the data packet is marked as abnormal and transferred to the error queue, and an alarm mechanism is triggered to notify the maintenance personnel.
[0086] By verifying the rationality of timestamps (such as whether the time is in reverse order or within a reasonable time range) and whether physical quantities such as voltage, current, and temperature conform to the battery's operating characteristics (such as whether the voltage exceeds the safety threshold or whether the temperature is abnormal), invalid data caused by sensor failures, communication interference, or software errors can be filtered out. This ensures that subsequent data analysis (such as battery performance evaluation and fault diagnosis) is based on accurate and reliable data sources, thereby improving the reliability of decision-making.
[0087] It should be understood that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart above may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0088] The following are embodiments of the apparatus described in this application, which can be used to execute the historical data writing method involved in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of the historical data writing method involved in this application.
[0089] Please see Figure 5 This application provides a historical data writing device 60, which includes: a thread calculation module 601, a thread pool creation module 602, a thread mapping creation module 603, a mapping query module 604, and a data writing module 605.
[0090] The thread calculation module 601 is used to obtain system resource parameters and calculate the optimal number of working threads T based on the current number of physical CPU cores P of the computer and the maximum number of file handles H allowed by the operating system.
[0091] Thread pool creation module 602 is used to create TH worker threads according to the optimal number of worker threads, each thread having a value of TH. i Create a dedicated first-in-first-out memory data queue Q for each thread. i ,i∈(1,…,T);
[0092] The thread mapping creation module 603 is used to create a global thread mapping dictionary; the identifier key of the thread mapping dictionary is an integer, with a value range from 0 to T-1; the unique associated data value of the thread mapping dictionary is the corresponding thread object TH. i and its queue Q i ;
[0093] The mapping query module 604 is used to receive battery data packets from the capacity formation system in real time, combine the key identification information in the battery data packets into a unique string, calculate the assignment key value R of the unique string, and query the thread mapping dictionary based on the assignment key value R.
[0094] Data writing module 605 is used to directly retrieve the associated thread TH based on the fact that the dispatch key value R exists in the thread mapping dictionary. i and queue Q i Write all the data from the battery data packet into Q. i At the end, start all TH worker threads, each thread TH i Execute the loop and listen to the dedicated queue Q. i Based on the dedicated queue Q i If not empty, write the data in the battery data package to the unique CSV file corresponding to the battery data package and close immediately.
[0095] It should be noted that the historical data writing device provided in the above embodiments is only illustrated by the division of the above functional modules when writing historical data to a fractional capacity system. In practical applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the historical data writing device will be divided into different functional modules to complete all or part of the functions described above. The above modules can be embedded in hardware or independent of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to the above modules.
[0096] Furthermore, the historical data writing device and the historical data writing method provided in the above embodiments belong to the same concept, and the specific way in which each module performs its operation has been described in detail in the method embodiments, and will not be repeated here.
[0097] Please see Figure 6 This application provides an electronic device 4000.
[0098] exist Figure 6 In this design, data interaction between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. This communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0099] Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.
[0100] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc. The memory 4003 may be a ROM (Read-Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or it may be an EEPROM (Electrically Erasable Programmable Read-Only Memory), a CD-ROM (Compact Disc Read-Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program instructions or code in the form of instructions or data structures and accessible by the electronic device 400, but not limited thereto.
[0101] The memory 4003 stores program instructions or code, and the processor 4001 can read the program instructions or code stored in the memory 4003 through the communication bus 4002.
[0102] When the program instructions or code are executed by the processor 4001, the historical data writing method in the above embodiments is implemented.
[0103] Furthermore, this application provides a storage medium storing program instructions or code, which is loaded and executed by a processor to implement the historical data writing method as described above.
[0104] This application provides a computer program product, which includes program instructions or code. The program instructions or code are stored in a storage medium. The processor of an electronic device reads the program instructions or code from the storage medium, loads and executes the program instructions or code, so that the electronic device implements the historical data writing method described above.
[0105] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A method for writing historical data, characterized in that, include: Step S1: Obtain system resource parameters and calculate the optimal number of worker threads T based on the current number of physical CPU cores P of the computer and the maximum number of file handles H allowed by the operating system; Step S2: Create TH worker threads according to the optimal number of worker threads, each thread having a capacity of TH. i Create a dedicated first-in-first-out memory data queue Q for each thread. i ,i∈(1,…,T); Step S3: Create a global thread mapping dictionary; the identifier key of the thread mapping dictionary is an integer, with a value range from 0 to T-1; the unique associated data value of the thread mapping dictionary is the corresponding thread object TH. i and its queue Q i ; Step S4: Receive battery data packets from the capacity formation system in real time, combine the key identification information in the battery data packets into a unique string, calculate the assignment key value R of the unique string, and query the thread mapping dictionary based on the assignment key value R. Step S5: Based on the fact that the dispatch key R exists in the thread mapping dictionary, directly obtain the associated thread TH. i and queue Q i Write all the data from the battery data packet into Q. i At the end, start all TH worker threads, each thread TH i Execute the loop and listen to the dedicated queue Q. i Based on the dedicated queue Q i If not empty, write the data in the battery data package to the unique CSV file corresponding to the battery data package and close immediately.
2. The historical data writing method according to claim 1, characterized in that, Step S1 includes: Call the system API to obtain the current number of physical CPU cores P and the maximum number of file handles H allowed by the operating system; The optimal number of worker threads T is calculated based on P and H, where T = min(P,H)-1.
3. The historical data writing method according to claim 1, characterized in that, Step S4 includes: Step S401: Receive a battery data packet from the formation and capacity testing system in real time; the data packet includes: timestamp, voltage, current, capacity, temperature, and key identification information: device ID and battery channel ID; Step S402: Combine the identification information into a unique string KeyStr = DeviceID + "_" + Channel ID, and use a 32-bit hash function to calculate the hash value HA of KeyStr; Step S403: Calculate the assigned key value R = HA%T based on the hash value HA of the unique string KeyStr, where % represents the modulo operation, and the assigned key value R is in the range [0, T-1]. Step S404: Using the dispatch key value R as a clue and querying the thread mapping dictionary, an existence result is obtained; the existence result is used to indicate whether the dispatch key value R exists in the thread mapping dictionary.
4. The historical data writing method according to claim 3, characterized in that, Step S4 further includes: If the existence result indicates that the dispatch key R does not exist in the thread mapping dictionary, then the dispatch key R is added to the thread mapping dictionary as an identifier Key, and its unique associated data value is associated with the thread TH pre-assigned for the dispatch key R in step S2. i and its queue Q i .
5. The historical data writing method according to claim 1, characterized in that, In step S5, based on the dedicated queue Q i If not empty, the data in the battery data package will be written to the unique CSV file corresponding to the battery data package, including: Step S501: From the dedicated queue Q i A battery data packet is extracted from the header and parsed to obtain the DeviceID, ChannelID, and measurement data from the data packet. Step S502: Construct a unique CSV file corresponding to the battery based on the DeviceID and ChannelID; Step S503: In response to opening the file handle, open the CSV file in append mode, append the timestamp, voltage, current, capacity, and temperature data in the data packet to the opened CSV file in CSV format, and immediately close the opened file handle.
6. The historical data writing method according to claim 3, characterized in that, The hash functions include, but are not limited to, MurmurHash3,FNV-1a.
7. The historical data writing method according to claim 1, characterized in that, In step S5, before writing the battery data packet to the CSV file, the data packet is checked for integrity, including checking the validity of the timestamp, voltage, current, and temperature range. If the check fails, the data packet is marked as abnormal and transferred to the error queue, and an alarm mechanism is triggered to notify the maintenance personnel.
8. A historical data writing device, characterized in that, The device includes: The thread calculation module is used to obtain system resource parameters and calculate the optimal number of worker threads T based on the current number of physical CPU cores P of the computer and the maximum number of file handles H allowed by the operating system. The thread pool creation module is used to create TH worker threads according to the optimal number of worker threads, with each thread having a value of TH. i Create a dedicated first-in-first-out memory data queue Q for each thread. i ,i∈(1,…,T); The thread mapping creation module is used to create a global thread mapping dictionary; the identifier key of the thread mapping dictionary is an integer, with a value range from 0 to T-1; the unique associated data value of the thread mapping dictionary is the corresponding thread object TH. i and its queue Q i ; The mapping query module is used to receive battery data packets from the capacity formation system in real time, combine the key identification information in the battery data packets into a unique string, calculate the assignment key value R of the unique string, and query the thread mapping dictionary based on the assignment key value R. The data writing module is used to directly retrieve the associated thread TH based on the fact that the dispatch key R exists in the thread mapping dictionary. i and queue Q i Write all the data from the battery data packet into Q. i At the end, start all TH worker threads, each thread TH i Execute the loop and listen to the dedicated queue Q. i Based on the dedicated queue Q i If not empty, write the data in the battery data package to the unique CSV file corresponding to the battery data package and close immediately.
9. An electronic device, characterized in that, include: At least one processor, at least one memory, wherein, The memory stores program instructions or code; The program instructions or code are loaded and executed by the processor, causing the electronic device to implement the historical data writing method as described in any one of claims 1 to 7.
10. A storage medium storing program instructions or code thereon, characterized in that, The program instructions or code are loaded and executed by the processor to implement the historical data writing method as described in any one of claims 1 to 7.