Storage battery monitoring control method, model training method and storage medium

Abstract

The invention provides a storage battery monitoring control method, a model training method and a storage medium, and relates to the technical field of communication, and the method can achieve the precise analysis and processing of the fault problem of a storage battery. The method comprises the following steps: acquiring state data of a battery chamber; wherein the battery chamber comprises at least one storage battery pack, and the storage battery pack comprises at least one single storage battery; inputting the state data of the battery chamber into the trained monitoring control model to obtain an action instruction of the battery chamber; the action instruction is used for indicating and adjusting the state of the storage battery pack and / or the single storage battery; wherein the monitoring control model is used for determining whether the storage battery pack and / or the single storage batteries break down or not based on the state data of the battery chamber, and determining an action instruction of the battery chamber under the condition that the storage battery pack and / or the single storage batteries break down; and issuing an action instruction.

Description

Technical Field

[0001] This application relates to the field of communication technologies, and in particular, to a method for monitoring and controlling a storage battery, a method for model training, and a storage medium. Background Art

[0002] In recent years, the integration of AI large model training and inference and cloud technologies has promoted the large-scale construction of data centers. In the communication industry, data centers are the core of business systems, and secure power supply is crucial. As the communication power supply reserve during power outage intervals, storage batteries are the key to the stable operation of communication equipment. Problems with storage battery power supply can affect the stable operation of IT equipment and pose potential safety hazards to business data operation, transmission, storage, and the reliable operation of systems.

[0003] Existing storage battery-related technologies mainly focus on predicting the capacity of storage batteries and online monitoring of abnormal storage battery groups. However, the fault handling operation process after detecting an abnormal storage battery group is not yet perfect, and the detection and fault location of storage battery groups are not accurate enough. Summary of the Invention

[0004] This application provides a method for monitoring and controlling a storage battery, a method for model training, and a storage medium, which improve the fault handling operation process after detecting an abnormal storage battery group and achieve accurate detection and fault location of storage battery groups.

[0005] In a first aspect, this application provides a method for monitoring and controlling a storage battery, including: obtaining status data of a battery room; where the battery room includes at least one storage battery group, and the storage battery group includes at least one single storage battery; inputting the status data of the battery room into a trained monitoring and control model to obtain an action instruction for the battery room; the action instruction is used to indicate adjusting the status of the storage battery group and / or the single storage battery; where the monitoring and control model is used to determine whether a fault occurs in the storage battery group and / or the single storage battery based on the status data of the battery room, and in the case of a fault, determine the action instruction for the battery room; and issuing the action instruction.

[0006] It can be understood that in existing storage battery-related technologies, the main focus is more on storage battery groups and there is a lack of a fault handling process after detecting an abnormal storage battery group. However, the method for monitoring and controlling a storage battery provided in this application can real-time monitor the status data of battery groups and single storage batteries in a battery room, and use the monitoring and control model of the storage battery to output the action instruction under this status data, improving the fault handling process of abnormal storage batteries, and the main focus and the issued action instruction are accurate to single storage batteries, achieving accurate detection and fault location of storage battery groups.

[0007] A possible implementation manner is that the status data includes: electrical characteristic data, operating status data, and environmental characteristic data.

[0008] In another possible implementation manner, after issuing an action instruction, the method further includes:

[0009] Receiving execution feedback information of the action instruction; the execution feedback information is used to indicate the execution situation of the action instruction and the state data of the battery room after the execution of the action instruction;

[0010] In the case where the execution feedback information indicates that the action instruction has been executed and the battery room has returned to the normal operation state, storing the state data of the battery room, the action instruction of the battery room, and the execution feedback information in the historical database; wherein, the data in the historical database is used for model training.

[0011] In yet another possible implementation manner, the monitoring and control model is trained based on the historical state data and historical action instructions of the battery room, wherein the historical state data serves as sample data and the historical action instructions serve as supervision information.

[0012] In a second aspect, the present application provides a model training method, the method including: obtaining the historical state data and historical action instructions of the battery room; wherein, the battery room includes at least one battery pack, and the battery pack includes at least one single battery; training the monitoring and control model to be trained based on the historical state data and historical action instructions to obtain a trained monitoring and control model; wherein, the monitoring and control model is used to determine whether a fault occurs in the battery pack and / or the single battery based on the state data of the battery room, and in the case of a fault, determining the action instruction of the battery room; the action instruction is used to indicate adjusting the state of the battery pack and / or the single battery.

[0013] It can be understood that the model training method provided by the present application, through training by using the historical state data and historical action instructions of the battery pack and single batteries in the battery room, obtains a monitoring and control model that can determine the optimal action instruction based on the real-time monitored state data. This model provides strong support for the precise regulation of single batteries, improves the intelligent level of battery room management, realizes the optimization of resource utilization, and enhances the stability of the entire battery supply system.

[0014] In a possible implementation manner, the monitoring and control model to be trained includes: a policy network, a target policy network, a value network, and a target value network; wherein, the target policy network is a copy network of the policy network; the target value network is a copy network of the value network; the target policy network, the value network, and the target value network are used to train the policy network to obtain a trained policy network; the trained monitoring and control model includes the trained policy network.

[0015] Another possible implementation method is to train the monitoring and control model to be trained based on historical state data and historical action instructions, including: inputting the first historical state data into the policy network to obtain the first action instruction for the battery room; inputting the first action instruction into the value network to obtain the value evaluation value of the first action instruction, where the value evaluation value is used to reflect the degree of benefit of the execution of the action instruction to the battery room; and adjusting the network parameters of the policy network based on the first value evaluation value.

[0016] Another possible implementation method, the method further includes: obtaining the second historical state data of the battery room after the execution of the first action instruction; inputting the second historical state data into the target policy network to obtain the second action instruction for the battery room; inputting the second action instruction into the target value network to obtain the value evaluation value of the second action instruction; and adjusting the network parameters of the value network based on the difference between the value evaluation value of the first action instruction and the value evaluation value of the second action instruction.

[0017] In a third aspect, the present application provides a battery monitoring and control device, which is applied to an intelligent control terminal of a data center. The device includes: an acquisition module, a processing module, and a sending module; the acquisition module is used to acquire the state data of the battery room, where the battery room includes at least one battery pack, and the battery pack includes at least one single battery; the processing module is used to input the state data of the battery room into the trained monitoring and control model to obtain the action instruction of the battery room, and the action instruction is used to indicate adjusting the state of the battery pack and / or the single battery; where the monitoring and control model is used to determine whether the battery pack and / or the single battery fails based on the state data of the battery room, and in the case of a failure, determine the action instruction of the battery room; the sending module is used to issue the action instruction.

[0018] In a possible implementation method, the state data includes: electrical characteristic data, operating state data, and environmental characteristic data.

[0019] In another possible implementation method, after issuing the action instruction, the acquisition module is further used to: receive the execution feedback information of the action instruction; the execution feedback information is used to indicate the execution situation of the action instruction and the state data of the battery room after the execution of the action instruction; in the case where the execution feedback information indicates that the action instruction has been executed and the battery room has returned to the normal operating state, the processing module is further used to: store the state data of the battery room, the action instruction of the battery room, and the execution feedback information in the historical database; where the data in the historical database is used for model training.

[0020] In another possible implementation method, the monitoring and control model is trained based on the historical state data and historical action instructions of the battery room, where the historical state data is used as sample data and the historical action instructions are used as supervision information.

[0021] Fourth aspect, the present application provides a model training device, which includes: an acquisition module and a training module; the acquisition module is configured to: acquire historical state data and historical action instructions of a battery chamber; wherein, the battery chamber includes at least one battery pack, and the battery pack includes at least one single battery; the training module is configured to: train a monitoring and control model to be trained based on the historical state data and historical action instructions to obtain a trained monitoring and control model; wherein, the monitoring and control model is configured to determine whether a battery pack and / or a single battery fails based on the state data of the battery chamber, and determine an action instruction of the battery chamber in the event of a failure; the action instruction is used to indicate adjusting the state of the battery pack and / or the single battery.

[0022] A possible implementation, the monitoring and control model to be trained includes: a policy network, a target policy network, a value network, and a target value network; wherein, the target policy network is a copy network of the policy network; the target value network is a copy network of the value network; the target policy network, the value network, and the target value network are used to train the policy network to obtain a trained policy network; the trained monitoring and control model includes the trained policy network.

[0023] Another possible implementation, the training module is specifically configured to: input the first historical state data into the policy network to obtain a first action instruction of the battery chamber; input the first action instruction into the value network to obtain a value evaluation value of the first action instruction, and the value evaluation value is used to reflect the degree of benefit of the execution of the action instruction to the battery chamber; adjust the network parameters of the policy network based on the first value evaluation value.

[0024] Yet another possible implementation, the training module is further configured to: acquire second historical state data of the battery chamber after the execution of the first action instruction; input the second historical state data into the target policy network to obtain a second action instruction of the battery chamber; input the second action instruction into the target value network to obtain a value evaluation value of the second action instruction; adjust the network parameters of the value network based on the difference between the value evaluation value of the first action instruction and the value evaluation value of the second action instruction.

[0025] Fifth aspect, the present application provides an electronic device, which includes: a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method of the first aspect or the second aspect described above.

[0026] Sixth aspect, the present application provides a chip system, which is applied to a battery monitoring and control device or a model training device; the chip system includes one or more interface circuits and one or more processors. The interface circuits and the processors are interconnected by lines; the interface circuits are configured to receive signals from the memory of the battery monitoring and control device or the model training device and send signals to the processors of the battery monitoring and control device or the model training device, and the signals include software instructions stored in the memory. When the processors execute the software instructions, the electronic device is caused to execute the method of the first aspect or the second aspect described above.

[0027] Seventh aspect, the present application provides a readable storage medium, which includes: software instructions; when the software instructions run in an electronic device, the electronic device is caused to implement the method of the first aspect or the second aspect described above.

[0028] Eighth aspect, the present application provides a computer program product, when the computer program product runs on an electronic device, the electronic device is caused to execute the steps of the related method described in the first aspect or the second aspect above to implement the method of the first aspect or the second aspect.

[0029] For the beneficial effects of the third aspect to the eighth aspect described above, reference may be made to the corresponding descriptions of the first aspect or the second aspect, and details are not repeated here. Description of the Drawings

[0030] Figure 1 It is a schematic diagram of the application environment of a battery monitoring and control method provided by the present application;

[0031] Figure 2 It is a schematic diagram of the architecture of a battery chamber acquisition terminal provided by the present application;

[0032] Figure 3 It is a schematic diagram of the architecture of a control room intelligent terminal provided by the present application;

[0033] Figure 4 It is a schematic diagram of the architecture of a battery chamber control terminal provided by the present application;

[0034] Figure 5 It is a schematic diagram of the flow of a battery monitoring and control method provided by the present application;

[0035] Figure 6 It is a schematic diagram of the flow of another battery monitoring and control method provided by the present application;

[0036] Figure 7 It is a schematic diagram of the flow of a model training method provided by the present application;

[0037] Figure 8 It is a schematic diagram of the architecture of a model training provided by the present application;

[0038] Figure 9 A schematic flowchart of another battery monitoring and control method provided for this application;

[0039] Figure 10 A schematic flowchart of another battery monitoring and control method provided for this application;

[0040] Figure 11 A schematic diagram of the composition of a battery monitoring and control device provided for this application;

[0041] Figure 12 A schematic diagram of the composition of a model training device provided for this application;

[0042] Figure 13 A schematic diagram of the composition of an electronic device provided for this application. Detailed implementation manners

[0043] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0044] It should be noted that in the embodiments of this application, words such as "exemplarily" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific manner.

[0045] In order to facilitate a clear description of the technical solutions in the embodiments of this application, in the embodiments of this application, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the words such as "first" and "second" do not limit the quantity and execution order.

[0046] In recent years, with the rapid development of artificial intelligence (AI), the urgent demand for data computing power in the training and inference of complex AI large models and the integration of artificial intelligence and cloud computing technologies has driven the large-scale construction of data centers. As the core area of each business system in the communication industry, a safe and reliable power supply is a prerequisite for maintaining the normal operation of the system. As an important communication power supply reserve during the power outage interval from the interruption of the mains power supply to the start of the generator set, the storage battery is an important guarantee for the normal operation of each communication device. Once a power supply problem occurs in the battery pack, it will affect the stable operation of IT equipment and pose a safety hazard to the operation, transmission, storage of business data and the reliable operation of the system.

[0047] The existing battery-related technologies mainly focus on predicting the capacity of the storage battery and on-line monitoring of abnormal battery packs. However, the fault handling operation process after detecting an abnormal battery pack is not yet perfect, and the detection and fault location of the battery pack are not accurate.

[0048] Based on this, the embodiment of the present application provides a battery monitoring and control method. Different from the existing battery-related technologies that focus on battery packs, this method is based on the status data of the battery room, uses a monitoring and control model to output action instructions under the status data and issues the action instructions, and can achieve precise point-to-point control of the detected abnormal single battery, realizing precise detection and fault location of the battery pack, and ensuring the stable operation of the power supply in the battery room.

[0049] The battery monitoring and control method provided by the present application can be applied to a Figure 1 battery monitoring and control system as shown. As Figure 1 shown, the battery monitoring and control system includes: a battery room acquisition terminal 100, a control room intelligent terminal 200, and a battery room control terminal 300. Among them, the control room intelligent terminal 200 is respectively connected to the battery room acquisition terminal 100 and the battery room control terminal 300.

[0050] The battery room acquisition terminal 100 is used to collect the status data of each battery room and send the collected data to the control room intelligent terminal 200.

[0051] Among them, the battery room includes at least one battery pack, and the battery pack includes at least one single battery.

[0052] In some embodiments, the battery room acquisition terminal 100 is further used to collect the execution situation of the action instruction and the status data after the execution of the action instruction when the action instruction is executed in the battery room.

[0053] The control room intelligent terminal 200 is used to obtain the status data of the battery room collected by the battery room acquisition terminal 100, and determine the battery room action instruction based on the status data of the battery room. The action instruction is used to indicate the adjustment of the status of the battery pack and / or the individual battery.

[0054] In some embodiments, a trained monitoring and control model is deployed in the control room intelligent terminal 200. The monitoring and control model is used to determine whether the battery pack and / or the individual battery fails based on the status data of the battery room, and determine the action instruction of the battery room in the event of a failure. Thus, the control room intelligent terminal 200 is specifically used to input the status data of the battery room into the trained monitoring and control model to obtain the action instruction of the battery room.

[0055] Exemplarily, the above monitoring and control model can be trained by the control room intelligent terminal 200; or, the above monitoring and control model can be trained by the model training device and deployed in the control room intelligent terminal 200.

[0056] In some embodiments, the control room intelligent terminal 200 is further used to send the generated action instruction of the battery room to the control terminal 300 of the battery room.

[0057] In some embodiments, the control room intelligent terminal 200 is further used to receive the execution feedback information of the action instruction of the battery room. The execution feedback information is used to indicate the execution situation of the action instruction and the status data of the battery room after the execution of the action instruction.

[0058] In some embodiments, the control room intelligent terminal 200 stores the obtained status data of the battery room in the way of positioning the individual battery.

[0059] Exemplarily, storing the status data in the way of positioning the individual battery can be implemented as follows:

[0060] Each building of the computer room buildings containing battery rooms in the data center park is named and positioned according to capital English letters, denoted as U. Each battery room in each building is labeled according to the floor F by numbers. Different battery packs L in the battery rooms on each floor are sorted and labeled according to lowercase English letters. The individual batteries n in each battery pack are positioned according to the digital numbering. Thus, the positioning U of all individual batteries in all computer room buildings in the data center park can be obtained L,F,n (For example, A 5,b,1 is the No. 1 individual battery in battery pack b of the battery room on the 5th floor of Building A), and the status data of each running battery is stored and backed up in the historical database with this positioning coordinate.

[0061] The control terminal 300 of the battery room is used to receive the action instruction of the control room intelligent terminal 200 and execute the action instruction.

[0062] Exemplarily, Figure 2 FIG. is a schematic diagram of an acquisition terminal architecture for a battery room provided by the present application, as Figure 2 shown. The above-mentioned battery room acquisition terminal 100 may include: a communication signal detection unit, a wireless communication unit, and a data monitoring and acquisition unit.

[0063] Among them, the communication signal detection unit determines whether there is wireless communication capability by monitoring the network signal of the data center. If so, wireless communication is adopted. If the network communication coverage condition is not met or the network signal is poor, a backup optical fiber communication method is adopted to ensure the timeliness and accuracy of data information acquisition and transmission.

[0064] The wireless communication unit provides wireless communication services for the battery room acquisition terminal 100 and the control room intelligent terminal 200. For example, the collected status data is sent to the control room intelligent terminal 200 by using an ultra-reliable and low-latency communication (Ultra Reliable&Low Latency Communication, uRLLC) network slice.

[0065] The data monitoring and acquisition unit is used to monitor and collect status data and execute feedback information, including monitoring and collecting status data such as electrical characteristic data, operating status data, and environmental characteristic data.

[0066] Exemplarily, the data monitoring and acquisition unit collects the execution feedback information by means of variable-frequency sampling. During the action execution time period after the action instruction is issued, the execution feedback information of the battery room after execution is sampled at a low frequency in the form of a heartbeat message. When the battery room returns to normal within this time period, the sampling of the execution feedback information ends. If the battery room has not returned to the normal operating state after the action instruction is issued, the data monitoring and acquisition unit uses high-frequency sampling to obtain the execution feedback information of the battery room and sends an alarm signal to the battery room intelligent terminal to indicate an abnormal operation.

[0067] Exemplarily, Figure 3 FIG. is a schematic diagram of a control room intelligent terminal architecture provided by the present application, as Figure 3 shown. The above-mentioned control room intelligent terminal 200 may include: a wireless communication unit, a data analysis and processing unit, a warning and alarm unit, and a historical database.

[0068] Among them, the wireless communication unit provides wireless communication services for the control room intelligent terminal 200, the battery room acquisition terminal 100, and the battery room control terminal 300.

[0069] The data analysis and processing unit is used to determine the action instruction of the battery room based on the status data of the battery room. For example, the status data of the battery room is input into the monitoring and control model to obtain the action instruction of the battery room.

[0070] The warning and alarm unit is used to send warning messages. For example, it sends warning messages when the charging current is too large.

[0071] The historical database is used to store the status data of the battery room, the action instructions of the battery room, and the execution feedback information.

[0072] Exemplarily, Figure 4 As shown in the schematic diagram of the battery room control terminal architecture provided by this application, Figure 4 as shown, the battery room control terminal 300 includes: a wireless communication unit and an action switch control unit.

[0073] Among them, the wireless communication unit provides wireless communication services for the battery room control terminal 300 and the control room intelligent terminal 200. For example, it receives the action instructions sent by the control room intelligent terminal 200 through the uRLLC network slice; the action switch control unit is used to execute the action instructions.

[0074] Exemplarily, the action switch control unit can be devices such as a voltage control switch and an air conditioner temperature control switch.

[0075] It should be noted that the system architecture described in the embodiments of this application is to more clearly illustrate the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided by the embodiments of this application. Those of ordinary skill in the art know that with the evolution of the system architecture, the technical solutions provided by the embodiments of this application are equally applicable to similar technical problems.

[0076] Figure 5 As shown in the flowchart of a battery monitoring and control method provided by an embodiment of this application. Figure 5 As shown, the battery monitoring and control method provided by this application can be implemented through the above-mentioned control room intelligent terminal 200, and specifically includes the following steps:

[0077] S101. Obtain the status data of the battery room.

[0078] Among them, the battery room includes at least one battery pack, and the battery pack includes at least one single battery.

[0079] In some embodiments, the status data includes: electrical characteristic data, operating status data, and environmental characteristic data.

[0080] Exemplarily, the electrical characteristic data includes: charge and discharge voltage data, charge and discharge current data, resistance, and charge and discharge times; the charge and discharge voltage data specifically includes the pre-discharge voltage, discharge voltage, floating charge voltage, and equalizing charge voltage of the battery pack and the single battery; the charge and discharge current data specifically includes the self-discharge current, working discharge current, floating charge current, and equalizing charge current of the battery pack and the single battery before discharge; the resistance is the resistance of the single battery.

[0081] Exemplarily, the operating state data includes the normal operating state information and abnormal / fault operating state information of individual storage batteries;

[0082] Exemplarily, the environmental characteristic data includes the surface temperature of individual storage batteries and the environmental temperature of the battery room.

[0083] Exemplarily, the state data of the battery room is obtained by receiving the electrical characteristic data, operating state data, and environmental characteristic data of each battery pack and each individual storage battery collected by the acquisition terminal in real time.

[0084] S102. Input the state data of the battery room into the trained monitoring and control model to obtain the action instruction of the battery room.

[0085] Among them, the action instruction is used to indicate adjusting the state of the battery pack and / or individual storage batteries; the monitoring and control model is used to determine whether the battery pack and / or individual storage batteries fail based on the state data of the battery room, and to determine the action instruction of the battery room in case of failure.

[0086] In some embodiments, the monitoring and control model is trained based on the historical state data and historical action instructions of the battery room, where the historical state data is used as sample data and the historical action instructions are used as supervision information.

[0087] Exemplarily, the monitoring and control model is the Actor network model in the trained Deep Deterministic Policy Gradient (DDPG) algorithm.

[0088] Exemplarily, the state data of the battery room is input into the trained Actor network model to obtain the action instruction of the battery room.

[0089] S103. Send down the action instruction.

[0090] Exemplarily, the action instruction can be to adjust the charge and discharge voltage of the storage battery, such as increasing or decreasing the charge and discharge voltage; it can be to adjust the temperature of the storage battery, such as reducing the air-conditioning temperature of the battery room.

[0091] It can be understood that for the storage battery monitoring and control method provided by this application, the sent-down action instruction carries the positioning information of the faulty or abnormal storage battery, and the battery room control terminal can accurately locate the position of the storage battery pointed to by the action instruction according to the positioning information and execute the action instruction. In this way, stable operation control of the battery room can be achieved.

[0092] In some embodiments, as Figure 6 shown, after step S103, the above method further includes the following steps:

[0093] S104. Receive the execution feedback information of the action instruction.

[0094] Among them, the execution feedback information is used to indicate the execution situation of the action instruction and the status data of the battery room after the execution of the action instruction.

[0095] Exemplarily, after the control room intelligent terminal issues an action instruction, it receives the execution feedback information of the action instruction collected by the acquisition terminal in the battery room. The execution feedback information includes: whether it is executed, the execution time, and the status data of the battery room after execution.

[0096] S105. When the execution feedback information indicates that the action instruction has been executed and the battery room has returned to the normal operation state, store the status data of the battery room, the action instruction of the battery room, and the execution feedback information in the historical database.

[0097] Among them, the data in the historical database can provide sample data for model training.

[0098] In some embodiments, when the execution feedback information indicates that the action instruction has been executed and the battery room has not returned to the normal operation state, repeat steps S102 - S104 until the battery room returns to the normal state.

[0099] Exemplarily, the model training method provided by the embodiments of the present application can be executed by the control room intelligent terminal 200 shown in Figure 1 , or by a training device (not shown in Figure 1 ), and the embodiments of the present application do not limit this.

[0100] Figure 7 This is a schematic flowchart of a model training method provided by the embodiments of the present application. As shown in Figure 7 , it specifically includes the following steps:

[0101] S201. Obtain the historical status data and historical action instructions of the battery room.

[0102] Among them, the battery room includes at least one battery pack, and the battery pack includes at least one single battery.

[0103] Exemplarily, obtain the historical status data and historical action instructions of the battery room from the historical database.

[0104] S202. Train the monitoring and control model to be trained based on the historical status data and historical action instructions to obtain a trained monitoring and control model.

[0105] Among them, the monitoring and control model is used to determine whether a failure has occurred in the battery pack and / or individual batteries based on the status data of the battery room, and to determine the action instruction of the battery room in the event of a failure; the action instruction is used to indicate the adjustment of the status of the battery pack and / or individual batteries.

[0106] In some embodiments, the monitoring and control model to be trained includes: a policy network, a target policy network, a value network, and a target value network; among them, the target policy network is a copy network of the policy network; the target value network is a copy network of the value network; the target policy network, the value network, and the target value network are used to train the policy network to obtain a trained policy network; the trained monitoring and control model includes the trained policy network.

[0107] In some embodiments, the above monitoring and control model further includes a memory experience storage pool, which is used to store experience data such as status data, action instructions, and parameters obtained by the model during the running and training process, and randomly sample the experience data in the memory experience storage pool as replay data to train the monitoring and control model, which can avoid the instability caused by the continuous relevant experience of the monitoring and control model.

[0108] In some embodiments, the above model training method includes:

[0109] a1. Input the first historical status data into the policy network to obtain the first action instruction of the battery room.

[0110] a2. Input the first action instruction into the value network to obtain the value evaluation value of the first action instruction, and the value evaluation value is used to reflect the degree of benefit of the execution of the action instruction to the battery room.

[0111] a3. Based on the first value evaluation value, adjust the network parameters of the policy network.

[0112] In some embodiments, the above model training method further includes:

[0113] b1. Obtain the second historical status data of the battery room after the execution of the first action instruction.

[0114] b2. Input the second historical status data into the target policy network to obtain the second action instruction of the battery room.

[0115] b3. Input the second action instruction into the target value network to obtain the value evaluation value of the second action instruction.

[0116] b4. Based on the difference between the value evaluation value of the first action instruction and the value evaluation value of the second action instruction, adjust the network parameters of the value network.

[0117] It can be understood that the network parameters of the value network are adjusted based on the difference between the value evaluation value of the first action instruction and the value evaluation value of the second action instruction; the network parameters of the policy network are adjusted based on the first value evaluation value of the value network. The monitoring and control model trained through iterative training has stability and can efficiently solve the problem of generating continuous action instructions in battery monitoring and control.

[0118] Exemplarily, the initial values of the network parameters of the target policy network can copy the network parameters of the policy network, and the initial values of the network parameters of the target value network can copy the network parameters of the value network. Subsequently, the parameter update of the target policy network module and the target value network module is realized in a soft update manner. The specific calculation methods refer to Formula (3) and Formula (6), which will not be elaborated here.

[0119] Exemplarily, the present application provides a schematic diagram of the architecture for model training, as Figure 8 shown, including: a policy network module, a target policy network module, a value network module, a target value network module, a memory experience storage pool, a simulation environment, and a historical database.

[0120] As Figure 8 shown, the training data can be sourced from the historical data generated by the interaction between the policy network module and the simulation environment. This historical data includes historical state data and historical action instructions; it can also be sourced from the historical database, which stores the historical data generated by the interaction between the model and the actual operating environment after the model is deployed. This historical data includes historical state data and historical action instructions.

[0121] As Figure 8 shown, the policy network module is used to analyze the sampled state data to generate action instructions. The value network module evaluates the value of this action instruction and adjusts the parameters of the policy network module based on this value evaluation value; the target policy network module and the target value network module respectively copy the network parameters of the corresponding modules; the target policy network module generates the next action instruction based on the next state data, and the target value network module generates the target value evaluation value based on the next state data, the next action instruction generated by the next state data, and the control value parameter of the target value network module. The parameters of the value network are updated using the difference between the value evaluation value and the target value evaluation value; the parameter update of the target policy network module and the target value network module is realized using a soft update calculation method. The model experience is stored in the memory experience storage pool and can be used for subsequent sampling and replay training.

[0122] Exemplarily, the policy network module is used to analyze the sampled state data to generate action instructions, and this generation process satisfies the following Formula (1):

[0123] a t =μ(s;θμ ) Formula (I)

[0124] Wherein, μ represents a policy function, s represents the current state data, and θ μ represents the control policy parameter of the policy network module, and a t represents the generated action instruction.

[0125] Exemplarily, the generated action instructions include actions such as adjusting the charging voltage and current of the battery pack, adjusting the temperature of the battery chamber air conditioner, and triggering the warning system.

[0126] Exemplarily, the target policy network module is used to generate the next action instruction based on the next state data, and this generation process satisfies the following formula (II):

[0127] a t+1 = μ'(s'; θ' μ ) Formula (II)

[0128] Wherein, μ' represents the target policy function, s' represents the state data after the execution of the action instruction generated by the policy network module, that is, the next state data, and θ' μ represents the control policy parameter of the target policy network module, and a t+1 represents the generated next action instruction.

[0129] Exemplarily, the update of the control policy parameter of the target policy network module adopts a soft update calculation method, and this calculation method satisfies the following formula (III):

[0130] θ’ μ,t+1 = λθ μ,t +(1 - λ)θ’ μ,t Formula (III)

[0131] Wherein, θ μ,t represents the current control policy parameter of the policy network module, θ' μ,t represents the current control policy parameter of the target policy network module, λ represents the soft update coefficient (or soft update step size), and the value is usually very small, such as 0.001, and θ' μ,t+1 represents the next control policy parameter of the target policy network module.

[0132] Exemplarily, the value network module is used to generate a value evaluation value based on the current state data, the action instruction generated from the current state data, and the control value parameter of the value network module, and this generation process satisfies the following formula (IV):

[0133] Q c = ω(s; a t ; θ ω ) Formula (IV)

[0134] Among them, s represents the current state data, a t represents the generated action instruction, θ ω represents the control value parameter of the value network module, ω represents the value function, Q c represents the value evaluation value.

[0135] Exemplarily, the target value network module is used to generate a target value evaluation value based on the next state data, the next action instruction generated from the next state data, and the control value parameter of the target value network module. This generation process satisfies the following formula (five):

[0136] Q c ′ = ω(s′; a t+1 ; θ′ ω ) Formula (five)

[0137] Among them, s′ represents the next state data, a t+1 represents the next action instruction, θ′ ω represents the control value parameter of the target value network module, ω represents the value function, Q c ′ represents the target value evaluation value.

[0138] Exemplarily, the update of the control value parameter of the target value network module adopts a soft update calculation method, and this calculation method satisfies the following formula (six):

[0139] θ’ ω,t+1 = λθ ω,t +(1 - λ)θ’ ω,t Formula (six)

[0140] Among them, θ ω,t represents the current control value parameter of the value network module, θ’ ω,t represents the current control value parameter of the target value network module, λ represents the soft update coefficient (or soft update step size), and the value is usually very small, such as 0.001, θ’ ω,t+1 represents the next control value parameter of the target value network module.

[0141] It can be understood that the target policy network module and the target value network module adopting the soft update method can provide a more stable target value for the policy network and the value network as a reference, avoiding oscillations and fluctuations during the model training process.

[0142] Next, a specific embodiment will be used to introduce the battery monitoring and control method of the present application. For example Figure 9 , taking the application of this method in the interaction process of the battery room control terminal, the battery room acquisition terminal, and the battery room control terminal as an example, it includes the following steps S301 to S310.

[0143] S301. The battery room acquisition terminal collects the status data of each battery pack and individual single-cell batteries in real time.

[0144] S302. The battery room acquisition terminal transmits the collected data to the intelligent room control terminal based on the network slicing of the uRLLC scenario.

[0145] S303. The intelligent room control terminal receives the status data collected by the battery room acquisition terminal and stores it in the way of single-cell battery positioning; constructs a DDPG neural network model to extract sample data from the historical database for training.

[0146] S304. The intelligent room control terminal inputs the real-time collected data into the trained DDPG neural network model, formulates action instructions through the policy network in the model, and retrieves the control terminal switch of the corresponding battery room according to the single-cell battery coordinate positioning.

[0147] S305. The intelligent room control terminal issues action instructions.

[0148] S306. The battery room control terminal executes the action instructions.

[0149] Exemplarily, the control terminal switch of the corresponding battery room in the battery room control terminal executes the above action instructions.

[0150] S307. The battery room acquisition terminal collects the action execution situation and the status data of the battery room after the action through the variable-frequency sampling method.

[0151] S308. The battery room acquisition terminal feeds back the data collected through the variable-frequency sampling method to the intelligent room control terminal.

[0152] S309. When it is determined that the battery room has returned to normal based on the data after the execution of the action instruction, the intelligent room control terminal aggregates and transmits key data information such as the status data at the time of the fault, the action instruction, the action execution situation, and the status data of the battery room after the action execution to the historical database for storage, for subsequent model training and learning, and generates a fault handling message for maintenance personnel to repair.

[0153] In some embodiments, when it is determined that the battery room has not returned to normal based on the data after the execution of the action instruction, steps S301 to S308 are repeated.

[0154] Next, another specific embodiment is used to introduce the battery monitoring and control method of the embodiment of the present application. The specific implementation process of this method is as follows Figure 10 shown.

[0155] S401. The communication signal detection unit of the battery room acquisition terminal detects the data center network signal.

[0156] S402. The battery room acquisition terminal determines whether the network signal has wireless communication capabilities. If yes, it executes step S403a; if no, it executes step S403b.

[0157] S403a. Communicate using the uRLLC network slice.

[0158] S403b. Communicate using the standby optical fiber.

[0159] S404. The battery room acquisition terminal collects the electrical characteristic data, operating status data of each battery pack and each single battery in the battery room, and the environmental characteristic data of the battery room in real time, and transmits the collected characteristic data to the intelligent control terminal in the data room control center for summarization.

[0160] S405. After receiving the data transmitted by the acquisition terminal, the intelligent control terminal backs up the data in the historical database in the way of positioning by single battery, and constructs a Deep Deterministic Policy Gradient (DDPG) algorithm, and extracts samples from the historical database with the historical electrical characteristic data and operating status data of the battery as the training set for comparative training analysis.

[0161] S406. Input the data collected in real time into the DDPG network model, and based on the current operating status, decide whether to issue an action instruction.

[0162] S407. Input the data collected in real time into the DDPG network model, and based on the current operating status, determine whether to issue an action instruction. If yes, it executes step S408; if no, it executes step S404.

[0163] S408. Based on the coordinates of the controlled battery, accurately issue and execute the action instruction to the corresponding battery room control terminal switch.

[0164] S409. The battery room acquisition terminal obtains the execution situation of the action instruction and the feedback of the working status of the battery room after the execution of the action instruction in the way of variable-frequency sampling, and feeds it back to the intelligent control terminal.

[0165] S410. Based on the feedback data, determine whether the battery room has returned to the normal working state. If yes, it executes step S411; if no, it executes step S404.

[0166] S411. Store the key data of the fault occurrence and handling in the historical database and generate a fault handling message for backup and archiving.

[0167] The above mainly introduces the solutions of the embodiments of the present disclosure from the perspective of methods. It can be understood that in order for the battery monitoring and control device or the model training device to implement the above functions, it includes at least one of the corresponding hardware structures and software modules for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present disclosure.

[0168] The embodiments of the present disclosure can divide the functional modules of the battery monitoring and control device or the model training device according to the above method embodiments. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one functional module. The above integrated module can be implemented in the form of hardware or software. It should be noted that the division of modules in the embodiments of the present disclosure is illustrative, only a logical function division, and there can be other division methods in actual implementation. The following takes the example of dividing each functional module corresponding to each function for illustration.

[0169] For example, Figure 11 is a schematic diagram of the composition of a battery monitoring and control device provided in an embodiment of the present application. As Figure 11 shown, the battery monitoring and control device 1100 includes: an acquisition module 1101, a processing module 1102, and a sending module 1103; the acquisition module 1101 is used to acquire the status data of the battery room, where the battery room includes at least one battery pack, and the battery pack includes at least one single battery; the processing module 1102 is used to input the status data of the battery room into the trained monitoring and control model to obtain an action instruction for the battery room, and the action instruction is used to indicate adjusting the status of the battery pack and / or the single battery; wherein, the monitoring and control model is used to determine whether the battery pack and / or the single battery fails based on the status data of the battery room, and determine the action instruction for the battery room in the case of a failure; the sending module 1103 is used to issue the action instruction.

[0170] A possible implementation manner is that the status data includes: electrical characteristic data, operating status data, and environmental characteristic data.

[0171] In another possible implementation manner, after the action instruction is issued, the acquisition module 1101 is further configured to: receive the execution feedback information of the action instruction; the execution feedback information is used to indicate the execution situation of the action instruction and the state data of the battery room after the execution of the action instruction; when the execution feedback information indicates that the action instruction has been executed and the battery room resumes normal operation, the processing module 1102 is further configured to: store the state data of the battery room, the action instruction of the battery room, and the execution feedback information in the historical database; wherein, the data in the historical database is used for model training.

[0172] In yet another possible implementation manner, the monitoring and control model is trained based on the historical state data and historical action instructions of the battery room, wherein the historical state data serves as sample data and the historical action instructions serve as supervision information.

[0173] For example, Figure 12 FIG. is a schematic diagram of the composition of a model training device provided in an embodiment of the present application. As shown in the figure, the model training device 1200 includes: an acquisition module 1201 and a training module 1202; the acquisition module 1201 is configured to: acquire the historical state data and historical action instructions of the battery room; wherein, the battery room includes at least one battery pack, and the battery pack includes at least one single battery; the training module 1202 is configured to: train the monitoring and control model to be trained based on the historical state data and historical action instructions to obtain a trained monitoring and control model; wherein, the monitoring and control model is used to determine whether a battery pack and / or a single battery fails based on the state data of the battery room, and to determine the action instruction of the battery room in the event of a failure; the action instruction is used to indicate adjusting the state of the battery pack and / or the single battery.

[0174] In a possible implementation manner, the monitoring and control model to be trained includes: a policy network, a target policy network, a value network, and a target value network; wherein, the target policy network is a copy network of the policy network; the target value network is a copy network of the value network; the target policy network, the value network, and the target value network are used to train the policy network to obtain a trained policy network; the trained monitoring and control model includes the trained policy network.

[0175] In another possible implementation manner, the training module 1202 is specifically configured to: input the first historical state data into the policy network to obtain the first action instruction of the battery room; input the first action instruction into the value network to obtain the value evaluation value of the first action instruction, and the value evaluation value is used to reflect the beneficial degree of the execution of the action instruction on the battery room; based on the first value evaluation value, adjust the network parameters of the policy network.

[0176] Another possible implementation is that the training module 1202 is further configured to: obtain the second historical state data of the battery chamber after the first action instruction is executed; input the second historical state data into the target policy network to obtain the second action instruction of the battery chamber; input the second action instruction into the target value network to obtain the value evaluation value of the second action instruction; and adjust the network parameters of the value network based on the difference between the value evaluation value of the first action instruction and the value evaluation value of the second action instruction.

[0177] In an exemplary embodiment, the embodiment of the present application further provides an electronic device, which may be the battery monitoring and control device or the model training device in the foregoing method embodiment. Figure 13 It is a schematic diagram of the composition of an electronic device provided by an embodiment of the present application. As Figure 13 shown, the electronic device may include: a processor 1301 and a memory 1302; the memory 1302 stores instructions executable by the processor 1301; when the processor 1301 is configured to execute the instructions, the electronic device or the network device or the manager implements the method described in the foregoing method embodiment.

[0178] In an exemplary embodiment, the embodiment of the present application further provides a computer-readable storage medium, on which computer program instructions are stored; when the computer program instructions are executed by a computer, the computer implements the method described in the foregoing embodiment. The computer-readable storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0179] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, when the computer program product runs on a computer, the computer is caused to execute the above-related method steps to implement the battery monitoring and control method in the above embodiment.

[0180] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A battery monitoring and control method, characterized in that: The method comprises: Acquiring status data of a battery chamber; wherein the battery chamber includes at least one storage battery pack, and the storage battery pack includes at least one single storage battery; Inputting the state data of the battery room into the trained monitoring and control model to obtain the action instructions of the battery room; the action instructions are used to instruct the adjustment of the state of the battery pack and / or the single battery; wherein the monitoring and control model is used to determine whether the battery pack and / or the single battery fails based on the state data of the battery room, and determine the action instructions of the battery room in the event of a failure; The action instruction is issued.

2. The method according to claim 1, characterized in that The status data includes: electrical characteristic data, operation status data, and environmental characteristic data.

3. The method according to claim 1, characterized in that After issuing the action instruction, the method further includes: receiving execution feedback information of the action instruction; the execution feedback information is used to indicate the execution status of the action instruction and the state data of the battery chamber after the execution of the action instruction; When the execution feedback information indicates that the action instruction has been executed and the battery chamber has resumed normal operation, the status data of the battery chamber, the action instruction of the battery chamber, and the execution feedback information are stored in a historical database; wherein the data in the historical database is used for model training.

4. The method according to claim 1, characterized in that: The monitoring and control model is trained based on historical status data and historical action instructions of the battery room, wherein the historical status data is used as sample data and the historical action instructions are used as supervision information.

5. A model training method, characterized in that: The method comprises: Acquire historical status data and historical action instructions of a battery room; wherein the battery room includes at least one storage battery pack, and the storage battery pack includes at least one single storage battery; The monitoring and control model to be trained is trained based on the historical status data and the historical action instructions to obtain a trained monitoring and control model; wherein the monitoring and control model is used to determine whether the battery pack and / or the single battery has failed based on the status data of the battery chamber, and determine the action instructions of the battery chamber in the event of a failure; the action instructions are used to instruct the adjustment of the status of the battery pack and / or the single battery.

6. The method according to claim 5, characterized in that The monitoring and control model to be trained includes: a policy network, a target policy network, a value network and a target value network; wherein the target policy network is a replica network of the policy network; and the target value network is a replica network of the value network; The target policy network, the value network and the target value network are used to train the policy network to obtain a trained policy network; The trained monitoring and control model includes the trained strategy network.

7. The method according to claim 6, characterized in that The training of the monitoring and control model to be trained based on the historical state data and the historical action instructions includes: Inputting the first historical state data into the strategy network to obtain a first action instruction for the battery room; Inputting the first action instruction into the value network to obtain a value evaluation value of the first action instruction, wherein the value evaluation value is used to reflect the degree of benefit of the execution of the action instruction to the battery room; Based on the first value assessment value, a network parameter of the policy network is adjusted.

8. The method according to claim 7, characterized in that The method further comprises: Acquire second historical state data of the battery chamber after the first action instruction is executed; Inputting the second historical state data into the target strategy network to obtain a second action instruction for the battery room; Inputting the second action instruction into the target value network to obtain a value evaluation value of the second action instruction; Based on the difference between the value evaluation value of the first action instruction and the value evaluation value of the second action instruction, a network parameter of the value network is adjusted.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 4 or claims 5 to 8.

10. A readable storage medium, characterized in that: The readable storage medium includes software instructions; When the software instructions are executed in an electronic device, the electronic device implements the method according to any one of claims 1 to 4 or claims 5 to 8.

11. A program product, characterized in that The program product includes a program, and when the program is run on an electronic device, the electronic device executes the method according to any one of claims 1 to 4 or claims 5 to 8.

Images