Mining lithium battery fire risk assessment method based on multi-modal deep learning

Abstract

The embodiment of the invention provides a mine lithium battery fire risk assessment method based on multi-modal deep learning. The method comprises the steps of obtaining mine image data, mine video data, mine temperature and air pressure data and battery temperature and air pressure data of a mine lithium battery; respectively performing feature extraction on the mine image data, the mine video data, the mine temperature and air pressure data and the battery temperature and air pressure data to correspondingly obtain key visual features, dynamic change features, mine features and battery features; performing feature fusion on the plurality of features to obtain a fused feature vector; inputting the fusion feature vector into a deep learning model, and constructing a mining lithium battery fire risk assessment model; based on a fire risk assessment model, training the training set, the verification set and the test set to obtain a trained fire risk assessment model; and based on the trained fire risk assessment model, after fire risk prediction is performed on the mining lithium battery, risk grade assessment is performed, and a corresponding risk early warning signal is generated.

Description

Technical Field

[0001] The present invention relates to the field of lithium battery fire risk assessment, and in particular to but is not limited to a method for assessing the fire risk of mining lithium batteries based on multimodal deep learning. Background Art

[0002] With the continuous growth of energy demand and the improvement of environmental protection awareness, lithium-ion batteries have become an important development direction for underground mining equipment due to their high energy density, long cycle life and environmental protection. However, lithium-ion batteries face special challenges in mining environments, such as high temperature, dust, mechanical impact and humidity, which increase the risk of thermal runaway and fire. Once thermal runaway occurs, the rapid rise in temperature inside the battery can trigger a violent chemical reaction, generate a large amount of heat and gas, which may cause fire or explosion.

[0003] The uniqueness of the mining environment makes it difficult to directly apply the traditional lithium-ion battery safety assessment methods. Currently, there are few battery safety tests in mining environments, which fail to fully consider the particularity of the mining environment, which may lead to inaccurate risk assessment. Once a fire occurs, it will not only cause serious damage to mining equipment, but also endanger the lives of miners.

[0004] Therefore, a suitable fire risk assessment method for mining lithium batteries is urgently needed. Summary of the invention

[0005] Based on the problems in the related art, an embodiment of the present invention provides a fire risk assessment method for mining lithium batteries based on multimodal deep learning.

[0006] The technical solution of the embodiment of the present invention is achieved as follows:

[0007] In a first aspect, an embodiment of the present invention provides a method for assessing the fire risk of a lithium battery for mining based on multimodal deep learning, the method comprising:

[0008] Acquire real-time mine environment data and battery temperature and pressure data of mining lithium batteries; the real-time mine environment data includes mine image data, mine video data and mine temperature and pressure data;

[0009] Performing feature extraction on the mine image data, the mine video data, the mine temperature and pressure data, and the battery temperature and pressure data, respectively, to obtain key visual features, dynamic change features, mine features, and battery features;

[0010] Performing feature fusion on the key visual features, the dynamic change features, the mine features and the battery features to obtain a fused feature vector;

[0011] Inputting the fused feature vector into a deep learning model to construct a fire risk assessment model for mining lithium batteries;

[0012] Based on the fire risk assessment model, training the training set, the validation set and the test set to obtain a trained fire risk assessment model;

[0013] Based on the trained fire risk assessment model, fire risk prediction is performed on the mining lithium battery to obtain a fire risk prediction result;

[0014] A risk level assessment is performed based on the fire risk prediction result to obtain a risk assessment level, and a corresponding risk warning signal is generated to implement corresponding response measures.

[0015] In a second aspect, an embodiment of the present invention provides a device for assessing the fire risk of a lithium battery for mining based on multimodal deep learning, the device comprising:

[0016] An acquisition module is used to acquire real-time mine environment data and battery temperature and pressure data of a mining lithium battery; the real-time mine environment data includes mine image data, mine video data, and mine temperature and pressure data;

[0017] An extraction module, used to extract features from the mine image data, the mine video data, the mine temperature and pressure data, and the battery temperature and pressure data, respectively, to obtain key visual features, dynamic change features, mine features, and battery features;

[0018] A fusion module, used for fusing the key visual features, the dynamic change features, the mine features and the battery features to obtain a fused feature vector;

[0019] A construction module is used to input the fused feature vector into a deep learning model to construct a fire risk assessment model for mining lithium batteries;

[0020] A training module, used to train a training set, a validation set and a test set based on the fire risk assessment model to obtain a trained fire risk assessment model;

[0021] A prediction module, used to predict the fire risk of the mining lithium battery based on the trained fire risk assessment model to obtain a fire risk prediction result;

[0022] The evaluation module is used to perform risk level evaluation based on the fire risk prediction result, obtain the risk evaluation level, and generate a corresponding risk warning signal to implement corresponding response measures.

[0023] In some embodiments, the extraction module is also used to extract the key visual features from the mine image data through a convolutional neural network; convolve the mine video data in the time dimension and the space dimension through a recurrent neural network or a 3D convolutional neural network to obtain the dynamic change characteristics; and extract features from the mine temperature and pressure data and the battery temperature and pressure data through a preset statistical method to obtain the mine characteristics and the battery characteristics.

[0024] In some embodiments, the extraction module is also used in the formula of the convolutional layer in the convolutional neural network as follows: in, is the activation value of the lth layer at position (i, j), is the weight of the convolution kernel, is the input of the previous layer, b (l) is the bias, σ is the activation function, and m and n represent the vertical and horizontal movement indexes on the input feature map during the convolution process, respectively.

[0025] In some embodiments, the extraction module is also used to determine the state update of the recurrent neural network unit by the following formula: t =σ(W f ·[h t-1 ,x t ]+b f ),i t =σ(W i ·[h t-1 ,x t ]+b i ), o t =σ(W o ·[h t-1 ,x t ]+b o ), h t =o t tanh(C t ), where f t ,i t , o t are the activation values ​​of the forget gate, input gate, and output gate, respectively, W f , W i , W C , W o are the weight matrices of the forget gate, input gate, candidate cell state, and output gate, respectively. f Used to control whether the current time step forgets the previous state, W i Used to control the impact of the current input on the state of the candidate cell, W C Used to generate new candidate cell states, Wo Used to control the output of the hidden state, b f , b i , b C , b o They are the bias items of the forget gate, input gate, candidate cell state and output gate, respectively. is the candidate cell state, C t is the current cell state, t and t-1 are the current time step and the previous time step respectively, h t is the hidden state of the current time step, h t-1 is the hidden state of the previous time step, the hidden state is the output of the recurrent neural network unit, which is used for the calculation of the next time step or the final output, and σ is the sigmoid activation function.

[0026] In some embodiments, the fusion module is also used to standardize the key visual features, the dynamically changing features, the mine features and the battery features to obtain standardized key visual features, dynamically changing features, mine features and battery features; and use a deep neural network to perform feature fusion on the standardized key visual features, dynamically changing features, mine features and battery features to obtain the fused feature vector.

[0027] In some embodiments, the fire risk prediction result includes at least a risk prediction value; the assessment module is further used to compare the risk prediction value with a preset risk level threshold to determine the risk assessment level; and based on the risk assessment level, generate the corresponding risk warning signal.

[0028] In a third aspect, an embodiment of the present invention provides a mine lithium battery fire risk assessment device based on multimodal deep learning, comprising: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the above-mentioned mine lithium battery fire risk assessment method based on multimodal deep learning.

[0029] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning.

[0030] The method for assessing the fire risk of lithium batteries for mining provided by the embodiment of the present invention based on multimodal deep learning collects the operating data of lithium batteries for mining, including temperature, voltage, current and external environmental parameters, as well as fire accident records, extracts key features after preprocessing the data, builds a fire risk assessment model, and then applies a machine learning algorithm to train and verify the model. Finally, the trained model is used to analyze the data collected in real time to predict the fire risk level of the lithium batteries for mining under the current working conditions. In this way, by collecting and analyzing a large amount of data, a fire risk assessment model is established, which can improve the accuracy and reliability of the fire risk prediction of lithium batteries for mining. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of a method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning provided by an embodiment of the present invention;

[0032] Figure 2 It is a structural schematic diagram of a mining lithium battery fire risk assessment system based on multimodal deep learning provided by an embodiment of the present invention;

[0033] Figure 3 It is a flow chart of another method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning provided by an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of the composition structure of a mining lithium battery fire risk assessment device based on multimodal deep learning provided in an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of the composition structure of a mining lithium battery fire risk assessment device based on multimodal deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0037] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meaning as those commonly understood by those skilled in the art to which the embodiments of the present invention pertain. The terms used in the embodiments of the present invention are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0038] The following describes an exemplary application of a mine lithium battery fire risk assessment device based on multimodal deep learning according to an embodiment of the present invention. The mine lithium battery fire risk assessment device based on multimodal deep learning provided by an embodiment of the present invention can be implemented as a terminal or a server. In one implementation, the mine lithium battery fire risk assessment device based on multimodal deep learning provided by an embodiment of the present invention can be implemented as various types of terminals such as laptops, tablet computers, desktop computers, and mobile devices; in another implementation, the mine lithium battery fire risk assessment device based on multimodal deep learning provided by an embodiment of the present invention can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present invention. Below, an exemplary application of a mining lithium battery fire risk assessment device based on multimodal deep learning when implemented as a server will be described.

[0039] The embodiment of the present invention provides a method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning. Figure 1 , Figure 1 This is a flow chart of a method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning provided by an embodiment of the present invention. Figure 1 The steps shown are explained.

[0040] Step S110, obtaining the real-time mine environment data and battery temperature and pressure data of the mining lithium battery; the real-time mine environment data includes mine image data, mine video data and mine temperature and pressure data.

[0041] In some embodiments, the real-time mine environment data refers to the environmental parameters acquired in real time inside the mine through sensors or other data acquisition devices. Here, the data acquisition device may be a high-resolution camera, a video surveillance system, or other image acquisition device. The sensor may be a temperature sensor, an air pressure sensor, a humidity sensor, a gas sensor, etc.

[0042] In some embodiments, the battery temperature and air pressure data refers to the surface temperature and surrounding air pressure data of the mining lithium battery during use.

[0043] In some embodiments, mine image data refers to real-time images of the interior of a mine acquired through a high-resolution camera or other image acquisition equipment, which helps monitor the condition of the mine and promptly identify safety hazards.

[0044] In some embodiments, mine video data is similar to image data, but the video data is continuous and can record the situation inside the mine in real time, providing more comprehensive monitoring.

[0045] In some embodiments, mine temperature and air pressure data refers to the temperature, air pressure, humidity and gas type inside the mine obtained through temperature sensors, air pressure sensors, humidity sensors and gas sensors. These data are mainly used to assess the safety status of the mine (such as whether there are risks such as gas accumulation and fire).

[0046] In some embodiments, the battery temperature and pressure data refers to the surface temperature and surrounding pressure data of the mining lithium battery during use. These data are used to evaluate the working status of the battery, predict the battery life, and promptly detect safety hazards of the battery (such as overheating, overpressure, etc.).

[0047] Step S120, performing feature extraction on the mine image data, the mine video data, the mine temperature and pressure data, and the battery temperature and pressure data, respectively, to obtain key visual features, dynamic change features, mine features, and battery features.

[0048] In the present invention, for mine image data, a convolutional neural network is used to extract key visual features, such as the shape and dynamics of flames and smoke. For mine video data, a temporal convolutional network or a long short-term memory network is used to extract dynamic change features and capture the time series information of fire development. For sensor data of mines and battery systems, that is, for data such as temperature, air pressure, voltage, pressure and gas, statistical features (mean, variance, peak, etc.) and trend and periodic features extracted from time series data are calculated.

[0049] Step S130, performing feature fusion on the key visual features, the dynamic change features, the mine features and the battery features to obtain a fused feature vector.

[0050] In some embodiments, the standardization process is helpful to eliminate the influence of different feature dimensions so that all features are on the same scale. Here, the standardization process method can be Min-Max standardization (i.e., scaling the feature value to a specified range) or Z-score standardization (i.e., converting the feature value to a distribution with a mean of 0 and a standard deviation of 1). Of course, other types of standardization processes may also be used, and the embodiments of the present invention do not specifically limit this.

[0051] Step S140, inputting the fused feature vector into a deep learning model to construct a fire risk assessment model for mining lithium batteries.

[0052] Step S150: Based on the fire risk assessment model, the training set, the validation set and the test set are trained to obtain a trained fire risk assessment model.

[0053] Step S160: Based on the trained fire risk assessment model, a fire risk prediction is performed on the mining lithium battery to obtain a fire risk prediction result.

[0054] Step S170, performing risk level assessment based on the fire risk prediction result to obtain a risk assessment level, and generating a corresponding risk warning signal to implement corresponding response measures.

[0055] The method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning provided by the embodiment of the present invention collects the operating data of lithium batteries for mining, including temperature, voltage, current and external environmental parameters, as well as historical fire accident records, pre-processes the data, extracts key features, builds a fire risk assessment model, and then applies a machine learning algorithm to train and verify the model. Finally, the trained model is used to analyze the real-time collected data to predict the fire risk level of the lithium battery for mining under the current working conditions. In this way, by collecting and analyzing a large amount of data, a fire risk assessment model is established, which can improve the accuracy and reliability of the fire risk prediction of lithium-ion batteries for mining. .

[0056] In some embodiments, the target acoustic signal includes an instantaneous acoustic change, an abnormal acoustic signal, and a transfer acoustic signal; the above step S120 can be implemented by the following steps S121 to S123:

[0057] Step S121, extracting the key visual features from the mine image data through a convolutional neural network.

[0058] Step S122, convolution of the mine video data in time dimension and space dimension is performed through a recurrent neural network or a 3D-CNN to obtain the dynamic change feature.

[0059] Step S123, extracting features of the mine temperature and pressure data and the battery temperature and pressure data by a preset statistical method to obtain the mine features and the battery features.

[0060] In some embodiments, the above step S121 may be implemented by the following contents:

[0061] The formula of the convolutional layer in the convolutional neural network is expressed as:

[0062]

[0063] in, is the activation value of the lth layer at position (i, j), is the weight of the convolution kernel, is the input of the previous layer, b (l) is the bias, σ is the activation function, and m and n represent the vertical and horizontal movement indexes on the input feature map during the convolution process, respectively.

[0064] In some embodiments, the above step S123 may be implemented by the following contents:

[0065] The state update of the recurrent neural network unit is determined by the following formula:

[0066] f t =σ(W f ·[h t-1 ,x t ]+b f ),

[0067] i t =σ(W i ·[h t-1 ,x t ]+b i ),

[0068]

[0069] o t =σ(W o ·[h t-1 ,x t ]+b o ),

[0070] h t =o t tanh(C t ),

[0071] Among them, f t ,i t , o t are the activation values ​​of the forget gate, input gate, and output gate, respectively, W f , W i , W C , W o are the weight matrices of the forget gate, input gate, candidate cell state, and output gate, respectively. f It is used to control whether the previous state is forgotten in the current time step. Wi is used to control the influence of the current input on the state of the candidate cell. C Used to generate new candidate cell states, W o Used to control the output of the hidden state, b f , bi , b C , b o They are the bias items of the forget gate, input gate, candidate cell state and output gate, respectively. is the candidate cell state, C t is the current cell state, t and t-1 are the current time step and the previous time step respectively, h t is the hidden state of the current time step, h t-1 is the hidden state of the previous time step, the hidden state is the output of the recurrent neural network unit, which is used for the calculation of the next time step or the final output, and σ is the sigmoid activation function.

[0072] In some embodiments, the above step S130 can be implemented by the following steps S131 to S132:

[0073] Step S131, standardizing the key visual features, the dynamic change features, the mine features and the battery features to obtain standardized key visual features, dynamic change features, mine features and battery features.

[0074] Step S132, using a deep neural network to perform feature fusion on the standardized key visual features, dynamic change features, mine features and battery features to obtain the fused feature vector.

[0075] In some embodiments, the above step S170 can be implemented by the following steps S171 to S172:

[0076] Step S171, comparing the risk prediction value with a preset risk level threshold to determine the risk assessment level.

[0077] Step S172: generating the corresponding risk warning signal based on the risk assessment level.

[0078] The following describes an exemplary application of an embodiment of the present invention in a practical application scenario.

[0079] Figure 2 A structural schematic diagram of a mine lithium battery fire risk assessment system based on multimodal deep learning provided in an embodiment of the present invention. The system includes a mine lithium battery fire data acquisition and preprocessing module, a fire risk assessment index set feature extraction module, a fire risk assessment index establishment module, a fire risk index weight calculation module, a fire risk index scoring module and a fire risk assessment and response module. Among them, the fire risk index weight calculation module and the fire risk index scoring module jointly evaluate the fire risk level. The fire risk assessment and response module sends an early warning signal to the relevant safety and emergency systems, and at the same time feeds back the assessment results for monitoring and optimization.

[0080] Figure 3 A flow chart of another method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning provided by an embodiment of the present invention, the method comprising:

[0081] Step 1: Collect relevant data of the mining lithium battery fire scene, which includes real-time mine environment data, such as image data, video data, temperature and air pressure data, and real-time data of the battery system such as temperature, voltage, pressure, gas, etc.

[0082] Step 2: Preprocess the collected fire scene data, then extract features for each type of data to generate several modal data sets, then fuse the features of the multimodal data sets to generate a new feature vector, and build a mine lithium battery fire risk assessment model.

[0083] Step 3: Based on the fire risk assessment model for lithium batteries used in mining, train the training set, validation set and test machine to obtain the fire risk assessment model that has been trained and validated.

[0084] Step 4: Predict the fire risk of mining lithium batteries based on a standard fire risk assessment model, obtain the risk assessment level based on the hazard factor evaluation, generate the corresponding risk warning signal and send it to the fire protection system, so as to carry out timely and effective response measures.

[0085] In the present invention, in step 1, high-resolution cameras and video monitoring systems installed in the mining area are used to collect image data and video data; distributed sensor grids are used to collect data such as temperature and air pressure in the mine; and relevant real-time data such as temperature, voltage, pressure, gas, etc. in the battery system are collected in real time through multiple sensors of the battery management unit (BMU), a key component in the battery management system (BMS).

[0086] In step 2, preprocessing the collected fire scene data includes:

[0087] Data cleaning: Analyze data based on statistical methods. First, calculate the mean and standard deviation of the entire data set. The deviation of each data point from the overall mean will be standardized by the standard deviation to obtain the Z score of each point. Data points with Z values ​​exceeding the threshold will be considered outliers. For outliers, according to the data analysis objectives, outliers are deleted, corrected, or replaced. Correction can be done by interpolation or data smoothing.

[0088] Feature extraction: Converting raw data into a format that the model can effectively process. Among them:

[0089] (1) Image data feature extraction: Use convolutional neural networks (CNN) to extract visual features from images. CNN can identify important features such as edges, shapes, and textures in images. Each convolution layer uses multiple convolution kernels (filters) to extract features from the input image. The formula can be expressed as:

[0090]

[0091] in, is the activation value of the lth layer at position (i, j), is the weight of the convolution kernel, is the input of the previous layer, b (l) is the bias, σ is the activation function (such as ReLU), and m and n represent the vertical and horizontal movement indexes on the input feature map during the convolution process, respectively.

[0092] Pooling layers (such as maximum pooling) are often inserted between convolutional layers to reduce feature dimensions and enhance the translation invariance of features. The formula is expressed as:

[0093] P ij =max1 a,b∈[0,k-1] A i·k+a,j·k+b

[0094] P ij is the output after the pooling operation, k is the size of the pooling window, and max is the operation of taking the maximum value in the k×k window.

[0095] (2) Video data feature extraction: Use 3D-CNN or recurrent neural network (LSTM, Long Short-Term Memory) to process video sequences and capture dynamic changes in time; 3D convolution operations can be performed not only in space but also in the temporal dimension to capture dynamic features in the video; LSTM is suitable for processing time series data and can remember long-term dependencies.

[0096] The state update of the LSTM unit is determined by the following formula:

[0097] f t =σ(W f ·[h t-1 ,x t ]+b f ),

[0098] i t =σ(W i ·[h t-1 ,x t ]+b i ),

[0099]

[0100] o t =σ(W o ·[h t-1 ,x t ]+b o ),

[0101] h t =o t tanh(C t ),

[0102] Among them, f t ,i t , o t are the activation values ​​of the forget gate, input gate, and output gate, respectively, W f , W i , W C , W o are the weight matrices of the forget gate, input gate, candidate cell state, and output gate, respectively. f Used to control whether the current time step forgets the previous state, W i Used to control the impact of the current input on the state of the candidate cell, W C Used to generate new candidate cell states, W o Used to control the output of the hidden state, b f , b i , b C , b o They are the bias items of the forget gate, input gate, candidate cell state and output gate, respectively. is the candidate cell state, C t is the current cell state, t and t-1 are the current time step and the previous time step respectively, h t is the hidden state of the current time step, h t-1 is the hidden state of the previous time step, the hidden state is the output of the recurrent neural network unit, which is used for the calculation of the next time step or the final output, and σ is the sigmoid activation function.

[0103] (3) Sensor data feature extraction: For sensor data such as temperature, pressure and gas, feature extraction can be performed through statistical methods such as mean, maximum, minimum and standard deviation.

[0104] Feature fusion of multimodal datasets: perform appropriate feature extraction for each data type after preprocessing. For example, for image data, use convolutional neural network (CNN) to extract key visual features, such as the shape and dynamics of flames and smoke; for video data, extract dynamic change features through temporal convolutional network (TCN) or long short-term memory network (LSTM) to capture the time series information of fire development; for sensor data, calculate statistical features (mean, variance, peak, etc.) as well as trend and periodic features extracted from time series data for data such as temperature, air pressure, voltage, pressure and gas.

[0105] The feature data are then standardized to ensure that they are on the same scale, helping to eliminate the impact of different magnitudes and distributions, making feature fusion more fair and effective. The formula is as follows:

[0106]

[0107] Then, we use a deep learning method, deep neural network (DNN), to fuse different feature sets. We combine all feature vectors into a long vector through feature concatenation and input it into the deep learning model:

[0108] v concat =[v image ;v video ;v sensor ]

[0109] Constructing a fire risk assessment model: Finally, the fused feature vector is used to construct and train a fire risk assessment model for lithium batteries used in mines. A deep neural network (DNN) is used in this patent.

[0110] In step 3, training and validating the fire risk assessment model includes:

[0111] Model training: distribute the data set:

[0112] Training set: used to train the model and adjust the model weight;

[0113] Validation set: used for performance evaluation during model training, helping to adjust hyperparameters and avoid overfitting;

[0114] Test set: Independent of the training and validation process, it is used to evaluate the performance of the model on unseen data and ensure the generalization ability of the model.

[0115] Next, train the model: use the training set to train the model, apply optimization algorithms such as gradient descent, and update the network parameters through back propagation; regularly use the validation set to check the model performance, adjust the learning rate, number of network layers, layer size or regularization parameters, etc.; monitor key indicators during training, such as accuracy, loss function value, recall rate and precision, to ensure that the model learns effectively.

[0116] Use grid search, random search, or Bayesian optimization methods to optimize hyperparameters and find the optimal model configuration.

[0117] Model validation and testing: Implement the k-fold cross-validation method to evaluate the stability and reliability of the model. This method can effectively improve the accuracy and robustness of model evaluation by splitting the dataset into different training and validation combinations multiple times; the selected final model configuration will be comprehensively tested on an independent test set to verify the performance of the model on unknown data. Focus on key indicators such as the model's accuracy, recall, and F1 score to ensure that the model can achieve the expected performance in practical applications; deeply analyze the model's performance under different conditions to identify possible sources of deviation and room for improvement. Evaluate the model's response time and prediction accuracy in various fire scenarios to ensure the model's application value.

[0118] Model deployment: Based on the results of testing and performance analysis, select the best model for deployment. Prepare the corresponding hardware and software environment to ensure the stable operation of the model in the actual operating environment. Implement an online monitoring mechanism to continuously track the effect of the model and make adjustments or optimizations when necessary.

[0119] In step 4, the fire risk prediction steps for mining lithium batteries include:

[0120] Fire risk prediction and assessment: Based on a fire risk assessment model that meets the standards, the real-time monitored data related to mining lithium batteries, including images, videos, and various sensor data (temperature, air pressure, voltage, pressure, gas, etc.), are input into the model. Ensure that the input data meets the format and specifications required by the model. Use the fire risk assessment model to analyze the preprocessed data and calculate the fire risk level in real time, which is divided into low risk, medium risk, and high risk. Set a threshold or decision boundary to classify the risk level. Compare the risk prediction value output by the model with the threshold to determine the final risk level.

[0121] Risk warning signal generation and transmission: Based on the risk level predicted by the model, the system automatically generates corresponding warning signals. These signals include visual and sound alarms, as well as emergency notifications sent to the mine safety management system. The warning signals are quickly transmitted to key areas, including the control center, safety management department, and on-site emergency response team through the mine's security communication network. According to the risk level and the predetermined emergency plan, the corresponding automated or manual response measures are initiated, such as activating the fire protection system, cutting off the power supply, and initiating evacuation procedures.

[0122] Monitoring and feedback: After the fire risk assessment system is fully deployed, continuously monitor its performance and effectiveness. Collect operational data, analyze the performance of the model in actual applications, and confirm the accuracy of warnings and response time. Adjust and optimize the fire risk assessment model based on monitoring results and user feedback. If necessary, update the model to adapt to new data patterns or improve the prediction algorithm. Regularly perform system maintenance and updates to adapt to environmental changes and technological developments to ensure the long-term effective operation of the system.

[0123] Figure 4 is a schematic diagram of the composition structure of a mining lithium battery fire risk assessment device based on multimodal deep learning provided by an embodiment of the present invention, such as Figure 4 As shown, a mine lithium battery fire risk assessment device 400 based on multimodal deep learning includes: an acquisition module 401, used to acquire the mine real-time environment data and battery temperature and pressure data of the mine lithium battery; the mine real-time environment data includes mine image data, mine video data and mine temperature and pressure data; a feature extraction 402, used to extract features of the mine image data, the mine video data, the mine temperature and pressure data and the battery temperature and pressure data, and obtain key visual features, dynamic change features, mine features and battery features; a fusion module 403, used to extract features of the key visual features, the dynamic change features, the mine features and the battery features. Perform feature fusion to obtain a fused feature vector; construct a module 404, which is used to input the fused feature vector into a deep learning model to construct a fire risk assessment model for lithium batteries used in mining; a training module 405, which is used to train a training set, a validation set and a test set based on the fire risk assessment model to obtain a trained fire risk assessment model; a prediction module 406, which is used to predict the fire risk of the lithium batteries used in mining based on the trained fire risk assessment model to obtain a fire risk prediction result; an evaluation module 407, which is used to evaluate the risk level based on the fire risk prediction result to obtain a risk assessment level, and generate a corresponding risk warning signal to implement corresponding response measures.

[0124] In some embodiments, the extraction module is also used to extract the key visual features from the mine image data through a convolutional neural network; convolve the mine video data in the time dimension and the space dimension through a recurrent neural network or a 3D convolutional neural network to obtain the dynamic change characteristics; and extract features from the mine temperature and pressure data and the battery temperature and pressure data through a preset statistical method to obtain the mine characteristics and the battery characteristics.

[0125] In some embodiments, the extraction module is also used in the formula of the convolutional layer in the convolutional neural network as follows: in, is the activation value of the lth layer at position (i, j), is the weight of the convolution kernel, is the input of the previous layer, b (l) is the bias, σ is the activation function, and m and n represent the vertical and horizontal movement indexes on the input feature map during the convolution process, respectively.

[0126] In some embodiments, the extraction module is also used to determine the state update of the recurrent neural network unit by the following formula: t =σ(W f ·[h t-1 ,x t ]+b f ),i t =σ(W i ·[h t-1 ,x t ]+b i ), o t =σ(W o ·[h t-1 ,x t ]+b o ), h t =o t tanh(C t ), where f t ,i t , o t are the activation values ​​of the forget gate, input gate, and output gate, respectively, W f , W i , W C , W o are the weight matrices of the forget gate, input gate, candidate cell state, and output gate, respectively. f Used to control whether the current time step forgets the previous state, W i Used to control the impact of the current input on the state of the candidate cell, W C Used to generate new candidate cell states, Wo Used to control the output of the hidden state, b f , b i , b C , b o They are the bias items of the forget gate, input gate, candidate cell state and output gate, respectively. is the candidate cell state, C t is the current cell state, t and t-1 are the current time step and the previous time step respectively, h t is the hidden state of the current time step, h t-1 is the hidden state of the previous time step, the hidden state is the output of the recurrent neural network unit, which is used for the calculation of the next time step or the final output, and σ is the sigmoid activation function.

[0127] In some embodiments, the fusion module is also used to standardize the key visual features, the dynamically changing features, the mine features and the battery features to obtain standardized key visual features, dynamically changing features, mine features and battery features; and use a deep neural network to perform feature fusion on the standardized key visual features, dynamically changing features, mine features and battery features to obtain the fused feature vector.

[0128] In some embodiments, the fire risk prediction result includes at least a risk prediction value; the assessment module is further used to compare the risk prediction value with a preset risk level threshold to determine the risk assessment level; and based on the risk assessment level, generate the corresponding risk warning signal.

[0129] It should be noted that the description of the device of the embodiment of the present invention is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the device embodiment, please refer to the description of the method embodiment of the present invention for understanding.

[0130] It should be noted that in the embodiment of the present invention, if the above-mentioned method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a terminal to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0131] Correspondingly, an embodiment of the present invention provides a mining lithium battery fire risk assessment device based on multimodal deep learning. Figure 5 is a schematic diagram of the composition structure of a mining lithium battery fire risk assessment device based on multimodal deep learning provided by an embodiment of the present invention, such as Figure 5 As shown, the multimodal deep learning-based mining lithium battery fire risk assessment device 500 at least includes: a processor 501 and a computer-readable storage medium 502 configured to store executable instructions, wherein the processor 501 generally controls the overall operation of the multimodal deep learning-based mining lithium battery fire risk assessment device 500. The computer-readable storage medium 502 is configured to store instructions and applications executable by the processor 501, and can also cache data to be processed or processed by each module in the processor 501 and the multimodal deep learning-based mining lithium battery fire risk assessment device 500, which can be implemented through flash memory (FLASH) or random access memory (RAM, Random Access Memory).

[0132] An embodiment of the present invention provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the embodiment of the present invention, for example, Figure 1 The method shown.

[0133] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above memories.

[0134] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0135] As an example, executable instructions may, but need not necessarily, correspond to a file in a file system, may be stored as part of a file storing other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0136] The above description is only an embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

[0137] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0138] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, an element defined by the statement "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed.

[0139] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning, characterized in that: The method comprises: Acquire real-time mine environment data and battery temperature and pressure data of mining lithium batteries; the real-time mine environment data includes mine image data, mine video data and mine temperature and pressure data; Performing feature extraction on the mine image data, the mine video data, the mine temperature and pressure data, and the battery temperature and pressure data, respectively, to obtain key visual features, dynamic change features, mine features, and battery features; Performing feature fusion on the key visual features, the dynamic change features, the mine features and the battery features to obtain a fused feature vector; Inputting the fused feature vector into a deep learning model to construct a fire risk assessment model for mining lithium batteries; Based on the fire risk assessment model, training the training set, the validation set and the test set to obtain a trained fire risk assessment model; Based on the trained fire risk assessment model, fire risk prediction is performed on the mining lithium battery to obtain a fire risk prediction result; A risk level assessment is performed based on the fire risk prediction result to obtain a risk assessment level, and a corresponding risk warning signal is generated to implement corresponding response measures.

2. The method according to claim 1, characterized in that: The feature extraction is performed on the mine image data, the mine video data, the mine temperature and pressure data, and the battery temperature and pressure data, respectively, to obtain key visual features, dynamic change features, mine features, and battery features, including: Extracting the key visual features from the mine image data by a convolutional neural network; Convolution of the mine video data in time dimension and space dimension is performed by a recurrent neural network or a 3D convolutional neural network to obtain the dynamic change feature; The mine characteristics and the battery characteristics are obtained by performing feature extraction on the mine temperature and pressure data and the battery temperature and pressure data through a preset statistical method.

3. The method according to claim 2, characterized in that The extracting the key visual features from the mine image data by using a convolutional neural network includes: The formula of the convolutional layer in the convolutional neural network is expressed as: in, is the activation value of the lth layer at position (i, j), is the weight of the convolution kernel, is the input of the previous layer, b (l) is the bias, σ is the activation function, and m and n represent the vertical and horizontal movement indexes on the input feature map during the convolution process, respectively.

4. The method according to claim 2, characterized in that: The mine video data is convolved in time and space dimensions by a recurrent neural network or a 3D-CNN to obtain the dynamic change features, including: The state update of the recurrent neural network unit is determined by the following formula: f t =σ(W f ·[h t-1 ,x t ]+b f ), i t =σ(W i ·[h t-1 ,x t ]+b i ), the t =σ(W o ·[h t-1 ,x t ]+b o ), h t =o t ·tanh(C t ), Among them, f t ,i t , o t are the activation values ​​of the forget gate, input gate, and output gate, respectively, W f , W i , W C , W o are the weight matrices of the forget gate, input gate, candidate cell state, and output gate, respectively. f Used to control whether the current time step forgets the previous state, W i Used to control the impact of the current input on the state of the candidate cell, W C Used to generate new candidate cell states, W o Used to control the output of the hidden state, b f , b i , b C , b o They are the bias items of the forget gate, input gate, candidate cell state and output gate, respectively. is the candidate cell state, C t is the current cell state, t and t-1 are the current time step and the previous time step respectively, h t is the hidden state of the current time step, h t-1 is the hidden state of the previous time step, the hidden state is the output of the recurrent neural network unit, which is used for the calculation of the next time step or the final output, and σ is the sigmoid activation function.

5. The method according to claim 1, characterized in that: The step of fusing the key visual features, the dynamic change features, the mine features and the battery features to obtain a fused feature vector includes: Standardizing the key visual features, the dynamic change features, the mine features, and the battery features to obtain standardized key visual features, dynamic change features, mine features, and battery features; A deep neural network is used to perform feature fusion on the standardized key visual features, dynamic change features, mine features and battery features to obtain the fused feature vector.

6. The method according to claim 1, characterized in that The fire risk prediction result at least includes a risk prediction value; the risk level assessment is performed based on the fire risk prediction result to obtain a risk assessment level, and a corresponding risk warning signal is generated to implement corresponding response measures, including: Comparing the risk prediction value with a preset risk level threshold to determine the risk assessment level; Based on the risk assessment level, the corresponding risk warning signal is generated.

7. A mine lithium battery fire risk assessment device based on multimodal deep learning, characterized in that: The device comprises: An acquisition module is used to acquire real-time mine environment data and battery temperature and pressure data of a mining lithium battery; the real-time mine environment data includes mine image data, mine video data, and mine temperature and pressure data; An extraction module, used to extract features from the mine image data, the mine video data, the mine temperature and pressure data, and the battery temperature and pressure data, respectively, to obtain key visual features, dynamic change features, mine features, and battery features; A fusion module, used for fusing the key visual features, the dynamic change features, the mine features and the battery features to obtain a fused feature vector; A construction module is used to input the fused feature vector into a deep learning model to construct a fire risk assessment model for mining lithium batteries; A training module, used to train a training set, a validation set and a test set based on the fire risk assessment model to obtain a trained fire risk assessment model; A prediction module, used to predict the fire risk of the mining lithium battery based on the trained fire risk assessment model to obtain a fire risk prediction result; The evaluation module is used to perform risk level evaluation based on the fire risk prediction result, obtain the risk evaluation level, and generate a corresponding risk warning signal to implement corresponding response measures.

8. A mine lithium battery fire risk assessment device based on multimodal deep learning, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the method for fire risk assessment of mining lithium batteries based on multimodal deep learning as described in any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the method for assessing the fire risk of lithium batteries for mining based on multimodal deep learning as described in any one of claims 1 to 6.

Images