Warehouse fire monitoring method and system based on image recognition

Abstract

The invention belongs to the technical field of fire monitoring, and discloses a warehouse fire monitoring method and system based on image recognition. The method comprises the following steps: based on a cloud data center, using an image recognition algorithm to construct a fire behavior recognition model, and using a digital twinborn technology to construct a warehouse digital twinborn model; based on an image data acquisition device, acquiring real-time image data of a monitoring area, encrypting the real-time image data through an edge computing gateway, and uploading the encrypted real-time image data to a cloud data center; and based on the cloud data center, according to the real-time image data, performing fire identification by using a fire identification model, and performing visual display on a generated real-time fire identification result by using a warehouse digital twin model. According to the invention, the problems of poor accuracy, limited monitoring range, insufficient real-time performance and lack of visual display in the prior art are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of fire monitoring, and in particular relates to a warehouse fire monitoring method and system based on image recognition. Background Art

[0002] With the continuous expansion of modern warehouses and the diversification of stored items, the potential safety hazards of warehouse fires have become increasingly prominent. In the first instance of a warehouse fire, an alarm needs to be sounded to remind workers to evacuate and notify warehouse managers to rescue personnel and property. With the development of monitoring technology and the popularization of sensor technology, warehouse fire monitoring through equipment such as smoke alarms and temperature sensors has become a major fire prevention measure.

[0003] The existing warehouse fire monitoring technology has the following defects: 1) Poor accuracy: Smoke alarms and temperature sensors are easily affected by environmental factors (such as dust, steam, temperature changes, etc.), resulting in high false alarm rates and poor accuracy, which brings inconvenience and trouble to warehouse management; 2) Limited monitoring range: Traditional sensor equipment can usually only monitor the area near it. For large warehouses, the monitoring range is limited and it is difficult to achieve full coverage of the entire warehouse; 3) Lack of real-time performance: Smoke alarms and temperature sensors can usually only provide alarms after a fire occurs, and cannot achieve real-time monitoring and early warning of fire conditions; 4) Lack of visual display: Traditional fire monitoring methods can usually only provide simple alarm signals and lack visual display of the fire, making it difficult for managers to quickly understand the fire situation. Summary of the invention

[0004] In order to solve the problems of poor accuracy, limited monitoring range, insufficient real-time performance and lack of visual display in the prior art, the present invention aims to provide a warehouse fire monitoring method and system based on image recognition.

[0005] The technical solution adopted by the present invention is: A warehouse fire monitoring method based on image recognition comprises the following steps: Based on the cloud data center, we use image recognition algorithms to build a fire identification model, and use digital twin technology to build a warehouse digital twin model; Based on the image data acquisition device, real-time image data of the monitoring area is collected, and the real-time image data is encrypted and uploaded to the cloud data center through the edge computing gateway; Based on the cloud data center, fire identification is performed using a fire identification model based on real-time image data, and the warehouse digital twin model is used to visualize the generated real-time fire identification results.

[0006] Furthermore, based on the cloud data center, an image recognition algorithm is used to build a fire recognition model, and a warehouse digital twin model is built using digital twin technology, including the following steps: Based on the cloud data center, a number of historical image data are collected, and the number of historical image data are preprocessed to obtain a number of preprocessed historical image data; Use the image recognition and transfer learning fusion algorithm to build the initial fire recognition model, and use the swarm intelligence optimization algorithm to generate the optimal initial network parameters of the fire recognition model; According to the optimal initial network parameters, the initial fire recognition model is improved, and several pre-processed historical image data are input for optimization training to obtain the final fire recognition model; According to the three-dimensional data of the warehouse, a corresponding three-dimensional simulation model of the warehouse is constructed, and image data streams are set for several digital entities in the three-dimensional simulation model of the warehouse to obtain a digital twin model of the warehouse.

[0007] Furthermore, an initial fire recognition model is constructed using an image recognition and transfer learning fusion algorithm, and a swarm intelligence optimization algorithm is used to generate the optimal initial network parameters of the fire recognition model, including the following steps: Use image recognition and transfer learning fusion algorithm to build an initial fire recognition model; The initial network parameters of the fire recognition model are encoded into individual vectors, and the swarm intelligence optimization algorithm is used to iteratively search for the best individual. The individual vector of the optimal individual is decoded to obtain the optimal initial network parameters of the fire recognition model.

[0008] Furthermore, the fire recognition model is constructed based on the ResNet-DANN-Elman algorithm, and the fire recognition model includes a fire recognition main network constructed based on the ResNet algorithm, a domain adversary constructed based on the DANN algorithm, and a deep learning classifier constructed based on the Elman algorithm. The fire recognition main network is connected to the transfer learning module and the deep learning classifier respectively.

[0009] Furthermore, the swarm intelligence optimization algorithm is an IFWA algorithm.

[0010] Furthermore, the initial network parameters of the fire recognition model are encoded into individual vectors, and the swarm intelligence optimization algorithm is used to iteratively search for the optimal individual, which includes the following steps: Taking minimizing the mean square error as the optimization objective, the initial network parameters of the fire recognition model are encoded into individual vectors of the IFWA algorithm; According to the optimization goal, set the fitness function of the IFWA algorithm, and set the IFWA population parameters and the maximum number of iterations; Initialization is performed according to the IFWA population parameters and individual vectors to generate a number of initial IFWA individuals of the initial IFWA population; According to the fitness function, the explosion radius, number of sparks and fitness value of the initial IFWA individual are obtained; According to the explosion radius, the number of sparks and the fitness value, a fireworks explosion is performed to obtain a number of updated IFWA individuals of the updated IFWA population; Using the Gaussian mutation algorithm, the initialized IFWA population is subjected to Gaussian mutation to generate a number of Gaussian mutated IFWA individuals of the Gaussian mutated IFWA population; Using a dynamic reverse learning algorithm, the initialized IFWA population is dynamically reversed to obtain a number of reverse IFWA individuals of the reverse IFWA population; Obtain the fitness value of each updated IFWA individual, Gaussian mutated IFWA individual and reverse IFWA individual, and take the IFWA individual with the minimum fitness value as the optimal individual; If the number of iterations of iterative optimization reaches the maximum number of iterations or the fitness value of the optimal individual meets the requirements, the optimal individual will be output.

[0011] Furthermore, based on the image data acquisition device, real-time image data of the monitoring area is collected, and the real-time image data is encrypted and uploaded to the cloud data center through the edge computing gateway, including the following steps: Generate a public-private key pair for the edge computing gateway based on a trusted institution, send the private key in the public-private key pair to the corresponding edge computing gateway, and publish the public key in the public-private key pair to the cloud data center; Based on the image data acquisition device, the initial real-time image data of the monitoring area is collected, and the initial real-time image data is sent to the edge computing gateway within the communication range; Based on the edge computing gateway, the initial real-time image data is preprocessed and compressed to obtain a real-time compressed package; The real-time compressed package is encrypted according to the private key to obtain the encrypted real-time compressed package, and the encrypted real-time compressed package is uploaded to the cloud data center.

[0012] Furthermore, based on the cloud data center, the fire identification model is used to identify the fire according to the real-time image data, and the warehouse digital twin model is used to visualize the generated real-time fire identification results, including the following steps: Based on the cloud data center, the encrypted real-time compressed package is decrypted and decompressed according to the public key to obtain the restored real-time image data and input it into the fire recognition model; According to the restored real-time image data, a fire situation recognition model is used to perform fire situation recognition to obtain a real-time fire situation recognition result; The restored real-time image data is written into the corresponding digital entity in the warehouse digital twin model as a real-time image data stream, and the real-time image data stream is visualized; If the real-time fire identification result indicates that a fire exists, a real-time fire alarm signal is generated, and the warehouse digital twin model is used to visualize the real-time fire identification result and the real-time fire alarm signal.

[0013] A warehouse fire monitoring system based on image recognition is used to implement a warehouse fire monitoring method. The system includes a cloud data center, several edge computing gateways and several image data acquisition devices. The cloud data center is communicated with the edge computing gateways of several monitoring areas respectively, and each edge computing gateway is communicated with several image data acquisition devices within the communication range respectively.

[0014] Furthermore, the cloud data center includes a model building unit and a fire situation identification and visualization display unit which are connected in sequence.

[0015] The beneficial effects of the present invention are: The present invention discloses a warehouse fire monitoring method and system based on image recognition. By using image recognition algorithm to construct an automated and intelligent fire recognition model, it can more accurately identify fires, reduce false alarms caused by environmental factors, improve monitoring accuracy and efficiency, and reduce cost investment; an edge-cloud collaboration mechanism is introduced to combine edge computing with a cloud data center, expand the monitoring range, and achieve comprehensive coverage of the entire warehouse, effectively solving the shortcomings of traditional warehouse fire monitoring in data processing capabilities and real-time performance, achieving real-time monitoring and early warning of fires, and reducing the difficulty of online monitoring; through digital twin technology, the present invention can display fire recognition results in a visual manner, helping operators to quickly understand the fire situation, improving intuitiveness and management convenience; image recognition technology can adapt to different environmental conditions, provide stable monitoring effects, reduce maintenance and replacement costs and time costs, effectively improve safety, and reduce fire hazards.

[0016] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the warehouse fire monitoring method based on image recognition in the present invention.

[0018] Figure 2 It is a structural block diagram of the warehouse fire monitoring system based on image recognition in the present invention. DETAILED DESCRIPTION

[0019] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.

[0020] Embodiment 1: like Figure 1 As shown, this embodiment provides a warehouse fire monitoring method based on image recognition, comprising the following steps: S1: Based on the cloud data center, use the image recognition algorithm to build a fire recognition model, and use the digital twin technology to build a warehouse digital twin model, including the following steps: S1-1: Based on the cloud data center, a number of historical image data are collected, and the number of historical image data are preprocessed to obtain a number of preprocessed historical image data; Preprocessing includes data cleaning, data enhancement, and size normalization. Data cleaning removes image data of low quality and incorrect labels. Data enhancement increases sample diversity and improves the generalization ability of the model by rotating, scaling, cropping, and color conversion. S1-2: Use the image recognition and transfer learning fusion algorithm to build an initial fire recognition model, and use the swarm intelligence optimization algorithm to generate the optimal initial network parameters of the fire recognition model, including the following steps: S1-2-1: Use image recognition and transfer learning fusion algorithm to build the initial fire recognition model; The fire recognition model is built based on the ResNet-DANN-Elman algorithm, and the fire recognition model includes a fire recognition main network built based on the Residual Network (ResNet) algorithm, a domain adversary built based on the Domain-Adversarial Neural Network (DANN) algorithm, and a deep learning classifier built based on the Elman algorithm. The fire recognition main network is connected to the transfer learning module and the deep learning classifier respectively; The main innovation of the residual network is the introduction of the concept of residual learning, which solves the gradient vanishing and gradient exploding problems in network training, allowing the network to be trained deeper and improving the model prediction accuracy. The domain adversary is used to distinguish the features of the source domain and the target domain, and try to distinguish which domain these features come from. Through this adversarial training, the main network of fire identification gradually learns to ignore domain-related information and only retains task-related information, improving prediction efficiency. The memory function of the deep learning classifier learns image features better through pre-training and performs classification prediction, further improving prediction accuracy and efficiency. The fire identification main network includes an input layer, a convolution layer, a residual block group and a global average pooling layer connected in sequence, the domain adversary includes a gradient reversal layer and a domain classification layer connected in sequence, the deep learning classifier includes a hidden layer, a connection layer and an output layer, the global average pooling layer of the fire identification main network is connected to the hidden layer of the deep learning classifier, and the residual block group of the fire identification main network is connected to the gradient reversal layer of the domain adversary; The input layer is used to receive input image data, the convolution layer is used to perform preliminary feature extraction on the image data, the residual block group includes multiple residual blocks, each group of residual blocks usually has the same number of output channels, and the size of the feature map is changed by changing the stride. Each residual block contains multiple convolutional layers and batch normalization layers, as well as a cross-layer data path. The design of the residual block helps solve the gradient vanishing problem in deep networks, allowing the network to be trained deeper. The global average pooling layer is used to convert the feature map into a fixed-length feature vector; The gradient reversal layer is located between the residual block group and the domain classification layer. Its function is to reverse the gradient during the back propagation process, so that the domain classifier can learn the features that distinguish the source domain from the target domain during the training process, while the main network for fire recognition is trained to generate domain-invariant features, thereby achieving adversarial training. The domain classification layer is used to predict whether the feature belongs to the source domain or the target domain based on the inverted features output by the gradient reversal layer; By utilizing the characteristics of transfer learning technology and based on a model pre-trained on a large data set, the model reduces the reliance on a large amount of labeled fire image data, effectively reducing the cost of data collection and annotation. By fine-tuning the model parameters, it can achieve high-performance recognition on a small amount of specific fire image data, improve the model's recognition ability in different fire image development situations, and can accurately identify fires in the early stages of development. The hidden layer contains ordinary neurons and successor neurons. Ordinary neurons are responsible for processing the input at the current moment, while successor neurons store the output of the hidden layer neurons at the previous moment as an additional input at the current moment, so that the network can remember the state at the previous moment and process sequence data. The successor layer does not directly receive external input, but stores the output of the hidden layer neurons at the previous moment. The number of neurons in the successor layer is usually the same as that in the hidden layer. The output of the successor layer is used as the input of the hidden layer, so that the network can capture the dynamic changes in the time series. The output layer is responsible for generating the final output of the network; S1-2-2: Encode the initial network parameters of the fire recognition model into individual vectors, use the Improved Fireworks Optimization Algorithm (IFWA) algorithm to iteratively optimize and generate the optimal individual, including the following steps: S1-2-2-1: Taking minimizing the mean square error as the optimization objective, the initial network parameters of the fire recognition model are encoded into individual vectors of the IFWA algorithm; S1-2-2-2: According to the optimization goal, set the fitness function of the IFWA algorithm, and set the IFWA population parameters and the maximum number of iterations; The formula is:

[0021] In the formula, is the fitness function; is the mean square error function; For the The true output value of samples; For the The ideal output value of samples; is the sample indicator; N is the total number of samples; S1-2-2-3: Initialize according to the IFWA population parameters and individual vectors to generate several initial IFWA individuals of the initial IFWA population; The formula is:

[0022] In the formula, is the initial IFWA individual of Circle chaos mapping; is the randomly generated initial IFWA individual; It is the IFWA individual indicator; S1-2-2-4: According to the fitness function, the explosion radius, number of sparks and fitness value of the initial IFWA individual are obtained; The formula is:

[0023] In the formula, For the initial IFWA individuals The number of sparks; is a numerical constant; is the maximum fitness value in the initialized IFWA population; For the initial IFWA individuals The fitness value of is an infinitesimal constant; a is the convergence factor; is a positive real number that is not 0;

[0024] In the formula, For the initial IFWA individuals Explosion radius; Adjusted constant for explosion radius; is the minimum fitness value in the initialized IFWA population;

[0025] In the formula, a is the convergence factor; tanh(.) is the hyperbolic tangent function; t , t max are the current number of iterations and the maximum number of iterations respectively; a max , a min are the maximum and minimum values ​​of the convergence factor, respectively; λ is the deceleration rate parameter, k is the decreasing cycle parameter, λ =-2 π , k = π ; The number of sparks determines the number of sub-fireworks produced after each firework explodes, and the explosion radius determines the distribution range of the sparks produced after the firework explodes in the solution space. In the early stage of iteration, a The larger the value of , the smaller the number of sparks of IFWA individuals and the larger the explosion radius, which helps to reduce the computational burden and is more widely distributed, which helps to explore more solution spaces. In the later stages of iteration, a smaller explosion radius helps to perform a fine search in a local area, and a larger number of sparks helps to increase the diversity of the search. S1-2-2-5: Perform fireworks explosion according to the explosion radius, the number of sparks and the fitness value to obtain a number of updated IFWA individuals of the updated IFWA population; The formula is:

[0026] In the formula, For updated IFWA individuals; A random number between -1 and 1; S1-2-2-6: Use the Gaussian mutation algorithm to perform Gaussian mutation on the initialized IFWA population to generate several Gaussian mutated IFWA individuals of the Gaussian mutated IFWA population; The formula is:

[0027] In the formula, is the IFWA individual of Gaussian variation; is a Gaussian distributed random number with mean and variance both 1; S1-2-2-7: Use the dynamic reverse learning algorithm to perform dynamic reverse learning on the initialized IFWA population to obtain several reverse IFWA individuals of the reverse IFWA population; The formula is:

[0028] In the formula, For reverse IFWA individuals; γ is the decreasing inertia coefficient; L max , L min are the maximum and minimum values ​​of the vector space respectively; S1-2-2-8: Obtain the fitness value of each updated IFWA individual, Gaussian mutated IFWA individual and reverse IFWA individual, and take the IFWA individual with the minimum fitness value as the optimal individual; S1-2-2-9: If the number of iterations of iterative optimization reaches the maximum number of iterations or the fitness value of the optimal individual meets the requirements, the optimal individual will be output; S1-2-3: Decode the individual vector of the optimal individual to obtain the optimal initial network parameters of the fire recognition model; The optimal initial network parameters include the optimal initial number of neurons, initial neuron weights, and initial neuron thresholds of the fire identification main network, domain adversary, and deep learning classifier; S1-3: According to the optimal initial network parameters, the initial fire recognition model is improved, and some pre-processed historical image data are input for optimization training to obtain the final fire recognition model, including the following steps: S1-3-1: According to the optimal initial network parameters, the initial fire situation recognition model is improved to obtain an improved fire situation recognition model; S1-3-2: Divide a number of pre-processed historical image data into a model training set and a model test set in a ratio of 7:3; S1-3-3: According to the model training set, the improved fire situation recognition model is optimized and trained to obtain the optimized fire situation recognition model, including the following steps: S1-3-3-1: The first loss function of the fire recognition main network (usually the cross entropy loss function of the fire recognition main network), the second loss function of the domain adversary (the loss function of the domain classification) and the third loss function of the deep learning classifier (the loss function of the classifier) ​​are weighted combined to obtain the joint loss function of the improved fire recognition model; S1-3-3-2: Use a number of preprocessed historical image data to pre-train the improved fire situation recognition model to obtain a pre-trained fire situation recognition model; S1-3-3-3: Input the preprocessed historical image data in the model training set into the pre-trained fire recognition model, and use the pre-trained fire recognition main network to extract the corresponding historical image data features; S1-3-3-4: Input the historical image data features into the deep learning classifier, perform classification prediction, and obtain the historical fire identification label; S1-3-3-5: Input the historical image data features into the gradient reversal layer of the domain adversary, perform gradient reversal on the historical image data features, and obtain the historical image data features after gradient reversal; S1-3-3-6: Input the historical image data features after gradient inversion into the domain classifier of the domain adversary to perform domain classification and obtain the historical domain classification label; S1-3-3-7: Based on the historical domain classification labels and historical fire identification labels, a joint loss function is used to obtain the historical loss value; S1-3-3-8: Based on the historical loss value, use the back propagation algorithm to update the model parameters of the pre-trained fire recognition model to obtain an updated fire recognition model; S1-3-3-9: Traverse all pre-processed historical image data in the model training set, and fine-tune the updated fire recognition model until the lowest historical loss value is output to obtain the optimized fire recognition model; S1-3-4: According to the model test set, the optimized fire situation recognition model is tested to obtain the model accuracy; S1-3-5: If the model accuracy is greater than the accuracy threshold, the final fire recognition model is output, otherwise, training optimization continues; S1-4: construct a corresponding warehouse 3D simulation model based on the warehouse 3D data, and set image data streams for several digital entities in the warehouse 3D simulation model to obtain a warehouse digital twin model; S2: Based on the image data acquisition device, collect real-time image data of the monitoring area, and encrypt and upload the real-time image data to the cloud data center through the edge computing gateway, including the following steps: S2-1: Generate a public-private key pair for the edge computing gateway based on a trusted institution, send the private key in the public-private key pair to the corresponding edge computing gateway, and publish the public key in the public-private key pair to the cloud data center; S2-2: Based on the image data acquisition device, collect initial real-time image data of the monitoring area, and send the initial real-time image data to the edge computing gateway within the communication range; S2-3: Based on the edge computing gateway, the initial real-time image data is preprocessed and compressed to obtain a real-time compressed package; S2-4: Encrypt the real-time compressed package according to the private key to obtain the encrypted real-time compressed package, and upload the encrypted real-time compressed package to the cloud data center; The formula is:

[0029] In the formula, It is a real-time compressed package after encryption; is an asymmetric encryption function; For real-time compression packages; Edge computing gateway The private key of Indicator for edge computing gateway; Encrypting the transmitted data prevents data from being intercepted and causing leakage of warehouse secrets, thus improving the security of data transmission; S3: Based on the cloud data center, the fire identification model is used to identify the fire according to the real-time image data, and the warehouse digital twin model is used to visualize the generated real-time fire identification results, including the following steps: S3-1: Based on the cloud data center, the encrypted real-time compressed package is decrypted and decompressed according to the public key to obtain the restored real-time image data and input it into the fire recognition model; The formula is:

[0030] In the formula, It is a real-time compressed package after decryption; is an asymmetric decryption function; It is a real-time compressed package after encryption; Edge computing gateway 's public key; S3-2: Based on the restored real-time image data, a fire situation recognition model is used to perform fire situation recognition to obtain a real-time fire situation recognition result; S3-3: Write the restored real-time image data as a real-time image data stream into the corresponding digital entity in the warehouse digital twin model, and visualize the real-time image data stream; S3-4: If the real-time fire identification result indicates that a fire exists, a real-time fire alarm signal is generated, and the warehouse digital twin model is used to visualize the real-time fire identification result and the real-time fire alarm signal.

[0031] Embodiment 2: like Figure 2As shown, this embodiment provides a warehouse fire monitoring system based on image recognition, which is used to implement a warehouse fire monitoring method. The system includes a cloud data center, several edge computing gateways, and several image data acquisition devices. The cloud data center is respectively connected to the edge computing gateways in several monitoring areas for communication, and each edge computing gateway is respectively connected to the image data acquisition devices within the communication range for communication; The cloud data center is used to build a fire identification model using an image recognition algorithm, and to build a warehouse digital twin model using digital twin technology; the fire identification model is used to identify fires based on real-time image data, and the warehouse digital twin model is used to visualize the generated real-time fire identification results; An image data acquisition device is used to collect real-time image data of the monitoring area and send the real-time image data to an edge computing gateway within the communication range; The edge computing gateway is used to pre-process and compress the initial real-time image data to obtain a real-time compressed package; encrypt the real-time compressed package to obtain an encrypted real-time compressed package, and upload the encrypted real-time compressed package to the cloud data center.

[0032] Preferably, the cloud data center includes a model building unit and a fire situation identification and visualization display unit connected in sequence; The model building unit uses image recognition algorithms to build a fire identification model and uses digital twin technology to build a warehouse digital twin model; The fire situation identification and visualization display unit uses the fire situation identification model to identify the fire situation based on the real-time image data, and uses the warehouse digital twin model to visualize the generated real-time fire situation identification results.

[0033] The present invention discloses a warehouse fire monitoring method and system based on image recognition. By using image recognition algorithm to construct an automated and intelligent fire recognition model, it can more accurately identify fires, reduce false alarms caused by environmental factors, improve monitoring accuracy and efficiency, and reduce cost investment; an edge-cloud collaboration mechanism is introduced to combine edge computing with a cloud data center, expand the monitoring range, and achieve comprehensive coverage of the entire warehouse, effectively solving the shortcomings of traditional warehouse fire monitoring in data processing capabilities and real-time performance, achieving real-time monitoring and early warning of fires, and reducing the difficulty of online monitoring; through digital twin technology, the present invention can display fire recognition results in a visual manner, helping operators to quickly understand the fire situation, improving intuitiveness and management convenience; image recognition technology can adapt to different environmental conditions, provide stable monitoring effects, reduce maintenance and replacement costs and time costs, effectively improve safety, and reduce fire hazards.

[0034] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0035] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0036] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0037] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0038] A person of ordinary skill in the art can understand that all or part of the steps in realizing the above-mentioned facts and methods can be completed by instructing the relevant hardware through a program, and the program involved or the program described can be stored in a computer-readable storage medium. When the program is executed, it includes the following steps: At this time, the corresponding method steps are derived, and the storage medium can be ROM / RAM, a disk, an optical disk, etc.

[0039] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A warehouse fire monitoring method based on image recognition, characterized in that: The steps include: Based on the cloud data center, we use image recognition algorithms to build a fire identification model, and use digital twin technology to build a warehouse digital twin model; Based on the image data acquisition device, real-time image data of the monitoring area is collected, and the real-time image data is encrypted and uploaded to the cloud data center through the edge computing gateway; Based on the cloud data center, fire identification is performed using a fire identification model based on real-time image data, and the warehouse digital twin model is used to visualize the generated real-time fire identification results.

2. A warehouse fire monitoring method based on image recognition according to claim 1, characterized in that: Based on the cloud data center, the image recognition algorithm is used to build a fire recognition model, and the digital twin technology is used to build a warehouse digital twin model, including the following steps: Based on the cloud data center, a number of historical image data are collected, and the number of historical image data are preprocessed to obtain a number of preprocessed historical image data; Use the image recognition and transfer learning fusion algorithm to build the initial fire recognition model, and use the swarm intelligence optimization algorithm to generate the optimal initial network parameters of the fire recognition model; According to the optimal initial network parameters, the initial fire recognition model is improved, and several pre-processed historical image data are input for optimization training to obtain the final fire recognition model; According to the three-dimensional data of the warehouse, a corresponding three-dimensional simulation model of the warehouse is constructed, and image data streams are set for several digital entities in the three-dimensional simulation model of the warehouse to obtain a digital twin model of the warehouse.

3. A warehouse fire monitoring method based on image recognition according to claim 2, characterized in that: The image recognition and transfer learning fusion algorithm is used to build an initial fire recognition model, and the swarm intelligence optimization algorithm is used to generate the optimal initial network parameters of the fire recognition model, including the following steps: Use image recognition and transfer learning fusion algorithm to build an initial fire recognition model; The initial network parameters of the fire recognition model are encoded into individual vectors, and the swarm intelligence optimization algorithm is used to iteratively search for the best individual. The individual vector of the optimal individual is decoded to obtain the optimal initial network parameters of the fire recognition model.

4. A warehouse fire monitoring method based on image recognition according to claim 3, characterized in that: The fire recognition model is constructed based on the ResNet-DANN-Elman algorithm, and the fire recognition model includes a fire recognition main network constructed based on the ResNet algorithm, a domain adversary constructed based on the DANN algorithm, and a deep learning classifier constructed based on the Elman algorithm. The fire recognition main network is connected to the transfer learning module and the deep learning classifier respectively.

5. The warehouse fire monitoring method based on image recognition according to claim 3 is characterized in that: The swarm intelligence optimization algorithm is the IFWA algorithm.

6. A warehouse fire monitoring method based on image recognition according to claim 5, characterized in that: The initial network parameters of the fire recognition model are encoded into individual vectors, and the swarm intelligence optimization algorithm is used to iteratively search for the best individual, including the following steps: Taking minimizing the mean square error as the optimization objective, the initial network parameters of the fire recognition model are encoded into individual vectors of the IFWA algorithm; According to the optimization goal, set the fitness function of the IFWA algorithm, and set the IFWA population parameters and the maximum number of iterations; Initialization is performed according to the IFWA population parameters and individual vectors to generate a number of initial IFWA individuals of the initial IFWA population; According to the fitness function, the explosion radius, number of sparks and fitness value of the initial IFWA individual are obtained; According to the explosion radius, the number of sparks and the fitness value, a fireworks explosion is performed to obtain a number of updated IFWA individuals of the updated IFWA population; Using the Gaussian mutation algorithm, the initialized IFWA population is subjected to Gaussian mutation to generate a number of Gaussian mutated IFWA individuals of the Gaussian mutated IFWA population; Using a dynamic reverse learning algorithm, the initialized IFWA population is dynamically reversed to obtain a number of reverse IFWA individuals of the reverse IFWA population; Obtain the fitness value of each updated IFWA individual, Gaussian mutated IFWA individual and reverse IFWA individual, and take the IFWA individual with the minimum fitness value as the optimal individual; If the number of iterations of iterative optimization reaches the maximum number of iterations or the fitness value of the optimal individual meets the requirements, the optimal individual will be output.

7. The warehouse fire monitoring method based on image recognition according to claim 2 is characterized in that: Based on the image data acquisition device, real-time image data of the monitoring area is collected, and the real-time image data is encrypted and uploaded to the cloud data center through the edge computing gateway, including the following steps: Generate a public-private key pair for the edge computing gateway based on a trusted institution, send the private key in the public-private key pair to the corresponding edge computing gateway, and publish the public key in the public-private key pair to the cloud data center; Based on the image data acquisition device, the initial real-time image data of the monitoring area is collected, and the initial real-time image data is sent to the edge computing gateway within the communication range; Based on the edge computing gateway, the initial real-time image data is preprocessed and compressed to obtain a real-time compressed package; The real-time compressed package is encrypted according to the private key to obtain the encrypted real-time compressed package, and the encrypted real-time compressed package is uploaded to the cloud data center.

8. The warehouse fire monitoring method based on image recognition according to claim 7 is characterized in that: Based on the cloud data center, the fire identification model is used to identify the fire according to the real-time image data, and the warehouse digital twin model is used to visualize the generated real-time fire identification results, including the following steps: Based on the cloud data center, the encrypted real-time compressed package is decrypted and decompressed according to the public key to obtain the restored real-time image data and input it into the fire recognition model; According to the restored real-time image data, a fire situation recognition model is used to perform fire situation recognition to obtain a real-time fire situation recognition result; The restored real-time image data is written into the corresponding digital entity in the warehouse digital twin model as a real-time image data stream, and the real-time image data stream is visualized; If the real-time fire identification result indicates that a fire exists, a real-time fire alarm signal is generated, and the warehouse digital twin model is used to visualize the real-time fire identification result and the real-time fire alarm signal.

9. A warehouse fire monitoring system based on image recognition, used to implement the warehouse fire monitoring method according to any one of claims 1 to 8, characterized in that: The system includes a cloud data center, several edge computing gateways and several image data acquisition devices. The cloud data center is respectively communicated with the edge computing gateways of several monitoring areas, and each of the edge computing gateways is respectively communicated with the several image data acquisition devices within the communication range.

10. The warehouse fire monitoring system based on image recognition according to claim 9, characterized in that: The cloud data center includes a model building unit and a fire situation identification and visualization display unit which are connected in sequence.

Images