Intelligent early warning system and method for fire in electric bicycle charging and parking places

Abstract

The present application provides an intelligent early warning system and method for fires in electric bicycle charging and parking places, which relates to the field of fire early warning. It uses image analysis and data processing technology based on artificial intelligence to perform multi-scale feature extraction and significant perception fusion on the multi-channel image of the state of the electric bicycle charging and parking place obtained by aggregating thermal infrared images and RGB images, and performs time-series correlation coding on the smoke concentration value data, so as to intelligently judge whether a fire is found based on the cross-modal features between the features obtained by significant fusion of the multi-channel image of the state of the electric bicycle charging and parking place and the time-series correlation features of the smoke concentration value. In this way, the fire risk can be evaluated more comprehensively, and the accuracy and timeliness of the fire warning can be improved, thereby realizing a more intelligent fire warning for electric bicycle charging and parking places.

Description

Technical Field

[0001] The present application relates to the field of fire warning, and more specifically, to an intelligent fire warning system and method for electric bicycle charging and parking places. Background Art

[0002] In recent years, electric vehicle fire accidents have occurred frequently, and the main cause of fire is attributed to electrical failures, especially safety hazards in the battery and charging process. Specifically, most electric vehicle bodies use flammable materials. Once a fire occurs, the fire spreads rapidly and produces a large amount of toxic smoke. These smoke not only reduce visibility, making it extremely difficult for people to escape, but also toxic gases may cause suffocation and other health hazards, resulting in serious casualties. Therefore, strengthening fire warning and emergency measures is crucial to protecting people's lives and property.

[0003] However, the fire warning systems in traditional electric bicycle charging and parking places mainly rely on a single type of detector, such as smoke detectors or temperature sensors. Although this method can monitor fires to a certain extent, it often lacks the ability to fully assess fire risks. For example, they may not be able to detect the early stages of fires, such as hidden fire sources caused by electrical faults, or fires without smoke generation. In addition, traditional early warning systems usually require manual analysis and inspection and maintenance of some fire detection data, which not only increases operating costs, but also may result in inaccurate analysis, inadequate inspections, or untimely maintenance due to human factors. Once a system fails, it may not be detected immediately, resulting in the failure of the early warning system, which will not be able to provide timely alarms when a fire occurs, increasing the risk of serious consequences caused by fire.

[0004] Therefore, an optimized fire warning solution for electric bicycle charging and parking places is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide an intelligent fire warning system and method for electric bicycle charging and parking places.

[0006] According to one aspect of the present application, an intelligent early warning system for fire in electric bicycle charging and parking places is provided, which comprises:

[0007] Collection unit, fire extinguishing unit, linkage unit and fire early warning control center;

[0008] The acquisition unit includes a thermal imaging binocular camera and a smoke detector, wherein the thermal imaging binocular camera is used to acquire thermal infrared images and RGB images of the electric bicycle charging and parking places, and the smoke detector is used to acquire the time queue of the smoke concentration value;

[0009] The fire warning control center is used to determine whether a fire is found based on the time queue of the thermal infrared image, the RGB image and the smoke density value, and generate a fire extinguishing control instruction in response to the fire being found;

[0010] The fire extinguishing unit includes an automatic sprinkler device and a suspended dry powder fire extinguisher, and the fire extinguishing unit is used to start the automatic sprinkler device and the suspended dry powder fire extinguisher in response to receiving the fire extinguishing control instruction;

[0011] The linkage unit is used to start the emergency light and shut down the charging pile in response to receiving the fire extinguishing control instruction.

[0012] In the above-mentioned intelligent early warning system for fires in electric bicycle charging and parking places, the fire early warning control center includes: a thermal infrared image RGB image aggregation module, which is used to aggregate the thermal infrared image and the RGB image along the channel dimension to obtain a multi-channel image of the state of the electric bicycle charging and parking place; a multi-scale feature extraction module for electric bicycle charging and parking places, which is used to input the multi-channel image of the state of the electric bicycle charging and parking place into a multi-scale feature extractor of the on-site state to obtain a shallow state feature map of the electric bicycle charging and parking place and a deep state feature map of the electric bicycle charging and parking place; a multi-scale saliency perception module for electric bicycle charging and parking places, which is used to input the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place The feature map is input into an attention-driven feature joint saliency perception network to obtain a multi-scale salient perception fusion feature map of an electric bicycle charging and parking place; a smoke concentration value sequence encoding module is used to input the time queue of the smoke concentration value into a sequence encoder to obtain a smoke concentration time series associated feature vector; a fire detection multimodal fusion module is used to input the multi-scale salient perception fusion feature map of an electric bicycle charging and parking place and the smoke concentration time series associated feature vector into a gated cross-modal joint encoder based on an imitation converter structure to obtain a fire detection multimodal fusion feature map; a fire detection result generation module is used to obtain a fire detection result based on the fire detection multimodal fusion feature map, and the fire detection result is used to indicate whether a fire is found.

[0013] In the above-mentioned intelligent early warning system for fires in electric bicycle charging and parking places, the multi-scale feature extraction module of the electric bicycle charging and parking place is used to: input the multi-channel image of the electric bicycle charging and parking place state into the field state multi-scale feature extractor based on the hollow pyramid network to obtain the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place.

[0014] In the above-mentioned intelligent early warning system for fires in electric bicycle charging and parking places, the multi-scale significant perception module of the electric bicycle charging and parking places includes: an electric bicycle charging and parking place feature map shape reshaping unit, which is used to reshape the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place to obtain the shallow state feature shape reshaping matrix of the electric bicycle charging and parking place and the deep state feature shape reshaping matrix of the electric bicycle charging and parking place; an electric bicycle charging and parking place shallow-deep state channel-by-channel interactive perception unit, which is used to reshape the shallow state feature shape reshaping matrix of the electric bicycle charging and parking place and the deep state feature shape reshaping matrix of the electric bicycle charging and parking place The deep state feature shape reshaping matrix of the parking place is input into the feature channel-by-channel interactive perception module to obtain the shallow-deep state dependency matrix of the electric bicycle charging and parking place and the deep-shallow state dependency matrix of the electric bicycle charging and parking place; the electric bicycle charging and parking place dependency matrix random deactivation processing unit is used to input the shallow-deep state dependency matrix of the electric bicycle charging and parking place and the deep-shallow state dependency matrix of the electric bicycle charging and parking place into the random deactivation module to obtain the pruned shallow-deep state dependency matrix of the electric bicycle charging and parking place and the pruned deep-shallow state dependency matrix of the electric bicycle charging and parking place; dependency optimization The deep state feature generation unit of the electric bicycle charging and parking place is used for matrix multiplication of the pruned shallow-deep state dependency matrix of the electric bicycle charging and parking place with the deep state feature shape reshaping matrix of the electric bicycle charging and parking place to obtain the dependency relationship optimized deep state feature matrix of the electric bicycle charging and parking place; the dependency relationship optimized shallow state feature generation unit of the electric bicycle charging and parking place is used for matrix multiplication of the pruned deep-shallow state dependency matrix of the electric bicycle charging and parking place with the shallow state feature shape reshaping matrix of the electric bicycle charging and parking place to obtain the dependency relationship optimized shallow state feature matrix of the electric bicycle charging and parking place; A relationship-optimized electric bicycle charging and parking place feature matrix shape reshaping unit is used to perform feature shape reshaping on the dependency-optimized electric bicycle charging and parking place shallow state feature matrix and the dependency-optimized electric bicycle charging and parking place deep state feature matrix to obtain an optimized electric bicycle charging and parking place shallow state feature map and an optimized electric bicycle charging and parking place deep state feature map; a multi-scale electric bicycle charging and parking place feature fusion unit is used to calculate the weighted sum of the optimized electric bicycle charging and parking place shallow state feature map and the optimized electric bicycle charging and parking place deep state feature map to obtain the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place.

[0015] In the above-mentioned intelligent early warning system for fire in electric bicycle charging and parking places, the electric bicycle charging and parking place shallow-deep state channel-by-channel interactive perception unit is used to: calculate the matrix product between the transposed matrix of the electric bicycle charging and parking place shallow state feature shape reshaping matrix and the electric bicycle charging and parking place deep state feature shape reshaping matrix to obtain a first correlation feature matrix; calculate the positional division of the scale of the first correlation feature matrix and the first correlation feature matrix to obtain a first correlation feature modulation matrix; calculate the matrix product between the transposed matrix of the electric bicycle charging and parking place deep state feature shape reshaping matrix and the electric bicycle charging and parking place shallow state feature shape reshaping matrix to obtain a second correlation feature matrix; calculate the positional division of the scale of the second correlation feature matrix and the second correlation feature matrix to obtain a second correlation feature modulation matrix; input the first correlation feature modulation matrix and the second correlation feature modulation matrix into the Softmax function respectively to obtain the electric bicycle charging and parking place shallow-deep state dependency matrix and the electric bicycle charging and parking place deep-shallow state dependency matrix.

[0016] In the above-mentioned intelligent early warning system for fire in electric bicycle charging and parking places, the smoke concentration value sequence encoding module is used to: input the time queue of the smoke concentration value into a sequence encoder based on the Bi-LSTM model to obtain the smoke concentration time series correlation feature vector.

[0017] In the above-mentioned intelligent early warning system for fire in electric bicycle charging and parking places, the fire detection multimodal fusion module includes: a multi-scale feature normalization convolution unit for electric bicycle charging and parking places, which is used to perform layer normalization and convolution coding processing on the multi-scale significant perception fusion feature map of the electric bicycle charging and parking places to obtain a multi-scale significant perception space deepening feature map of the electric bicycle charging and parking places; a multi-scale significant perception space deepening feature backup unit for electric bicycle charging and parking places, which is used to copy the multi-scale significant perception space deepening feature map of the electric bicycle charging and parking places to obtain a backup multi-scale significant perception space deepening feature map of the electric bicycle charging and parking places; a multi-scale significant perception space deepening feature shape reshaping unit for electric bicycle charging and parking places, which is used to reshape the feature shape of the backup multi-scale significant perception space deepening feature map of the electric bicycle charging and parking places and the multi-scale significant perception space deepening feature map of the electric bicycle charging and parking places respectively to obtain a backup multi-scale significant perception space deepening feature matrix of the electric bicycle charging and parking places and a multi-scale significant perception space deepening feature matrix of the electric bicycle charging and parking places as a simulated query feature matrix and a simulated key feature matrix; a multi-scale significant perception local autocorrelation library matrix calculation unit for electric bicycle charging and parking places, which is used to calculate the The product of the imitation query feature matrix and the transposed matrix of the imitation key feature matrix is ​​used to obtain a multi-scale significant perception local autocorrelation library matrix of electric bicycle charging and parking places; a multi-scale significant perception attention gated autocorrelation analysis unit of electric bicycle charging and parking places is used to input the multi-scale significant perception local autocorrelation library matrix of electric bicycle charging and parking places into an attention gating unit based on a GELU function to obtain a multi-scale significant perception gated local autocorrelation library weight matrix of electric bicycle charging and parking places; a smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight calculation unit is used to use the smoke concentration time series correlation feature matrix to obtain a multi-scale significant perception gated local autocorrelation library weight matrix of electric bicycle charging and parking places. The eigenvector is used as the value eigenvector, and the product between the weight matrix of the multi-scale significant perception gated local autocorrelation library of the electric bicycle charging and parking place and the time series correlation eigenvector of the smoke concentration is calculated to obtain the smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight vector; the multi-scale significant perception fusion feature weighting unit of the electric bicycle charging and parking place is used to use the smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight vector as the weight vector, and perform channel-by-channel weighting on the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain the fire detection multi-modal fusion feature map.

[0018] In the above-mentioned intelligent early warning system for fire in electric bicycle charging and parking places, the multi-scale feature normalization convolution unit of the electric bicycle charging and parking place is used to: perform layer normalization processing on the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain a layer-normalized multi-scale significant perception fusion feature map of the electric bicycle charging and parking place; perform point convolution encoding and spatial convolution encoding on the layer-normalized multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain the multi-scale significant perception spatial deepening feature map of the electric bicycle charging and parking place.

[0019] In the above-mentioned intelligent fire warning system for electric bicycle charging and parking places, the fire detection result generation module is used to: input the fire detection multimodal fusion feature map into the classifier-based fire detection module to obtain the fire detection result, and the fire detection result is used to indicate whether a fire is detected.

[0020] According to another aspect of the present application, a fire intelligent early warning method for electric bicycle charging and parking places is provided, which includes:

[0021] Collect thermal infrared images and RGB images of electric bicycle charging and parking areas, as well as the time queue of smoke concentration values;

[0022] determining whether a fire is found based on the thermal infrared image, the RGB image and the time sequence of the smoke density value, and generating a fire extinguishing control instruction in response to the fire being found;

[0023] In response to receiving the fire extinguishing control instruction, starting the automatic sprinkler equipment and the suspended dry powder fire extinguisher;

[0024] In response to receiving the fire extinguishing control instruction, the emergency light is activated and the charging pile is closed.

[0025] Compared with the prior art, the electric bicycle charging and parking place fire intelligent early warning system and method provided by the present application adopts the image analysis and data processing technology based on artificial intelligence to extract multi-scale features and significant perception fusion of the multi-channel image of the electric bicycle charging and parking place status obtained by aggregating thermal infrared images and RGB images, and performs time-series correlation coding on the smoke concentration value data, so as to intelligently judge whether a fire is found based on the cross-modal features between the features obtained by significant fusion of the multi-channel image of the electric bicycle charging and parking place status and the time-series correlation features of the smoke concentration value. In this way, the fire risk can be evaluated more comprehensively, and the accuracy and timeliness of the fire early warning can be improved, thereby realizing a more intelligent fire early warning for electric bicycle charging and parking places. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0027] Figure 1 4 is a block diagram of an intelligent early warning system for fire in electric bicycle charging and parking areas according to an embodiment of the present application.

[0028] Figure 2 This is a block diagram of a fire warning control center in an intelligent fire warning system for electric bicycle charging and parking places according to an embodiment of the present application.

[0029] Figure 3 This is a schematic diagram of the data flow of the fire warning control center in the intelligent fire warning system for electric bicycle charging and parking places according to an embodiment of the present application.

[0030] Figure 4 This is a scenario application diagram of an intelligent fire warning system for electric bicycle charging and parking places according to other embodiments of the present application.

[0031] Figure 5 This is a flow chart of an intelligent early warning method for fire in an electric bicycle charging and parking area according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0033] Electric vehicle fire accidents are frequent, mainly due to electrical failures and the use of flammable materials. Traditional fire warning systems rely on a single detector, which makes it difficult to fully assess fire risks and requires manual maintenance, resulting in inaccurate analysis and untimely maintenance.

[0034] Based on this, the present application proposes an intelligent fire warning system for electric bicycle charging and parking places. Figure 1 FIG. 1 is a block diagram of an intelligent early warning system for fires in electric bicycle charging and parking areas according to an embodiment of the present application. Figure 1As shown, in the intelligent fire warning system 100 for electric bicycle charging and parking places, it includes: a collection unit 110, a fire extinguishing unit 130, a linkage unit 140 and a fire warning control center 120; the collection unit 110 includes a thermal imaging binocular camera and a smoke detector, the thermal imaging binocular camera is used to collect thermal infrared images and RGB images of the electric bicycle charging and parking places, and the smoke detector is used to collect the time queue of smoke concentration values; the fire warning control center 120 is used to determine whether a fire is found based on the thermal infrared image, the RGB image and the time queue of the smoke concentration value, and generate a start fire extinguishing control instruction in response to the discovery of a fire; the fire extinguishing unit 130 includes an automatic sprinkler device and a suspended dry powder fire extinguisher, and the fire extinguishing unit 130 is used to start the automatic sprinkler device and the suspended dry powder fire extinguisher in response to receiving the fire extinguishing control instruction; the linkage unit 140 is used to start the emergency light and close the charging pile in response to receiving the fire extinguishing control instruction.

[0035] It should be understandable that, considering that the thermal imaging binocular camera can detect the thermal radiation of an object and generate a thermal infrared image, it can show the temperature distribution of different areas of the detected object. When a fire occurs, the temperature of an electric bicycle will rise abnormally in some areas. In order to detect the fire in time, a thermal imaging binocular camera is needed to collect thermal infrared images of the electric bicycle charging and parking places. In addition, RGB images of the electric bicycle charging and parking places need to be collected. The RGB images provide visual information of the electric bicycle, which helps to determine the stage of fire development. An increase in smoke concentration is usually direct evidence of a fire. Smoke detectors are responsible for collecting smoke concentration data in the air, which also helps to detect fires in electric bicycle charging and parking places in time.

[0036] The combination of thermal imaging binocular cameras and smoke detectors forms a multi-dimensional monitoring method. In order to identify fires in a timely and accurate manner, it is necessary to analyze the collected image data and smoke concentration data, that is, to upload the collected monitoring data to the fire warning control center, where a deep learning algorithm will be used to analyze the monitoring data in real time, and then determine the fire situation in the electric bicycle charging and parking area, and generate a fire extinguishing control command based on the fire situation obtained by real-time analysis.

[0037] The generated fire extinguishing control command will be sent to the relevant equipment through the central control server. When the automatic sprinkler equipment, hanging dry powder fire extinguisher, emergency light, charging pile and other related equipment receive the command, the sprinkler equipment will start spraying, the dry powder fire extinguisher will automatically release the dry powder fire extinguishing agent, the emergency light will turn on, and the charging pile will be turned off. This automated emergency response mechanism can ensure that effective fire extinguishing measures can be taken quickly when a fire occurs, reducing the threat of fire to the safety of personnel and property.

[0038] Correspondingly, the fire warning control center aggregates the thermal infrared image and RGB image collected by the thermal imaging binocular camera to obtain a multi-channel image of the state of the electric bicycle charging and parking place, obtains the time queue of the smoke concentration value, and uses the image analysis and data processing technology based on artificial intelligence to extract multi-scale features and significant perception fusion of the multi-channel image of the state of the electric bicycle charging and parking place, and performs time series correlation coding on the smoke concentration value, so as to intelligently judge whether a fire is found according to the cross-modal features between the features obtained by the significant fusion of the multi-channel image of the state of the electric bicycle charging and parking place and the time series correlation features of the smoke concentration value. In this way, temperature anomalies, visible light changes and smoke concentrations can be monitored simultaneously to more comprehensively evaluate fire risks from multiple angles. At the same time, artificial intelligence processing technology is used to accurately identify hidden fires, improve the accuracy and timeliness of warnings, thereby reducing the need for manual intervention, thereby realizing a more intelligent fire warning for electric bicycle charging and parking places.

[0039] Figure 2 This is a block diagram of a fire warning control center in an intelligent fire warning system for electric bicycle charging and parking places according to an embodiment of the present application. Figure 3 This is a schematic diagram of the data flow of the fire warning control center in the intelligent fire warning system for electric bicycle charging and parking places according to the embodiment of the present application. Figure 2 and Figure 3As shown, in the fire warning control center 120, it includes: a thermal infrared image RGB image aggregation module 121, which is used to aggregate the thermal infrared image and the RGB image along the channel dimension to obtain a multi-channel image of the state of the electric bicycle charging and parking place; a multi-scale feature extraction module 122 for the electric bicycle charging and parking place, which is used to input the multi-channel image of the state of the electric bicycle charging and parking place into the multi-scale feature extractor of the scene state to obtain a shallow state feature map of the electric bicycle charging and parking place and a deep state feature map of the electric bicycle charging and parking place; a multi-scale saliency perception module 123 for the electric bicycle charging and parking place, which is used to input the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place into the multi-scale saliency perception module 124 based on the annotation. The intention-driven feature joint saliency perception network is used to obtain a multi-scale salient perception fusion feature map of the electric bicycle charging and parking place; the smoke concentration value sequence encoding module 124 is used to input the time queue of the smoke concentration value into the sequence encoder to obtain the smoke concentration time series correlation feature vector; the fire detection multimodal fusion module 125 is used to input the multi-scale salient perception fusion feature map of the electric bicycle charging and parking place and the smoke concentration time series correlation feature vector into the gated cross-modal joint encoder based on the simulated converter structure to obtain the fire detection multimodal fusion feature map; the fire detection result generation module 126 is used to obtain the fire detection result based on the fire detection multimodal fusion feature map, and the fire detection result is used to indicate whether a fire is found.

[0040] In an embodiment of the present application, the thermal infrared image RGB image aggregation module 121 is used to aggregate the thermal infrared image and the RGB image along the channel dimension to obtain a multi-channel image of the state of the electric bicycle charging and parking place. Accordingly, considering that the thermal infrared image and the RGB image respectively express the temperature distribution of the surface of the electric bicycle and the visual information of the surface, such as color, texture, etc. In order to obtain richer data and thus more accurately identify and analyze the scene, in the technical solution of the present application, the thermal infrared image and the RGB image are aggregated along the channel dimension to obtain a multi-channel image of the state of the electric bicycle charging and parking place. In other words, a single thermal infrared image may not provide enough information to determine whether a fire has occurred, especially when the temperature change is not obvious or there are other heat sources interfering. RGB images can provide additional visual clues to help the system make more accurate judgments.

[0041] In an embodiment of the present application, the multi-scale feature extraction module 122 of the electric bicycle charging and parking place is used to input the multi-channel image of the electric bicycle charging and parking place status into the multi-scale feature extractor of the on-site status to obtain the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place. Specifically, in an embodiment of the present application, the multi-scale feature extraction module of the electric bicycle charging and parking place is used to: input the multi-channel image of the electric bicycle charging and parking place status into the multi-scale feature extractor of the on-site status based on the hollow pyramid network to obtain the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place. It should be understood that the multi-channel image of the electric bicycle charging and parking place status contains features of different scales, such as shallow color, texture, edge and other features, and deep semantic features such as key behaviors of electric bicycles. Based on this, in the technical solution of the present application, the multi-channel image of the state of the electric bicycle charging and parking place is input into the multi-scale feature extractor of the field state based on the dilated pyramid network to capture and extract feature information of different scales, and obtain the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place. It should be understood that the dilated pyramid network is a deep learning network structure that combines the ideas of dilated convolution and feature pyramid. The dilated convolution increases the receptive field of the convolution kernel without increasing the number of parameters and the amount of calculation by inserting "holes" (i.e., zero padding) between the elements of the standard convolution kernel. This feature enables the dilated convolution to capture a wider range of contextual information while maintaining a high-resolution feature map. The dilated pyramid network further utilizes this feature of the dilated convolution to construct a multi-scale feature pyramid structure. In this structure, feature maps of different levels are extracted through dilated convolution, and each layer retains rich semantic information and spatial details, and has different receptive fields. Through mechanisms such as hollow lateral connections, these feature maps at different levels are effectively fused together to generate a multi-level feature representation that contains rich semantic information and retains sufficient detail information. Considering that the characteristics of fire (such as flames and smoke) may appear at different scales, by inputting the multi-channel image of the state of the electric bicycle charging and parking area into the multi-scale feature extractor of the field state based on the hollow pyramid network, it can ensure that the information of different scales in the input multi-channel image is paid attention to at the same time, thereby enhancing the sensitivity of fire detection.

[0042] In the embodiment of the present application, the multi-scale significant perception module 123 of the electric bicycle charging and parking place is used to input the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place into the feature joint significant perception network based on attention driving to obtain the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place. Accordingly, considering that there is a mutual joint interaction relationship between the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place, and each feature map has significant feature information about its own scale, in order to more accurately and comprehensively identify the overall significant state characteristics of the electric bicycle charging and parking place, and improve the accuracy of fire detection, in the technical solution of the present application, the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place are input into the feature joint significant perception network based on attention driving to obtain the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place. In particular, the feature joint significant perception network based on attention driving adopts a dual interactive attention mechanism to enhance the interaction between feature maps, thereby improving the expressiveness of features.

[0043] In detail, first, the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place are reshaped to obtain the shallow state feature shape reshaping matrix of the electric bicycle charging and parking place and the deep state feature shape reshaping matrix of the electric bicycle charging and parking place, which provides a basis for subsequent feature interaction. It should be understood that by reshaping the two features here, their original integrity in channels and space can be retained, while ensuring that the feature maps of different levels have a unified dimension in subsequent processing. Then, the two feature matrices after shape reshaping are input into the feature channel-by-channel interactive perception module to interact with each other in dependencies, so as to learn the dependency relationship between the electric bicycle charging and parking places of different scale channels, understand the relationship between different features, and obtain the shallow-deep state dependency matrix of the electric bicycle charging and parking place and the deep-shallow state dependency matrix of the electric bicycle charging and parking place. Then, the two dependency feature matrices are input into the random inactivation module for pruning to obtain the pruned shallow-deep state dependency matrix of the electric bicycle charging and parking place and the pruned deep-shallow state dependency matrix of the electric bicycle charging and parking place. That is, random dropout is a regularization technique that can reduce the overfitting of the model to the data by randomly "turning off" some neurons in the network. The pruning process can reveal the key features of the model by identifying and removing unimportant parameters. Subsequently, the pruned features are used to optimize the shallow state feature shape reshaping matrix and the deep state feature shape reshaping matrix of the electric bicycle charging and parking place, respectively, to improve the expression of significant key features in the shallow and deep layers. Finally, the optimized features are restored to the form of feature maps and weighted fused to obtain multi-scale significant perception fusion feature maps of electric bicycle charging and parking places. In this way, the key and effective complementary information in the feature maps of electric bicycle charging and parking places of different scales can be effectively combined and highlighted, thereby improving the comprehensiveness of fire detection.

[0044] Specifically, in the embodiment of the present application, the multi-scale significant perception module of the electric bicycle charging and parking place includes: an electric bicycle charging and parking place feature map shape reshaping unit, which is used to reshape the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place to obtain the shallow state feature shape reshaping matrix of the electric bicycle charging and parking place and the deep state feature shape reshaping matrix of the electric bicycle charging and parking place; an electric bicycle charging and parking place shallow-deep state channel-by-channel interactive perception unit, which is used to reshape the shallow state feature shape reshaping matrix of the electric bicycle charging and parking place and the deep state feature shape reshaping matrix of the electric bicycle charging and parking place The feature shape reshaping matrix is ​​input into the feature channel-by-channel interactive perception module to obtain the shallow-deep state dependency matrix of the electric bicycle charging and parking places and the deep-shallow state dependency matrix of the electric bicycle charging and parking places; the electric bicycle charging and parking place dependency matrix random deactivation processing unit is used to input the shallow-deep state dependency matrix of the electric bicycle charging and parking places and the deep-shallow state dependency matrix of the electric bicycle charging and parking places into the random deactivation module to obtain the pruned shallow-deep state dependency matrix of the electric bicycle charging and parking places and the pruned deep-shallow state dependency matrix of the electric bicycle charging and parking places; the dependency relationship optimization electric bicycle A charging and parking place deep state feature generation unit is used to perform matrix multiplication on the pruned electric bicycle charging and parking place shallow-deep state dependency matrix and the electric bicycle charging and parking place deep state feature shape reshaping matrix to obtain a dependency relationship optimized electric bicycle charging and parking place deep state feature matrix; a dependency relationship optimized electric bicycle charging and parking place shallow state feature generation unit is used to perform matrix multiplication on the pruned electric bicycle charging and parking place deep-shallow state dependency matrix and the electric bicycle charging and parking place shallow state feature shape reshaping matrix to obtain a dependency relationship optimized electric bicycle charging and parking place shallow state feature matrix; The optimized electric bicycle charging and parking place feature matrix shape reshaping unit is used to perform feature shape reshaping on the dependency-optimized electric bicycle charging and parking place shallow state feature matrix and the dependency-optimized electric bicycle charging and parking place deep state feature matrix to obtain an optimized electric bicycle charging and parking place shallow state feature map and an optimized electric bicycle charging and parking place deep state feature map; the multi-scale electric bicycle charging and parking place feature fusion unit is used to calculate the weighted sum of the optimized electric bicycle charging and parking place shallow state feature map and the optimized electric bicycle charging and parking place deep state feature map to obtain the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place.

[0045] More specifically, in an embodiment of the present application, the electric bicycle charging and parking place shallow-deep state channel-by-channel interactive perception unit is used to: calculate the matrix product between the transposed matrix of the electric bicycle charging and parking place shallow state feature shape reshaping matrix and the electric bicycle charging and parking place deep state feature shape reshaping matrix to obtain a first correlation feature matrix; calculate the positional division of the scale of the first correlation feature matrix and the first correlation feature matrix to obtain a first correlation feature modulation matrix; calculate the matrix product between the transposed matrix of the electric bicycle charging and parking place deep state feature shape reshaping matrix and the electric bicycle charging and parking place shallow state feature shape reshaping matrix to obtain a second correlation feature matrix; calculate the positional division of the scale of the second correlation feature matrix and the second correlation feature matrix to obtain a second correlation feature modulation matrix; input the first correlation feature modulation matrix and the second correlation feature modulation matrix into the Softmax function respectively to obtain the electric bicycle charging and parking place shallow-deep state dependency matrix and the electric bicycle charging and parking place deep-shallow state dependency matrix.

[0046] In the embodiment of the present application, specifically, the multi-scale feature extraction module of the electric bicycle charging and parking place is used to: input the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place into the feature joint saliency perception network based on attention driving, and process them with the following saliency joint perception formula to obtain the multi-scale saliency perception fusion feature map of the electric bicycle charging and parking place; wherein the saliency joint perception formula is:

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] in, and They are respectively a shallow state characteristic diagram of the electric bicycle charging and parking place and a deep state characteristic diagram of the electric bicycle charging and parking place, For the reshape operation, and are respectively the shallow state characteristic shape reshaping matrix of the electric bicycle charging and parking place and the deep state characteristic shape reshaping matrix of the electric bicycle charging and parking place, is the matrix product, and is the transposed matrix of the shallow state characteristic shape reshaping matrix of the electric bicycle charging and parking place and the deep state characteristic shape reshaping matrix of the electric bicycle charging and parking place, and are the scales of the first correlation feature matrix and the second correlation feature matrix, respectively, for function, and The shallow-deep state dependency matrix of the electric bicycle charging and parking places and the deep-shallow state dependency matrix of the electric bicycle charging and parking places, For pruning, and are respectively the shallow-deep state dependency matrix of the pruned electric bicycle charging and parking places and the deep-shallow state dependency matrix of the pruned electric bicycle charging and parking places, and The dependency relationship is used to optimize the deep state feature matrix of the electric bicycle charging and parking places and the dependency relationship is used to optimize the shallow state feature matrix of the electric bicycle charging and parking places, respectively. and They are respectively the shallow state characteristic diagram of the optimized electric bicycle charging and parking place and the deep state characteristic diagram of the optimized electric bicycle charging and parking place, and is the weight hyperparameter, It is a multi-scale salient perception fusion feature map of the electric bicycle charging and parking place.

[0059] In the embodiment of the present application, the smoke concentration value sequence encoding module 124 is used to input the time queue of the smoke concentration value into the sequence encoder to obtain the smoke concentration time series correlation feature vector. Specifically, in the embodiment of the present application, the smoke concentration value sequence encoding module is used to: input the time queue of the smoke concentration value into the sequence encoder based on the Bi-LSTM model to obtain the smoke concentration time series correlation feature vector. It should be understood that the smoke concentration values ​​between different time spans in the time queue of the smoke concentration value have different time series correlation relationships, and LSTM can effectively capture the long-term dependency in the input sequence data by introducing memory units and gating mechanisms (input gates, forget gates, and output gates). The Bi-LSTM model composed of two LSTMs can not only process the input sequence data from front to back, but also process the sequence data from back to front, thereby more effectively capturing the long-term dependency of the input sequence data. Based on this, in order to reflect the dynamic change trend and fluctuation of the smoke concentration value in the time series, in the technical solution of the present application, the time queue of the smoke concentration value is input into the sequence encoder based on the Bi-LSTM model to extract and capture the correlation between different time spans, and obtain the smoke concentration time series correlation feature vector.

[0060] In an embodiment of the present application, the fire detection multimodal fusion module 125 is used to input the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place and the smoke concentration time series correlation feature vector into a gated cross-modal joint encoder based on an imitation transformer structure to obtain a fire detection multimodal fusion feature map. Accordingly, considering that the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place highlights the significant visual information in the image, the smoke concentration time series correlation feature vector reflects the time series dynamic change trend of the smoke concentration. Therefore, in order to fuse these features of different sources and types together to obtain a more comprehensive feature representation of the fire situation, in the technical solution of the present application, the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place and the smoke concentration time series correlation feature vector are input into a gated cross-modal joint encoder based on an imitation transformer structure to obtain a fire detection multimodal fusion feature map. It is worth mentioning that the gated cross-modal joint encoder based on the imitation transformer structure achieves efficient integration between different modal features by imitating the self-attention mechanism of the Transformer architecture and combining the gating mechanism. Specifically, firstly, the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place is layer-normalized to eliminate the differences between different channels. Then, the layer-normalized features are sequentially subjected to point convolution and spatial convolution encoding to enhance the channel feature expression ability of the feature map, enhance the features by learning the complex relationship between channels, and extract the spatial information in the feature map to obtain the multi-scale significant perception spatial deepening feature map of the electric bicycle charging and parking place. After that, in order to retain the spatial structure features, the spatial deepening feature map is copied to obtain the backup multi-scale significant perception spatial deepening feature map of the electric bicycle charging and parking place. Then, the copied features and spatial features are reshaped to obtain the backup multi-scale significant perception spatial deepening feature matrix of the electric bicycle charging and parking place and the multi-scale significant perception spatial deepening feature matrix of the electric bicycle charging and parking place and as the imitation query feature matrix and the imitation key feature matrix. After that, the autocorrelation matrix between the imitation query feature matrix and the imitation key feature matrix is ​​calculated to capture the correlation between the electric bicycle charging and parking places and obtain the multi-scale significant perception local autocorrelation library matrix of the electric bicycle charging and parking places. Furthermore, the attention gating unit based on the GELU function is used to dynamically adjust the autocorrelation weight of each eigenvalue in the autocorrelation library matrix to achieve adaptive selection and enhancement of features to obtain the multi-scale significant perception gated local autocorrelation library weight matrix of electric bicycle charging and parking places. Subsequently, the smoke concentration time series correlation feature vector and the local autocorrelation matrix are matrix multiplied to achieve cross-modal correlation fusion to obtain the smoke concentration-electric bicycle charging and parking place cross-modal interaction fusion condition weight vector.Finally, the multi-scale salient perception fusion feature map of electric bicycle charging and parking places is weighted by the fusion condition weight vector to obtain a multi-modal fusion feature map of fire detection that includes the multi-scale salient features of electric bicycle charging and parking places and the time series features of smoke concentration.

[0061] Specifically, in the embodiment of the present application, the fire detection multimodal fusion module includes: a multi-scale feature normalization convolution unit of an electric bicycle charging and parking place, which is used to perform layer normalization and convolution coding processing on the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain a multi-scale significant perception space deepening feature map of the electric bicycle charging and parking place; a multi-scale significant perception space deepening feature backup unit of an electric bicycle charging and parking place, which is used to copy the multi-scale significant perception space deepening feature map of the electric bicycle charging and parking place to obtain a backup multi-scale significant perception space deepening feature map of the electric bicycle charging and parking place; a multi-scale significant perception space deepening feature shape reshaping unit of an electric bicycle charging and parking place, which is used to reshape the feature shape of the backup multi-scale significant perception space deepening feature map of the electric bicycle charging and parking place and the multi-scale significant perception space deepening feature map of the electric bicycle charging and parking place respectively to obtain a backup multi-scale significant perception space deepening feature matrix of the electric bicycle charging and parking place and a multi-scale significant perception space deepening feature matrix of the electric bicycle charging and parking place as a simulated query feature matrix and a simulated key feature matrix; a multi-scale significant perception local autocorrelation library matrix calculation unit of an electric bicycle charging and parking place, which is used to calculate the simulated query feature matrix. The product of the feature matrix and the transposed matrix of the imitation key feature matrix is ​​used to obtain the multi-scale significant perception local autocorrelation library matrix of the electric bicycle charging and parking places; the multi-scale significant perception attention gated autocorrelation analysis unit of the electric bicycle charging and parking places is used to input the multi-scale significant perception local autocorrelation library matrix of the electric bicycle charging and parking places into the attention gating unit based on the GELU function to obtain the multi-scale significant perception gated local autocorrelation library weight matrix of the electric bicycle charging and parking places; the smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight calculation unit is used to use the smoke concentration time series correlation feature to The quantity is taken as the value feature vector, and the product between the weight matrix of the multi-scale significant perception gated local autocorrelation library of the electric bicycle charging and parking place and the time series correlation feature vector of the smoke concentration is calculated to obtain the smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight vector; the multi-scale significant perception fusion feature weighting unit of the electric bicycle charging and parking place is used to use the smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight vector as the weight vector, and perform channel-by-channel weighting on the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain the fire detection multi-modal fusion feature map.

[0062] More specifically, in an embodiment of the present application, the multi-scale feature normalization convolution unit of the electric bicycle charging and parking place is used to: perform layer normalization processing on the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain a layer-normalized multi-scale significant perception fusion feature map of the electric bicycle charging and parking place; perform point convolution encoding and spatial convolution encoding on the layer-normalized multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain the multi-scale significant perception spatial deepening feature map of the electric bicycle charging and parking place.

[0063] In the embodiment of the present application, specifically, the fire detection multimodal fusion module is used to: input the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place and the smoke concentration time series correlation feature vector into a gated cross-modal joint encoder based on an imitation converter structure, and process them with the following cross-modal joint encoding formula to obtain the fire detection multimodal fusion feature map; wherein the cross-modal joint encoding formula is:

[0064]

[0065]

[0066]

[0067]

[0068]

[0069] in, is a multi-scale salient perception fusion feature map of the electric bicycle charging and parking place, is the layer normalization operation, is point convolutional coding, The convolution kernel is The convolution operation, is a multi-scale significant perception space deepening feature map of the electric bicycle charging and parking place, For copy operation, is a multi-scale significant perception space deepening feature map of the backup electric bicycle charging and parking place, For the reshape operation, is the transpose of the feature matrix, is matrix multiplication, is the GELU function, is the weight matrix of the multi-scale significant perception gated local autocorrelation library of the electric bicycle charging and parking place, is the smoke concentration time series correlation feature vector, is the weight vector of the cross-modal interaction fusion condition of smoke concentration and electric bicycle charging and parking place, It is the multimodal fusion feature map of fire detection.

[0070] In the embodiment of the present application, the fire detection result generation module 126 is used to obtain a fire detection result based on the fire detection multimodal fusion feature map, and the fire detection result is used to indicate whether a fire is found. Specifically, in the embodiment of the present application, the fire detection result generation module is used to: input the fire detection multimodal fusion feature map into the fire detection module based on the classifier to obtain the fire detection result, and the fire detection result is used to indicate whether a fire is found. That is, the fire detection multimodal fusion feature map obtained by cross-modal joint encoding of the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place and the smoke concentration time series correlation feature vector is used for classification processing to intelligently determine whether a fire is found. In this way, temperature anomalies, visible light changes and smoke concentrations can be monitored simultaneously to more comprehensively evaluate fire risks from multiple angles. At the same time, artificial intelligence processing technology is used to accurately identify hidden fires, improve the accuracy and timeliness of early warnings, thereby reducing the need for manual intervention, thereby providing a more intelligent fire early warning system for electric bicycle charging and parking places.

[0071] In a preferred example, considering that the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place and the time-series correlation feature vector of the smoke concentration respectively represent the multi-scale multi-depth image joint perception semantic features of the multi-channel image of the electric bicycle charging and parking place state obtained by aggregating the thermal infrared image of the electric bicycle charging and parking place and the RGB image along the channel dimension and the time-series long-range-short-range correlation features of the smoke concentration, therefore, when performing gated cross-modal joint encoding based on the converter-like structure, the multi-modal semantic feature diversified set expression distribution of the fire detection multi-modal fusion feature map will be caused by the feature modality differences and feature encoding differences. Therefore, it is expected to improve the balance between the accuracy and completeness of the class regression mapping when the fire detection multi-modal fusion feature map is input into the classifier-based fire detection module for classification regression, thereby improving the accuracy of the fire detection results.

[0072] Based on this, in the preferred example, inputting the fire detection multimodal fusion feature map into the classifier-based fire detection module to obtain the fire detection result specifically includes: determining the fire detection multimodal fusion maximum eigenvalue and the fire detection multimodal fusion minimum eigenvalue of the fire detection multimodal fusion feature map, and calculating the fire detection multimodal fusion mean and the fire detection multimodal fusion standard deviation of the feature set of the fire detection multimodal fusion feature map; calculating the quotient of the fire detection multimodal fusion mean and the fire detection multimodal fusion standard deviation to obtain the fire detection multimodal fusion statistical normalization value; calculating the inverse of each eigenvalue of the fire detection multimodal fusion feature map, and comparing it with the fire detection multimodal fusion maximum eigenvalue. After multiplying the difference with the minimum eigenvalue of the fire detection multimodal fusion, perform point-wise subtraction from the fire detection multimodal fusion statistical normalization value to obtain a fire detection multimodal fusion distribution approximation map; calculate an exponential function with a natural constant as the base for each eigenvalue of the fire detection multimodal fusion distribution approximation map, and perform point-wise addition with the fire detection multimodal fusion statistical normalization value to obtain a fire detection multimodal fusion probability approximation map; calculate the base-two logarithm of the absolute value of each eigenvalue of the fire detection multimodal fusion probability approximation map to obtain an optimized fire detection multimodal fusion feature map; input the optimized fire detection multimodal fusion feature map into the classifier-based fire detection module to obtain the fire detection result.

[0073] Among them, the optimization expression of the multi-modal fusion feature map of fire detection is:

[0074]

[0075] in, is the fire detection multimodal fusion feature map, , , and They are the feature sets of the multimodal fusion feature graph of fire detection The mean, standard deviation, minimum and maximum values ​​of is the logarithm to base 2, Indicates adding by position point. represents the natural exponential function, It means point multiplication by position. It means to subtract by position point. To calculate the inverse of each eigenvalue of the fire detection multimodal fusion feature map, Represents the optimized fire detection multimodal fusion feature map.

[0076] That is, by performing random statistical normalization on the diversified feature set of the fire detection multimodal fusion feature map, the confidence space constructed based on the overall feature value of the fire detection multimodal fusion feature map is approximated by the response hypothesis test of each feature value of the fire detection multimodal fusion feature map in a standardized continuous probability density distribution, thereby establishing the target reachability of the discretized feature distribution of the fire detection multimodal fusion feature map to the unified regression target, so as to achieve the balanced executableness between mapping accuracy and mapping completeness in the classification regression process based on the discretized feature distribution of the fire detection multimodal fusion feature map, and improve the accuracy of the fire detection result obtained by inputting the fire detection multimodal fusion feature map into the classifier-based fire detection module. In this way, temperature anomalies, visible light changes and smoke concentrations can be monitored simultaneously to more comprehensively evaluate fire risks from multiple angles. At the same time, artificial intelligence processing technology is used to accurately identify hidden fires, improve the accuracy and timeliness of early warnings, thereby reducing the need for manual intervention, thereby providing a more intelligent fire early warning system for electric bicycle charging and parking places.

[0077] In summary, the intelligent early warning system 100 for fires in electric bicycle charging and parking places based on the embodiment of the present application is explained, which uses image analysis and data processing technology based on artificial intelligence to perform multi-scale feature extraction and significant perception fusion on the multi-channel image of the state of the electric bicycle charging and parking place obtained by aggregating thermal infrared images and RGB images, and performs time-series correlation encoding on the smoke concentration value data, so as to intelligently judge whether a fire is found based on the cross-modal features between the features obtained by significant fusion of the multi-channel image of the state of the electric bicycle charging and parking place and the time-series correlation features of the smoke concentration value. In this way, the fire risk can be evaluated more comprehensively, and the accuracy and timeliness of the fire warning can be improved, thereby realizing a more intelligent fire warning for electric bicycle charging and parking places.

[0078] In other embodiments of the present application, an intelligent fire warning system for electric bicycle charging and parking places is also provided. Figure 4 This is a scene application diagram of the intelligent early warning system for fire in electric bicycle charging and parking places according to other embodiments of the present application. Figure 4 As shown in the application scenario, firstly, the networked monitoring devices (such as Figure 4 10) and smoke alarms (such as Figure 4 20) to collect environmental monitoring images and smoke concentration values ​​of electric bicycle charging and parking places in real time. Then, the collected environmental monitoring images and smoke concentration values ​​of the electric bicycle charging and parking places are input into the intelligent control host (such as the intelligent early warning algorithm for fire in electric bicycle charging and parking places) deployed. Figure 430), wherein the electric bicycle charging and parking place fire intelligent early warning algorithm can process the environmental monitoring image and smoke concentration value of the electric bicycle charging and parking place to determine whether a fire is found, and generate a control instruction in response to the discovery of a fire. The intelligent control host sends the control instruction to each device to start the relevant device to quickly and efficiently respond to the fire. For example, a water tank (such as Figure 4 The automatic spraying equipment (such as 11) shown in FIG. Figure 4 13) and closed sprinkler pipes (such as Figure 4 15) will automatically open to spray; hanging dry powder fire extinguisher (such as Figure 4 12) will automatically open to spray dry powder, electric vehicle charging piles (such as Figure 4 16) shown in the figure, power off operation is performed, and fire extinguishing agent (such as Figure 4 The cabinet 14) shown in the figure will automatically open for fire extinguishing, and the smoke alarm will activate the alarm function when a fire is detected.

[0079] Figure 5 Flow chart of the intelligent early warning method for fire in electric bicycle charging and parking places according to an embodiment of the present application. Figure 5 As shown, in the intelligent early warning method for fire in the electric bicycle charging and parking place, the method includes: S110, collecting thermal infrared images and RGB images of the electric bicycle charging and parking place and collecting a time queue of smoke concentration values; S120, based on the thermal infrared image, the RGB image and the time queue of the smoke concentration value, determining whether a fire is found, and generating a start fire extinguishing control instruction in response to the discovery of a fire; S130, in response to receiving the fire extinguishing control instruction, starting the automatic sprinkler equipment and the suspended dry powder fire extinguisher; S140, in response to receiving the fire extinguishing control instruction, starting the emergency light and closing the charging pile.

[0080] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned intelligent early warning method for fire in electric bicycle charging and parking places have been referred to above. Figures 1 to 3 The description of the intelligent fire warning system for electric bicycle charging and parking places has been introduced in detail, and therefore, its repeated description will be omitted.

[0081] In summary, the intelligent early warning method for fires in electric bicycle charging and parking places based on the embodiment of the present application is explained, which uses image analysis and data processing technology based on artificial intelligence to perform multi-scale feature extraction and significant perception fusion on the multi-channel image of the state of the electric bicycle charging and parking place obtained by aggregating thermal infrared images and RGB images, and performs time-series correlation encoding on the smoke concentration value data, so as to intelligently judge whether a fire is found based on the cross-modal features between the features obtained by significant fusion of the multi-channel image of the state of the electric bicycle charging and parking place and the time-series correlation features of the smoke concentration value. In this way, the fire risk can be evaluated more comprehensively, and the accuracy and timeliness of the fire warning can be improved, thereby realizing a more intelligent fire warning for electric bicycle charging and parking places.

Claims

1. An intelligent early warning system for fires in electric bicycle charging and parking areas, characterized in that: include: Collection unit, fire extinguishing unit, linkage unit and fire early warning control center; The acquisition unit includes a thermal imaging binocular camera and a smoke detector, wherein the thermal imaging binocular camera is used to acquire thermal infrared images and RGB images of the electric bicycle charging and parking places, and the smoke detector is used to acquire the time queue of the smoke concentration value; The fire warning control center is used to determine whether a fire is found based on the time queue of the thermal infrared image, the RGB image and the smoke density value, and generate a fire extinguishing control instruction in response to the fire being found; The fire extinguishing unit includes an automatic sprinkler device and a suspended dry powder fire extinguisher, and the fire extinguishing unit is used to start the automatic sprinkler device and the suspended dry powder fire extinguisher in response to receiving the fire extinguishing control instruction; The linkage unit is used to start the emergency light and shut down the charging pile in response to receiving the fire extinguishing control instruction; Wherein, the fire early warning control center includes: A thermal infrared image RGB image aggregation module, used for aggregating the thermal infrared image and the RGB image along the channel dimension to obtain a multi-channel image of the state of the electric bicycle charging and parking place; An electric bicycle charging and parking place multi-scale feature extraction module is used to input the electric bicycle charging and parking place state multi-channel image into the field state multi-scale feature extractor to obtain an electric bicycle charging and parking place shallow state feature map and an electric bicycle charging and parking place deep state feature map; A multi-scale salient perception module for electric bicycle charging and parking places, used for inputting the shallow state feature map of the electric bicycle charging and parking places and the deep state feature map of the electric bicycle charging and parking places into an attention-driven feature joint saliency perception network to obtain a multi-scale salient perception fusion feature map of the electric bicycle charging and parking places; A smoke concentration value sequence encoding module, used for inputting the time queue of the smoke concentration value into a sequence encoder to obtain a smoke concentration time series correlation feature vector; A fire detection multimodal fusion module, used for inputting the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place and the smoke concentration time series correlation feature vector into a gated cross-modal joint encoder based on an imitation converter structure to obtain a fire detection multimodal fusion feature map; A fire detection result generating module, used to obtain a fire detection result based on the fire detection multimodal fusion feature map, wherein the fire detection result is used to indicate whether a fire is found; Wherein, the fire detection multimodal fusion module includes: The electric bicycle charging and parking place multi-scale feature normalization convolution unit is used to perform layer normalization and convolution coding processing on the electric bicycle charging and parking place multi-scale significant perception fusion feature map to obtain the electric bicycle charging and parking place multi-scale significant perception space deepening feature map; A multi-scale significant perception space deepening feature backup unit for electric bicycle charging and parking places, used to copy the multi-scale significant perception space deepening feature map of the electric bicycle charging and parking places to obtain a backup multi-scale significant perception space deepening feature map of the electric bicycle charging and parking places; The electric bicycle charging and parking place multi-scale significant perception space deepening feature shape reshaping unit is used to reshape the feature shape of the backup electric bicycle charging and parking place multi-scale significant perception space deepening feature map and the electric bicycle charging and parking place multi-scale significant perception space deepening feature map respectively to obtain the backup electric bicycle charging and parking place multi-scale significant perception space deepening feature matrix and the electric bicycle charging and parking place multi-scale significant perception space deepening feature matrix as the simulated query feature matrix and the simulated key feature matrix; A multi-scale significant perception local autocorrelation library matrix calculation unit for electric bicycle charging and parking places, used to calculate the product between the imitation query feature matrix and the transposed matrix of the imitation key feature matrix to obtain a multi-scale significant perception local autocorrelation library matrix for electric bicycle charging and parking places; An electric bicycle charging and parking place multi-scale significant perception attention gated autocorrelation analysis unit is used to input the electric bicycle charging and parking place multi-scale significant perception local autocorrelation library matrix into the attention gated unit based on the GELU function to obtain the electric bicycle charging and parking place multi-scale significant perception gated local autocorrelation library weight matrix; A smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight calculation unit is used to calculate the product between the multi-scale significant perception gated local autocorrelation library weight matrix of the electric bicycle charging and parking place and the smoke concentration time series associated feature vector using the smoke concentration time series associated feature vector as the value feature vector to obtain the smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight vector; The multi-scale significant perception fusion feature weighting unit of the electric bicycle charging and parking place is used to use the smoke concentration-electric bicycle charging and parking place cross-modal interactive fusion condition weight vector as the weight vector, and perform channel-by-channel weighting on the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain the fire detection multi-modal fusion feature map.

2. The intelligent early warning system for fire in electric bicycle charging and parking places according to claim 1 is characterized in that: The multi-scale feature extraction module of the electric bicycle charging and parking place is used to: input the multi-channel image of the electric bicycle charging and parking place state into the field state multi-scale feature extractor based on the hollow pyramid network to obtain the shallow state feature map of the electric bicycle charging and parking place and the deep state feature map of the electric bicycle charging and parking place.

3. The intelligent early warning system for fire in electric bicycle charging and parking places according to claim 2 is characterized in that: The multi-scale saliency perception module of the electric bicycle charging and parking place includes: An electric bicycle charging and parking place characteristic map shape reshaping unit is used to perform characteristic shape reshaping on the electric bicycle charging and parking place shallow state characteristic map and the electric bicycle charging and parking place deep state characteristic map to obtain an electric bicycle charging and parking place shallow state characteristic shape reshaping matrix and an electric bicycle charging and parking place deep state characteristic shape reshaping matrix; An electric bicycle charging and parking place shallow-deep state channel-by-channel interactive perception unit is used to input the electric bicycle charging and parking place shallow state feature shape reshaping matrix and the electric bicycle charging and parking place deep state feature shape reshaping matrix into a feature channel-by-channel interactive perception module to obtain an electric bicycle charging and parking place shallow-deep state dependency matrix and an electric bicycle charging and parking place deep-shallow state dependency matrix; An electric bicycle charging and parking place dependency matrix random deactivation processing unit is used to input the electric bicycle charging and parking place shallow-deep state dependency matrix and the electric bicycle charging and parking place deep-shallow state dependency matrix into a random deactivation module to obtain a pruned electric bicycle charging and parking place shallow-deep state dependency matrix and a pruned electric bicycle charging and parking place deep-shallow state dependency matrix; A dependency-optimized electric bicycle charging and parking place deep state feature generating unit is used to perform matrix multiplication of the pruned electric bicycle charging and parking place shallow-deep state dependency matrix and the electric bicycle charging and parking place deep state feature shape reshaping matrix to obtain a dependency-optimized electric bicycle charging and parking place deep state feature matrix; A dependency-optimized electric bicycle charging and parking place shallow state feature generating unit is used to perform matrix multiplication of the pruned electric bicycle charging and parking place deep-shallow state dependency matrix and the electric bicycle charging and parking place shallow state feature shape reshaping matrix to obtain a dependency-optimized electric bicycle charging and parking place shallow state feature matrix; A dependency relationship optimized electric bicycle charging and parking place feature matrix shape reshaping unit is used to perform feature shape reshaping on the dependency relationship optimized electric bicycle charging and parking place shallow state feature matrix and the dependency relationship optimized electric bicycle charging and parking place deep state feature matrix to obtain an optimized electric bicycle charging and parking place shallow state feature map and an optimized electric bicycle charging and parking place deep state feature map; The multi-scale electric bicycle charging and parking place feature fusion unit is used to calculate the weighted sum of the optimized electric bicycle charging and parking place shallow state feature map and the optimized electric bicycle charging and parking place deep state feature map to obtain the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place.

4. The intelligent early warning system for fire in electric bicycle charging and parking places according to claim 3 is characterized in that: The shallow-deep state channel-by-channel interactive perception unit of the electric bicycle charging and parking place is used for: Calculating the matrix product between the transposed matrix of the shallow state characteristic shape reshaping matrix of the electric bicycle charging and parking place and the deep state characteristic shape reshaping matrix of the electric bicycle charging and parking place to obtain a first correlation characteristic matrix; calculating a position-wise division of the first correlation feature matrix and the scale of the first correlation feature matrix to obtain a first correlation feature modulation matrix; Calculating the matrix product between the transposed matrix of the deep state characteristic shape reshaping matrix of the electric bicycle charging and parking place and the shallow state characteristic shape reshaping matrix of the electric bicycle charging and parking place to obtain a second related characteristic matrix; calculating a position-wise division of the scale of the second correlation feature matrix and the second correlation feature matrix to obtain a second correlation feature modulation matrix; The first correlation feature modulation matrix and the second correlation feature modulation matrix are respectively input into the Softmax function to obtain the shallow-deep state dependency matrix of the electric bicycle charging and parking place and the deep-shallow state dependency matrix of the electric bicycle charging and parking place.

5. The intelligent early warning system for fire in electric bicycle charging and parking places according to claim 4 is characterized in that: The smoke concentration value sequence encoding module is used to: input the time queue of the smoke concentration value into a sequence encoder based on the Bi-LSTM model to obtain the smoke concentration time series correlation feature vector.

6. The intelligent early warning system for fire in electric bicycle charging and parking places according to claim 5 is characterized in that: The multi-scale feature normalization convolution unit of the electric bicycle charging and parking place is used to: Performing layer normalization processing on the multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain a layer normalized multi-scale significant perception fusion feature map of the electric bicycle charging and parking place; Point convolution encoding and spatial convolution encoding are performed on the layer-normalized multi-scale significant perception fusion feature map of the electric bicycle charging and parking place to obtain the multi-scale significant perception spatial deepening feature map of the electric bicycle charging and parking place.

7. The intelligent early warning system for fire in electric bicycle charging and parking places according to claim 6 is characterized in that: The fire detection result generating module is used to: input the fire detection multimodal fusion feature map into the classifier-based fire detection module to obtain the fire detection result, and the fire detection result is used to indicate whether a fire is found.

8. An intelligent early warning method for fires in electric bicycle charging and parking places, using the intelligent early warning system for fires in electric bicycle charging and parking places as claimed in claim 1, characterized in that: include: Collect thermal infrared images and RGB images of electric bicycle charging and parking areas, as well as the time queue of smoke concentration values; determining whether a fire is found based on the thermal infrared image, the RGB image and the time sequence of the smoke density value, and generating a fire extinguishing control instruction in response to the fire being found; In response to receiving the fire extinguishing control instruction, starting the automatic sprinkler equipment and the suspended dry powder fire extinguisher; In response to receiving the fire extinguishing control instruction, the emergency light is activated and the charging pile is closed.

Images