Disaster detection method, system and electronic equipment

Abstract

The present application provides a disaster detection method, system and electronic device, which are applied to target tower facilities, wherein the target tower facilities are provided with at least one edge gateway device and multiple pan-tilt cameras, and each of the pan-tilt cameras is provided with at least one monitoring area through a preset point. The method comprises: the cloud device sends the container image package of the disaster detection task to the edge gateway device, and the disaster detection task includes an inspection rule, and the inspection rule specifies the target preset point corresponding to the monitoring area. The edge gateway device accesses the pan-tilt camera corresponding to the target preset point according to the inspection rule of the disaster detection task sent by the cloud device, so as to collect images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image collection data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification results to the cloud device. The cloud device performs display based on the disaster detection data.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a disaster detection method, system and electronic equipment. Background Art

[0002] Existing natural disaster detection is mostly completed through manual inspections, which are insufficient in efficiency and coverage.

[0003] Therefore, how to effectively detect natural disasters is an issue that needs to be addressed urgently. Summary of the invention

[0004] The purpose of this application is to provide a disaster detection method, system and electronic equipment that can effectively perform disaster detection.

[0005] In order to achieve the above purpose, the embodiment of the present application is implemented as follows:

[0006] In a first aspect, a disaster detection method is provided, which is applied to a target tower facility, wherein the target tower facility is provided with at least one edge gateway device and a plurality of pan-tilt cameras, each of the pan-tilt cameras being provided with at least one monitoring area through a preset point, comprising:

[0007] The cloud device sends the container image package of the disaster detection task to the edge gateway device, where the disaster detection task includes an inspection rule, and the inspection rule specifies a target preset point corresponding to the monitoring area;

[0008] The edge gateway device accesses the pan / tilt camera corresponding to the target preset point according to the inspection rules of the disaster detection task issued by the cloud device to collect images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image collection data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification result to the cloud device;

[0009] The cloud device displays based on the disaster detection data;

[0010] Among them, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the disaster detection data includes a screen capture image marked with a disaster label; the cloud device initiates a manual verification of the disaster label marked on the screen capture image, and when the manual verification determines that the disaster label is incorrect, the screen capture image is feature extracted based on the large model to record the feature vector extracted from the large model feature into the feature vector library; when the disaster identification result indicates that a disaster has occurred, the edge gateway device determines the similarity between the feature vector of the screen acquisition data and the feature vector recorded in the feature vector library, and if the similarity reaches a preset standard, the disaster identification result is filtered.

[0011] In a second aspect, a disaster detection system is provided, comprising: a cloud device, at least one edge gateway device arranged at a target tower facility, and a plurality of pan-tilt cameras arranged at the target tower facility, each of the pan-tilt cameras being provided with at least one monitoring area through a preset point; wherein:

[0012] The cloud device sends the container image package of the disaster detection task to the edge gateway device, where the disaster detection task includes an inspection rule, and the inspection rule specifies a target preset point corresponding to the monitoring area;

[0013] The edge gateway device accesses the pan / tilt camera corresponding to the target preset point according to the inspection rules of the disaster detection task issued by the cloud device to collect images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image collection data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification result to the cloud device;

[0014] The cloud device displays based on the disaster detection data;

[0015] Among them, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the disaster detection data includes a screen capture image marked with a disaster label; the cloud device initiates a manual verification of the disaster label marked on the screen capture image, and when the manual verification determines that the disaster label is incorrect, the screen capture image is feature extracted based on the large model to record the feature vector extracted from the large model feature into the feature vector library; when the disaster identification result indicates that a disaster has occurred, the edge gateway device determines the similarity between the feature vector of the screen acquisition data and the feature vector recorded in the feature vector library, and if the similarity reaches a preset standard, the disaster identification result is filtered.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to execute the method described in the first aspect.

[0017] According to a fourth aspect, a computer program product is provided, the computer program product comprising a computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute the method described in the first aspect.

[0018] The embodiment of the present application takes advantage of the fact that tower facilities are located in remote areas and have good views, and proposes a natural disaster detection solution deployed in tower facilities. The solution specifically configures an edge gateway device and a pan-tilt camera on the tower facility side. The pan-tilt camera is set with at least one monitoring area through a preset point, and the container image package of the disaster detection task is sent to the edge gateway device with the help of a cloud device, so that the edge gateway device accesses the pan-tilt camera of the corresponding monitoring area for image acquisition according to the inspection rules of the disaster detection task sent by the cloud device, and performs disaster identification on the image acquisition data based on the deep learning model, so that the disaster detection data including the disaster identification results can be uploaded to the cloud device for display by the cloud device. Compared with the traditional manual inspection method, the embodiment of the present application can achieve or nearly achieve full-time, high-coverage disaster detection, and there is no need to consider traffic issues, so it has high practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is a schematic diagram of a first flow chart of the disaster detection method according to an embodiment of the present application.

[0021] Figure 2 This is a schematic diagram of the architecture of a disaster detection system for power facilities according to an embodiment of the present application.

[0022] Figure 3 This is a second flow chart of the disaster detection method according to an embodiment of the present application.

[0023] Figure 4 This is a third flow chart of the disaster detection method according to an embodiment of the present application.

[0024] Figure 5 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The present application aims to propose a disaster detection scheme to solve the current problem that natural disasters are difficult to inspect. Considering the characteristics of tower facilities being located in remote areas and having good views, the natural disaster detection scheme of the present application is carried out on the tower facility side. The scheme specifically configures an edge gateway device and a pan-tilt camera on the tower facility side. The pan-tilt camera is provided with at least one monitoring area through a preset point. The container image package of the disaster detection task is sent to the edge gateway device with the help of a cloud device, so that the edge gateway device accesses the pan-tilt camera of the corresponding monitoring area for image acquisition according to the inspection rules of the disaster detection task sent by the cloud device, and performs disaster identification on the image acquisition data based on a deep learning model, so that the disaster detection data including the disaster identification results can be uploaded to the cloud device for display by the cloud device. Compared with the traditional manual inspection method, the embodiment of the present application can achieve or nearly achieve full-time, high-coverage disaster detection, and does not need to consider traffic issues, so it has high practicality.

[0026] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0027] An embodiment of the present application provides a disaster detection method, which is applied to a target tower facility. Figure 1 Schematic diagram of the process of the disaster detection method, including the following steps:

[0028] S102, the cloud device sends the container image package of the disaster detection task to the edge gateway device, the disaster detection task includes an inspection rule, and the inspection rule specifies a target preset point corresponding to the monitoring area.

[0029] In this embodiment, the target tower facilities can be, but are not limited to, high-voltage wire towers, communication towers, etc. Such facilities are generally not located in places with inconvenient transportation, such as mountainous areas, suburbs, etc., and have a better field of view due to their height, which is suitable for disaster detection. The PTZ camera is set with at least one monitoring area through preset points, and the monitoring area is an area near the target tower facilities.

[0030] In this embodiment, the cloud device stores disaster detection tasks configured with artificial intelligence algorithms for different disaster types (such as fire, ice, landslide, etc.), and each disaster detection task is presented in the form of a container (docker) image. That is, the cloud device can send the disaster detection task to the edge gateway device in the form of a container image.

[0031] S104, the edge gateway device accesses the pan-tilt camera corresponding to the target preset point in accordance with the inspection rules of the disaster detection task issued by the cloud device, so as to collect images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image collection data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification results to the cloud device.

[0032] In this embodiment, the deep learning model can be trained by a cloud device and sent down for configuration to the edge gateway device (including the application of the deep learning model). The edge gateway device mainly runs AI visual calculations related to the deep learning model, so there are no high requirements on hardware.

[0033] Taking cable ice recognition as an example, the training principle of the deep learning model is: the cable ice recognition results (true values) corresponding to the cable ice recognition samples are used as training labels to label the cable ice recognition samples; then, the labeled cable ice recognition samples are input into the deep learning model to predict the corresponding cable ice recognition results. Next, the loss between the cable ice recognition results predicted by the deep learning model and the true value cable ice recognition results indicated by the training labels is calculated, and the model parameters of the deep learning model are optimized with the purpose of reducing the loss. After multiple rounds of iterations of the model parameters, the cable ice recognition results predicted by the deep learning model are consistent with the true values, thus having the ability to predict.

[0034] Correspondingly, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the disaster detection data includes a screen capture image annotated with a disaster label. After receiving the disaster detection data, the cloud device can initiate a manual verification of the disaster label annotated with the screen capture image, and when the manual verification determines that the disaster label is incorrect, the screen capture image is feature extracted based on the large model to record the feature vector extracted from the large model feature into the feature vector library.

[0035] On the one hand, the cloud device can train the deep learning model based on the captured images with incorrect disaster labels, and send the model parameters of the trained deep learning model to the edge gateway device, so that the edge gateway device can adjust the locally configured target deep learning model based on the model parameters sent by the cloud device, thereby forming a dynamic iteration mechanism for the target deep learning model.

[0036] On the other hand, the feature vector library on the cloud device side can also be used by the edge gateway device. During each disaster detection task, if the edge gateway device determines the disaster identification result of the disaster, it can determine the similarity between the feature vector of the picture acquisition data and the feature vector recorded in the feature vector library. If the similarity reaches the preset standard, it means that the current picture acquisition data may not be accurately identified by the deep learning model for disaster identification, and there is a possibility of false detection, so the disaster identification result is filtered.

[0037] On the basis of the above, in order to further realize the judgment of false detection. The target tower facilities can also be equipped with dual-spectrum thermal imaging cameras; if the disaster detection task is used to detect fires, the edge gateway device can also perform thermal imaging fireworks detection based on the dual-spectrum thermal imaging camera when the disaster detection data indicates that a fire has occurred in the monitoring area corresponding to the target preset point. If the fireworks detection result does not match the disaster identification result (the fireworks detection result is that no fire has occurred), the disaster identification result can be modified to no fire. For weather with severe fog in mountainous areas, the target deep learning model is easily identified as a fire. The introduction of dual-spectrum thermal imaging cameras for thermal imaging fireworks detection can promptly discover the false detection problem of the target deep learning model.

[0038] To this end, this embodiment also places disaster measurement IoT sensor devices (preferably capable of being captured by the pan-tilt camera image) at the target tower facilities and / or in areas prone to natural disasters near the target tower facilities, such as mud level meters, liquid level meters, rain gauges, etc. When the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the cloud device obtains disaster measurement data from the corresponding IoT sensor device, and performs false alarm verification on the disaster detection data based on the disaster measurement data.

[0039] S106, the cloud device displays based on the disaster detection data.

[0040] Specifically, the cloud device of this embodiment can display the collected data through the screen interface. On this basis, if the disaster detection data indicates that a disaster has occurred, the cloud device can perform an early warning function to remind relevant maintenance personnel to handle it.

[0041] In actual applications, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the cloud device can also send an instruction to the edge gateway device to perform high-frequency close-up reshooting of the monitoring area corresponding to the target preset point; correspondingly, the edge gateway device performs high-frequency close-up reshooting of the monitoring area corresponding to the target preset point in accordance with the issued high-frequency close-up reshooting instruction, and uploads the high-frequency close-up reshooting data to the cloud device for display by the cloud device.

[0042] The disaster detection method of the embodiment is described in detail below in combination with practical applications.

[0043] The disaster detection method of this embodiment is Figure 2 The disaster detection system shown is implemented, and the disaster detection system mainly includes: cloud devices, edge gateway devices and Internet of Things sensor devices.

[0044] The main business of the natural disaster detection system is as follows:

[0045] 1: Configure and connect the edge gateway and IoT devices in the cloud, actively request to obtain edge gateway data, and wait to receive data reported by IoT devices.

[0046] 2: The cloud obtains business data such as the edge gateway camera channel and preset points, configures different algorithm task docker images for different preset points, sets up inspection plans and sends them to the edge gateway for execution.

[0047] 3: The edge gateway uses private collaboration to control the camera to rotate and zoom the pan / tilt according to the configured business inspection rules, and notifies the corresponding algorithm to start detection at different preset points. After the preset point stay time is up, the algorithm is notified to stop detection, saving computing resources.

[0048] 4: After the algorithm container task on the edge gateway is running, it pulls the stream for the configured camera channel, waits for the notification of the preset point in place, continuously extracts frames to obtain decoded pictures, performs algorithm recognition, extracts DINO large model feature vectors from the recognition results, and performs error filtering with the local feature library, and then reports the alarm pictures, videos, and feature vectors to the cloud. Receive the stop detection command and wait for the inspection notification.

[0049] Some algorithms need to be integrated with IoT sensor device data or other data for detection, so they need to be further combined with other data for algorithm alarm filtering. For example, this solution uses a thermal imaging dual-spectrum PTZ camera for fireworks detection. The fireworks algorithm needs to analyze normal spectrum image data and thermal imaging data to reduce false alarms caused by foggy weather in mountainous areas.

[0050] 5: In addition to displaying algorithm alarm data, cloud services also collect, analyze, and display snapshot data from regular inspections and IoT device data. IoT devices are used to assist AI in visual analysis and to provide early warning and observation. Data analysis based on different device types is used to draw trend charts for timely disaster warnings.

[0051] 6: Install specific IoT detection equipment (such as mud level meters) in areas prone to landslides and mud-rock flows, and set camera preset points. When the cloud receives abnormal data, it notifies the edge gateway to control the gimbal to rotate to the preset point for zooming in, and conducts high-frequency snapshot monitoring, and notifies supervisors.

[0052] 7: The cloud business has designed a manual annotation entry for false alarms, set up a false alarm library, automatically optimized the training model in the background, and automatically sent the false alarm feature vector and the updated model to the edge gateway for algorithm detection result filtering and algorithm model update.

[0053] 8: The cloud supports user management, permission management, alarm push management (WeChat applets, SMS, etc.), emqx IoT MQTT message forwarding subscription service, data training annotation and other supporting functions.

[0054] Specifically, the main functions of cloud device image annotation and model self-training are as follows:

[0055] 1: Dataset management: The data set is divided into training set, validation set and test set. It supports users to upload compressed packages of different data sets on the server management interface, and supports receiving sample images and annotation data reported by edge gateway devices, storing them in cloud devices and classifying them. The server deploys the open source image and video annotation tool CVAT, which is convenient for users to load images for annotation using the web.

[0056] 2: Automatic model training: Based on the automatic training platform of PyTorch, given the source, format and processing method of the data set, select the YOLOX model as the basic disaster identification model for customization, write the training script and training parameters for automatic training, and use torch.onnx.export to convert the model to ONNX general format after training. Due to business needs to support different types of edge gateways such as GPU and NPU, for example, NPU devices need to use the RKNN Toolkit provided by Rockchip to convert ONNX to RKNN format.

[0057] 3: Based on the feature vector library of the DINO large model, when a user submits a negative sample error image, the feature value of the target object is automatically analyzed and calculated and stored in the cloud feature library, and pushed to the local feature library of the edge gateway that submitted the negative sample for comparison and filtering. At the same time, the image is submitted to the model automatic training module for model iterative optimization, and after the optimization is completed, it is updated in batches to all edge gateways.

[0058] 4: False alarm management: users can manually select alarm images for confirmation, submit erroneous data to the false alarm library, and let the background automatically extract error feature vectors and prepare for subsequent automatic model updates. The false alarm library can be managed, including adding false alarms, deleting false alarms, and other operations.

[0059] The main functions of the edge gateway device are as follows:

[0060] 1: Business management: Built-in web services facilitate local configuration for edge gateway users, including network settings, username and password configuration, cloud-edge interaction configuration, algorithm parameter configuration, alarm reporting configuration, etc.

[0061] 2: Camera control: Responsible for using the private SDK or public onvif protocol provided by cameras of different brands to control the PTZ camera to realize inspection, zooming, snapshot and other functions.

[0062] 3: Algorithm application management: responsible for receiving the algorithm application docker sent from the cloud and creating it on the specified port to facilitate sending control protocol messages into the container.

[0063] 4: Inspection task management: Responsible for accepting camera inspection tasks sent from the cloud, running different natural disaster detection algorithm containers in combination with camera channel preset points, and sending control instructions to the algorithm application container.

[0064] 5: Algorithm alarm management: Alarms generated by all algorithm applications can be viewed locally and forwarded to the cloud, supporting offline retransmission mechanism.

[0065] 6: Algorithm model update and feature vector library update: responsible for receiving cloud instructions to update the algorithm model and local feature vector library.

[0066] Among them, the edge gateway device inspection algorithm task process is as follows Figure 3 As shown, after receiving the disaster detection task sent by the cloud device, the surveillance cameras of all relevant preset points will start the docker of the disaster detection task and collect pictures (pull the stream). The retention time of the picture collection should be sufficient. In addition, if the surveillance camera configuration of the preset point does not have the docker of the disaster detection task, it will be enabled after downloading.

[0067] The main functions of the algorithm application of the edge gateway device are as follows:

[0068] 1: Use the open source codec library ffmpeg, rtsp parsing library, etc. to pull the stream, and use the hardware codec library of the edge gateway device itself to decode the image and output the BGR format memory data.

[0069] 2: Continuously extract frames to read image data in the streaming media decoding queue, send it to the YOLOX model for inference calculation, and output disaster detection results. The disaster detection results are then sent to the DINO large model for feature vector extraction. After extraction, they are compared with the local feature vector library. Only those that meet the threshold are output for alarm. The alarm is stored in the local database of the edge gateway device, including the original image, annotated image, feature data, etc., and is reported to the cloud for archiving in a timely manner.

[0070] 3: Specific algorithms for natural disasters have specially added fusion analysis of thermal imaging data and sensor data to reduce the problems of high false alarms or non-recognition of pure visual analysis in complex meteorological environments and cumbersome manual processing of false alarms, thereby improving the overall availability of the system.

[0071] 4: Algorithm applications are dockerized to facilitate management, distribution, creation, and updating. The algorithm applications currently developed for the natural disaster detection system include wire ice detection, landslide detection, and forest fire detection, which can be continuously updated in the future.

[0072] In addition, in the algorithm application of the edge gateway device, if it is fire detection, thermal imaging smoke and fire detection can also be performed based on the dual-spectrum thermal imaging camera; as mentioned earlier, if the smoke and fire detection result does not match the disaster identification result, it is determined to be a false alarm.

[0073] In addition, the update process of YOLOX model iteration is as follows Figure 4 As shown, including:

[0074] 1: Cloud administrators first divide the data set into training set, validation set and test set based on the basic content. The training set is used to train the model, the validation set is used to adjust hyperparameters and monitor model performance, and the test set is used to finally evaluate the model performance.

[0075] 2: The background automatically detects whether the training set database has changed. If there is a change, a new trainable data set is created. The cloud uses the COCO data format as the data set format. The original simple data content imported for the first time will be automatically converted to the COCO data format.

[0076] 3: Configure YOLOX model parameters, specify the path of the dataset, the number of categories, model hyperparameters, etc. At the same time, set the batch size, learning rate, number of iterations, etc. of the training. Automatically import the changed dataset content into the PyTorch framework, and define the model using the preset loss function, optimizer and other parameters as preparation before training.

[0077] 4: After configuring the environment and data, use the command line tools or custom scripts provided by YOLO to train the model.

[0078] 5: During or after training, the validation set is automatically used to verify the performance of the model. The model performance is automatically judged based on the set average precision (mAP), accuracy, recall, precision and other indicators. If it does not meet the standards, the algorithm configuration personnel are notified to check and update the parameter configuration for adjustment, and the generation of the final usable model for the device is stopped.

[0079] 6: If there is a test set and a threshold is set, the generalization ability of the model will continue to be automatically evaluated. If the standard is not met, subsequent model conversion will be stopped and the algorithm configuration personnel will be notified.

[0080] 7: If the standard is met, the background will automatically convert the model and save it in ONNX and RKNN formats respectively, and the database will record the latest model version number.

[0081] 8: Push the latest model (model parameters) to the online edge gateway device according to the push update rules.

[0082] On the other hand, another embodiment of the present application also provides a disaster detection system, referring to Figure 2 As shown, the disaster detection system mainly includes: a cloud device, at least one edge gateway device arranged at a target tower facility, and a plurality of pan-tilt cameras arranged at the target tower facility, each of the pan-tilt cameras being arranged with at least one monitoring area through a preset point; namely:

[0083] The cloud device sends the container image package of the disaster detection task to the edge gateway device, where the disaster detection task includes an inspection rule, and the inspection rule specifies a target preset point corresponding to the monitoring area;

[0084] The edge gateway device accesses the pan / tilt camera corresponding to the target preset point according to the inspection rules of the disaster detection task issued by the cloud device to collect images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image collection data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification result to the cloud device;

[0085] In addition, the cloud device performs display based on the disaster detection data.

[0086] Among them, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the disaster detection data includes a screen capture image marked with a disaster label; the cloud device initiates a manual verification of the disaster label marked on the screen capture image, and when the manual verification determines that the disaster label is incorrect, the screen capture image is feature extracted based on the large model to record the feature vector extracted from the large model feature into the feature vector library; when the disaster identification result indicates that a disaster has occurred, the edge gateway device determines the similarity between the feature vector of the screen acquisition data and the feature vector recorded in the feature vector library, and if the similarity reaches a preset standard, the disaster identification result is filtered.

[0087] Optionally, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the cloud device sends an instruction to the edge gateway device to perform high-frequency close-up reshooting of the monitoring area corresponding to the target preset point; the edge gateway device performs high-frequency close-up reshooting of the monitoring area corresponding to the target preset point in accordance with the issued high-frequency close-up reshooting instruction, and uploads the high-frequency close-up reshooting data to the cloud device.

[0088] Optionally, the target tower facility is provided with an IoT sensor device for disaster measurement; when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the cloud device obtains disaster measurement data from the IoT sensor device, and performs false alarm verification on the disaster detection data based on the disaster measurement data.

[0089] Optionally, the cloud device trains the deep learning model based on the captured images with incorrect disaster labels, and sends the model parameters of the trained deep learning model to the edge gateway device; the edge gateway device adjusts the locally configured deep learning model based on the model parameters sent by the cloud device.

[0090] The target tower facility is provided with a dual-spectrum thermal imaging camera; if the disaster detection task is used to detect fire, the edge gateway device is also used, before uploading the disaster detection data including the disaster identification result to the cloud device, to: when the disaster detection data indicates that a fire has occurred in the monitoring area corresponding to the target preset point, to perform thermal imaging of fireworks and fire detection based on the dual-spectrum thermal imaging camera; if the fireworks and fire detection result does not match the disaster identification result, to modify the disaster identification result to indicate that no fire has occurred.

[0091] It should be noted that the disaster detection system of this embodiment can be used as Figure 1 The execution subject of the method shown can thus realize Figure 1 The steps and functions of the method shown.

[0092] Figure 5 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 5 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0093] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0094] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0095] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming the above-mentioned cloud device and edge gateway device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0096] The cloud device sends the container image package of the disaster detection task to the edge gateway device, and the disaster detection task includes an inspection rule, and the inspection rule specifies a target preset point corresponding to the monitoring area.

[0097] The edge gateway device accesses the pan-tilt camera corresponding to the target preset point in accordance with the inspection rules of the disaster detection task issued by the cloud device, so as to capture images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image capture data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification results to the cloud device.

[0098] The cloud device displays based on the disaster detection data;

[0099] Among them, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the disaster detection data includes a screen capture image marked with a disaster label; the cloud device initiates a manual verification of the disaster label marked on the screen capture image, and when the manual verification determines that the disaster label is incorrect, the screen capture image is feature extracted based on the large model to record the feature vector extracted from the large model feature into the feature vector library; when the disaster identification result indicates that a disaster has occurred, the edge gateway device determines the similarity between the feature vector of the screen acquisition data and the feature vector recorded in the feature vector library, and if the similarity reaches a preset standard, the disaster identification result is filtered.

[0100] The above application Figure 1 The method disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in one or more embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with one or more embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0101] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0102] An embodiment of the present application further provides a computer program product, which includes a computer-readable storage medium storing a computer program.

[0103] In one implementation, the computer program is operable to cause a computer to perform the following operations:

[0104] The cloud device sends the container image package of the disaster detection task to the edge gateway device, and the disaster detection task includes an inspection rule, and the inspection rule specifies a target preset point corresponding to the monitoring area.

[0105] The edge gateway device accesses the pan-tilt camera corresponding to the target preset point in accordance with the inspection rules of the disaster detection task issued by the cloud device, so as to capture images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image capture data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification results to the cloud device.

[0106] The cloud device displays based on the disaster detection data;

[0107] Among them, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the disaster detection data includes a screen capture image marked with a disaster label; the cloud device initiates a manual verification of the disaster label marked on the screen capture image, and when the manual verification determines that the disaster label is incorrect, the screen capture image is feature extracted based on the large model to record the feature vector extracted from the large model feature into the feature vector library; when the disaster identification result indicates that a disaster has occurred, the edge gateway device determines the similarity between the feature vector of the screen acquisition data and the feature vector recorded in the feature vector library, and if the similarity reaches a preset standard, the disaster identification result is filtered.

[0108] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the present application shall be included in the protection scope of one or more embodiments of the present application.

[0109] The systems, devices, and modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0110] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0111] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0112] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A disaster detection method, applied to a target tower facility, characterized in that: The target tower facility is provided with at least one edge gateway device and a plurality of pan-tilt cameras, each of the pan-tilt cameras is provided with at least one monitoring area through preset points, including: The cloud device sends the container image package of the disaster detection task to the edge gateway device, wherein the disaster detection task includes an inspection rule, and the inspection rule specifies a target preset point corresponding to the monitoring area; The edge gateway device accesses the pan / tilt camera corresponding to the target preset point according to the inspection rules of the disaster detection task issued by the cloud device to collect images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image collection data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification result to the cloud device; The cloud device displays based on the disaster detection data; Among them, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the disaster detection data includes a screen capture image marked with a disaster label; the cloud device initiates a manual verification of the disaster label marked on the screen capture image, and when the manual verification determines that the disaster label is incorrect, the screen capture image is feature extracted based on the large model to record the feature vector extracted from the large model feature into the feature vector library; when the disaster identification result indicates that a disaster has occurred, the edge gateway device determines the similarity between the feature vector of the screen acquisition data and the feature vector recorded in the feature vector library, and if the similarity reaches a preset standard, the disaster identification result is filtered.

2. The method according to claim 1, characterized in that: Also includes: When the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the cloud device sends an instruction to the edge gateway device to perform high-frequency close-up reshooting on the monitoring area corresponding to the target preset point; The edge gateway device performs high-frequency close-up reshooting on the monitoring area corresponding to the target preset point according to the issued high-frequency close-up reshooting instruction, and uploads the high-frequency close-up reshooting data to the cloud device.

3. The method according to claim 1, characterized in that The target tower facility is provided with an Internet of Things sensor device for disaster measurement; the method further comprises: When the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the cloud device obtains disaster measurement data from the Internet of Things sensor device, and performs false alarm verification on the disaster detection data based on the disaster measurement data.

4. The method according to claim 1, characterized in that: Also includes: The cloud device trains the deep learning model based on the captured image with incorrect disaster label, and sends the model parameters of the trained deep learning model to the edge gateway device; The edge gateway device adjusts the locally configured deep learning model based on the model parameters sent by the cloud device.

5. The method according to claim 1, characterized in that The target tower facility is provided with a dual-spectrum thermal imaging camera; if the disaster detection task is used to detect fire, before the edge gateway device uploads the disaster detection data including the disaster identification result to the cloud device, the method further includes: When the disaster detection data indicates that a fire has occurred in the monitoring area corresponding to the target preset point, the edge gateway device performs thermal imaging smoke and fire detection based on the dual-spectrum thermal imaging camera. If the smoke and fire detection result does not match the disaster identification result, the disaster identification result is modified to indicate that no fire has occurred.

6. A disaster detection system, characterized in that: include: A cloud device, at least one edge gateway device arranged at a target tower facility, and a plurality of pan-tilt cameras arranged at the target tower facility, each of the pan-tilt cameras being arranged with at least one monitoring area through a preset point; wherein: The cloud device sends the container image package of the disaster detection task to the edge gateway device, where the disaster detection task includes an inspection rule, and the inspection rule specifies a target preset point corresponding to the monitoring area; The edge gateway device accesses the pan / tilt camera corresponding to the target preset point according to the inspection rules of the disaster detection task issued by the cloud device to collect images of the monitoring area corresponding to the target preset point, and performs disaster identification on the image collection data based on the locally configured deep learning model, so as to upload the disaster detection data including the disaster identification result to the cloud device; The cloud device displays based on the disaster detection data; Among them, when the disaster detection data indicates that a natural disaster has occurred in the monitoring area corresponding to the target preset point, the disaster detection data includes a screen capture image marked with a disaster label; the cloud device initiates a manual verification of the disaster label marked on the screen capture image, and when the manual verification determines that the disaster label is incorrect, the screen capture image is feature extracted based on the large model to record the feature vector extracted from the large model feature into the feature vector library; when the disaster identification result indicates that a disaster has occurred, the edge gateway device determines the similarity between the feature vector of the screen acquisition data and the feature vector recorded in the feature vector library, and if the similarity reaches a preset standard, the disaster identification result is filtered.

7. An electronic device comprising: processor; and a memory arranged to store computer executable instructions which, when executed, cause the processor to perform a method as claimed in any one of claims 1 to 5.

8. A computer program product, comprising a computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute the method according to any one of claims 1 to 5.

Images