Plant abnormal event detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a plant abnormal event detection method and device, electronic equipment and a storage medium. The method comprises: acquiring a video of a plant roof scene shot by a camera; determining a detection target of the plant roof scene; obtaining the world coordinates of the detection target through a calibration algorithm; encrypting the video privacy information of the detection target through a federated learning model; identifying the detection target through a YOLOV7 target detection model to obtain detection information; and determining whether a plant abnormal event occurs according to the detection information. Based on this, the embodiments of the present application use the federated learning algorithm model to protect the video information privacy of the target while identifying the detection target through the YOLOV7 target detection model, so as to timely determine the abnormal event occurring in the plant roof scene and ensure the safety of the plant. Therefore, the embodiments of the present application use the federated learning algorithm model to reduce the risk of user privacy leakage while detecting abnormal events in the plant scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of factory building safety management, and in particular to a factory building abnormal event detection method and device, an electronic device and a storage medium. BACKGROUND

[0002] In view of the occurrence of abnormal events in the factory building of the factory area, the existing detection method cannot determine whether an abnormal event occurs on the top periphery of the factory building. The existing factory building abnormal event detection is only for indoor and outdoor abnormal event detection. At present, the detection of the top of the factory building house still relies on manual detection. The manual detection is low in efficiency and single in effect. It can only detect a certain specific abnormal event in a certain specific scene. The manual detection range is limited, and the detection effect is not real-time. Moreover, the labor cost is high, and the inconvenience caused by day and night changes needs to be overcome. The factory building abnormal situation cannot be detected all day. In addition, since the factory building has confidentiality requirements, the existing factory building roof abnormal event detection method has the risk of user privacy leakage. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a factory building abnormal event detection method and device, an electronic device and a storage medium, which can reduce the risk of user privacy leakage while detecting abnormal events in the factory building scene.

[0004] To achieve the above-mentioned purpose, the first aspect of the embodiments of the present application provides a factory building abnormal event detection method, which comprises:

[0005] acquiring a video of a factory building roof scene shot by a camera;

[0006] determining a detection target of the factory building roof scene;

[0007] obtaining the world coordinates of the detection target through a calibration algorithm;

[0008] encrypting the video privacy information of the detection target through a federated learning model;

[0009] identifying the detection target through a YOLOV7 target detection model to obtain detection information;

[0010] judging whether a factory building abnormal event occurs according to the detection information.

[0011] In some embodiments, the acquiring of the video of the factory building roof scene shot by the camera comprises:

[0012] shooting the factory building roof scene through the camera arranged on the unmanned aerial vehicle to obtain the video.

[0013] In some embodiments, before the camera captures the video of the factory roof scene, the method further comprises:

[0014] defining different factory roof scenes;

[0015] determining the types of factory abnormal events corresponding to different factory roof scenes.

[0016] In some embodiments, the world coordinates of the detection target obtained by the calibration algorithm comprise:

[0017] obtaining the internal parameters of the camera;

[0018] calculating the translation vector of the camera model perpendicular to the factory roof;

[0019] calculating the rotation matrix of the world coordinate system around the image coordinate system;

[0020] obtaining the coordinate conversion relationship between the image coordinates and the world coordinates in combination with the actual position of the camera;

[0021] obtaining the world coordinates of the detection target based on the coordinate conversion relationship.

[0022] In some embodiments, the training method of the factory abnormal event detection model is as follows:

[0023] constructing a federated learning model;

[0024] collecting detection data in the field;

[0025] determining the types of factory abnormal events to be detected;

[0026] training the federated learning model through federated learning until the target detection accuracy is reached, thereby obtaining the factory abnormal event detection model.

[0027] In some embodiments, the method of determining whether a factory abnormal event occurs according to the detection information comprises:

[0028] when detecting whether the factory roof is missing, comparing the color steel image used to build the factory roof with the preset complete roof picture, and if the comparison is inconsistent, it is determined that the factory abnormal event of missing factory roof occurs;

[0029] when detecting whether there are foreign objects on the factory roof, different types of objects are identified through the YOLOV7 target detection model, and if an abnormal object is identified, it is determined that the factory abnormal event of foreign objects on the factory roof occurs;

[0030] When detecting whether there is an abnormality in the reinforced steel pipe on the factory roof, the shape of the reinforced steel pipe detected based on the Canny edge detection algorithm is compared with the shape of the normal steel pipe. If the comparison is inconsistent, it is determined that there is an abnormality in the reinforced steel pipe on the factory roof.

[0031] When detecting whether there is an abnormality in the wires on the factory roof, the real-time information of the wires detected based on the Canny edge detection algorithm is compared with the laying position of normal wires. If the comparison is inconsistent, it is determined that there is an abnormality in the wires on the factory roof.

[0032] When detecting whether the factory roof has geometric deformation, the geometric shape of the factory roof obtained by the geometric detection algorithm is compared with the geometric shape of the normal factory roof. If the comparison is inconsistent, it is determined that the factory roof has geometrically deformed and an abnormal factory event has occurred.

[0033] In some embodiments, after determining whether an abnormal event in the factory building occurs according to the detection information, the method further includes:

[0034] Provide early warning and report abnormal events in the factory.

[0035] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for detecting abnormal events in a factory building, the device comprising:

[0036] An acquisition module is used to acquire images of the factory roof scene captured by the camera;

[0037] Determination module, used to determine the detection target of the factory roof scene;

[0038] A calibration module, configured to obtain the world coordinates of the detection target through a calibration algorithm;

[0039] An encryption module, configured to encrypt the image privacy information of the detection target through a federated learning model;

[0040] An identification module is used to identify the detection target through the YOLOV7 target detection model and obtain detection information;

[0041] The judgment module is used to judge whether an abnormal event in the factory building occurs based on the detection information.

[0042] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0043] To achieve the above object, a fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method in the first aspect.

[0044] The method and device for detecting abnormal events in a factory building, the electronic device and the storage medium provided by the present application obtain a video of a roof scene of a factory building; determine a detection target of the roof scene of the factory building; obtain the world coordinates of the detection target through a calibration algorithm; encrypt the video privacy information of the detection target through a federated learning model; identify the detection target through a YOLOV7 target detection model to obtain detection information; and determine whether an abnormal event in the factory building occurs according to the detection information. Based on this, compared with the existing manual detection method, the embodiment of the present application adopts machine vision technology and a federated learning model, captures the target on the roof of the factory building while analyzing the video of the roof scene of the factory building, obtains the world coordinates of the target through a calibration algorithm, protects the video information privacy of the target through a federated learning algorithm model, and identifies the detection target through a YOLOV7 target detection model to detect the real-time situation of the factory building, thereby realizing a machine vision-based method for detecting abnormal events in a train factory building from the perspective of video, which can timely determine abnormal events occurring in the roof scene of the factory building to ensure the safety of the factory building. Therefore, the embodiment of the present application uses a federated learning algorithm model to reduce the risk of user privacy leakage while detecting abnormal events in the scene of the factory building. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of the method for detecting abnormal events in a factory building provided by the embodiment of the present application;

[0046] Figure 2 is a sub-flowchart of the method for detecting abnormal events in a factory building provided by the embodiment of the present application;

[0047] Figure 3 is a sub-flowchart of the method for detecting abnormal events in a factory building provided by the embodiment of the present application;

[0048] Figure 4 is a flowchart of the method for training a model for detecting abnormal events in a factory building provided by the embodiment of the present application;

[0049] Figure 5 is a structural schematic diagram of the device for detecting abnormal events in a factory building provided by the embodiment of the present application;

[0050] Figure 6 is a hardware structural schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0052] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the sequence in the flowchart. The terms "first", "second", and the like in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0053] In view of the technical problems of low detection efficiency and risk of user privacy leakage in the prior art, the embodiments of the present application provide a factory abnormal event detection method and device, an electronic device and a storage medium, a video of a roof scene of a factory is obtained by a camera; a detection target of the roof scene of the factory is determined; a world coordinate of the detection target is obtained through a calibration algorithm; video privacy information of the detection target is encrypted through a federated learning model; the detection target is identified through a YOLOV7 target detection model to obtain detection information; and whether a factory abnormal event occurs is determined according to the detection information. Based on this, compared with the existing manual detection method, the embodiments of the present application use machine vision technology and a federated learning model, capture the target of the roof of the factory while analyzing the video of the roof scene of the factory, obtain the world coordinate of the target through a calibration algorithm, protect the video information privacy of the target by using the federated learning algorithm model, and identify the detection target through the YOLOV7 target detection model to detect the real-time situation of the factory, thereby realizing a machine vision-based train factory abnormal event detection method from the perspective of video, which can timely determine the abnormal event occurring in the roof scene of the factory to ensure the safety of the factory. Therefore, the embodiments of the present application use the federated learning algorithm model to reduce the risk of user privacy leakage while detecting abnormal events in the factory scene.

[0054] The factory abnormal event detection method and device, the electronic device and the storage medium provided by the embodiments of the present application are specifically described as follows. First, the factory abnormal event detection method in the embodiments of the present application is described.

[0055] Figure 1 is an optional flowchart of the factory abnormal event detection method provided by the embodiments of the present application, Figure 1 The method in can include but is not limited to including steps S101 to S106.

[0056] Step S101, obtaining a video of a roof scene of a factory shot by a camera;

[0057] Step S102, determine the detection target of the factory roof scene;

[0058] Step S103, obtain the world coordinates of the detection target through the calibration algorithm;

[0059] Step S104, encrypt the video privacy information of the detection target through the federated learning model;

[0060] Step S105, identify the detection target through the YOLOV7 target detection model to obtain detection information;

[0061] Step S106, determine whether an abnormal event of the factory has occurred according to the detection information.

[0062] In some embodiments, a camera is used to take pictures of the factory roof scene that needs to be detected. Since different detection scenes correspond to different abnormal events, different scenes need to be defined. The definition refers to defining different components of the factory roof, such as color steel roof, steel pipe for reinforcing roof, and electric wire and other roof components.

[0063] In some embodiments, an unmanned aerial vehicle is used to take pictures of the scene and detect abnormalities of the house. The unmanned aerial vehicle can quickly detect the overall appearance of the factory roof, and can take pictures of the factory roof from different angles to obtain more complete information of the factory roof.

[0064] In some embodiments, the detected factory roof scene includes but is not limited to: factory roof and factory roof components in the factory area, factory building and overall appearance, etc.

[0065] In some embodiments, the abnormal events corresponding to the factory roof scene include but are not limited to: factory roof abnormal events in the factory area, including whether the factory roof is missing, whether there are foreign matters on the factory roof, whether the reinforcing steel pipe of the factory roof is broken, missing, or bent, whether the electric wire of the factory roof is broken or deformed, and whether the overall appearance of the factory roof is geometrically deformed, etc.

[0066] In some embodiments, in order to improve the detection accuracy of the abnormal events of the factory roof in the factory area, the camera is calibrated in the embodiments of the present application, and the actual position of the detected target is determined through the calibration algorithm. First, the internal parameters of the camera are obtained, then the translation vector of the camera model perpendicular to the factory roof is calculated, and the rotation matrix of the world coordinate system around the image coordinate system is calculated, and the coordinate conversion relationship between the image coordinates and the world coordinates can be obtained by combining the actual position of the camera. It should be noted that the position of the world coordinates is the position of the target in the real space.

[0067] In some embodiments, when acquiring a target, a YOLOV7 target detection model that can detect multiple targets is used. This model can identify multiple different targets at the same time. Compared with other detection algorithm models, this model can provide more real-time detection information in video detection, and can transmit the information in the surveillance video to the YOLOV7 target detection model in real time. It should be noted that the YOLO algorithm is the most typical representative of the one-stage target detection algorithm. It recognizes and locates objects based on deep neural networks, runs very fast, and can be used in real-time systems. Among them, YOLOV7 is currently the most advanced algorithm in the YOLO series, surpassing the previous YOLO series in accuracy and speed.

[0068] In some embodiments, when selecting a detection model, due to the confidentiality of factory buildings, in order to maximize the privacy of user data, this application selects a federated learning model as the detection framework for the detection model. Federated learning, also known as collaborative learning, can perform large-scale training on devices that generate data, and these sensitive data are retained by the owner of the data, collected locally, and trained locally. After local training, the central training coordinator obtains the training contribution of each node by obtaining updates to the distributed model, but does not access the actual sensitive data.

[0069] In some embodiments, federated learning has different types of learning models, such as horizontal, vertical, and transfer. The essence of horizontal federated learning is the union of samples, which is suitable for scenarios where the participants have the same business model but reach different customers, that is, there is a lot of feature overlap and little user overlap; the essence of vertical federated learning is the union of features, which is suitable for scenarios where there is a lot of user overlap and little feature overlap; when there is little overlap in features and samples between participants, federated transfer learning can be considered; this application involves the roofs of factory buildings in a factory area. Since different factory roofs overlap with each other, the embodiments of this application use horizontal federated learning as a benchmark for model training.

[0070] See also Figure 2 In some embodiments, before step S101, the following steps may be included but not limited to steps S201 to S202:

[0071] Step S201, defining different factory roof scenes;

[0072] Step S202 : determining the types of abnormal factory events corresponding to different factory roof scenes.

[0073] In some embodiments, a camera is used to capture the factory roof scene that needs to be inspected. Since different inspection scenes correspond to different abnormal events, different scenes need to be defined. Definition refers to defining the different components of the factory roof, such as: color steel roof, steel pipes for reinforcing the roof, and other roof components such as wires.

[0074] Referring to Figure 3 In some embodiments, step S103 can include but is not limited to steps S301-S305:

[0075] Step S301, obtaining the internal parameters of the camera;

[0076] Step S302, calculating the translation vector of the camera model perpendicular to the factory roof;

[0077] Step S303, calculating the rotation matrix of the world coordinate system around the image coordinate system;

[0078] Step S304, obtaining the coordinate conversion relationship between the image coordinates and the world coordinates in combination with the actual position of the camera;

[0079] Step S305, obtaining the world coordinates of the detected target based on the coordinate conversion relationship.

[0080] In some embodiments, in order to improve the detection accuracy of the factory roof abnormal event, the camera is calibrated in the embodiments of the present application, and the actual position of the detected target is determined through a calibration algorithm. First, the internal parameters of the camera are obtained, then the translation vector of the camera model perpendicular to the factory roof is calculated, and then the rotation matrix of the world coordinate system around the image coordinate system is calculated, and in combination with the actual position of the camera, the coordinate conversion relationship between the image coordinates and the world coordinates can be obtained. It should be noted that the position of the world coordinates is the position of the target in the real space.

[0081] Referring to Figure 4 In some embodiments, the training method of the factory abnormal event detection model can include but is not limited to steps S401-S404:

[0082] Step S401, constructing a federated learning model;

[0083] Step S402, collecting detection data on site;

[0084] Step S403, determining the types of factory abnormal events to be detected;

[0085] Step S404, training the federated learning model through federated learning until the target detection accuracy is reached, and obtaining the factory abnormal event detection model.

[0086] In some embodiments, the application selects a federated learning model as the detection framework of the detection model. After the detection model is constructed, data is input into the federated learning model for data training to obtain the detection accuracy of the detection model. When the model is data trained, 80% of the data is used for training and 20% of the data is used for testing. If the required detection accuracy is not reached, the model is continuously trained by data until the required accuracy is reached.

[0087] In some embodiments, step S105 can include but is not limited to steps S501 to S505:

[0088] Step S501, when detecting whether the roof of the factory building is missing, by comparing the color steel image used to build the roof of the factory building with the preset complete roof picture of the building, if the comparison is inconsistent, it is judged that the abnormal event of the factory building roof missing occurs;

[0089] Step S502, when detecting whether there is foreign matter on the roof of the factory building, different kinds of objects are identified by a YOLOV7 target detection model, and if an abnormal object is identified, it is judged that the abnormal event of the factory building roof with foreign matter occurs;

[0090] Step S503, when detecting whether the reinforcing steel pipe of the roof of the factory building has an abnormal situation, by comparing the shape of the reinforcing steel pipe detected based on the Canny edge detection algorithm with the shape of the normal steel pipe, if the comparison is inconsistent, it is judged that the abnormal event of the reinforcing steel pipe of the roof of the factory building occurs;

[0091] Step S504, when detecting whether the electric wire of the roof of the factory building has an abnormal situation, by comparing the real-time information of the electric wire detected based on the Canny edge detection algorithm with the laying position of the normal electric wire, if the comparison is inconsistent, it is judged that the abnormal event of the electric wire of the roof of the factory building occurs;

[0092] Step S505, when detecting whether the roof of the factory building has geometric deformation, by comparing the geometric shape of the roof of the factory building detected based on the geometric detection algorithm with the geometric shape of the normal roof of the factory building, if the comparison is inconsistent, it is judged that the abnormal event of the roof of the factory building occurs.

[0093] In some embodiments, whether the roof of the factory building is missing is detected: the roof of the factory building is detected by a YOLOV7 model. Since the roof of the factory building is built with a whole color steel, only the color steel used for building needs to be detected for defects. When the defects are detected, the application uses a template matching algorithm to determine whether there are defects. By comparing the complete roof picture of the building, it is judged whether the roof of the building photographed by the unmanned aerial vehicle at the present stage has missing and other abnormal events.

[0094] In some embodiments, when detecting whether there are foreign matters on the roof of the factory building, since the factory building roof exists in an open environment, foreign matters are easy to fall on the factory building, which may pose a threat to the safety of the factory building roof if not handled for a long time. The YOLOV7 algorithm is used to detect the roof of the factory area, and YOLOV7 can detect different types of objects. The YOLOV7 is used to detect the types of objects on the roof of the factory building to see whether there are abnormal objects.

[0095] In some embodiments, the reinforced steel pipe on the roof of the factory building in the factory area is detected for fracture, loss, or bending. When detecting whether the reinforced steel pipe is fractured, lost, or bent, the shape of the reinforced steel pipe under normal conditions is first labeled, including the length of the steel pipe and the curvature of the steel pipe. The reinforced steel pipe on the roof of the factory building is used to prevent the roof from being blown off in windy weather. The Canny edge detection algorithm is used to detect the steel pipe. After obtaining the shape of the reinforced steel pipe, the normal steel pipe is compared to determine whether the reinforced steel pipe has abnormal conditions such as loss and deformation.

[0096] In some embodiments, the condition of whether the electric wire on the roof of the factory building in the factory area is fractured or deformed is detected. First, the position of the electric wire laid on the factory building is determined, and the length and curvature of the electric wire are labeled. The Canny edge detection algorithm is used to obtain real-time information of the electric wire. The normal laying position is compared to determine whether the electric wire has an abnormal event.

[0097] In some embodiments, whether the overall appearance of the roof of the factory building in the factory area is deformed is detected. The factory building roof has a certain geometric shape. By detecting the external geometric shape of the factory building, it can be determined whether the roof of the factory building is deformed. When the external shape of the factory building roof is deformed, it has great harmfulness to the factory building. The geometric shape of the roof of the factory building can be obtained by using an unmanned aerial vehicle and a geometric detection algorithm, and then the deformation of the roof of the factory building is detected to ensure the safety of the building.

[0098] Based on this, the embodiments of the present application can realize the detection of different types of abnormal events in the same scene, not only detecting whether the roof is missing, but also detecting whether there are foreign matters on the roof, whether the reinforced steel pipe is abnormal, whether there are foreign matters on the roof, and whether the roof is geometrically deformed, to better monitor whether the factory building is safe.

[0099] It should be noted that the abnormal events corresponding to the scene of the roof of the factory building include but are not limited to: the abnormal events of the roof of the factory building in the factory area include whether the roof of the factory building is missing, whether there are foreign matters on the roof of the factory building in the factory area, whether the reinforced steel pipe on the roof of the factory building in the factory area is fractured, lost, or bent, whether the electric wire on the roof of the factory building in the factory area is fractured or deformed, and whether the overall appearance of the roof of the factory building in the factory area is geometrically deformed.

[0100] In some embodiments, step S106 can be followed by, but is not limited to, step S107:

[0101] Step S107, warning and reporting the factory abnormal event.

[0102] In some embodiments, by adopting machine vision technology and federated learning model, while analyzing the video of the factory roof scene, the target on the factory roof is captured, the world coordinates of the target are obtained through the calibration algorithm, and the video information privacy of the target is protected through the federated learning algorithm model, while the YOLOV7 target detection model is used to identify and detect the target, so as to detect the real-time situation of the factory, thereby realizing a machine vision-based train factory abnormal event detection method from the video perspective, which can timely judge the abnormal event occurring in the factory roof scene, and ensure the safety of the factory. Through machine vision, all components on the roof of the factory building are collected, and it is detected whether there is foreign matter on the roof of the building, and it is comprehensively judged whether an abnormal event occurs on the roof of the building. The event occurring is transmitted to the relevant department in real time, which plays a role of real-time detection and real-time warning.

[0103] Based on this, the application provides a method for detecting multiple factory roof abnormal events by using machine vision. The method uses a federated learning algorithm to encrypt user data through a federated learning model to ensure the privacy and security of user data. The application uses machine vision detection, which not only collects the roof of the factory building, but also collects other roof components of the building, to jointly determine whether an abnormal event occurs on the roof of the building.

[0104] Referring to Figure 5 The embodiments of the application also provide a factory abnormal event detection device, which can implement the above factory abnormal event detection method. The device comprises:

[0105] The acquisition module 510 is configured to acquire a video of a camera shooting a factory roof scene.

[0106] The determination module 520 is configured to determine a detection target of the factory roof scene.

[0107] The calibration module 530 is configured to obtain the world coordinates of the detection target through a calibration algorithm.

[0108] The encryption module 540 is configured to encrypt the video privacy information of the detection target through a federated learning model.

[0109] The identification module 550 is configured to identify the detection target through a YOLOV7 target detection model to obtain detection information.

[0110] The judgment module 560 is configured to judge whether a factory abnormal event occurs according to the detection information.

[0111] Based on this, the plant abnormal event detection device provided in the embodiments of the present application acquires the video of the roof scene of the plant shot by the camera; determines the detection target of the roof scene of the plant; obtains the world coordinates of the detection target through the calibration algorithm; encrypts the video privacy information of the detection target through the federated learning model; identifies the detection target through the YOLOV7 target detection model to obtain the detection information; and determines whether a plant abnormal event occurs according to the detection information. The embodiments of the present application acquire the video of the roof scene of the plant shot by the camera; determine the detection target of the roof scene of the plant; obtain the world coordinates of the detection target through the calibration algorithm; encrypt the video privacy information of the detection target through the federated learning model; identify the detection target through the YOLOV7 target detection model to obtain the detection information; and determine whether a plant abnormal event occurs according to the detection information. Based on this, compared with the existing manual detection method, the embodiments of the present application use machine vision technology and a federated learning model to analyze the video of the roof scene of the plant, capture the target on the roof of the plant, obtain the world coordinates of the target through the calibration algorithm, protect the video information privacy of the target through the federated learning algorithm model, identify the detection target through the YOLOV7 target detection model, and detect the real-time situation of the plant, thereby realizing a machine vision-based detection method for train plant abnormal events from the perspective of video, which can timely determine abnormal events occurring in the roof scene of the plant to ensure the safety of the plant. Therefore, the embodiments of the present application use the federated learning algorithm model to reduce the risk of user privacy leakage while detecting abnormal events in the plant scene.

[0112] The specific implementation of the plant abnormal event detection device is basically the same as that of the above-mentioned specific embodiments of the plant abnormal event detection method, and will not be repeated here.

[0113] The embodiments of the present application also provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned plant abnormal event detection method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0114] Please refer to Figure 6 , Figure 6 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0115] The processor 601 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0116] The memory 602 can be implemented by a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 602 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 602 and are called and executed by the processor 601 to implement the plant abnormal event detection method of the embodiments of the present application, that is, by acquiring a video of a roof scene of a plant shot by a camera; determining a detection target of the roof scene of the plant; obtaining a world coordinate of the detection target by a calibration algorithm; encrypting video privacy information of the detection target by a federated learning model; identifying the detection target by a YOLOV7 target detection model to obtain detection information; and determining whether a plant abnormal event occurs according to the detection information. Based on this, compared with the existing manual detection method, the embodiments of the present application use machine vision technology and a federated learning model to analyze the video of the roof scene of the plant, capture the target on the roof of the plant, obtain the world coordinate of the target by a calibration algorithm, protect the video information privacy of the target by a federated learning algorithm model, and identify the detection target by a YOLOV7 target detection model to detect the real-time situation of the plant, thereby realizing a machine vision-based train plant abnormal event detection method from the perspective of video, which can timely determine the abnormal event occurring in the roof scene of the plant to ensure the safety of the plant. Therefore, the embodiments of the present application use a federated learning algorithm model to reduce the risk of user privacy leakage while detecting abnormal events in the plant scene.

[0117] The input / output interface 603 is configured to realize information input and output.

[0118] The communication interface 604 is configured to realize the communication interaction between the device and other devices, which can realize communication through a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0119] The bus is configured to transmit information between various components (for example, the processor 601, the memory 602, the input / output interface 603, and the communication interface 604) of the device.

[0120] The processor 601, the memory 602, the input / output interface 603, and the communication interface 604 are communicatively connected with each other through a bus.

[0121] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the abnormal event detection method for a factory building.

[0122] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0123] The embodiment of the present application provides the abnormal event detection method for a factory building, the abnormal event detection device for a factory building, the electronic equipment and the storage medium. The video of the roof scene of the factory building is obtained by a camera; the detection target of the roof scene of the factory building is determined; the world coordinates of the detection target are obtained by a calibration algorithm; the video privacy information of the detection target is encrypted by a federated learning model; the detection target is identified by a YOLOV7 target detection model to obtain detection information; and whether an abnormal event of the factory building occurs is determined according to the detection information. Based on this, compared with the existing manual detection method, the machine vision technology and the federated learning model are adopted in the embodiment of the present application. While the video of the roof scene of the factory building is analyzed, the target of the roof of the factory building is captured. The world coordinates of the target are obtained by a calibration algorithm. The video information privacy of the target is protected by using the federated learning algorithm model. The detection target is identified by the YOLOV7 target detection model to detect the real-time situation of the factory building. Therefore, the embodiment of the present application realizes a machine vision-based train factory building abnormal event detection method from the perspective of video, can timely judge the abnormal event appearing in the roof scene of the factory building, and ensures the safety of the factory building. Therefore, the embodiment of the present application uses the federated learning algorithm model to reduce the risk of user privacy leakage while detecting the abnormal event in the factory building scene.

[0124] Those skilled in the art can understand that all or some of the steps in the method disclosed above can be implemented by software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable programs, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, as known to those skilled in the art, communication media generally includes computer readable programs, data structures, program modules or other data in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium.

[0125] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0126] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine some steps or different steps.

[0127] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, i.e., can be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0128] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0129] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on circuitry availability, based on stated preference or the like, and that "default" or other orderings are thus permissible. Further, the terms "comprise", "comprising", "include", "including", and the like, are specifically intended to be open-ended. That is, references to individual steps and the like do not suhstantially exclude the presence of two or more of a given step or its integral presence in the process, method, system, article, or apparatus having been made with a wider scope. The use of notation such as "first", "second", "third", etc. does not generally limit the areas, but can be used for clarity, and merely establishes the order unless otherwise stated below.

[0130] It should be understood that, in the application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0131] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0132] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0133] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0134] If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store programs.

[0135] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A plant abnormal event detection method characterized by, The method comprises: acquiring a video of a factory roof scene shot by a camera; determining a detection target of the factory roof scene; obtaining the world coordinates of the detection target through a calibration algorithm; encrypting the video privacy information of the detection target through a federated learning model; identifying the detection target through a YOLOV7 target detection model to obtain detection information; judging whether a factory abnormal event occurs according to the detection information; wherein the judging whether a factory abnormal event occurs according to the detection information comprises: when detecting whether the factory roof is missing, comparing the color steel image used for building the factory roof with the preset complete roof picture, and if the comparison is inconsistent, it is judged that the factory abnormal event of factory roof missing occurs; when detecting whether there are foreign matters on the factory roof, identifying different kinds of objects through the YOLOV7 target detection model, and if an abnormal object is identified, it is judged that the factory abnormal event of foreign matters on the factory roof occurs; when detecting whether the reinforced steel pipe of the factory roof has an abnormal situation, comparing the shape of the reinforced steel pipe detected based on the Canny edge detection algorithm with the shape of the normal steel pipe, and if the comparison is inconsistent, it is judged that the factory abnormal event of the reinforced steel pipe of the factory roof having an abnormal situation occurs; when detecting whether the electric wire of the factory roof has an abnormal situation, comparing the real-time information of the electric wire detected based on the Canny edge detection algorithm with the laying position of the normal electric wire, and if the comparison is inconsistent, it is judged that the factory abnormal event of the electric wire of the factory roof having an abnormal situation occurs; when detecting whether the factory roof has geometric deformation, comparing the geometric shape of the factory roof detected based on the geometric detection algorithm with the geometric shape of the normal factory roof, and if the comparison is inconsistent, it is judged that the factory abnormal event of the factory roof having geometric deformation occurs.

2. The method of claim 1, wherein, The acquiring a video of a factory roof scene shot by a camera comprises: shooting the factory roof scene through the camera arranged on the unmanned aerial vehicle to obtain the video.

3. The method of claim 1, wherein, Before the acquiring a video of a factory roof scene shot by a camera, it further comprises: defining different factory roof scenes; determining the types of factory abnormal events corresponding to different factory roof scenes.

4. The method of claim 1, wherein, The obtaining the world coordinates of the detection target through a calibration algorithm comprises: obtaining the internal parameters of the camera; calculating the translation vector of the camera model perpendicular to the factory roof; calculating the rotation matrix of the world coordinate system around the image coordinate system; obtaining the coordinate conversion relationship between the image coordinates and the world coordinates in combination with the actual position of the camera; obtaining the world coordinates of the detection target based on the coordinate conversion relationship.

5. The method of claim 4, wherein, The training method of the factory abnormal event detection model is as follows: constructing a federated learning model; collecting detection data in the field; determining the types of factory abnormal events to be detected; training the federated learning model through federated learning until the target detection accuracy is reached to obtain the factory abnormal event detection model.

6. The method of claim 1, wherein, After the judging whether a factory abnormal event occurs according to the detection information, it further comprises: The abnormal event of the factory building is prewarned and reported.

7. A plant abnormality event detection device characterized by comprising: The device comprises: An acquisition module is configured to acquire a video of a roof scene of a factory building captured by a camera; A determination module is configured to determine a detection target of the roof scene of the factory building; A calibration module is configured to obtain world coordinates of the detection target through a calibration algorithm; An encryption module is configured to encrypt video privacy information of the detection target through a federated learning model; An identification module is configured to identify the detection target through a YOLOV7 target detection model to obtain detection information; A judgment module is configured to determine whether an abnormal event of the factory building occurs according to the detection information; The determination whether the abnormal event of the factory building occurs according to the detection information comprises: When detecting whether the roof of the factory building is missing, comparing an image of a color steel used for building the roof of the factory building with a preset complete roof picture, and in the case of inconsistency, determining that the abnormal event of the factory building of missing roof occurs; When detecting whether there is a foreign object on the roof of the factory building, identifying different types of objects through the YOLOV7 target detection model, and in the case of identifying an abnormal object, determining that the abnormal event of the factory building of foreign objects on the roof occurs; When detecting whether the reinforced steel pipe of the roof of the factory building is abnormal, comparing the shape of the reinforced steel pipe detected based on a Canny edge detection algorithm with the shape of a normal steel pipe, and in the case of inconsistency, determining that the abnormal event of the factory building of abnormal condition of the reinforced steel pipe of the roof occurs; When detecting whether the electric wire of the roof of the factory building is abnormal, comparing real-time information of the electric wire detected based on the Canny edge detection algorithm with the laying position of a normal electric wire, and in the case of inconsistency, determining that the abnormal event of the factory building of abnormal condition of the electric wire of the roof occurs; When detecting whether the roof of the factory building is geometrically deformed, comparing the geometric shape of the roof of the factory building detected based on a geometric detection algorithm with the geometric shape of a normal roof of the factory building, and in the case of inconsistency, determining that the abnormal event of the factory building of geometric deformation of the roof occurs.

8. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the factory building abnormal event detection method of any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to realize the factory building abnormal event detection method of any one of claims 1 to 6.

Images