Real-time video monitoring anomaly detection method and system based on deep learning

Abstract

The invention relates to the technical field of video monitoring, in particular to a real-time video monitoring anomaly detection method and system based on deep learning. The real-time video monitoring anomaly detection method based on deep learning comprises the steps of storage database establishment, information acquisition, coal briquette transportation monitoring model establishment, anomaly judgment, conveyor belt conveying speed calculation, primary conveyor belt anomaly judgment, secondary conveyor belt anomaly judgment, conveying anomaly output, conveying normal output, cache cleaning and the like. The real-time video monitoring anomaly detection system comprises a camera, an acquisition module, a storage module, a judgment module I, a judgment module II, a judgment module III, a calculation module, a judgment module IV, a judgment module V, an output module I, an output module II and a cleaning module. And the accuracy of monitoring the conveying abnormity of the coal briquettes is further improved.

Description

Technical Field

[0001] The present application relates to the technical field of video surveillance, and in particular to a real-time video surveillance anomaly detection method and system based on deep learning. Background Art

[0002] Video anomaly detection is a process that uses computer vision and machine learning techniques to analyze videos captured by surveillance cameras to identify and report abnormal behaviors or events. Anomalies usually refer to events that are inconsistent with normal conditions or expectations. They can be considered as unusual appearance or movement attributes, with characteristics such as uncertainty and irregularity.

[0003] With the popularization of cameras and underground coal mine safety monitoring systems, video monitoring has been widely used underground. At present, in the existing technology, when monitoring the coal mining process, the transportation status of the mined coal blocks on the conveyor belt is captured and located to monitor abnormal situations during the transportation of coal blocks.

[0004] However, during the transportation process, the coal blocks may collapse on the conveyor belt due to the vibration of the conveyor belt and other reasons. When the camera captures the collapse of the coal block, the image of the collapsed coal block will be inconsistent with the original image when processed, and an abnormal coal block alarm will be issued. However, sometimes, the coal blocks collapse on the conveyor belt but are still transported by the conveyor belt without any abnormality, which increases the probability of false alarm of abnormal monitoring during the coal block transportation process. Summary of the invention

[0005] In order to reduce the false alarm probability of abnormal monitoring during coal block transportation, the present invention provides a real-time video monitoring anomaly detection method and system based on deep learning.

[0006] In a first aspect, the present invention provides a real-time video surveillance anomaly detection method based on deep learning, which adopts the following technical solution: A real-time video surveillance anomaly detection method based on deep learning, comprising the following steps: Establishing a storage database: used to store the information obtained in the information obtaining step; Information acquisition: acquiring image information of m conveyors, image information of n coal blocks, and image information of i conveyor belts, and performing image noise reduction and image enhancement processing on the image information of m conveyors, image information of n coal blocks, and image information of i conveyor belts, thereby acquiring conveyor image comparison model A, coal block image comparison model B, and conveyor belt image comparison model C; Establish a coal block transportation monitoring model: conduct real-time video monitoring of the coal blocks in transportation, and obtain the conveyor contour image information Mx, conveyor belt contour image information Ix and coal block contour image information Nx during the monitoring process, segment the conveyor belt contour image on the conveyor contour image, and then obtain the position information of the conveyor belt on the conveyor, segment the coal block image information on the conveyor belt contour image, and then obtain the position information of the coal block on the conveyor belt, and perform the abnormality judgment step; Abnormal judgment: including the initial judgment step and the secondary judgment step: Initial judgment: if the coal block contour image information Nx exists in the conveyor belt contour image information Ix, the secondary judgment step is performed; if the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, the conveying abnormality output step is performed; Secondary judgment: when the coal block contour image information Nx exists in the contour image information Ix of the conveyor belt, the continuity judgment of the coal block contour image information Nx is performed. If the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the normal conveying step is executed; if the coal block contour image information Nx does not exist continuously on the contour image information Ix, the abnormal conveying step is executed; Abnormal conveying output: when the coal block contour image information Nx does not exist in the contour image information Ix of the conveyor belt, a warning of "coal block not mined, conveyor belt idling" is output; When the coal block contour image information Nx is not continuous on the contour image information Ix, a warning of "coal block mining, mining speed is sometimes fast and sometimes slow" is output; Normal transportation and output: The coal blocks are transported normally and continuously through the conveyor belt.

[0007] By adopting the above calculation scheme, the conveyor image, conveyor belt image and coal block image are first trained by the deep learning method, and then the control model is obtained. The abnormal state of the coal block during transportation is judged by the coal block information on the conveyor belt, and the cause of the abnormal state of the coal block is obtained. Since the coal block will be affected by the vibration of the conveyor belt during transportation, it may cause the coal block to collapse. When the coal block collapses, whether there is an abnormal condition of the coal block on the conveyor belt is monitored in real time, thereby improving the accuracy of monitoring the abnormal coal block transportation. Optionally, the abnormality judgment step further includes three judgment steps, and a conveyor belt conveying speed calculation step and an initial conveyor belt abnormality judgment step are further provided after the abnormality judgment step; Three judgments: when the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the edge of the conveyor belt contour image information Ix is judged. If the coal block contour image information Nx exists on the edge of the conveyor belt contour image information Ix, the conveying abnormality output step is executed. At the same time, the conveying abnormality step outputs "coal block overloading"; If there is no coal block contour image information Nx at the edge of the conveyor belt contour image information Ix, the conveyor belt conveying speed calculation is performed: Conveyor belt conveying speed calculation: Calculate the conveyor belt conveying speed V0: The calculation model of the conveyor belt conveying speed V0 is as follows: ; Where T0 is the time when the coal block is first detected, T1 is the time when the coal block is secondly detected, and L is the distance the coal block moves between T0 and T1; Initial conveyor belt abnormality judgment: obtain the roller speed V1 of the conveyor. If V0=V1, execute the normal conveying step. If V0≠V1, execute the abnormal conveying step. In the abnormal conveying step, a warning "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping" is also output.

[0008] Optionally, a second conveyor belt abnormality judgment step is further provided after the three judgment steps; Secondary conveyor belt abnormality judgment: judge the position of the conveyor belt to determine whether the position of the conveyor belt contour image in the conveyor contour image has changed. If the position of the conveyor belt contour image in the conveyor contour image has changed, the conveyor abnormality output step is executed, and at the same time, the conveyor abnormality output step outputs a warning of "conveyor belt deviation"; If the position of the conveyor belt profile image in the conveyor machine profile image does not change, the normal conveying step is performed.

[0009] Optionally, a cache cleaning step is further provided after the normal output delivery step; Cache cleaning: When the coal blocks on the conveyor belt are transported normally and continuously, the normal coal block transportation information that has been photographed is deleted.

[0010] In a second aspect, the present invention provides a real-time video surveillance anomaly detection system, which adopts the following technical solution: A real-time video monitoring anomaly detection system includes a camera, an acquisition module, a storage module, a judgment module I, a judgment module II, an output module I and an output module II; Storage module: the input end is electrically connected to the output end of the acquisition module and the output end of the camera, and is used to store the information acquired by the acquisition module and the camera; Acquisition module: The output end is electrically connected to the input end of the storage module, and is used to acquire the image information of m conveyors, the image information of n coal blocks, and the image information of i conveyor belts, and perform image noise reduction and image enhancement processing on the image information of m conveyors, the image information of n coal blocks, and the image information of i conveyor belts, thereby acquiring the conveyor image comparison model A, the coal block image comparison model B, and the conveyor belt image comparison model C, and storing the acquired information in the storage module; Camera: The output end is connected to the input end of the storage module by electrical signals, and is used to perform real-time video monitoring of the coal blocks in transportation, and obtain the conveyor contour image information Mx, the conveyor belt contour image information Ix and the coal block contour image information Nx during the monitoring process, segment the conveyor belt contour image on the conveyor contour image, and then obtain the position information of the conveyor belt on the conveyor, segment the coal block image information on the conveyor belt contour image, and then obtain the position information of the coal block on the conveyor belt, and execute the judgment module I; Judgment module I: the input end is electrically connected to the output end of the storage module, and the output end is electrically connected to the input end of the judgment module II and the input end of the output module I, calling the information stored in the storage module and judging the conveying status of the coal block: If the coal block contour image information Nx exists in the conveyor belt contour image information Ix, the judgment module II is executed; if the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, the output module I is executed; Judgment module II: the output end is electrically connected to the input end of output module I and the input end of output module II. When the coal block contour image information Nx exists in the contour image information Ix of the conveyor belt, the continuity of the coal block contour information Nx is judged. If the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the output module II is executed. If the coal block contour image information Nx does not exist continuously on the contour image information Ix, the output module I is executed. Output module I: when the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, it outputs a warning of "the coal block has not been mined, and the conveyor belt is idling"; When the coal block contour image information Nx is not continuous on the contour image information Ix, a warning of "coal block mining, mining speed is sometimes fast and sometimes slow" is output; Output module II: The coal blocks are transported normally and continuously via the conveyor belt.

[0011] Optionally, a judgment module III, a calculation module and a judgment module IV are further provided after the judgment module II: Judgment module III: the output end is connected to the input end of the calculation module by electrical signals. When the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the edge of the conveyor belt contour image information Ix is judged. If the coal block contour image information Nx exists on the edge of the conveyor belt contour image information Ix, the output module I is executed. At the same time, the output module I outputs "coal block overloading"; If the coal block contour image information Nx does not exist at the edge of the conveyor belt contour image information Ix, the calculation module is executed; Calculation module: The output end is connected to the input end of the judgment module IV for calculating the conveying speed V0 of the conveyor belt: The calculation model of the conveyor belt conveying speed V0 is as follows: ; Where T0 is the time when the coal block is first detected, T1 is the time when the coal block is secondly detected, and L is the distance the coal block moves between T0 and T1; Judgment module IV: used to judge the speed of the conveyor belt and obtain the roller speed V1 of the conveyor. If V0=V1, the output module II is executed. If V0≠V1, the output module I is executed, and the output module I also outputs a warning that "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping".

[0012] Optionally, a judgment module V is further provided after the judgment module III: Judgment module V: judges the position of the conveyor belt, and judges whether the position of the conveyor belt contour image in the conveyor contour image has changed. If the position of the conveyor belt contour image in the conveyor contour image has changed, the output module I is executed, and at the same time, the output module I outputs the warning of "conveyor belt deviation" in the abnormal output step; If the position of the conveyor belt profile image in the conveyor machine profile image does not change, the output module II is executed.

[0013] Optionally, a cleaning module is further provided after the output module II; Cleaning module: The input end is connected with the output end of the output module II by electrical signals. When the coal blocks on the conveyor belt are conveyed normally and continuously by the conveyor belt, the normal coal block conveying information that has been photographed is deleted.

[0014] In summary, the present invention includes at least one of the following beneficial technical effects: 1. Through the deep learning method, the conveyor images, conveyor belt images and coal block images are first trained to obtain the control model. The abnormal state of the coal blocks during transportation is judged by the information of the coal blocks on the conveyor belt, and the reasons for the abnormal state of the coal blocks are obtained. Since the coal blocks will be affected by the vibration of the conveyor belt during transportation, it may cause the coal blocks to collapse. When the coal blocks collapse, whether there are abnormal conditions of the coal blocks on the conveyor belt is monitored in real time, thereby improving the accuracy of monitoring the abnormal coal block transportation.

[0015] 2. Through real-time monitoring of the conveyor belt and image comparison of the running conveyor belt, the deviation information of the conveyor belt can be obtained, and the deviated conveyor belt can be corrected, thereby improving the coal block transportation efficiency.

[0016] 3. By comparing the speed of the conveyor and roller with the speed of the coal blocks conveyed by the conveyor belt, it is possible to monitor whether the conveyor belt is slipping, and issue an abnormal instruction through an alarm. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of Example 1 of the present application; Figure 2 This is a system structure diagram of Example 2 of the present application. DETAILED DESCRIPTION

[0018] The following combination Figure 1 to Figure 2 The present invention is described in further detail.

[0019] Embodiment 1: This embodiment of the present application provides a real-time video surveillance anomaly detection method based on deep learning. The following technical solutions are adopted: Reference Figure 1 , a real-time video surveillance anomaly detection method based on deep learning, comprising the following steps: S1: Establishing a storage database: used to store the information obtained in the information obtaining step.

[0020] S2: Information acquisition: Obtain image information of m conveyors, image information of n coal blocks and image information of i conveyor belts, and perform image denoising and image enhancement processing on the image information of m conveyors, image information of n coal blocks and image information of i conveyor belts, and then obtain conveyor image comparison model A, coal block image comparison model B and conveyor belt image comparison model C.

[0021] S3: Establish a coal block transportation monitoring model: perform real-time video monitoring of the coal blocks in transportation, and obtain the conveyor contour image information Mx, conveyor belt contour image information Ix and coal block contour image information Nx during the monitoring process, segment the conveyor belt contour image on the conveyor contour image, and then obtain the position information of the conveyor belt on the conveyor, segment the coal block image information on the conveyor belt contour image, and then obtain the position information of the coal blocks on the conveyor belt, and perform abnormal judgment steps.

[0022] (For example, when real-time video monitoring of coal blocks in transport is performed, an image of a conveyor is acquired, wherein the image of the conveyor includes an image of a conveyor belt, and the image of the conveyor belt includes an image of the coal blocks).

[0023] S4: Abnormality judgment: including the first judgment step, the second judgment step and the third judgment step; S41: Initial judgment: If the coal block contour image information Nx exists in the conveyor belt contour image information Ix, a secondary judgment step is performed; if the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, a conveying abnormality output step is performed.

[0024] S42: Secondary judgment: When there is coal block contour image information Nx in the contour image information Ix of the conveyor belt, the continuity of the coal block contour image information Nx is judged. If the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, three judgment steps are executed. If the coal block contour image information Nx does not exist continuously on the contour image information Ix, the conveying abnormality step is executed.

[0025] S43: Three judgments: When the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the edge of the conveyor belt contour image information Ix is judged. If the coal block contour image information Nx exists on the edge of the conveyor belt contour image information Ix, the conveying abnormality output step is executed. At the same time, the conveying abnormality step outputs "coal block overloading". If the coal block contour image information Nx does not exist on the edge of the conveyor belt contour image information Ix, the conveying speed calculation step of the conveyor belt is executed.

[0026] S5: Calculation of conveyor belt conveying speed: Calculate the conveyor belt conveying speed V0: The calculation model of the conveyor belt conveying speed V0 is as follows: ; In the formula, T0 is the time when the coal block is first detected, T1 is the time when the coal block is secondly detected, and L is the distance the coal block moves between T0 and T1.

[0027] S6: Initial conveyor belt abnormality judgment: obtain the roller speed V1 of the conveyor. If V0=V1, execute the secondary conveyor belt abnormality judgment step. If V0≠V1, execute the conveyor abnormality output step, and the conveyor abnormality step also outputs a warning that "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping."

[0028] S7: Secondary conveyor belt abnormality judgment: judge the position of the conveyor belt to determine whether the position of the conveyor belt contour image in the conveyor contour image has changed. If the position of the conveyor belt contour image in the conveyor contour image has changed, the conveyor abnormality output step is executed, and the conveyor abnormality output step outputs a warning of "conveyor belt deviation" at the same time; If the position of the conveyor belt profile image in the conveyor machine profile image does not change, the conveyor normal output step is performed.

[0029] S8: Abnormal conveying output: when the coal block contour image information Nx does not exist in the contour image information Ix of the conveyor belt, a warning of "the coal block is not mined, and the conveyor belt is idling" is output; When the coal block contour image information Nx is not continuous on the contour image information Ix, a warning of "coal block mining, mining speed is sometimes fast and sometimes slow" is output; There is coal block contour image information Nx at the edge of the conveyor belt contour image information Ix, and a warning of "coal block overloading" is output; When V0≠V1, the warning "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping" is output; The position of the conveyor belt profile image in the conveyor machine profile image changes, and a "conveyor belt deviation" warning is output.

[0030] S9: Normal transport output: The coal blocks are transported normally and continuously through the conveyor belt.

[0031] S10: Cache cleaning: When the coal blocks on the conveyor belt are conveyed normally and continuously by the conveyor belt, the captured normal coal block conveying information is deleted.

[0032] The implementation principle of a real-time video monitoring anomaly detection method based on deep learning in an embodiment of the present application is as follows: the conveyor image, conveyor belt image and coal block image are first trained by the deep learning method to obtain a control model, and the abnormal state of the coal block during transportation is judged by the coal block information on the photographed conveyor belt, and the cause of the abnormal state of the coal block is obtained. Since the coal block will be affected by the vibration of the conveyor belt during transportation, it may cause the coal block to collapse. When the coal block collapses, whether there is an abnormal condition of the coal block on the conveyor belt, by real-time monitoring of the conveyor belt and image comparison of the running conveyor belt, the deviation information of the conveyor belt can be obtained, and the deviated conveyor belt can be corrected, thereby improving the coal block transportation efficiency; by comparing the rotation speed of the conveyor and roller with the speed of the conveyor belt conveying the coal block, it is monitored whether the conveyor belt slips, and an abnormal instruction is issued through an alarm.

[0033] Embodiment 2, the present invention provides a real-time video monitoring anomaly detection system, which adopts the following technical solution: Reference Figure 2 , a real-time video monitoring anomaly detection system, including a camera, an acquisition module, a storage module, a judgment module I, a judgment module II, a judgment module III, a calculation module, a judgment module IV, a judgment module V, an output module I, an output module II and a cleaning module: Storage module: the input end is electrically connected to the output end of the acquisition module and the output end of the camera, and is used to store the information acquired by the acquisition module and the camera; Acquisition module: The output end is electrically connected to the input end of the storage module, and is used to acquire the image information of m conveyors, the image information of n coal blocks, and the image information of i conveyor belts, and perform image noise reduction and image enhancement processing on the image information of m conveyors, the image information of n coal blocks, and the image information of i conveyor belts, thereby acquiring the conveyor image comparison model A, the coal block image comparison model B, and the conveyor belt image comparison model C, and storing the acquired information in the storage module; Camera: The output end is connected to the input end of the storage module by electrical signals, and is used to perform real-time video monitoring of the coal blocks in transportation, and obtain the conveyor contour image information Mx, the conveyor belt contour image information Ix and the coal block contour image information Nx during the monitoring process, segment the conveyor belt contour image on the conveyor contour image, and then obtain the position information of the conveyor belt on the conveyor, segment the coal block image information on the conveyor belt contour image, and then obtain the position information of the coal block on the conveyor belt, and execute the judgment module I; Judgment module I: the input end is electrically connected to the output end of the storage module, and the output end is electrically connected to the input end of the judgment module II and the input end of the output module I, calling the information stored in the storage module and judging the conveying status of the coal block: If the coal block contour image information Nx exists in the conveyor belt contour image information Ix, the judgment module II is executed; if the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, the output module I is executed; Judgment module II: the output end is electrically connected to the input end of output module I and the input end of output module II. When the coal block contour image information Nx exists in the contour image information Ix of the conveyor belt, the continuity of the coal block contour information Nx is judged. If the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the output module II is executed. If the coal block contour image information Nx does not exist continuously on the contour image information Ix, the output module I is executed. Judgment module III: the output end is connected to the input end of the calculation module by electrical signals. When the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the edge of the conveyor belt contour image information Ix is judged. If the coal block contour image information Nx exists on the edge of the conveyor belt contour image information Ix, the output module I is executed. At the same time, the output module I outputs "coal block overloading"; If the coal block contour image information Nx does not exist at the edge of the conveyor belt contour image information Ix, the calculation module is executed; Calculation module: The output end is connected to the input end of the judgment module IV for calculating the conveying speed V0 of the conveyor belt: The calculation model of the conveyor belt conveying speed V0 is as follows: ; Where T0 is the time when the coal block is first detected, T1 is the time when the coal block is secondly detected, and L is the distance the coal block moves between T0 and T1; Judgment module IV: used to judge the speed of the conveyor belt and obtain the roller speed V1 of the conveyor. If V0=V1, the output module II is executed. If V0≠V1, the output module I is executed, and the output module I also outputs a warning that "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping"; Judgment module V: judges the position of the conveyor belt, and judges whether the position of the conveyor belt contour image in the conveyor contour image has changed. If the position of the conveyor belt contour image in the conveyor contour image has changed, the output module I is executed, and at the same time, the output module I outputs the warning of "conveyor belt deviation" in the abnormal output step; If the position of the conveyor belt profile image in the conveyor machine profile image does not change, the output module II is executed.

[0034] Output module I: when the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, it outputs a warning of "the coal block has not been mined, and the conveyor belt is idling"; When the coal block contour image information Nx is not continuous on the contour image information Ix, a warning of "coal block mining, mining speed is sometimes fast and sometimes slow" is output; There is coal block contour image information Nx at the edge of the conveyor belt contour image information Ix, and a warning of "coal block overloading" is output; When V0≠V1, the warning "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping" is output; The position of the conveyor belt profile image in the conveyor machine profile image changes, and a "conveyor belt deviation" warning is output.

[0035] Output module II: The coal blocks are transported normally and continuously via the conveyor belt.

[0036] Cleaning module: The input end is connected with the output end of the output module II by electrical signals. When the coal blocks on the conveyor belt are conveyed normally and continuously by the conveyor belt, the normal coal block conveying information that has been photographed is deleted.

[0037] The implementation principle of a real-time video monitoring anomaly detection system in an embodiment of the present application is as follows: a storage module stores information acquired by an acquisition module and a camera; the acquisition module acquires image information of m conveyors, image information of n coal blocks, and image information of i conveyor belts, and performs image noise reduction and image enhancement processing on the image information of m conveyors, image information of n coal blocks, and image information of i conveyor belts, thereby acquiring conveyor image comparison model A, coal block image comparison model B, and conveyor belt image comparison model C, and storing the acquired information in the storage module; the camera performs real-time video monitoring of the coal blocks in transportation, and acquires the transport information during the monitoring process. The conveyor machine contour image information Mx, the conveyor belt contour image information Ix and the coal block contour image information Nx are used to segment the conveyor belt contour image on the conveyor contour image, thereby obtaining the position information of the conveyor belt on the conveyor, and segment the coal block image information on the conveyor belt contour image, thereby obtaining the position information of the coal block on the conveyor belt. If the coal block contour image information Nx exists in the conveyor belt contour image information Ix, the judgment module II is executed; if the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, the output module I is executed; the judgment module II performs continuity judgment on the coal block contour information Nx. If the coal block contour image information Nx is not present in the conveyor belt contour image information Ix, the output module I is executed; the judgment module II performs continuity judgment on the coal block contour information Nx. If the image information Nx exists continuously on the conveyor belt contour image information Ix, the output module II is executed; if the coal block contour image information Nx does not exist continuously on the contour image information Ix, the output module I is executed; the judgment module III judges the edge of the conveyor belt contour image information Ix; if the coal block contour image information Nx exists on the edge of the conveyor belt contour image information Ix, the output module I is executed, and at the same time, the output module I outputs "coal block overloading"; if the coal block contour image information Nx does not exist on the edge of the conveyor belt contour image information Ix, the calculation module is executed; the calculation module calculates the conveying speed V0 of the conveyor belt: the judgment module IV obtains The roller speed of the conveyor is V1. If V0=V1, the output module II is executed. If V0≠V1, the output module I is executed, and the output module I also outputs a warning that "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping"; the judgment module V judges the position of the conveyor belt to determine whether the position of the conveyor belt contour image in the conveyor contour image has changed. If the position of the conveyor belt contour image in the conveyor contour image has changed, the output module I is executed, and at the same time, the output module I conveys an abnormal output step to output a warning that "the conveyor belt is deviating"; if the position of the conveyor belt contour image in the conveyor contour image has not changed, the output module II is executed.

[0038] When the output module I outputs the contour image information Ix of the conveyor belt and the coal block contour image information Nx does not exist, the output module I outputs a warning of "the coal block has not been mined, and the conveyor belt is idling"; When the coal block contour image information Nx is not continuous on the contour image information Ix, a warning of "coal block mining, mining speed is sometimes fast and sometimes slow" is output. When the coal blocks on the conveyor belt are normally and continuously transported by the conveyor belt, the normal coal block transportation information that has been photographed is deleted.

[0039] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time video surveillance anomaly detection method based on deep learning, characterized in that: The following steps are involved: Establishing a storage database: used to store the information obtained in the information obtaining step; Information acquisition: acquiring image information of m conveyors, image information of n coal blocks, and image information of i conveyor belts, and performing image noise reduction and image enhancement processing on the image information of m conveyors, image information of n coal blocks, and image information of i conveyor belts, thereby acquiring conveyor image comparison model A, coal block image comparison model B, and conveyor belt image comparison model C; Establish a coal block transportation monitoring model: conduct real-time video monitoring of the coal blocks in transportation, and obtain the conveyor contour image information Mx, conveyor belt contour image information Ix and coal block contour image information Nx during the monitoring process, segment the conveyor belt contour image on the conveyor contour image, and then obtain the position information of the conveyor belt on the conveyor, segment the coal block image information on the conveyor belt contour image, and then obtain the position information of the coal block on the conveyor belt, and perform the abnormality judgment step; Abnormality judgment: according to the coal block contour image information Nx existing on the conveyor belt contour image information Ix, abnormality judgment is made on the conveying of coal blocks, and a conveying abnormality output step or a conveying normal output step is executed; Abnormal conveying output: abnormal conveying of coal blocks through the conveyor belt; Normal transportation and output: The coal blocks are transported normally and continuously through the conveyor belt.

2. The real-time video surveillance anomaly detection method based on deep learning according to claim 1 is characterized in that: The abnormality judgment step includes a primary judgment step and a secondary judgment step: Initial judgment: if the coal block contour image information Nx exists in the conveyor belt contour image information Ix, the secondary judgment step is performed; if the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, the conveying abnormality output step is performed; Secondary judgment: when the coal block contour image information Nx exists in the contour image information Ix of the conveyor belt, the continuity judgment of the coal block contour image information Nx is performed. If the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the normal conveying step is executed; if the coal block contour image information Nx does not exist continuously on the contour image information Ix, the abnormal conveying step is executed; The abnormal conveying output step outputs a warning of "the coal block is not mined, and the conveyor belt is idling" when the coal block contour image information Nx does not exist in the contour image information Ix of the conveyor belt; When the coal block contour image information Nx is discontinuous on the contour image information Ix, a warning of "coal block mining, the mining speed is sometimes fast and sometimes slow" is output.

3. The real-time video surveillance anomaly detection method based on deep learning according to claim 2 is characterized in that: The abnormality judgment step further includes three judgment steps, after which a conveyor belt conveying speed calculation step and a first conveyor belt abnormality judgment step are provided; Three judgments: when the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the edge of the conveyor belt contour image information Ix is judged. If the coal block contour image information Nx exists on the edge of the conveyor belt contour image information Ix, the conveying abnormality output step is executed. At the same time, the conveying abnormality step outputs "coal block overloading"; If the coal block contour image information Nx does not exist at the edge of the conveyor belt contour image information Ix, the conveyor belt conveying speed calculation is performed: Conveyor belt conveying speed calculation: Calculate the conveyor belt conveying speed V0: The calculation model of the conveyor belt conveying speed V0 is as follows: ; Where T0 is the time when the coal block is first detected, T1 is the time when the coal block is secondly detected, and L is the distance the coal block moves between T0 and T1; Initial conveyor belt abnormality judgment: obtain the roller speed V1 of the conveyor. If V0=V1, execute the normal conveying step. If V0≠V1, execute the abnormal conveying step. In the abnormal conveying step, a warning "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping" is also output.

4. The method for real-time video surveillance anomaly detection based on deep learning according to claim 3 is characterized in that: After the three judgment steps, a second conveyor belt abnormality judgment step is also provided; Secondary conveyor belt abnormality judgment: judge the position of the conveyor belt to determine whether the position of the conveyor belt contour image in the conveyor contour image has changed. If the position of the conveyor belt contour image in the conveyor contour image has changed, the conveyor abnormality output step is executed, and at the same time, the conveyor abnormality output step outputs a "conveyor belt deviation" warning; If the position of the conveyor belt profile image in the conveyor machine profile image does not change, the normal conveying step is performed.

5. The real-time video surveillance anomaly detection method based on deep learning according to claim 4 is characterized in that: A cache cleaning step is also provided after the normal output delivery step; Cache cleaning: When the coal blocks on the conveyor belt are transported normally and continuously, the normal coal block transportation information that has been photographed is deleted.

6. A real-time video surveillance anomaly detection system, applying the real-time video surveillance anomaly detection method based on deep learning as described in any one of claims 1 to 5, characterized in that: It includes a camera, an acquisition module, a storage module, a judgment module I, a judgment module II, an output module I and an output module II; Storage module: the input end is electrically connected to the output end of the acquisition module and the output end of the camera, and is used to store the information acquired by the acquisition module and the camera; Acquisition module: The output end is electrically connected to the input end of the storage module, and is used to acquire the image information of m conveyors, the image information of n coal blocks, and the image information of i conveyor belts, and perform image noise reduction and image enhancement processing on the image information of m conveyors, the image information of n coal blocks, and the image information of i conveyor belts, thereby acquiring the conveyor image comparison model A, the coal block image comparison model B, and the conveyor belt image comparison model C, and storing the acquired information in the storage module; Camera: The output end is connected to the input end of the storage module by electrical signals, and is used to perform real-time video monitoring of the coal blocks in transportation, and obtain the conveyor contour image information Mx, the conveyor belt contour image information Ix and the coal block contour image information Nx during the monitoring process, segment the conveyor belt contour image on the conveyor contour image, and then obtain the position information of the conveyor belt on the conveyor, segment the coal block image information on the conveyor belt contour image, and then obtain the position information of the coal block on the conveyor belt, and execute the judgment module I; Judgment module I: the input end is electrically connected to the output end of the storage module, and the output end is electrically connected to the input end of the judgment module II and the input end of the output module I, calling the information stored in the storage module and judging the conveying status of the coal block: If the coal block contour image information Nx exists in the conveyor belt contour image information Ix, the judgment module II is executed; if the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, the output module I is executed; Judgment module II: the output end is electrically connected to the input end of output module I and the input end of output module II. When the coal block contour image information Nx exists in the contour image information Ix of the conveyor belt, the continuity of the coal block contour information Nx is judged. If the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the output module II is executed. If the coal block contour image information Nx does not exist continuously on the contour image information Ix, the output module I is executed. Output module I: when the coal block contour image information Nx does not exist in the conveyor belt contour image information Ix, it outputs a warning of "the coal block has not been mined, and the conveyor belt is idling"; When the coal block contour image information Nx is not continuous on the contour image information Ix, a warning of "coal block mining, mining speed is sometimes fast and sometimes slow" is output; Output module II: The coal blocks are transported normally and continuously via the conveyor belt.

7. A real-time video monitoring anomaly detection system according to claim 6, characterized in that: The judgment module II is further followed by a judgment module III, a calculation module and a judgment module IV: Judgment module III: the output end is connected to the input end of the calculation module by electrical signals. When the coal block contour image information Nx exists continuously on the conveyor belt contour image information Ix, the edge of the conveyor belt contour image information Ix is judged. If the coal block contour image information Nx exists on the edge of the conveyor belt contour image information Ix, the output module I is executed. At the same time, the output module I outputs "coal block overloading"; If the coal block contour image information Nx does not exist at the edge of the conveyor belt contour image information Ix, the calculation module is executed; Calculation module: The output end is connected to the input end of the judgment module IV for calculating the conveying speed V0 of the conveyor belt: Judgment module IV: used to judge the speed of the conveyor belt and obtain the roller speed V1 of the conveyor. If V0=V1, the output module II is executed. If V0≠V1, the output module I is executed, and the output module I also outputs a warning that "the conveyor belt speed is inconsistent with the roller speed, and the conveyor belt is slipping".

8. A real-time video surveillance anomaly detection system according to claim 7, characterized in that: A judgment module V is also provided after the judgment module III: Judgment module V: judges the position of the conveyor belt, and judges whether the position of the conveyor belt contour image in the conveyor contour image has changed. If the position of the conveyor belt contour image in the conveyor contour image has changed, the output module I is executed, and at the same time, the output module I outputs the warning of "conveyor belt deviation" in the abnormal output step; If the position of the conveyor belt profile image in the conveyor machine profile image does not change, the output module II is executed.

9. A real-time video surveillance anomaly detection system according to claim 8, characterized in that: A cleaning module is also provided after the output module II; Cleaning module: The input end is connected with the output end of the output module II by electrical signals. When the coal blocks on the conveyor belt are conveyed normally and continuously by the conveyor belt, the normal coal block conveying information that has been photographed is deleted.

Images