Large-scene multi-target on-site holographic intelligent monitoring comprehensive management system

Abstract

The invention relates to the technical field of intelligent monitoring, in particular to a large-scene multi-target on-site holographic intelligent monitoring comprehensive management system, which comprises a device determination module, a plurality of initial monitoring devices and a control module, the model construction module obtains and analyzes a plurality of initial monitoring video data, and constructs an initial monitoring model; the grade determination module determines an initial number of preset monitoring targets and determines an initial importance grade; the anomaly recognition module determines an initial monitoring frequency, performs monitoring based on the initial monitoring frequency, and recognizes an initial anomaly event based on an initial monitoring result; the degree analysis module analyzes the initial abnormal event and gives an alarm based on the comparison result of the initial severity, or adjusts the initial monitoring frequency, monitors the sub-region corresponding to the abnormal event in real time based on the adjusted monitoring frequency, analyzes the real-time monitoring result, gives an alarm based on the analysis result, or continues monitoring. According to the invention, the comprehensive accuracy of monitoring is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to a large-scale multi-target on-site holographic intelligent monitoring integrated management system. Background Art

[0002] When dealing with large-scale multi-target monitoring, traditional monitoring systems often face problems such as complex equipment installation, many monitoring blind spots, untimely abnormal identification, and slow response. Especially in complex environments such as open-pit coal mines, due to the vast scenes, diverse targets and large dynamic changes, traditional monitoring methods are difficult to meet the needs of efficient and accurate safety management.

[0003] The patent document with Chinese patent application number CN117111519A discloses a coal production safety intelligent monitoring system and method, including: a central control unit: responsible for receiving and integrating data from each sensor node; a sensor node group: the sensor node group is distributed in each production area of ​​the coal mine; a data calibration module: calibrates and converts the received raw data, and converts it into a unified format and unit; an adaptive environmental modeling module: based on the unified format data from the data calibration module, constructs a real-time environmental model of the mine; a real-time risk assessment module: based on the real-time environmental model from the adaptive environmental modeling module, performs a risk assessment on the current mine environment; an alarm module: takes alarm measures according to the risk level.

[0004] The existing technology for dynamic tracking and behavior analysis of targets in complex scenes often relies on preset rules and templates, which are difficult to adapt to the rapid changes and complex behavior patterns of targets in the scenes, resulting in insufficient accuracy and reliability of the analysis results, and causing the problem of low accuracy in large-scene multi-target monitoring. Summary of the invention

[0005] To this end, the present invention provides a large-scene, multi-target on-site holographic intelligent monitoring integrated management system, which can solve the problem of low monitoring accuracy.

[0006] To achieve the above objectives, the present invention provides a large-scale multi-target on-site holographic intelligent monitoring integrated management system, the system comprising:

[0007] an equipment determination module, configured to install a number of initial monitoring devices based on preset installation parameters, and the equipment determination module is further configured to adjust the preset installation parameters based on the abnormal monitoring area and the area to be monitored to obtain actual installation parameters, and to adjust the number of initial monitoring devices based on the actual installation parameters to obtain a number of actual monitoring devices;

[0008] a model building module, connected to the device determination module, for obtaining a number of initial monitoring video data of a number of actual monitoring devices, and building an initial monitoring model of the area to be monitored based on analysis results of the number of initial monitoring video data to determine a number of sub-areas in the initial monitoring model;

[0009] A level determination module, connected to the model building module, for determining initial importance levels of several sub-areas according to the identified and determined initial numbers of preset monitoring targets in the several sub-areas;

[0010] an abnormality identification module, connected to the level determination module, for determining an initial monitoring frequency for each of the sub-areas based on the initial importance level, and identifying an initial abnormal event according to an initial monitoring result for each of the sub-areas based on the initial monitoring frequency;

[0011] The degree analysis module is connected to the abnormality identification module, and is used to analyze the initial abnormal event, obtain the initial severity, and issue an alarm based on the comparison result of the initial severity, or adjust the initial monitoring frequency to obtain the adjusted monitoring frequency. The degree analysis module is also used to analyze the monitoring results obtained during the real-time monitoring of the sub-area corresponding to the abnormal event based on the adjusted monitoring frequency, and to issue an alarm based on the analysis results, or continue monitoring.

[0012] Furthermore, the device determination module includes:

[0013] an overlap determination unit, configured to determine an effective monitoring area of ​​any of the initial monitoring devices, and to analyze a plurality of effective monitoring areas to determine a plurality of overlapping monitoring areas;

[0014] a non-monitoring determination unit, configured to determine a number of non-monitoring areas based on the number of the effective monitoring areas and the areas to be monitored;

[0015] A parameter determination unit is connected to the overlap determination unit and the non-monitoring determination unit, and is used to analyze a number of the overlapped monitoring areas and a number of the non-monitoring areas, and to adjust the preset installation parameters based on the analysis results to obtain actual installation parameters.

[0016] Furthermore, the parameter determination unit includes:

[0017] an adjusting subunit, used for adjusting a preset installation angle and / or a preset installation position of the initial monitoring device so as to minimize a total value of the plurality of overlapping monitoring areas;

[0018] The adding subunit is connected to the adjusting subunit and is used to detect the adjusted unmonitored area to determine the regional environmental characteristics of the unmonitored area and to add monitoring equipment according to the regional environmental characteristics.

[0019] Furthermore, the adjustment subunit includes:

[0020] A severity analysis block is used to analyze a number of overlapping monitoring areas and determine the severity of the overlap according to the analysis results;

[0021] a priority determination block, connected to the degree analysis block, for determining adjustment priorities of a plurality of the initial monitoring devices based on the overlap severity;

[0022] an adjustment block connected to the priority determination block, for adjusting the preset installation angle and / or the preset installation position of each of the initial monitoring devices based on the adjustment priority, so as to reduce the overlapping monitoring area formed by the initial monitoring devices adjacent thereto;

[0023] The repeat block is connected with the degree analysis block, the priority determination block and the adjustment block, and is used to repeatedly execute the functions of the above blocks until the total value of the overlapping monitoring areas reaches a minimum value.

[0024] Furthermore, the model building module includes:

[0025] A data acquisition unit, used for acquiring a corresponding number of initial monitoring video data based on a number of the real-time detection devices;

[0026] a data processing unit connected to the data acquisition unit, configured to use a computer vision algorithm to identify a plurality of preset monitoring targets in a plurality of initial monitoring video data to obtain preset monitoring target data, and to use a multi-target tracking algorithm to track a plurality of preset monitoring targets, and to use a deep learning model to identify the behaviors of a plurality of preset monitoring targets to obtain target behavior data;

[0027] A construction unit is connected to the data processing unit, and is used to determine the monitoring boundary according to the several initial monitoring video data, to divide the area to be monitored into several sub-areas according to the several monitoring boundaries, and to extract the regional environmental characteristics in the several sub-areas to construct several sub-scene models, and to combine the several sub-scene models to obtain the overall scene model, to construct several monitoring target models based on the several preset monitoring target data and their corresponding target behavior data, and to construct the initial monitoring model based on the overall scene model and the monitoring target model.

[0028] Furthermore, the level determination module includes:

[0029] A target recognition unit, used for recognizing a preset monitoring target in any of the sub-areas based on a computer vision algorithm;

[0030] The level determination unit is connected to the target recognition unit and is used to count the initial number of preset detection targets in any of the sub-areas, sort the initial numbers from large to small, and determine the initial importance level corresponding to each sub-area according to the sorting result.

[0031] Furthermore, the abnormality identification module includes:

[0032] A behavior determination unit, used to determine actual behavior data of any preset monitoring target based on a number of the initial monitoring results;

[0033] An environment determination unit, configured to determine actual environment data in any of the sub-areas based on a number of the initial monitoring results;

[0034] An abnormality determination unit is connected to the behavior determination unit and the environment determination unit, and is used to compare the actual behavior data with the preset behavior data to obtain a behavior comparison result, compare the actual environment data with the preset environment data to obtain an environment comparison result, and determine the initial abnormal event based on the behavior comparison result and the environment comparison result.

[0035] Furthermore, the degree analysis module includes:

[0036] a severity determination unit, configured to analyze the initial abnormal event and determine an initial severity according to the analysis result;

[0037] a comparison unit, connected to the degree determination unit, for comparing the initial severity with the preset severity to obtain a comparison result;

[0038] The adjustment unit is connected to the comparison unit and is used to calculate an adjustment parameter when the initial severity is less than a preset severity, and adjust the initial monitoring frequency based on the adjustment parameter.

[0039] Furthermore, the degree determination unit includes:

[0040] A behavior calculation subunit, used to calculate a behavior score based on a deviation value between the actual behavior data and the preset behavior data;

[0041] An environment calculation subunit, used to calculate an environment score based on a deviation value between the actual environment data and the preset environment data;

[0042] The severity determination subunit is connected to the behavior calculation subunit and the environment calculation subunit, and is used to calculate an abnormality score based on the behavior score and the environment score, and determine the initial severity based on the abnormality score.

[0043] Furthermore, the adjustment unit includes:

[0044] a parameter calculation subunit, configured to calculate the adjustment parameter based on the initial severity and the preset severity;

[0045] The adjustment subunit is connected to the parameter calculation subunit and is used to calculate based on the adjustment parameter and the initial monitoring frequency to obtain the adjusted monitoring frequency.

[0046] Compared with the prior art, the beneficial effect of the present invention is that, by setting the equipment determination module, the initial monitoring equipment can be quickly deployed according to the preset installation parameters, and the installation parameters can be dynamically adjusted based on the actual monitoring needs, so as to ensure the accurate deployment and efficient use of the monitoring equipment. By setting the model construction module, the initial monitoring model of the area to be monitored can be automatically constructed by analyzing the initial monitoring video data, and the key sub-areas can be identified, so as to improve the accuracy of monitoring. By setting the level determination module, the initial importance level of the monitoring target can be determined according to the number of preset monitoring targets in each sub-area, and differentiated monitoring frequencies can be formulated accordingly, so that the system can give priority to high-risk areas, improve monitoring efficiency, and avoid waste of resources. By setting the abnormal identification module, the initial abnormal events can be discovered and identified in time through real-time monitoring of each sub-area. By setting the degree analysis module, the severity of the abnormal events can be further analyzed, and corresponding alarm measures or monitoring frequencies can be taken according to the analysis results, so as to help achieve rapid response and effective management of abnormal events. The system can not only perform alarm processing, but also adjust the initial monitoring frequency as needed, and even re-divide the monitoring model through the analysis of real-time monitoring results. The continuous optimization and iteration capabilities enable the system to continuously adapt to changes in the monitoring environment and maintain efficient and accurate monitoring effects.

[0047] In particular, by setting an overlapping determination unit to analyze the effective monitoring area of ​​each initial monitoring device, it is possible to accurately identify which areas are covered by multiple devices at the same time, namely, overlapping monitoring areas, which helps to avoid waste of resources. After identifying the overlapping areas, the device position or parameters are adjusted according to actual needs to reduce unnecessary overlap, thereby improving the overall monitoring efficiency. By setting a non-monitoring determination unit to compare the effective monitoring area with the area to be monitored, it is possible to clearly identify which areas have not yet been monitored, namely, non-monitored areas, which helps to ensure full coverage of the area to be monitored. By setting a parameter determination unit based on the analysis results of the overlapping monitoring area and the non-monitoring area, the preset installation parameters are dynamically adjusted to obtain more reasonable and effective actual installation parameters, which helps to improve the coverage and monitoring accuracy of the monitoring system, and can also improve the stability and reliability of the system.

[0048] In particular, by setting the degree analysis block to analyze each overlapping monitoring area and determining the severity of the overlap based on a certain algorithm or standard, an important basis is provided for subsequent adjustments, ensuring the pertinence and effectiveness of the adjustments; by setting the priority determination block to determine the adjustment order of the initial monitoring equipment based on the evaluation results of the overlap severity, giving priority to the areas with the most serious overlap, which helps to quickly reduce the overall overlap level and improve the adjustment efficiency; by setting the adjustment block, the preset installation angle and / or preset installation position of the initial monitoring equipment is adjusted according to the adjustment priority, the overlapping monitoring area is reduced, and the layout of the monitoring equipment is optimized; by reducing the overlapping monitoring area, it helps to improve the overall performance of the monitoring system and improve the monitoring accuracy and efficiency; by setting the repeated block, the functions of the degree analysis, priority determination and adjustment blocks are repeatedly executed to ensure that after multiple adjustments, the total value of the overlapping monitoring area reaches the minimum value, thereby obtaining the optimal monitoring equipment layout plan, which helps to eliminate potential layout problems and improve the stability and reliability of the monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A structural block diagram of a large-scale multi-target on-site holographic intelligent monitoring integrated management system provided by an embodiment of the present invention;

[0050] Figure 2 A structural block diagram of a device determination module in a large-scene multi-target on-site holographic intelligent monitoring integrated management system provided by an embodiment of the present invention;

[0051] Figure 3 A structural block diagram of a model building module in a large-scene multi-target on-site holographic intelligent monitoring integrated management system provided by an embodiment of the present invention;

[0052] Figure 4 A structural block diagram of the abnormality recognition module in the large-scene multi-target on-site holographic intelligent monitoring integrated management system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0054] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0055] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be understood as a limitation on the present invention.

[0056] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0057] See also Figure 1 As shown, an embodiment of the present invention provides a large-scene multi-target on-site holographic intelligent monitoring integrated management system, the system comprising:

[0058] The device determination module 10 is used to install a number of initial monitoring devices based on preset installation parameters, and the device determination module is further used to adjust the preset installation parameters based on the abnormal monitoring area and the area to be monitored to obtain actual installation parameters, and to adjust the number of initial monitoring devices based on the actual installation parameters to obtain a number of actual monitoring devices;

[0059] A model building module 20, connected to the device determination module 10, is used to obtain a number of initial monitoring video data of a number of actual monitoring devices, and to build an initial monitoring model of the area to be monitored based on analysis results of the initial monitoring video data to determine a number of sub-areas in the initial monitoring model;

[0060] A level determination module 30, connected to the model building module 20, is used to determine the initial importance levels of the plurality of sub-areas according to the identified and determined initial numbers of preset monitoring targets in the plurality of sub-areas;

[0061] an abnormality identification module 40, connected to the level determination module 30, for determining an initial monitoring frequency for each of the sub-areas based on the initial importance level, and identifying an initial abnormal event according to an initial monitoring result for each of the sub-areas based on the initial monitoring frequency;

[0062] The degree analysis module 50 is connected to the abnormality identification module 40, and is used to analyze the initial abnormal event, obtain the initial severity, and issue an alarm based on the comparison result of the initial severity, or adjust the initial monitoring frequency to obtain the adjusted monitoring frequency. The degree analysis module is also used to analyze the monitoring results obtained during the real-time monitoring of the sub-area corresponding to the abnormal event based on the adjusted monitoring frequency, and to issue an alarm based on the analysis results, or continue monitoring.

[0063] Specifically, the embodiment of the present invention configures the system to prevent the occurrence of open-pit coal mine safety accidents, such as predicting safety issues of slopes, vehicles, personnel, etc., and issuing early warning signals in a timely manner, thereby preventing such incidents from happening again.

[0064] Specifically, the embodiment of the present invention can quickly deploy the initial monitoring equipment according to the preset installation parameters by setting the equipment determination module, and dynamically adjust the installation parameters based on the actual monitoring needs, so as to ensure the accurate deployment and efficient use of the monitoring equipment. By setting the model construction module, the initial monitoring model of the area to be monitored can be automatically constructed by analyzing the initial monitoring video data, and the key sub-areas can be identified, thereby improving the accuracy of monitoring. By setting the level determination module, the initial importance level of the monitoring target can be determined according to the number of preset monitoring targets in each sub-area, and differentiated monitoring frequencies can be formulated accordingly, so that the system can give priority to high-risk areas, improve monitoring efficiency, and avoid waste of resources. By setting the abnormal identification module, the initial abnormal events can be discovered and identified in time through real-time monitoring of each sub-area. By setting the degree analysis module, the severity of the abnormal events can be further analyzed, and corresponding alarm measures or monitoring frequencies can be taken according to the analysis results, which is helpful to achieve rapid response and effective management of abnormal events. The system can not only perform alarm processing, but also adjust the initial monitoring frequency as needed, and even re-divide the monitoring model through the analysis of real-time monitoring results. The continuous optimization and iteration capabilities enable the system to continuously adapt to changes in the monitoring environment and maintain efficient and accurate monitoring effects.

[0065] See also Figure 2 As shown, the device determination module 10 includes:

[0066] An overlap determination unit 11, used to determine an effective monitoring area of ​​any of the initial monitoring devices, and to analyze a plurality of effective monitoring areas to determine a plurality of overlapping monitoring areas;

[0067] A non-monitoring determination unit 12, configured to determine a number of non-monitoring areas based on the number of effective monitoring areas and the areas to be monitored;

[0068] The parameter determination unit 13 is connected to the overlap determination unit 11 and the non-monitoring determination unit 12, and is used to analyze a number of the overlapped monitoring areas and a number of the non-monitoring areas, and to adjust the preset installation parameters based on the analysis results to obtain actual installation parameters.

[0069] Specifically, the embodiment of the present invention determines the effective monitoring area of ​​any of the initial monitoring devices including:

[0070] Acquire monitoring data of any of the initial monitoring devices, and construct a monitoring screen based on the monitoring data;

[0071] An effective monitoring area of ​​any of the initial monitoring devices is determined based on the monitoring screen.

[0072] Specifically, the step of determining the plurality of overlapping monitoring areas and the plurality of unmonitored areas in the embodiment of the present invention includes:

[0073] Comparing the plurality of monitoring images, and determining the plurality of overlapping monitoring areas according to the comparison results;

[0074] Compare the plurality of effective monitoring areas with the areas to be monitored, and determine a plurality of unmonitored areas according to the comparison results.

[0075] Specifically, the embodiment of the present invention analyzes the effective monitoring area of ​​each initial monitoring device by setting an overlapping determination unit, and can accurately identify which areas are covered by multiple devices at the same time, that is, overlapping monitoring areas, which helps to avoid waste of resources. After identifying the overlapping areas, the device position or parameters are adjusted according to actual needs to reduce unnecessary overlap, thereby improving the overall monitoring efficiency. By setting a non-monitoring determination unit to compare the effective monitoring area with the area to be monitored, it can clearly identify which areas have not yet been monitored, that is, non-monitored areas, which helps to ensure full coverage of the area to be monitored. By setting a parameter determination unit based on the analysis results of the overlapping monitoring area and the non-monitoring area, the preset installation parameters are dynamically adjusted to obtain more reasonable and effective actual installation parameters, which helps to improve the coverage and monitoring accuracy of the monitoring system, and can also improve the stability and reliability of the system.

[0076] Specifically, the parameter determination unit includes:

[0077] an adjusting subunit, used for adjusting a preset installation angle and / or a preset installation position of the initial monitoring device so as to minimize a total value of the plurality of overlapping monitoring areas;

[0078] The adding subunit is connected to the adjusting subunit and is used to detect the adjusted unmonitored area to determine the regional environmental characteristics of the unmonitored area and to add monitoring equipment according to the regional environmental characteristics.

[0079] Specifically, the adding subunit described in the embodiment of the present invention includes:

[0080] A detection unit, used to detect the adjusted unmonitored area and analyze the size of the unmonitored area;

[0081] A selection unit, connected to the detection unit, is used to select the appropriate type and quantity of monitoring equipment according to the size of the area, ensuring that the monitoring equipment can effectively cover the unmonitored area;

[0082] The deployment unit is connected to the selection unit and is used to perform physical deployment of the monitoring equipment according to the selection result, including installation, debugging and calibration steps, to ensure that the monitoring equipment can work normally and provide accurate monitoring data.

[0083] Specifically, the embodiments of the present invention significantly reduce the total value of overlapping monitoring areas by setting an adjustment subunit to adjust the preset installation angle and / or preset installation position of the initial monitoring device, which helps to avoid waste of resources. Reducing overlap means that each monitoring device can focus more on its unique monitoring area, thereby improving the overall monitoring efficiency. By setting an addition subunit to detect the remaining unmonitored areas, new monitoring devices are added in a targeted manner according to regional environmental characteristics to fill the monitoring gaps, ensuring comprehensive coverage of the area to be monitored, improving the integrity and reliability of the system, and selecting appropriate monitoring equipment according to regional environmental characteristics to ensure maximization of the monitoring effect.

[0084] Specifically, the adjustment subunit includes:

[0085] A severity analysis block is used to analyze a number of overlapping monitoring areas and determine the severity of the overlap according to the analysis results;

[0086] a priority determination block, connected to the degree analysis block, for determining adjustment priorities of a plurality of the initial monitoring devices based on the overlap severity;

[0087] an adjustment block connected to the priority determination block, for adjusting the preset installation angle and / or the preset installation position of each of the initial monitoring devices based on the adjustment priority, so as to reduce the overlapping monitoring area formed by the initial monitoring devices adjacent thereto;

[0088] The repeat block is connected with the degree analysis block, the priority determination block and the adjustment block, and is used to repeatedly execute the functions of the above blocks until the total value of the overlapping monitoring areas reaches a minimum value.

[0089] Specifically, the step of determining the severity of overlap according to the analysis result in the embodiment of the present invention includes:

[0090] Calculating the overlapping area of ​​any of the overlapping detection areas;

[0091] Counting the number of initial monitoring devices corresponding to any of the rewriting detection areas, and using the statistical result as the overlap number;

[0092] An overlap score is calculated based on the overlap area and the overlap quantity to determine the overlap severity according to the overlap score.

[0093] Specifically, the overlap score in the embodiment of the present invention=(overlap area / preset overlap area)×(overlap number / preset overlap number);

[0094] The preset overlapping area is 1 / 100 of the area of ​​the area to be monitored, and the preset overlapping number is 2.

[0095] Specifically, the embodiment of the present invention determines the overlap severity according to the overlap score by sorting the overlap scores from large to small, and the sorting result corresponds to the overlap severity level classification, and the area with the highest score is regarded as the most serious overlap area.

[0096] Specifically, the embodiment of the present invention analyzes each overlapping monitoring area by setting a degree analysis block, and determines the severity of the overlap based on a certain algorithm or standard, which provides an important basis for subsequent adjustments and ensures the pertinence and effectiveness of the adjustments. By setting a priority determination block, the adjustment order of the initial monitoring equipment is determined based on the evaluation results of the severity of the overlap, and the areas with the most serious overlap are given priority, which helps to quickly reduce the overall overlap level and improve the adjustment efficiency. By setting an adjustment block, the preset installation angle and / or preset installation position of the initial monitoring equipment is adjusted according to the adjustment priority, the overlapping monitoring area is reduced, and the layout of the monitoring equipment is optimized. By reducing the overlapping monitoring area, it helps to improve the overall performance of the monitoring system and improve the monitoring accuracy and efficiency. By setting a repeated block, the functions of the degree analysis, priority determination and adjustment blocks are repeatedly executed to ensure that after multiple adjustments, the total value of the overlapping monitoring area reaches the minimum value, thereby obtaining the optimal monitoring equipment layout plan, which helps to eliminate potential layout problems and improve the stability and reliability of the monitoring system.

[0097] See also Figure 3 As shown, the model building module 20 includes:

[0098] A data acquisition unit 21, used for acquiring a corresponding number of initial monitoring video data based on a number of the real-time detection devices;

[0099] A data processing unit 22, connected to the data acquisition unit 21, is used to use a computer vision algorithm to identify a plurality of preset monitoring targets in a plurality of initial monitoring video data to obtain preset monitoring target data, and to use a multi-target tracking algorithm to track a plurality of preset monitoring targets, and to use a deep learning model to identify the behaviors of a plurality of preset monitoring targets to obtain target behavior data;

[0100] The construction unit 23 is connected to the data processing unit 23, and is used to determine the monitoring boundary according to the several initial monitoring video data, to divide the area to be monitored into several sub-areas according to the several monitoring boundaries, and to extract the regional environmental characteristics in the several sub-areas to construct several sub-scene models, and to combine the several sub-scene models to obtain the overall scene model, to construct several monitoring target models based on the several preset monitoring target data and their corresponding target behavior data, and to construct the initial monitoring model based on the overall scene model and the monitoring target model.

[0101] Specifically, an example of the model building module 20 in the embodiment of the present invention is:

[0102] The data processing unit uses a computer vision algorithm to process the collected initial monitoring video data. For example, various preset monitoring targets in open-pit coal mines, such as miners, transport vehicles, and excavation equipment, are identified through image recognition technology. For image data, the algorithm can detect the contour, color, texture and other features of the target to determine the location and category of the target. A multi-target tracking algorithm is used to track several preset monitoring targets that have been identified. The tracking algorithm continuously tracks and locates the target based on the target's appearance features, movement speed, direction and other information. A deep learning model is used to identify the behavior of the preset monitoring targets. For example, the working status of miners, the speed and direction of transport vehicles, the operating mode of excavation equipment, etc. can be identified. The deep learning model can automatically extract the characteristics of the target and accurately classify and identify the target's behavior by learning from a large amount of labeled data.

[0103] The construction unit determines the monitoring boundary of the open-pit coal mine based on the collected initial monitoring video data. For example, the corresponding monitoring boundary of the open-pit coal mine is determined according to the video boundary in the initial monitoring video data. The monitored area is divided into several sub-areas based on the boundaries of several initial monitoring video data. Several sub-scene models are constructed based on regional environmental characteristics. Each sub-scene model corresponds to a specific sub-area. The regional environmental characteristics include terrain characteristics, etc. These sub-scene models are combined to form a comprehensive total scene model. The total scene model is combined with the monitoring target model to form a complete initial monitoring model.

[0104] Specifically, the embodiment of the present invention determines the monitoring boundary according to the initial monitoring video data, including:

[0105] Identify edge information in the initial monitoring video data based on an edge detection algorithm to determine the boundaries of the video screen;

[0106] The boundary of the video screen is used as the monitoring boundary.

[0107] Specifically, the embodiment of the present invention can use a deep learning model, which can be a recurrent neural network (RNN) or a long short-term memory network (LSTM), to perform time series analysis on the target's behavior, thereby identifying the target's behavior pattern.

[0108] Specifically, the computer vision algorithms of the embodiments of the present invention include YOLO, SSD, etc.

[0109] Specifically, the embodiments of the present invention acquire monitoring video data from the detection device in real time by setting a data acquisition unit, thereby ensuring the timeliness and accuracy of the data, providing an accurate data basis for subsequent data processing and model building, and collecting data from multiple real-time detection devices to ensure the comprehensiveness and completeness of the data, thereby improving the accuracy and reliability of the model. By setting the data processing unit to use computer vision algorithms and multi-target tracking algorithms, it is possible to accurately identify and track preset monitoring targets in the video, help to quickly locate targets in complex scenes, and obtain their motion trajectory and state information, identify target behaviors through deep learning models, understand the target's behavior patterns and intentions, and provide guidance for subsequent It provides an accurate data basis for security analysis and anomaly detection. By setting up construction units to determine the monitoring boundaries based on the initial monitoring video data and dividing the area to be monitored into several sub-areas, it helps to decompose complex scenes into more manageable parts, facilitate subsequent analysis and modeling, extract environmental features in sub-areas, and build sub-scene models based on these features to provide accurate data for subsequent monitoring and early warning. The sub-scene models are combined to form an overall scene model, and a monitoring target model is built based on the preset monitoring target data and its behavior data. This allows the model to reflect the structure and characteristics of the overall scene, and to focus on specific monitoring targets and their behavior changes, thereby improving the accuracy of model construction.

[0110] Specifically, the level determination module 30 includes:

[0111] A target recognition unit, used for recognizing a preset monitoring target in any of the sub-areas based on a computer vision algorithm;

[0112] The level determination unit is connected to the target recognition unit and is used to count the initial number of preset detection targets in any of the sub-areas, sort the initial numbers from large to small, and determine the initial importance level corresponding to each sub-area according to the sorting result.

[0113] Specifically, the target recognition unit described in the embodiment of the present invention adopts the YOLO algorithm to perform target detection. The YOLO algorithm can quickly and accurately identify multiple targets in the video, and can process video stream data in real time. The YOLO algorithm divides the image into grids, each grid is responsible for predicting the target whose center point falls within the grid, and each grid predicts the bounding box and probability. In this way, the YOLO algorithm can simultaneously perform target detection and positioning, greatly improving the detection speed and accuracy.

[0114] See also Figure 4 As shown, the abnormality identification module 40 includes:

[0115] A behavior determination unit 41, used to determine actual behavior data of any preset monitoring target based on a number of the initial monitoring results;

[0116] An environment determination unit 42, configured to determine actual environment data in any of the sub-areas based on a number of the initial monitoring results;

[0117] The abnormality determination unit 43 is connected to the behavior determination unit 41 and the environment determination unit 42, and is used to compare the actual behavior data with the preset behavior data to obtain a behavior comparison result, compare the actual environment data with the preset environment data to obtain an environment comparison result, and determine the initial abnormal event based on the behavior comparison result and the environment comparison result.

[0118] Specifically, the preset behavior data in the embodiment of the present invention is a normal behavior range corresponding to a preset monitoring target, which is determined based on historical work behavior data. For example, the preset behavior data includes a normal working mode of a miner, a reasonable driving speed range of a transport vehicle, a normal operating process of an excavation device, etc.;

[0119] The preset environmental data is determined based on the regional environmental characteristics collected when the initial monitoring model is constructed.

[0120] Specifically, the behavior determination unit described in the embodiment of the present invention adopts a convolutional neural network (CNN) to perform behavior analysis. CNN performs well in processing image data and can extract key behavior features from video frames. It can classify and identify the target's behavior through a trained model to determine whether it meets the range of preset behavior data.

[0121] Specifically, the embodiments of the present invention set a behavior determination unit to accurately determine the actual behavior data of the preset monitoring target based on the initial monitoring results, which helps to grasp the state changes of the monitoring target in real time and provide an accurate basis for subsequent abnormality analysis. By setting an environment determination unit to determine the actual environmental data in the sub-area based on the initial monitoring results in real time, data support is provided for building a more comprehensive and accurate monitoring model. By setting an abnormality determination unit to compare the actual behavior data with the preset behavior data, and the actual environmental data with the preset environmental data, a comprehensive judgment of the initial abnormal event is made, which helps to improve the accuracy and reliability of abnormality identification and improve the accuracy of the entire monitoring system.

[0122] Specifically, the degree analysis module 50 includes:

[0123] a severity determination unit, configured to analyze the initial abnormal event and determine an initial severity according to the analysis result;

[0124] a comparison unit, connected to the degree determination unit, for comparing the initial severity with the preset severity to obtain a comparison result;

[0125] The adjustment unit is connected to the comparison unit and is used to calculate an adjustment parameter when the initial severity is less than a preset severity, and adjust the initial monitoring frequency based on the adjustment parameter.

[0126] Specifically, the preset severity level in the embodiment of the present invention is 0.7.

[0127] Specifically, the degree determination unit includes:

[0128] A behavior calculation subunit, used to calculate a behavior score based on a deviation value between the actual behavior data and the preset behavior data;

[0129] An environment calculation subunit, used to calculate an environment score based on a deviation value between the actual environment data and the preset environment data;

[0130] The severity determination subunit is connected to the behavior calculation subunit and the environment calculation subunit, and is used to calculate an abnormality score based on the behavior score and the environment score, and determine the initial severity based on the abnormality score.

[0131] Specifically, an example of the behavior calculation subunit in the embodiment of the present invention is behavior score = deviation value between actual behavior data and preset behavior data;

[0132] An example of the environment calculation subunit is environment score = deviation value between actual environment data and preset environment data;

[0133] The initial severity=α×behavior score+β×environment score, wherein α and β are preset weight coefficients, and α+β=1.

[0134] Specifically, the deviation value between the actual behavior data and the preset behavior data described in the embodiment of the present invention can be calculated by calculating the similarity deviation value between the actual behavior data and the preset behavior data. For example, the preset behavior data includes the normal working mode of the miner, the reasonable driving speed range of the transport vehicle, the normal operation process of the excavation equipment, etc. The working mode of the actual miner is matched with the normal working mode of the miner to obtain the similarity value, and (1-similarity value) is determined as the deviation value between the actual behavior data and the preset behavior data;

[0135] The deviation value between the actual environment data and the preset environment data can be calculated by calculating the similarity deviation value between the actual environment data and the preset environment data. For example, the preset environment data is determined based on the regional environment characteristics collected when the initial monitoring model is constructed. The actual regional environment characteristics are compared with the regional environment characteristics collected when the initial monitoring model is constructed, and the similarity matching result between the two is calculated, and (1-similarity value) is used as the deviation value between the actual environment data and the preset environment data.

[0136] Specifically, the similarity calculated in the embodiment of the present invention may be calculated by a cosine similarity algorithm.

[0137] Specifically, the adjustment unit includes:

[0138] a parameter calculation subunit, configured to calculate the adjustment parameter based on the initial severity and the preset severity;

[0139] The adjustment subunit is connected to the parameter calculation subunit and is used to calculate based on the adjustment parameter and the initial monitoring frequency to obtain the adjusted monitoring frequency.

[0140] Specifically, the adjustment parameter in the embodiment of the present invention = initial severity / preset severity;

[0141] The adjusted monitoring frequency=adjustment parameter×initial monitoring frequency.

[0142] Specifically, the embodiment of the present invention analyzes the real-time monitoring results, issues an alarm based on the analysis results, or continues monitoring, including:

[0143] Based on a number of real-time monitoring results, real-time abnormal events are identified and the real-time severity of the real-time abnormal events is determined. When the real-time severity is greater than or equal to the preset severity, an alarm is issued. When the real-time severity is less than the preset severity, monitoring continues.

[0144] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0145] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A large-scale multi-target on-site holographic intelligent monitoring integrated management system, characterized in that: include: an equipment determination module, configured to install a number of initial monitoring equipment based on preset installation parameters, and further configured to adjust the preset installation parameters based on the abnormal monitoring area and the area to be monitored to obtain actual installation parameters, and to adjust the number of initial monitoring equipment based on the actual installation parameters to obtain a number of actual monitoring equipment; a model building module, connected to the device determination module, for obtaining a number of initial monitoring video data of a number of actual monitoring devices, and building an initial monitoring model of the area to be monitored based on analysis results of the number of initial monitoring video data to determine a number of sub-areas in the initial monitoring model; A level determination module, connected to the model building module, for determining initial importance levels of several sub-areas according to the identified and determined initial numbers of preset monitoring targets in the several sub-areas; an abnormality identification module, connected to the level determination module, for determining an initial monitoring frequency for each of the sub-areas based on the initial importance level, and identifying an initial abnormal event according to an initial monitoring result for each of the sub-areas based on the initial monitoring frequency; The degree analysis module is connected to the abnormality identification module, and is used to analyze the initial abnormal event, obtain the initial severity, and issue an alarm based on the comparison result of the initial severity, or adjust the initial monitoring frequency to obtain the adjusted monitoring frequency. The degree analysis module is also used to analyze the monitoring results obtained during the real-time monitoring of the sub-area corresponding to the abnormal event based on the adjusted monitoring frequency, and to issue an alarm based on the analysis results, or continue monitoring.

2. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 1 is characterized in that: The device determination module includes: an overlap determination unit, configured to determine an effective monitoring area of ​​any of the initial monitoring devices, and to analyze a plurality of effective monitoring areas to determine a plurality of overlapping monitoring areas; a non-monitoring determination unit, configured to determine a number of non-monitoring areas based on the number of the effective monitoring areas and the areas to be monitored; A parameter determination unit is connected to the overlap determination unit and the non-monitoring determination unit, and is used to analyze a number of the overlapped monitoring areas and a number of the non-monitoring areas, and to adjust the preset installation parameters based on the analysis results to obtain actual installation parameters.

3. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 2 is characterized in that: The parameter determination unit comprises: an adjusting subunit, used for adjusting a preset installation angle and / or a preset installation position of the initial monitoring device so as to minimize a total value of the plurality of overlapping monitoring areas; The adding subunit is connected to the adjusting subunit and is used to detect the adjusted unmonitored area to determine the regional environmental characteristics of the unmonitored area and to add monitoring equipment according to the regional environmental characteristics.

4. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 3 is characterized in that: The adjustment subunit comprises: A severity analysis block is used to analyze a number of overlapping monitoring areas and determine the severity of the overlap according to the analysis results; a priority determination block, connected to the degree analysis block, for determining adjustment priorities of a plurality of the initial monitoring devices based on the overlap severity; an adjustment block connected to the priority determination block, for adjusting the preset installation angle and / or the preset installation position of each of the initial monitoring devices based on the adjustment priority, so as to reduce the overlapping monitoring area formed by the initial monitoring devices adjacent thereto; The repeat block is connected with the degree analysis block, the priority determination block and the adjustment block, and is used to repeatedly execute the functions of the above blocks until the total value of the overlapping monitoring areas reaches a minimum value.

5. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 4 is characterized in that: The model building module includes: A data acquisition unit, used for acquiring a corresponding number of initial monitoring video data based on a number of the real-time detection devices; a data processing unit connected to the data acquisition unit, configured to use a computer vision algorithm to identify a plurality of preset monitoring targets in a plurality of initial monitoring video data to obtain preset monitoring target data, and to use a multi-target tracking algorithm to track a plurality of preset monitoring targets, and to use a deep learning model to identify the behaviors of a plurality of preset monitoring targets to obtain target behavior data; A construction unit is connected to the data processing unit, and is used to determine the monitoring boundary according to the several initial monitoring video data, to divide the area to be monitored into several sub-areas according to the several monitoring boundaries, and to extract the regional environmental characteristics in the several sub-areas to construct several sub-scene models, and to combine the several sub-scene models to obtain the overall scene model, to construct several monitoring target models based on the several preset monitoring target data and their corresponding target behavior data, and to construct the initial monitoring model based on the overall scene model and the monitoring target model.

6. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 5 is characterized in that: The level determination module comprises: A target recognition unit, used for recognizing a preset monitoring target in any of the sub-areas based on a computer vision algorithm; The level determination unit is connected to the target recognition unit and is used to count the initial number of preset detection targets in any of the sub-areas, sort the initial numbers from large to small, and determine the initial importance level corresponding to each sub-area according to the sorting result.

7. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 6 is characterized in that: The abnormality identification module comprises: A behavior determination unit, used to determine actual behavior data of any preset monitoring target based on a number of the initial monitoring results; An environment determination unit, configured to determine actual environment data in any of the sub-areas based on a number of the initial monitoring results; An abnormality determination unit is connected to the behavior determination unit and the environment determination unit, and is used to compare the actual behavior data with the preset behavior data to obtain a behavior comparison result, compare the actual environment data with the preset environment data to obtain an environment comparison result, and determine the initial abnormal event based on the behavior comparison result and the environment comparison result.

8. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 7 is characterized in that: The degree analysis module includes: a severity determination unit, configured to analyze the initial abnormal event and determine an initial severity according to the analysis result; a comparison unit, connected to the degree determination unit, for comparing the initial severity with the preset severity to obtain a comparison result; The adjustment unit is connected to the comparison unit and is used to calculate an adjustment parameter when the initial severity is less than a preset severity, and adjust the initial monitoring frequency based on the adjustment parameter.

9. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 8 is characterized in that: The degree determination unit comprises: A behavior calculation subunit, used to calculate a behavior score based on a deviation value between the actual behavior data and the preset behavior data; An environment calculation subunit, used to calculate an environment score based on a deviation value between the actual environment data and the preset environment data; The severity determination subunit is connected to the behavior calculation subunit and the environment calculation subunit, and is used to calculate an abnormality score based on the behavior score and the environment score, and determine the initial severity based on the abnormality score.

10. The large-scale multi-target on-site holographic intelligent monitoring integrated management system according to claim 9 is characterized in that: The adjustment unit comprises: a parameter calculation subunit, configured to calculate the adjustment parameter based on the initial severity and the preset severity; The adjustment subunit is connected to the parameter calculation subunit and is used to calculate based on the adjustment parameter and the initial monitoring frequency to obtain the adjusted monitoring frequency.

Images