A real-time monitoring system based on a sensor array

Abstract

The application discloses a real-time monitoring system based on a sensing array, relates to the technical field of Internet of Things, and effectively improves the accuracy of monitoring results. The application establishes a plurality of data-level local background image models according to historical multi-modal data, splices and overlaps local background multi-dimensional image models generated by different sensing arrays to obtain a multi-modal scene monitoring model, establishes a multi-modal event dynamic three-dimensional model and corresponding event feature tracks according to a plurality of historical multi-modal behavior data of a plurality of preset abnormal behavior events, further generates event feature track data sets of the abnormal behavior events, generates corresponding real-time scene-event multi-modal interaction tracks according to real-time multi-modal scene data, matches the real-time scene-event multi-modal interaction tracks with the event feature track data sets, and judges an abnormal behavior event currently occurring in a monitoring scene according to a matching result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things, and particularly relates to a real-time monitoring system based on a sensing array. BACKGROUND

[0002] The existing monitoring technology mainly realizes real-time monitoring through a single optical camera, and cannot realize multi-level and multi-angle scene monitoring, so that some areas cannot be effectively monitored, and thus the monitoring effect and accuracy cannot reach an ideal degree.

[0003] A sensing array is a system composed of multiple sensors, which is used for monitoring and detecting activities and events in a specific area. These sensors can be various types, such as infrared sensors, microwave sensors, sound sensors, etc., which can detect different physical quantities or events, such as movement, temperature change, sound, etc. The application field of the sensing array is very wide, from home security systems to industrial monitoring, environmental monitoring and military use, etc.

[0004] Therefore, how to realize multi-dimensional monitoring of the monitoring scene while ensuring the accuracy of the monitoring result is a difficulty in the prior art, and thus a real-time monitoring system based on a sensing array is provided. SUMMARY

[0005] In order to solve the above technical problems, the purpose of the present application is to provide a real-time monitoring system based on a sensing array.

[0006] In order to achieve the above purpose, the present application provides the following technical scheme:

[0007] A real-time monitoring system based on a sensing array comprises a cloud computing platform, which is in communication connection with a scene data acquisition module, a monitoring model establishment module, a behavior event analysis module and a scene monitoring module.

[0008] The scene data acquisition module is used for setting a sensing array and a data acquisition period in a monitoring scene, and then acquiring historical multi-modal data and real-time multi-modal data of the monitoring scene in each data acquisition period through the sensing array.

[0009] The monitoring model establishment module is used for establishing a plurality of data-level local background image models according to the historical multi-modal data, and then mutually overlapping the local background image models of each data level to obtain a local background multi-dimensional image model, and simultaneously splicing and overlapping the local background multi-dimensional image models generated by different sensing arrays to obtain a multi-modal scene monitoring model of the monitoring scene.

[0010] The behavior event analysis module is configured to establish a multi-modal event dynamic three-dimensional model and corresponding event feature trajectory according to a plurality of historical multi-modal behavior data of a plurality of preset abnormal behavior events, and further generate an event feature trajectory data set of a corresponding abnormal behavior event according to the multi-modal event dynamic three-dimensional model and the corresponding event feature trajectory;

[0011] The scene monitoring module is configured to update a multi-modal scene monitoring model according to real-time multi-modal scene data, generate a real-time multi-modal event dynamic three-dimensional model input into the updated multi-modal scene monitoring model, and further generate a corresponding real-time scene-event multi-modal interaction trajectory. The real-time scene-event multi-modal interaction trajectory is matched with each event feature trajectory data set, and an abnormal behavior event currently occurring in the monitored scene is determined according to a matching result.

[0012] Further, the multi-modal data collection process includes:

[0013] The scene data collection module sets a plurality of groups of sensing arrays in the monitored scene, each group of sensing arrays is composed of a plurality of sensors, and each group of sensing arrays is provided with a sensor setting number;

[0014] Each sensor is provided with the same data collection period, and the data collection ranges of the same type of sensors in adjacent sensing arrays partially overlap;

[0015] When the data collection period starts, each sensor in each sensing array collects data in its data collection range, and the collected data is sent to the scene data collection module when the data collection period ends;

[0016] The scene data collection module sets a number to the corresponding uploaded data according to the number of the sensors, integrates the uploaded data with the same first number of subscripts, and further obtains historical multi-modal scene data and real-time multi-modal scene data corresponding to the sensing arrays.

[0017] Further, the establishment process of the local background image model includes:

[0018] The monitoring model establishment module first extracts historical scene optical video data from the historical multi-modal scene data, divides the historical scene optical video data into a plurality of historical scene optical image data according to frames, further segments a plurality of scene feature targets from each historical scene optical image data, sequentially superimposes all historical scene optical image data corresponding to the same scene feature target in the same historical scene optical video data, and establishes a corresponding scene target dynamic model according to a superimposition result;

[0019] The scene target dynamic models are spliced to obtain corresponding local scene image dynamic models, a three-dimensional coordinate system is established, and the local scene image dynamic models are mapped on the three-dimensional coordinate system. According to the number of historical scene optical image data corresponding to the historical scene optical video data, a plurality of time nodes are set;

[0020] A plurality of feature space coordinates are randomly set on each local scene target dynamic model, and then the displacement trajectory of each local scene target dynamic model in the local scene image dynamic model is obtained when the feature space coordinates change with the time nodes;

[0021] The displacement trajectories of the same scene feature target generated by the same sensing array under different data acquisition periods are compared with each other. If the displacement trajectories of the scene feature target under each data acquisition period exist and change periodically, and the similarity of the displacement trajectories under each period is more than 95%, the corresponding scene feature target is set as a scene background target label, otherwise no operation is performed;

[0022] The scene feature targets in each local scene image dynamic model without a scene background target label are removed, and the remaining scene feature targets are retained to obtain a local background optical image model corresponding to the historical optical video data. Similar to the process of establishing the local background optical image model, the corresponding local background image model is established according to each data from the same multi-modal data except the historical optical video data.

[0023] Further, the process of establishing the multi-modal scene monitoring model includes:

[0024] Each local background model corresponding to each multi-modal data is overlapped and mapped to obtain a corresponding local background multi-dimensional image model. According to the actual spatial distribution of each sensing array in the monitoring scene, the local background multi-dimensional image models generated by different sensing arrays are spliced and overlapped to obtain a multi-modal scene monitoring model of the monitoring scene, and the multi-modal scene monitoring model is sent to a behavior event analysis module and a scene monitoring module.

[0025] Further, the process of establishing the multi-modal event dynamic three-dimensional model includes:

[0026] Similar to the process of extracting each scene feature target from historical multi-modal scene data, a plurality of event feature targets are extracted from each historical multi-modal behavior data. Each event feature target is matched with a scene feature target in the multi-modal scene monitoring model, and the corresponding event feature target is removed according to the matching result.

[0027] A plurality of action recognition points are set on the reserved event feature target, and then a corresponding multi-modal event dynamic three-dimensional model is established according to the historical multi-modal event data and the reserved event feature target.

[0028] Further, the event feature trajectory data set establishment process includes:

[0029] A three-dimensional coordinate system is established, the multi-modal event dynamic three-dimensional model and the multi-modal scene monitoring model are overlapped and mapped on the three-dimensional coordinate system, and then according to the trajectory generated by the action recognition points on the multi-modal event dynamic three-dimensional model when the multi-modal event dynamic three-dimensional model is displaced in the multi-modal scene monitoring model, a corresponding event feature trajectory is generated.

[0030] Meanwhile, the scene feature targets in the interaction between the multi-modal event dynamic three-dimensional model and the multi-modal scene monitoring model in the displacement process are labeled, and then an event feature trajectory data set corresponding to an abnormal behavior event is generated.

[0031] The event feature trajectory data set is composed of a plurality of scene-event multi-modal interaction trajectory ranges.

[0032] Further, the scene-event multi-modal interaction trajectory range generation process includes:

[0033] A plurality of model regions of the same size are divided in each scene feature target, and the model region is composed of a plurality of layers of data.

[0034] When the multi-modal event dynamic three-dimensional model interacts with any scene feature target over time, the corresponding scene feature target is displaced or locally deformed, and then the displacement trajectories of each model region are counted and integrated to form a corresponding scene-event multi-modal interaction trajectory.

[0035] The scene-event multi-modal interaction trajectories corresponding to the same pair of event feature targets and scene feature targets in each historical multi-modal behavior data of the same abnormal behavior event are overlapped with each other, and then the scene-event multi-modal interaction trajectory range corresponding to each event feature target and scene feature target in each abnormal behavior event is obtained.

[0036] Further, the process of real-time monitoring of the monitoring scene according to the event feature trajectory data set includes:

[0037] The scene monitoring module adopts a process similar to the construction of the multi-modal scene monitoring model, the multi-modal event dynamic three-dimensional model, and the scene-event multi-modal interaction trajectory, and constructs a real-time multi-modal scene monitoring model and a real-time multi-modal event dynamic three-dimensional model according to real-time multi-modal scene data.

[0038] According to the real-time multi-modal scene monitoring model, the multi-modal scene monitoring model is updated, and the real-time multi-modal event dynamic three-dimensional model is input into the updated multi-modal scene monitoring model, and then the corresponding real-time scene-event multi-modal interaction track is generated.

[0039] The real-time scene-event multi-modal interaction track is matched with each event feature track data set, if each level data of the real-time scene-event multi-modal interaction track is in the scene-event multi-modal interaction track range in an event feature track data set, it is judged that the monitoring occurs corresponding abnormal behavior event, and then the corresponding behavior warning prompt is generated and sent to the monitoring personnel, otherwise no operation is done.

[0040] Compared with the prior art, the beneficial effects of the present application are:

[0041] The present application establishes a plurality of data level local background image models according to historical multi-modal data, splices and overlaps the local background multi-dimensional image models generated by different sensing arrays to obtain a multi-modal scene monitoring model, establishes a multi-modal event dynamic three-dimensional model and corresponding event feature track according to a plurality of historical multi-modal behavior data of a plurality of preset abnormal behavior events, and then generates an event feature track data set of each abnormal behavior event, generates a corresponding real-time scene-event multi-modal interaction track according to real-time multi-modal scene data, matches the real-time scene-event multi-modal interaction track with each event feature track data set, and judges the abnormal behavior event currently occurring in the monitoring scene according to the matching result, thereby realizing multi-dimensional monitoring of the monitoring scene while ensuring the accuracy of the monitoring result. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application.

[0043] Figure 1 The schematic diagram of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described in detail below. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0045] As Figure 1As shown, a real-time monitoring system based on a sensing array includes a cloud computing platform, which is communicatively connected with a scene data acquisition module, a monitoring model establishment module, a behavior event analysis module, and a scene monitoring module;

[0046] The scene data acquisition module is configured to set a sensing array and a data acquisition period in a monitoring scene, and then acquire historical multi-modal data and real-time multi-modal data of the monitoring scene in each data acquisition period through the sensing array;

[0047] The monitoring model establishment module is configured to establish a plurality of data-level local background image models according to the historical multi-modal data, and then obtain a local background multi-dimensional image model by mutually overlapping the local background image models of each data level, and further obtain a multi-modal scene monitoring model of the monitoring scene by overlapping the local background multi-dimensional image models generated by different sensing arrays;

[0048] The behavior event analysis module is configured to establish a multi-modal event dynamic three-dimensional model and a corresponding event feature trajectory according to a plurality of historical multi-modal behavior data of a plurality of preset abnormal behavior events, and then generate an event feature trajectory dataset of a corresponding abnormal behavior event according to the multi-modal event dynamic three-dimensional model and the corresponding event feature trajectory;

[0049] The scene monitoring module is configured to update the multi-modal scene monitoring model according to real-time multi-modal scene data, and simultaneously generate a real-time multi-modal event dynamic three-dimensional model and input the real-time multi-modal event dynamic three-dimensional model into the updated multi-modal scene monitoring model, and then generate a corresponding real-time scene-event multi-modal interaction trajectory, match the real-time scene-event multi-modal interaction trajectory with each event feature trajectory dataset, and determine an abnormal behavior event currently occurring in the monitoring scene according to a matching result.

[0050] Further, the working principle of the present application is illustrated by the following embodiments:

[0051] The scene data acquisition module sets a plurality of groups of sensing arrays in the monitoring scene, and the sensing array is composed of a plurality of sensors, wherein the types of sensors include temperature sensors, cameras, laser sensors, etc.

[0052] Then, the scene data acquisition module sets a number s 1，1 , s 1，2 , s 1，3 , …, s m，n for the sensors in each group of sensing arrays, wherein s m，n represents the nth sensor in the mth group of sensing arrays, and m and n are natural numbers greater than 0.

[0053] The same data acquisition period is set for each sensor, and the data acquisition ranges of the same type of sensors in the sensor array at adjacent positions have a partial overlap;

[0054] When the data acquisition period starts, the sensors in each sensor array acquire data in their data acquisition ranges, and send the acquired data to the scene data acquisition module at the end of the data acquisition period;

[0055] The scene data acquisition module sets a number for the corresponding uploaded data according to the number of the sensors, and integrates the uploaded data whose first digit of the number index is the same, thereby obtaining historical multi-modal scene data and real-time multi-modal scene data corresponding to the sensor array;

[0056] It should be noted that the multi-modal data includes scene optical video data, scene infrared video time, scene laser signal spectrum, etc.

[0057] Further, the scene data acquisition module sends the historical multi-modal scene data acquired by each sensor array to the monitoring model establishment module;

[0058] The monitoring model establishment module first extracts historical scene optical video data from the historical multi-modal scene data, divides the historical scene optical video data into a plurality of historical scene optical image data according to frames, and then segments a plurality of scene feature targets from each historical scene optical image data. The historical scene optical image data corresponding to the same scene feature target in the same historical scene optical video data is sequentially superimposed, and a corresponding scene target dynamic model is established according to the superimposed result;

[0059] It should be noted that the scene feature target includes objects such as roads, trees, buildings, etc. in the scene, for example, the monitoring scene is a city road, and the scene feature target includes objects such as traffic arteries, vehicles, pedestrians, traffic lights, plants, etc.

[0060] The plurality of scene target dynamic models are spliced to obtain a corresponding local scene image dynamic model, a three-dimensional coordinate system is established, and the local scene image dynamic model is mapped on the three-dimensional coordinate system, and a plurality of time nodes are set according to the number of historical scene optical image data corresponding to the historical scene optical video data;

[0061] A plurality of feature space coordinates are randomly set on each local scene target dynamic model, and the displacement trajectory of each local scene target dynamic model in the local scene image dynamic model when the feature space coordinates change with the time nodes is obtained;

[0062] The displacement trajectories of the same scene feature target corresponding to the scene optical video data generated by the same sensor array in different data collection periods are compared with each other, if the displacement trajectories of the scene feature target in each data collection period exist and change periodically, and the similarity of the displacement trajectories in each period is above 95%, the corresponding scene feature target is set as a scene background target label, otherwise no operation is performed;

[0063] The scene feature targets without the scene background target label in each local scene image dynamic model are removed, and the remaining scene feature targets are retained, thereby obtaining a local background optical image model corresponding to the historical optical video data.

[0064] Further, similar to the process of establishing a local background optical image model, a corresponding local background image model is established according to each data from the same multi-modal data except the historical optical video data, for example, a local background infrared image model, a local background three-dimensional image model, etc.

[0065] Since each historical data from the same multi-modal data is generated by each sensor in the same sensor array, and the collection range and collection time of each sensor are consistent, the local background models corresponding to each historical data exist correlation;

[0066] Therefore, each local background model corresponding to each multi-modal data is overlapped and mapped, thereby obtaining a corresponding local background multi-dimensional image model.

[0067] At the same time, since there is an intersection between the data collection ranges of each sensor array, the local background multi-dimensional image models generated by different sensor arrays are spliced and overlapped according to the actual spatial distribution of each sensor array in the monitoring scene, thereby obtaining a multi-modal scene monitoring model of the monitoring scene, and the multi-modal scene monitoring is sent to the behavior event analysis module and the scene monitoring module.

[0068] Further, the behavior event analysis module is preset with a plurality of historical multi-modal behavior data of a plurality of abnormal behavior events, for example, vehicle collision, oil and gas pipe explosion, pedestrian fight, etc.

[0069] Similar to the process of extracting each scene feature target from the historical multi-modal scene data, a plurality of event feature targets are extracted from each historical multi-modal behavior data, and each event feature target is matched with the scene feature target in the multi-modal scene monitoring model, if there is a matching result between the scene feature target and the event feature target, the corresponding event feature target is removed, otherwise no operation is performed.

[0070] It should be noted that the event feature target can be a car, a pedestrian, an animal, etc.

[0071] a plurality of action recognition points are set on the reserved event feature target, and a corresponding multi-modal event dynamic three-dimensional model is established according to each historical multi-modal event data and the reserved event feature target;

[0072] A three-dimensional coordinate system is established, the multi-modal event dynamic three-dimensional model and the multi-modal scene monitoring model are overlapped and mapped on the three-dimensional coordinate system, and then when the multi-modal event dynamic three-dimensional model is displaced in the multi-modal scene monitoring model, a corresponding event feature trajectory is generated according to the trajectory generated by the action recognition point on the multi-modal event dynamic three-dimensional model;

[0073] Meanwhile, the scene feature target in the interaction between the multi-modal event dynamic three-dimensional model and the multi-modal scene monitoring model in the displacement process is labeled, and then an event feature trajectory dataset of a corresponding abnormal behavior event is generated;

[0074] The generation process of the event feature trajectory dataset of the scene abnormal behavior event includes:

[0075] A plurality of model regions of the same size are divided in each scene feature target, and the model region is composed of a plurality of layers of data;

[0076] When the multi-modal event dynamic three-dimensional model interacts with any scene feature target over time, the corresponding scene feature target is displaced or locally deformed, which causes the displacement of the model region in the scene feature target and the change of each layer of data in the model region at different levels, and then the displacement trajectory of each model region is counted and integrated as a corresponding scene-event multi-modal interaction trajectory. For example, the abnormal behavior event is vehicle collision. Before the collision, the temperature distribution of the corresponding vehicle is fixed and the temperature value is stable. When the collision occurs, it may cause the temperature of the local area of the vehicle or the plant to rise sharply, which causes the corresponding temperature data layer in the corresponding model region to change dramatically;

[0077] The scene-event multi-modal interaction trajectories corresponding to the same pair of event feature target and scene feature target in each historical multi-modal behavior data of the same abnormal behavior event are overlapped with each other, and then the scene-event multi-modal interaction trajectory range corresponding to each event feature target and scene feature target in each abnormal behavior event is obtained;

[0078] All scene-event interaction trajectory ranges are integrated to obtain an event feature trajectory dataset of a corresponding abnormal behavior event.

[0079] Further, the behavior event analysis module sends the event feature trajectory dataset of all abnormal behavior events to the scene monitoring module;

[0080] The scene monitoring module adopts a process similar to constructing a multi-modal scene monitoring model, a multi-modal event dynamic three-dimensional model and a scene-event multi-modal interaction track, constructs a real-time multi-modal scene monitoring model and a real-time multi-modal event dynamic three-dimensional model according to real-time multi-modal scene data;

[0081] According to the real-time multi-modal scene monitoring model, the multi-modal scene monitoring model is updated, and the real-time multi-modal event dynamic three-dimensional model is input into the updated multi-modal scene monitoring model, and then a corresponding real-time scene-event multi-modal interaction track is generated;

[0082] The real-time scene-event multi-modal interaction track is matched with each event feature track data set. If each level of data of the real-time scene-event multi-modal interaction track is in the scene-event multi-modal interaction track range in an event feature track data set, it is judged that the monitoring occurs a corresponding abnormal behavior event, and then a corresponding behavior warning prompt is generated and sent to the monitoring personnel, otherwise no operation is performed.

[0083] The above embodiments are only used to illustrate the technical method of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical method of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical method of the present application.

Claims

1. A real-time monitoring system based on a sensor array comprising a cloud computing platform, characterized in that, The cloud computing platform is communicatively connected with a scene data acquisition module, a monitoring model establishment module, a behavior event analysis module and a scene monitoring module; The scene data acquisition module is configured to set a sensing array and a data acquisition period in a monitoring scene, and then acquire historical multi-modal data and real-time multi-modal data of the monitoring scene in each data acquisition period through the sensing array; The monitoring model establishment module is configured to establish a plurality of data-level local background image models according to the historical multi-modal data, and then obtain a local background multi-dimensional image model by mutually overlapping the local background image models of each data level, and further obtain a multi-modal scene monitoring model of the monitoring scene by overlapping the local background multi-dimensional image models generated by different sensing arrays; The behavior event analysis module is configured to establish a multi-modal event dynamic three-dimensional model and a corresponding event feature track according to a plurality of historical multi-modal behavior data of a plurality of preset abnormal behavior events, and then generate an event feature track data set of a corresponding abnormal behavior event according to the multi-modal event dynamic three-dimensional model and the corresponding event feature track, wherein the event feature track data set is composed of a plurality of scene-event multi-modal interaction track ranges; The scene monitoring module is configured to update the multi-modal scene monitoring model according to real-time multi-modal scene data, and simultaneously generate a real-time multi-modal event dynamic three-dimensional model and input it into the updated multi-modal scene monitoring model, and then generate a corresponding real-time scene-event multi-modal interaction track, match the real-time scene-event multi-modal interaction track with each event feature track data set, and determine an abnormal behavior event currently occurring in the monitoring scene according to a matching result; The generation process of the scene-event multi-modal interaction track range includes: dividing a plurality of model regions of the same size within each scene feature target, wherein the model region is composed of a plurality of layers of data; when the multi-modal event dynamic three-dimensional model interacts with any scene feature target as time changes, the corresponding scene feature target is displaced or locally deformed, and then the displacement tracks of each model region are counted and integrated to obtain a corresponding scene-event multi-modal interaction track; overlapping the scene-event multi-modal interaction tracks corresponding to the same pair of event feature targets and scene feature targets in each historical multi-modal behavior data of the same abnormal behavior event, and then obtaining the scene-event multi-modal interaction track range corresponding to each event feature target and scene feature target in each abnormal behavior event.

2. A real-time monitoring system based on sensor array according to claim 1, characterized in that, The acquisition process of the multi-modal data includes: The scene data acquisition module sets a plurality of sensing arrays in the monitoring scene, wherein each sensing array is composed of a plurality of sensors, and the sensors in each sensing array are numbered; each sensor is set with the same data acquisition period, and the data acquisition ranges of the same type of sensors in adjacent sensing arrays partially overlap; when the data acquisition period starts, the sensors in each sensing array acquire data in their data acquisition ranges, and send the acquired data to the scene data acquisition module when the data acquisition period ends; The scene data acquisition module sets a number on corresponding uploaded data according to the number of the sensor and integrates, and then obtains historical multi-modal scene data and real-time multi-modal scene data corresponding to the sensing array.

3. A real-time monitoring system based on sensor array according to claim 2, characterized in that, The process of establishing the local background image model of each data level includes: First, generate a plurality of scene feature targets from the historical multi-modal scene data, and then establish a corresponding scene target dynamic model according to each type of data in the historical multi-modal scene data; Splice each scene target dynamic model to obtain a corresponding local scene image dynamic model, establish a three-dimensional coordinate system, and map the local scene image dynamic model on the three-dimensional coordinate system; Randomly set a plurality of feature space coordinates on each local scene target dynamic model, and then obtain the displacement trajectory of each local scene target dynamic model in the local scene image dynamic model when the feature space coordinates change with the time node; Compare the displacement trajectories of the same scene feature target generated by the same sensing array at different data acquisition cycles, and set scene background target labels for the corresponding scene feature targets according to the comparison results; Remove the scene feature targets in each local scene image dynamic model that do not have scene background target labels, and retain the remaining scene feature targets, and then obtain the corresponding local background image model of each data level.

4. A real-time monitoring system based on sensor array according to claim 3, characterized in that, The process of establishing the multi-modal scene monitoring model includes: Map each local background model generated by each multi-modal data, and then obtain a corresponding local background multi-dimensional image model, and splice and overlap the local background multi-dimensional image models generated by different sensing arrays according to the actual spatial distribution of each sensing array in the monitoring scene, and then obtain a multi-modal scene monitoring model of the monitoring scene.

5. A real-time monitoring system based on sensor array according to claim 4, characterized in that, The process of establishing the multi-modal event dynamic three-dimensional model includes: Extract a plurality of event feature targets from each historical multi-modal behavior data by using a process similar to that of extracting each scene feature target from the historical multi-modal scene data, match each event feature target with the scene feature target in the multi-modal scene monitoring model, and remove the corresponding event feature target according to the matching result; Set a plurality of action recognition points on the retained event feature targets, and establish a corresponding multi-modal event dynamic three-dimensional model according to each historical multi-modal event data and the retained event feature targets.

6. A real-time monitoring system based on sensor array according to claim 5, characterized in that, The process of establishing the event feature trajectory data set includes: Establish a three-dimensional coordinate system, and map the multi-modal event dynamic three-dimensional model and the multi-modal scene monitoring model on the three-dimensional coordinate system, and then generate a corresponding event feature trajectory according to the trajectory generated by the action recognition points on the multi-modal event dynamic three-dimensional model when the multi-modal event dynamic three-dimensional model displaces in the multi-modal scene monitoring model; Label the scene feature targets in the multi-modal event dynamic three-dimensional model that interact with the multi-modal scene monitoring model during displacement, and then generate an event feature trajectory data set corresponding to the abnormal behavior event.

7. A real-time monitoring system based on sensor array according to claim 6, characterized in that, The process of real-time monitoring of the monitoring scene according to the event feature trajectory data set includes: The scene monitoring module adopts a process similar to constructing a multi-modal scene monitoring model, a multi-modal event dynamic three-dimensional model, and a scene-event multi-modal interaction track, constructs a real-time multi-modal scene monitoring model and a real-time multi-modal event dynamic three-dimensional model according to real-time multi-modal scene data; According to the real-time multi-modal scene monitoring model, the multi-modal scene monitoring model is updated, and the real-time multi-modal event dynamic three-dimensional model is input into the updated multi-modal scene monitoring model, and then the corresponding real-time scene-event multi-modal interaction track is generated; The real-time scene-event multi-modal interaction track is matched with each event feature track data set. If the data of each level of the real-time scene-event multi-modal interaction track is in the scene-event multi-modal interaction track range in an event feature track data set, it is judged that the monitoring occurs a corresponding abnormal behavior event, and then a corresponding behavior warning prompt is generated and sent to the monitoring personnel, otherwise no operation is performed.

Images