Dynamic monitoring alarm condition identification system based on three-dimensional panoramic space fusion

Abstract

The invention provides a dynamic monitoring alarm condition recognition system based on three-dimensional panoramic space fusion, and relates to the technical field of data fusion, and the system comprises a collection module which is used for collecting initial multi-source data corresponding to each target sub-region in a plurality of target sub-regions contained in a target region; the preprocessing module is used for carrying out distributed data preprocessing to obtain to-be-fused image data and to-be-fused three-dimensional point cloud data; the fusion module is used for performing three-dimensional panoramic space fusion to obtain a three-dimensional panoramic space model corresponding to the target area; the feature acquisition module is used for acquiring behavior feature information of the moving target contained in the target area in the target time period; and the identification module is used for obtaining an alarm condition identification result of the target area. According to the invention, the real-time performance of the whole system can be improved, monitoring and identification of alarm conditions can be better realized, and personal injury and property loss can be avoided.

Description

Technical Field

[0001] The present application relates to the technical field of data fusion, and in particular to a dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion. Background Art

[0002] With the acceleration of urbanization and the increasing frequency of social activities, public security is facing more and more challenges. Traditional monitoring systems have played an important role in ensuring security, but they have also gradually exposed many limitations.

[0003] In terms of data collection, traditional monitoring mainly relies on two-dimensional video images, which provide information limited to the plane perspective and lack the description of the depth information and three-dimensional spatial structure of the monitoring scene. This makes it difficult to accurately judge the location, size and relationship of the target in complex scenes, such as large public places and city streets, and easily causes problems such as misjudgment of target occlusion and inaccurate spatial positioning. For example, in crowded places, it is difficult to determine the specific distribution and behavior status of people on different floors or in different depth areas only through two-dimensional images.

[0004] For data processing, the image processing algorithms of traditional monitoring systems are relatively simple, usually only performing basic image enhancement and target detection, and lack effective integration of multi-source data. When faced with complex environmental interference (such as lighting changes, shadows, bad weather, etc.), image quality deteriorates, which in turn greatly reduces the accuracy of target detection and recognition. At the same time, due to the lack of in-depth mining of semantic information, it is impossible to intelligently understand and classify objects and behaviors in the scene, making it difficult to accurately judge the police situation. For example, it is impossible to automatically distinguish between normal crowd gatherings and suspicious crowd gatherings, and it is impossible to accurately identify the types of illegal driving of vehicles.

[0005] In summary, dynamic monitoring systems based on two-dimensional and three-dimensional data have emerged. However, due to the huge amount of data and numerous data processing tasks, the preprocessing stage after data collection consumes a lot of time and resources, resulting in poor real-time performance of dynamic monitoring. Therefore, there is an urgent need for an efficient and fast data preprocessing system for large volumes and multiple data dimensions in the process of dynamic alarm monitoring. Summary of the invention

[0006] In response to the above technical problems, the present application provides a dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion, which at least partially solves the problems existing in the prior art.

[0007] In a first aspect of the present application, a dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion is provided, the system comprising:

[0008] An acquisition module, used for acquiring initial multi-source data corresponding to each target sub-region in a plurality of target sub-regions contained in the target region; wherein the initial multi-source data includes initial image data and initial three-dimensional point cloud data;

[0009] A preprocessing module is used to perform distributed data preprocessing on the initial image data and the initial three-dimensional point cloud data according to the distributed computing framework to obtain the image data to be fused and the three-dimensional point cloud data to be fused; wherein the distributed data preprocessing on the initial image data includes several image processing subtasks; the distributed data preprocessing on the initial three-dimensional point cloud data includes several data processing subtasks; the above-mentioned distributed computing framework includes a task allocation node and several computing nodes; the above-mentioned task allocation node is used to allocate the image processing subtasks and data processing subtasks contained in the target area to several computing nodes according to the computing resource demand priority of each image processing subtask or each data processing subtask and the current load rate corresponding to each computing node; the computing resource demand priority is determined according to the CPU demand degree and the memory demand degree;

[0010] A fusion module is used to perform 3D panoramic spatial fusion on the above-mentioned image data to be fused and the 3D point cloud data to be fused to obtain a 3D panoramic spatial model corresponding to the target area; wherein the above-mentioned 3D panoramic spatial fusion includes geometric fusion and semantic fusion;

[0011] A feature acquisition module, used to acquire the behavior feature information of the moving targets contained in the target area within the target time period according to the above three-dimensional panoramic space model;

[0012] The recognition module is used to obtain the warning situation recognition result of the target area based on the above-mentioned behavior characteristic information and preset warning situation recognition rules.

[0013] This application has at least the following beneficial effects:

[0014] The present application provides a dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion. For the data (initial image data and initial three-dimensional point cloud data) collected for three-dimensional panoramic space fusion with large volume and multi-source characteristics, in order to save processing time and improve data preprocessing efficiency, a distributed computing framework is used for processing during data preprocessing. The distributed computing framework has a task allocation node and several computing nodes, wherein the task allocation node is used to allocate all data to several computing nodes for parallel processing. When allocating data, on the one hand, the different demand rates for computing resources of the computing device due to different factors such as computational complexity are considered for each processing task itself, and on the other hand, the current load of each computing node is considered. The above two situations are comprehensively considered for data distribution. While enabling the corresponding processing subtasks to be processed in parallel, tasks are allocated according to the characteristics of the processing subtasks and the load of each computing node, and computing resources are reasonably used, saving the overall data processing time, and fully and reasonably utilizing computing resources.

[0015] Finally, based on the pre-processed data, three-dimensional panoramic space fusion is performed to dynamically monitor the police situation. Due to the fast data processing speed, the real-time performance of the entire system can be improved, and the monitoring and identification of police situations can be better realized to avoid personal injury and property loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A structural block diagram of a dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present application, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0021] Please refer to Figure 1 As shown, the embodiment of the present application provides a dynamic monitoring alarm recognition system 100 based on three-dimensional panoramic space fusion, and the system includes the following modules:

[0022] The acquisition module 110 is used to acquire initial multi-source data corresponding to each target sub-region in a plurality of target sub-regions included in the target region; wherein the initial multi-source data includes initial image data and initial three-dimensional point cloud data.

[0023] Specifically, the target area can be a park, a school, a factory, etc., and the method of dividing several target sub-areas can be any area division method known to technical personnel in this field, so that the total amount of data in each target sub-area is basically similar.

[0024] It should be noted that the initial image data is obtained according to the following method: using a high-definition camera network to collect all-round and multi-angle video images of the monitoring area to ensure coverage of the entire target scene. The initial three-dimensional point cloud data is obtained according to the following method: using a three-dimensional laser scanner or a depth camera and other equipment to collect three-dimensional point cloud data of the monitoring area to obtain the geometric shape information of the target area.

[0025] It is understandable that for key areas or specific objects, sensors (such as motion sensors, temperature sensors, etc.) are deployed to collect additional physical data, which will be used as auxiliary information for alarm judgment.

[0026] The preprocessing module 120 is used to perform distributed data preprocessing on the initial image data and the initial three-dimensional point cloud data according to the distributed computing framework to obtain the image data to be fused and the three-dimensional point cloud data to be fused; wherein, the distributed data preprocessing on the initial image data includes several image processing subtasks; the distributed data preprocessing on the initial three-dimensional point cloud data includes several data processing subtasks; the above-mentioned distributed computing framework includes a task allocation node and several computing nodes; the above-mentioned task allocation node is used to allocate the image processing subtasks and data processing subtasks contained in the target area to several computing nodes according to the computing resource requirement priority of each image processing subtask or each data processing subtask and the current load rate corresponding to each computing node; the computing resource requirement priority is determined according to the CPU requirement and the memory requirement.

[0027] Specifically, as an example: image processing subtasks can be image filtering, image enhancement, key frame extraction, etc.; data processing subtasks can be point cloud filtering, point cloud registration, point cloud simplification, etc. The above-mentioned distributed computing framework includes a task allocation node and several computing nodes, wherein the task allocation node is used to send the multi-source data corresponding to each target sub-area to several computing nodes for data preprocessing. As an example: Apache Spark is used as a distributed computing framework to build a cluster consisting of multiple computing nodes. The cluster includes a master node (Master Node) for task scheduling and resource management, and multiple slave nodes (Worker Node) for actual data processing task execution. The nodes are connected by a high-speed network to ensure the efficiency of data transmission.

[0028] The corresponding data is sent to the computing node according to the computing resource demand priority of each image processing subtask or each data processing subtask and the current load rate corresponding to each computing node. That is, in this application, on the one hand, it is considered that each processing task itself has different demand rates for computing resources of the computing device due to different factors such as computational complexity. As an example: Image filtering: Complex image bilateral filtering algorithms require complex weighted calculations for each pixel in the image and its surrounding pixels, involving a large number of mathematical operations. In this case, the computing power of the CPU is the key factor limiting the speed of task execution. If the CPU resources are insufficient, the task execution will be very slow. Simple image enhancement tasks (such as simple contrast and brightness adjustment) mainly perform basic arithmetic operations on image pixels, and the computational complexity is relatively low. However, these tasks still require a certain amount of memory space to store intermediate results and original image data when processing image data. Therefore, for this type of task, the amount of free memory is considered.

[0029] On the other hand, the current load of each computing node is taken into consideration. The above two situations are considered comprehensively for data distribution. While the corresponding processing subtasks can be processed in parallel, tasks are allocated according to the characteristics of the processing subtasks and the load of each computing node, which makes reasonable use of computing resources, saves overall data processing time, and fully and reasonably utilizes computing resources.

[0030] The fusion module 130 is used to perform three-dimensional panoramic spatial fusion on the above-mentioned image data to be fused and the three-dimensional point cloud data to be fused to obtain a three-dimensional panoramic spatial model corresponding to the target area; wherein the above-mentioned three-dimensional panoramic spatial fusion includes geometric fusion and semantic fusion.

[0031] Specifically, geometric fusion is first performed: the processed 3D point cloud data is fused with the Geographic Information System (GIS) data of the monitored area, the point cloud model is accurately mapped to the real geographic coordinate space, and its exact position and posture on the map are determined.

[0032] Based on the three-dimensional point cloud model, the two-dimensional coordinate information of the object in the key frame image is mapped to the three-dimensional space through projection transformation, the geometric association between the image and the three-dimensional point cloud is realized, and a three-dimensional panoramic space model with real geometric shape and texture information is constructed.

[0033] Secondly, semantic fusion is performed. First, deep learning algorithms (such as convolutional neural networks) are used to perform semantic segmentation on key frame images, identify different object categories in the image (such as people, vehicles, buildings, etc.), and assign semantic labels to each object. The semantic segmentation results are fused with the object geometry information in the 3D point cloud data, and semantic attributes are added to each object or area in the 3D panoramic space, so that the entire scene has rich semantic information, which is convenient for subsequent police situation analysis and identification.

[0034] The feature acquisition module 140 is used to acquire the behavior feature information of the moving targets contained in the target area within the target time period according to the above three-dimensional panoramic space model.

[0035] Specifically, first, real-time data update is performed: new video images and three-dimensional point cloud data are continuously collected, and according to the above data processing and spatial fusion steps, the new data is integrated into the constructed three-dimensional panoramic space model in real time to achieve real-time update of dynamic monitoring scenes. Secondly, moving target detection and tracking are performed, that is, in the three-dimensional panoramic space, the optical flow method, background difference method or deep learning-based target detection algorithm (such as YOLO, FasterR-CNN, etc.) is used to detect moving targets (such as pedestrians, vehicles, etc.) for video image sequences. Then, the detected moving targets are tracked, and the position and motion state of the target in the continuous frames are determined through the feature matching of the target (such as color features, shape features, motion trajectory features, etc.) and tracking algorithms (such as Kalman filtering, particle filtering, etc.), and the motion trajectory model of the target is established. Finally, behavioral feature extraction is performed, that is, for moving targets, their behavioral features are extracted. For example, the walking speed, walking direction, posture changes and other features of pedestrians are analyzed; for vehicles, the speed, curvature of the driving trajectory, vehicle type and other features are extracted.

[0036] At the same time, combined with the data collected by sensors (such as the target movement speed detected by the motion sensor, the regional temperature change detected by the temperature sensor, etc.), the target's behavioral characteristic information is further enriched.

[0037] The identification module 150 is used to obtain the warning situation identification result of the target area according to the above-mentioned behavior characteristic information and the preset warning situation identification rules.

[0038] Specifically, the preset alarm recognition rules can be that in a specific area, if unidentified persons are detected wandering or gathering for a long time and the number of people exceeds a set threshold, it is judged as a suspicious gathering alarm; if a vehicle is driving in a prohibited area or the speed is abnormally fast and the trajectory is unstable, it is judged as an illegal driving alarm; if the temperature in a specific area is detected to be abnormally high and accompanied by characteristics such as smoke, it is judged as a fire alarm, etc.

[0039] The extracted moving target behavior characteristics and other relevant data are matched and judged in real time with the set alarm recognition rules. Through logical reasoning, data analysis and model calculation, it is determined whether an alarm occurs and the type and severity of the alarm.

[0040] Once an alarm is identified, the early warning mechanism is triggered immediately. According to the type and severity of the alarm, the alarm information is sent to the monitoring personnel through various means (such as sound and light alarm, SMS notification, system pop-up window, etc.), and the location and related information of the alarm are highlighted in the 3D panoramic space model, so that the monitoring personnel can quickly locate and handle the alarm.

[0041] In summary, the present application provides a dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion. For the data (initial image data and initial three-dimensional point cloud data) collected for three-dimensional panoramic space fusion with large volume and multi-source characteristics, in order to save processing time and improve data preprocessing efficiency, a distributed computing framework is used for processing during data preprocessing. The distributed computing framework has a task allocation node and several computing nodes, wherein the task allocation node is used to allocate all data to several computing nodes for parallel processing. When allocating data, on the one hand, the different demand rates for computing resources of the computing device due to different factors such as computational complexity are considered for each processing task itself, and on the other hand, the current load of each computing node is considered. The above two situations are comprehensively considered for data distribution. While enabling the corresponding processing subtasks to be processed in parallel, tasks are allocated according to the characteristics of the processing subtasks and the load of each computing node, and computing resources are used reasonably, saving the overall data processing time, and fully and reasonably utilizing computing resources.

[0042] Finally, based on the pre-processed data, three-dimensional panoramic space fusion is performed to dynamically monitor the police situation. Due to the fast data processing speed, the real-time performance of the entire system can be improved, and the monitoring and identification of police situations can be better realized to avoid personal injury and property loss.

[0043] In an exemplary embodiment of the present application, the preprocessing module includes:

[0044] The subtask acquisition unit is used to obtain a subtask data list set Z = (Z1, Z2, ..., Z i , …, Z n ), i = 1, 2, ..., n, where n is the number of processing subtasks, n = n t +n s ; Among them; n t is the number of image processing subtasks; n s is the number of data processing subtasks; Z iis the subtask data list set corresponding to the i-th processing subtask; Z i =(Z i,1 , Z i,2 , …, Z i，j , …, Z i，m );j=1,2,…,m;m is the number of target sub-regions;Z i，j It is the image data list or 3D point cloud data list corresponding to the i-th processing subtask in the j-th target sub-region.

[0045] Specifically, the processing tasks performed during preprocessing of the initial image data are divided into several image processing subtasks, and the processing tasks performed during preprocessing of the initial three-dimensional point cloud data are divided into several data processing subtasks to obtain a subtask data list set. Here, a list of data to be processed corresponding to each processing subtask in each target sub-area is obtained. As an example: For the processing subtask: point cloud registration task, when processing the initial image data and three-dimensional point cloud data collected from the j-th target sub-area, each processing subtask must be executed once, but when each processing subtask is executed, the corresponding basic data (initial image data and three-dimensional point cloud data) may be different. Therefore, in this application, for a large target area, it is divided into several target sub-areas and then each processing subtask is executed on the data corresponding to each target sub-area, which preliminarily reduces the data processing volume.

[0046] The load rate acquisition unit is used to acquire the current load rate of each computing node to obtain a load rate list F = (F1, F2, ..., F x , …, F y ), where y is the number of computing nodes; F x is the current load rate corresponding to the x-th computing node.

[0047] Specifically, F x Meet the following conditions:

[0048] F x =α x ×Fcpu x +β x ×Fmem x

[0049] Among them, Fcpu x is the current CPU load rate of the xth computing node; Fmem x is the current memory load rate of the xth computing node; α x is the CPU weight of the x-th computing node; β x is the memory weight of the xth computing node; α x +β x =1;Fcpu xMeet the following conditions:

[0050] Fcpu x =Ucpu x / Ccpu x

[0051] Among them, Ucpu x is the current CPU usage of the xth computing node; Ccpu x is the number of cores of the x-th computing node.

[0052] Fmem x Meet the following conditions:

[0053] Fmem x =MU x / MT x

[0054] Among them, MU x is the currently used memory of the xth computing node; MT x is the total memory of the x-th computing node.

[0055] In this embodiment, the current load rate of each computing node is defined from the two aspects of CPU load rate and memory load rate because the computing node is mainly composed of two core resources, CPU and memory, which play a key role in the task execution process. The CPU (central processing unit) is the computing core and control core of the computer, responsible for executing program instructions, data processing and calculation. The memory (main memory) is used to store running programs and data, and is the storage area where the CPU can quickly access data. In a distributed computing environment, the use of these two resources directly affects the execution efficiency of the task and the overall performance of the node.

[0056] And α x is the CPU weight of the xth computing node, β x is the memory weight of the x-th computing node, where α x It can be determined based on the hardware configuration of the xth computing node. If the CPU performance of the computing node is good and the memory performance is relatively poor, then the corresponding CPU weight is greater. On the contrary, if the memory performance of another computing node is better and the CPU performance is poor, then the corresponding memory weight is greater. Among them, the factors affecting CPU performance can be the number of cores, the number of threads, the main frequency, the cache size, and the architecture; and the factors affecting memory can be capacity, frequency, type, timing, etc.

[0057] In addition, it should be noted that, in this embodiment, as an example, the current CPU usage is determined as follows: a single-core CPU executes tasks for 6 seconds in 10 seconds, so its CPU usage is 60%. The current CPU load rate is related to the CPU usage rate and the number of CPU cores, that is, when the CPU usage rates of two computing nodes are the same, the current CPU load rate of the computing node with a larger number of corresponding cores is smaller, because the more cores it has, the more remaining computing resources.

[0058] In summary, the current CPU load rate determined by this application can make the obtained current CPU load rate more objective and accurate by comprehensively evaluating the CPU and memory loads.

[0059] A priority acquisition unit is used to acquire a computing resource requirement priority list Y=(Y1, Y2, ..., Y i , …, Y n );Y i is the computing resource demand priority corresponding to the i-th processing subtask; the above computing resource demand priority includes a first priority, a second priority and a third priority; wherein, the CPU demand CPUX1 corresponding to the above first priority meets the following conditions: CPUX1≥CPUY; CPUY is the first CPU demand threshold; the CPU demand CPUX2 corresponding to the above second priority meets the following conditions: CPUE<CPUX2<CPUY; CPUE is the second CPU demand threshold; and the memory demand MX2 corresponding to the second priority meets the following conditions: MX2≥MY; MY is the memory demand threshold; the CPU demand CPUX3 corresponding to the above third priority meets the following conditions: CPUX3≤CPUE; and the memory demand MX3 corresponding to the third priority meets the following conditions: MX3≥MY.

[0060] Specifically, in this embodiment, since each processing subtask has different task characteristics, its requirements for the CPU and memory of the computing node are different. Therefore, this embodiment defines the overall requirements of each processing subtask for the computing node with different priorities, wherein the first priority is computationally intensive. Taking the complex image bilateral filtering algorithm as an example, it requires complex weighted calculations for each pixel in the image and its surrounding pixels, involving a large number of mathematical operations. In this case, the computing power of the CPU is the key factor limiting the speed of task execution. If the CPU resources are insufficient, the task execution will be very slow. Therefore, for the processing subtasks of the first priority, giving priority to the CPU occupancy rate can ensure that these tasks are assigned to nodes that can complete the calculation quickly. The second priority is to have certain requirements for both computing resources and memory resources. It is necessary to consider the synergy of CPU and memory resources during the task execution process. The third priority, such as simple image enhancement tasks (such as simple contrast and brightness adjustment), mainly performs basic arithmetic operations on image pixels, and the computational complexity is relatively low. However, these tasks still require a certain amount of memory space to store intermediate results and original image data when processing image data. Therefore, for low-priority tasks, the amount of free memory is considered.

[0061] The allocation unit is used to control the task allocation node to allocate the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-area of ​​the n processing subtasks to y computing nodes for distributed data preprocessing according to F and Y, so as to obtain the image data to be fused and the three-dimensional point cloud data to be fused.

[0062] Specifically, in one embodiment, for the first priority, the allocation unit includes:

[0063] The first allocation subunit is used if Y i is the first priority, then the first target computing node list YJ is obtained according to F i =(YJ i，1 , Y.J. i，2 , …, YJ i，a , …, YJ i，b ); a = 1, 2, ..., b; where b is the number of the first target computing nodes; b < y; YJ i，a is the node identifier of the ath first target computing node corresponding to the i-th processing subtask; F i，a <YF; YF is the first load rate threshold; Ccpu i，a >YCcpu i ;YCcpu i Y i Corresponding key core number; F i，a The current load rate of the ath first target computing node corresponding to the i-th processing subtask; Ccpui，a The number of cores of the a-th first target computing node corresponding to the i-th processing subtask.

[0064] The first weight allocation subunit is used to allocate weights according to YJ i , get the first node allocation weight list YP i =(YP i，1 , Y.P. i，2 , …, YP i，a , …, YP i，b ), among which, YP i，a Assign a weight to the processing subtask of the ath first target computing node corresponding to the i-th processing subtask; YP i，a Meet the following conditions: YP i，a =[(1-F i，a )×Ccpu i，a ] / {Σ b a＝1 [(1-F i，a )×Ccpu i，a ]}.

[0065] The first quantity allocation subunit is used to allocate the quantity according to YP i , get the first node allocation quantity list YS i =(YS i，1 , YS i，2 , …, YS i，a , …, YS i，b );YS i，a The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the a-th first target computing node; YS i，a Meet the following conditions: YS i，a =rounddown(n×YP i，a ); rounddown() is the default rounding function.

[0066] The first processing subunit is used to process the i , distribute the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to b first target computing nodes for distributed data preprocessing.

[0067] In this embodiment, when weighting the processing subtasks of the first priority, the ones with a smaller current load rate (lighter node load and larger remaining computing resources) are first selected. Since the processing subtasks of the first priority have higher requirements for the CPU, in this embodiment, the number of CPU cores obtained is greater than Y. iThe computing node with the corresponding critical core number is used as the first target computing node. Here, each processing subtask has a corresponding critical core number, which is the number of cores with better efficiency when processing the task. As an example: the first load rate threshold can be 0.5. Furthermore, since the computing power of the CPU in the processing subtask corresponding to the first priority is a key factor limiting the task execution speed, in this embodiment, only the load rate and CPU core number of the computing node are considered when assigning weights to each settlement node, so that when allocating data for the processing subtask of the first priority, the amount of computing tasks obtained by each computing node is the most appropriate. Improve the overall data processing efficiency.

[0068] In one embodiment, for the second priority level, the allocation unit includes:

[0069] The second allocation subunit is used if Y i is the second priority, then the second target computing node list EJ is obtained according to F i =(EJ i，1 , E.J. i，2 , …, E. J. i，c , …, E. J. i，d );c=1,2,…,d;wherein d is the number of the second target computing nodes;d<y;EJ i，c is the node identifier of the cth second target computing node corresponding to the i-th processing subtask; F i，c <EF; EF>YF; EF is the second load rate threshold; F i，c The current load rate of the node corresponding to the cth second target of the i-th processing subtask is calculated;

[0070] The second weight allocation subunit is used to allocate the weights according to EJ i , get the second node allocation weight list EP i =(EP i，1 , E.P. i，2 , …, EP i，c , …, EP i，d );Among them, EP i，c Assign a weight to the processing subtask of the cth second target computing node corresponding to the i-th processing subtask; EP i，c Meet the following conditions: EP i，c =[(1-F i，c )×(Ccpu i，c +MFR i，c )] / {Σ d c＝1 [(1-F i，c )×(Ccpu i，c +MFR i，c )]}; MFR i，cis the current free memory of the cth second target computing node corresponding to the i-th processing subtask; MFR i，c =MT i，c -MU i，c ;MT i，c is the current total memory of the cth second target computing node corresponding to the i-th processing subtask; MU i，c is the currently used memory of the cth second target computing node corresponding to the i-th processing subtask;

[0071] The second quantity allocation subunit is used to allocate the quantity according to the EP i , get the second node allocation quantity list ES i =(ES i，1 , E.S. i，2 , …, ES i，c , …, ES i，d ), where ES i，c The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the c-th second target computing node; ES i，c Meet the following conditions: ES i，c =rounddown(n×EP i，c );

[0072] The second processing subunit is used to i , distribute the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to d second target computing nodes for distributed data preprocessing.

[0073] In this embodiment, when weights are assigned to the processing subtasks of the second priority, the load rate of the selected computing nodes can be smaller than that of the first priority, but it cannot be too small. As an example, the first load rate threshold can be 0.7. And because the second priority has certain requirements for both CPU and memory, when weights are assigned to each computing node, not only the number of CPU cores is considered, but also the influence of the amount of free memory on weight assignment is considered, so that the obtained weight assignment result is more in line with the demand characteristics of the second priority processing subtasks for computing resources.

[0074] In one embodiment, for the third priority level, the allocation unit includes: a third allocation subunit, which is used to i is the third priority, then the third target computing node list HJ is obtained according to F i =(HJ i，1 , H.J. i，2 , …, HJ i，e , …, HJ i，f); e = 1, 2, ..., f; where f is the number of the third target computing nodes; f < y; HJ i，e is the node identifier of the e-th third target computing node corresponding to the i-th processing subtask; F i，e <HF; HF>EF; HF is the third load rate threshold; F i，e The current load rate of the e-th third target computing node corresponding to the i-th processing subtask;

[0075] The third weight allocation subunit is used to allocate the weight according to HJ i , get the third node allocation weight list HP i =(HP i，1 , HP i，2 ,…,HP i，e ,…,HP i，f ), where HP i，e Assign a weight to the processing subtask of the eth third target computing node corresponding to the i-th processing subtask; HP i，e Meet the following conditions: HP i，e =[(1-F i，e )×MFR i，e ] / {Σ f e＝1 [(1-F i，e )×MFR i，e ]};MFR i，e is the current free memory of the e-th third target computing node corresponding to the i-th processing subtask; MFR i，e =MT i，e -MU i，e ;MT i，e is the current total memory of the e-th third target computing node corresponding to the i-th processing subtask; MU i，e is the currently used memory of the e-th third target computing node corresponding to the i-th processing subtask;

[0076] The third quantity allocation subunit is used to allocate the amount of i , get the third node allocation quantity list HS i =(HS i，1 , HS i，2 ,…,HS i，e ,…,HS i，f ), where HS i，e The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the e-th third target computing node; HS i，e Meet the following conditions: HS i，e =rounddown(n×HP i，e );

[0077] The third processing subunit is used to process the i , distribute the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to f third target computing nodes for distributed data preprocessing.

[0078] In this embodiment, compared with the above two embodiments, the requirement for the number of CPU cores is lower, but the requirement for the amount of memory is certain, because when allocating data to each target sub-area for the third priority processing subtask, the influence of the amount of free memory of the computing node is considered. As an example, the third load rate threshold may be 0.9.

[0079] In summary, this application sets different data allocation methods for different types of processing subtasks, fully considering the requirements of each data subtask for computing resources and the load of each computing node itself, and handles different situations differently, making the final data allocation more reasonable and the computing efficiency of the distributed computing framework higher.

[0080] In addition, the above figures are only schematic illustrations of the processes included in the system according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0081] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0082] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion, characterized in that: The system comprises: An acquisition module, used for acquiring initial multi-source data corresponding to each target sub-region in a plurality of target sub-regions contained in the target region; wherein the initial multi-source data includes initial image data and initial three-dimensional point cloud data; A preprocessing module is used to perform distributed data preprocessing on the initial image data and the initial three-dimensional point cloud data according to the distributed computing framework to obtain the image data to be fused and the three-dimensional point cloud data to be fused; wherein the distributed data preprocessing on the initial image data includes several image processing subtasks; the distributed data preprocessing on the initial three-dimensional point cloud data includes several data processing subtasks; the distributed computing framework includes a task allocation node and several computing nodes; the task allocation node is used to allocate the image processing subtasks and data processing subtasks contained in the target area to several computing nodes according to the computing resource demand priority of each image processing subtask or each data processing subtask and the current load rate corresponding to each computing node; the computing resource demand priority is determined according to the CPU demand and memory demand; A fusion module, used for performing 3D panoramic spatial fusion on the image data to be fused and the 3D point cloud data to be fused, so as to obtain a 3D panoramic spatial model corresponding to the target area; wherein the 3D panoramic spatial fusion includes geometric fusion and semantic fusion; A feature acquisition module, used to acquire behavior feature information of moving targets contained in a target area within a target time period according to the three-dimensional panoramic space model; The recognition module is used to obtain the warning situation recognition result of the target area according to the behavior characteristic information and the preset warning situation recognition rules.

2. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 1 is characterized in that: The preprocessing module comprises: The subtask acquisition unit is used to obtain a subtask data list set Z = (Z1, Z2, ..., Z i , …, Z n ), i = 1, 2, ..., n, where n is the number of processing subtasks, n = n t +n s ; Among them; n t is the number of image processing subtasks; n s is the number of data processing subtasks; Z i is the subtask data list set corresponding to the i-th processing subtask; Z i =(Z i,1 , Z i,2 , …, Z i，j , …, Z i，m );j=1,2,…,m;m is the number of target sub-regions;Z i，j The image data list or 3D point cloud data list corresponding to the i-th processing subtask in the j-th target sub-area; The load rate acquisition unit is used to acquire the current load rate of each computing node to obtain a load rate list F = (F1, F2, ..., F x , …, F y ), where y is the number of computing nodes; F x is the current load rate corresponding to the xth computing node; A priority acquisition unit is used to acquire a computing resource requirement priority list Y=(Y1, Y2, ..., Y i , …, Y n );Y i is the computing resource demand priority corresponding to the i-th processing subtask; the computing resource demand priority includes a first priority, a second priority and a third priority; wherein the CPU demand CPUX1 corresponding to the first priority meets the following conditions: CPUX1≥CPUY; CPUY is the first CPU demand threshold; the CPU demand CPUX2 corresponding to the second priority meets the following conditions: CPUE<CPUX2<CPUY; CPUE is the second CPU demand threshold; and the memory demand MX2 corresponding to the second priority meets the following conditions: MX2≥MY; MY is the memory demand threshold; the CPU demand CPUX3 corresponding to the third priority meets the following conditions: CPUX3≤CPUE; and the memory demand MX3 corresponding to the third priority meets the following conditions: MX3≥MY; The allocation unit is used to control the task allocation node to allocate the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-area of ​​the n processing subtasks to y computing nodes for distributed data preprocessing according to F and Y, so as to obtain the image data to be fused and the three-dimensional point cloud data to be fused.

3. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 2 is characterized in that: F x Meet the following conditions: F x =a x ×Fcpu x +b x ×Fmem x Among them, Fcpu x is the current CPU load rate of the xth computing node; Fmem x is the current memory load rate of the xth computing node; α x is the CPU weight of the x-th computing node; β x is the memory weight of the xth computing node; α x +β x =1.

4. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 3 is characterized in that: Fcpu x Meet the following conditions: Fcpu x =Ucpu x / Ccpu x Among them, Ucpu x is the current CPU usage of the xth computing node; Ccpu x is the number of cores of the x-th computing node.

5. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 4 is characterized in that: Fmem x Meet the following conditions: Fmem x =MU x / MT x Among them, MU x is the currently used memory of the xth computing node; MT x is the total memory of the x-th computing node.

6. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 5 is characterized in that: The distribution unit comprises: The first allocation subunit is used if Y i is the first priority, then the first target computing node list YJ is obtained according to F i =(YJ i，1 , Y.J. i，2 , …, YJ i，a , …, YJ i，b ); a = 1, 2, ..., b; where b is the number of the first target computing nodes; b < y; YJ i，a is the node identifier of the ath first target computing node corresponding to the i-th processing subtask; F i，a <YF; YF is the first load rate threshold; Ccpu i，a >YCcpu i ;YCcpu i Y i Corresponding key core number; F i，a The current load rate of the ath first target computing node corresponding to the i-th processing subtask; Ccpu i，a The number of cores of the a-th first target computing node corresponding to the i-th processing subtask; The first weight allocation subunit is used to allocate weights according to YJ i , get the first node allocation weight list YP i =(YP i，1 , Y.P. i，2 , …, YP i，a , …, YP i，b ); among them, YP i，a Assign a weight to the processing subtask of the ath first target computing node corresponding to the i-th processing subtask; YP i，a Meet the following conditions: YP i，a =[(1-F i，a )×Ccpu i，a ] / {Σ b a＝1 [(1-F i，a )×Ccpu i，a ]}; The first quantity allocation subunit is used to allocate the quantity according to YP i , get the first node allocation quantity list YS i =(YS i，1 , YS i，2 , …, YS i，a , …, YS i，b );YS i，a The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the a-th first target computing node; YS i，a Meet the following conditions: YS i，a =rounddown(n×YP i，a ); rounddown() is the preset rounding function; The first processing subunit is used to process the i , distribute the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to b first target computing nodes for distributed data preprocessing.

7. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 5 is characterized in that: The distribution unit comprises: The second allocation subunit is used if Y i is the second priority, then the second target computing node list EJ is obtained according to F i =(EJ i，1 , E.J. i，2 , …, E. J. i，c , …, E. J. i，d );c=1,2,…,d;wherein d is the number of the second target computing nodes;d<y;EJ i，c is the node identifier of the cth second target computing node corresponding to the i-th processing subtask; F i，c <EF; EF>YF; EF is the second load rate threshold; F i，c The current load rate of the node corresponding to the cth second target of the i-th processing subtask is calculated; The second weight allocation subunit is used to allocate the weights according to EJ i , get the second node allocation weight list EP i =(EP i，1 , E.P. i，2 , …, EP i，c , …, EP i，d );Among them, EP i，c Assign a weight to the processing subtask of the cth second target computing node corresponding to the i-th processing subtask; EP i，c Meet the following conditions: EP i，c =[(1-F i，c )×(Ccpu i，c +MFR i，c )] / {Σ d c＝1 [(1-F i，c )×(Ccpu i，c +MFR i，c )]}; MFR i，c is the current free memory of the cth second target computing node corresponding to the i-th processing subtask; MFR i，c =MT i，c -MU i，c ;MT i，c is the current total memory of the cth second target computing node corresponding to the i-th processing subtask; MU i，c is the currently used memory of the cth second target computing node corresponding to the i-th processing subtask; The second quantity allocation subunit is used to allocate the quantity according to the EP i , get the second node allocation quantity list ES i =(ES i，1 , E.S. i，2 , …, ES i，c , …, ES i，d ), where ES i，c The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the c-th second target computing node; ES i，c Meet the following conditions: ES i，c =rounddown(n×EP i，c ); The second processing subunit is used to i , distribute the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to d second target computing nodes for distributed data preprocessing.

8. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 5 is characterized in that: The distribution unit comprises: The third allocation subunit is used if Y i is the third priority, then the third target computing node list HJ is obtained according to F i =(HJ i，1 , H.J. i，2 , …, HJ i，e , …, HJ i，f ); e = 1, 2, ..., f; where f is the number of the third target computing nodes; f < y; HJ i，e is the node identifier of the e-th third target computing node corresponding to the i-th processing subtask; F i，e <HF; HF>EF; HF is the third load rate threshold; F i，e The current load rate of the e-th third target computing node corresponding to the i-th processing subtask; The third weight allocation subunit is used to allocate the weight according to HJ i , get the third node allocation weight list HP i =(HP i，1 , HP i，2 ,…,HP i，e ,…,HP i，f ), where HP i，e Assign a weight to the processing subtask of the eth third target computing node corresponding to the i-th processing subtask; HP i，e Meet the following conditions: HP i，e =[(1-F i，e )×MFR i，e ] / {Σ f e＝1 [(1-F i，e )×MFR i，e ]};MFR i，e is the current free memory of the e-th third target computing node corresponding to the i-th processing subtask; MFR i，e =MT i，e -MU i，e ;MT i，e is the current total memory of the e-th third target computing node corresponding to the i-th processing subtask; MU i，e is the currently used memory of the e-th third target computing node corresponding to the i-th processing subtask; The third quantity allocation subunit is used to allocate the amount of i , get the third node allocation quantity list HS i =(HS i，1 , HS i，2 ,…,HS i，e ,…,HS i，f ), where HS i，e The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the e-th third target computing node; HS i，e Meet the following conditions: HS i，e =rounddown(n×HP i，e ); The third processing subunit is used to process the i , distribute the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to f third target computing nodes for distributed data preprocessing.

Images