A dynamic monitoring alarm identification system based on three-dimensional panoramic space fusion

Abstract

The application provides a kind of dynamic monitoring alarm identification system based on three-dimensional panoramic space fusion, it is related to data fusion technical field, including: acquisition module, for collecting the initial multi-source data corresponding to each target sub-region in the target region including several target sub-regions;Preprocessing module, for distributed data preprocessing, obtain the image data to be fused and the three-dimensional point cloud data to be fused;Fusion module, for three-dimensional panoramic space fusion, obtain the three-dimensional panoramic space model corresponding to target region;Feature acquisition module, for obtaining the behavior characteristic information of moving target contained in target region in target time period;Identification module, for obtaining the alarm identification result of target region.The application can improve the real-time performance of the whole system, better realize the monitoring and identification of alarm, avoid causing personal injury and property loss.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data fusion, in particular to a dynamic monitoring alarm identification system based on three-dimensional panoramic space fusion. BACKGROUND

[0002] With the acceleration of urbanization and the increasing frequency of social activities, public safety is facing more and more challenges. Traditional monitoring systems have played an important role in ensuring safety, but have gradually exposed many limitations.

[0003] In terms of data collection, traditional monitoring mainly relies on two-dimensional video images, which provide information limited to a plane perspective, lacking a description of the depth information and three-dimensional spatial structure of the monitoring scene. This makes it difficult to accurately determine the position, size and mutual relationship of targets in complex scenes, such as large public places, city streets, etc., and is prone to problems such as target occlusion misjudgment and inaccurate spatial positioning. For example, in places with a large number of people, it is difficult to determine the specific distribution and behavior state of personnel in different floors or different depth areas through two-dimensional images alone.

[0004] For data processing, the image processing algorithms of traditional monitoring systems are relatively simple, usually only performing basic image enhancement and target detection, and lack effective integration of multi-source data. When facing complex environmental interference (such as changes in light, shadows, bad weather, etc.), image quality decreases, which greatly reduces the accuracy of target detection and recognition. At the same time, due to the lack of in-depth mining of semantic information, it is difficult to intelligently understand and classify objects and behaviors in the scene, making it difficult to accurately judge the alarm. For example, it is difficult to automatically distinguish between normal crowd gathering and suspicious crowd gathering behavior, and accurately identify the type of illegal driving of vehicles.

[0005] In summary, dynamic monitoring systems based on two-dimensional data and three-dimensional data have emerged, but due to the large amount of data and numerous data processing tasks, the pre-processing stage after data collection will consume a large amount of time and resources, resulting in poor real-time performance of dynamic monitoring. Therefore, there is an urgent need for an efficient and fast data preprocessing system for large amounts of data and multiple data dimensions in the process of dynamic alarm monitoring. SUMMARY

[0006] To solve the above technical problems, the present application provides a dynamic monitoring alarm identification system based on three-dimensional panoramic space fusion, which at least partially solves the problems existing in the prior art.

[0007] In a first aspect of the present application, a dynamic monitoring alarm identification system based on three-dimensional panoramic space fusion is provided, which comprises:

[0008] The acquisition module is configured to acquire initial multi-source data corresponding to each target sub-region in the target region; wherein the initial multi-source data includes initial image data and initial three-dimensional point cloud data;

[0009] The preprocessing module is configured to perform distributed data preprocessing on the initial image data and the initial three-dimensional point cloud data according to a distributed computing framework, to obtain to-be-fused image data and to-be-fused three-dimensional point cloud data; wherein the distributed data preprocessing on the initial image data includes a plurality of image processing sub-tasks; the distributed data preprocessing on the initial three-dimensional point cloud data includes a plurality of data processing sub-tasks; the distributed computing framework includes one task allocation node and a plurality of computing nodes; the task allocation node is configured to allocate the image processing sub-tasks and the data processing sub-tasks included in the target region to the plurality of computing nodes according to a computing resource demand priority of each image processing sub-task or each data processing sub-task and a current load rate of each computing node; the computing resource demand priority is determined according to CPU demand and memory demand;

[0010] The fusion module is configured to perform three-dimensional panoramic space fusion on the to-be-fused image data and the to-be-fused three-dimensional point cloud data, to obtain a three-dimensional panoramic space model corresponding to the target region; wherein the three-dimensional panoramic space fusion includes geometric fusion and semantic fusion;

[0011] The feature acquisition module is configured to acquire behavior feature information of a moving target included in the target region in a target time period according to the three-dimensional panoramic space model;

[0012] The recognition module is configured to obtain an alarm recognition result of the target region according to the behavior feature information and a preset alarm recognition rule.

[0013] The present application has at least the following beneficial effects:

[0014] The application provides a dynamic monitoring alarm identification system based on three-dimensional panoramic space fusion. For the collected data (initial image data and initial three-dimensional point cloud data) with large amount and multi-source characteristics for three-dimensional panoramic space fusion, in the data preprocessing, in order to save processing time and improve data preprocessing efficiency, a distributed computing framework is used for processing. The distributed computing framework has a task allocation node and a plurality of computing nodes. The task allocation node is used for allocating all data to the plurality of computing nodes for parallel processing. In the data allocation, on one hand, the different requirements of each processing task itself for the computing resources of the computing device due to different reasons such as computing complexity are considered, and on the other hand, the current load of each computing node is considered. The data is delivered by comprehensively considering the above two conditions. The corresponding processing subtasks can be processed in parallel, and the processing subtasks are allocated according to the characteristics of the processing subtasks and the load of each computing node, so that the computing resources are reasonably used, the overall data processing time is saved, and the computing resources are fully and reasonably used.

[0015] Finally, the dynamic monitoring alarm is performed based on the preprocessed data after three-dimensional panoramic space fusion. Due to the fast data processing speed, the real-time performance of the whole system can be improved, the monitoring and identification of the alarm can be better realized, and personal injury and property loss can be avoided. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 The structural diagram of the dynamic monitoring alarm identification system based on three-dimensional panoramic space fusion provided by the embodiments of the application is shown. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the application will be described clearly and completely with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0019] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described accompanying drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0020] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms and that any specific structure and / or function described herein is merely illustrative. Based on the teachings herein one skilled in the art should appreciate that an aspect described herein can be implemented independently of any other aspects and that two or more of these aspects can be combined in various ways. For example, an apparatus can be implemented or a method can be practiced using any number of the aspects set forth herein. In addition, such an apparatus can be implemented or such a method can be practiced using other structure and / or functionality in addition to or other than one or more of the aspects set forth herein.

[0021] Reference is made to Figure 1 As shown in the drawings, the embodiments of the present application provide a dynamic monitoring alarm identification system 100 based on three-dimensional panoramic space fusion, which comprises the following modules:

[0022] The acquisition module 110 is configured to acquire initial multi-source data corresponding to each target sub-region in a plurality of target sub-regions included in a target region; wherein the initial multi-source data comprises initial image data and initial three-dimensional point cloud data.

[0023] Specifically, the target region can be a park, a school, a factory, etc., and the division method of the plurality of target sub-regions can be any region division method known to those skilled in the art, so that the total amount of data in each target sub-region obtained is substantially similar.

[0024] It should be noted that the initial image data is obtained according to the following method: using a high-definition camera network to perform omnidirectional and multi-angle video image acquisition on the monitoring region to ensure coverage of the entire target scene. The initial three-dimensional point cloud data is obtained according to the following method: using a three-dimensional laser scanner or a depth camera to collect three-dimensional point cloud data of the monitoring region to obtain geometric shape information of the target region.

[0025] It can be understood that for the key area or specific object, sensors such as motion sensors, temperature sensors, etc. are deployed to collect additional physical data which will be used as auxiliary information for alarm judgment.

[0026] The preprocessing module 120 is configured to perform distributed data preprocessing on the initial image data and the initial three-dimensional point cloud data according to a distributed computing framework, to obtain to-be-fused image data and to-be-fused three-dimensional point cloud data; wherein the distributed data preprocessing on the initial image data includes a plurality of image processing subtasks; the distributed data preprocessing on the initial three-dimensional point cloud data includes a plurality of data processing subtasks; the distributed computing framework includes one task allocation node and a plurality of computing nodes; the task allocation node is configured to allocate image processing subtasks and data processing subtasks contained in a target region to the plurality of computing nodes according to a computing resource demand priority of each image processing subtask or each data processing subtask and a current load rate of each computing node; the computing resource demand priority is determined according to CPU demand and memory demand.

[0027] Specifically, as an example: the image processing subtask can be image filtering, image enhancement, key frame extraction, etc.; the data processing subtask can be point cloud filtering, point cloud registration, point cloud simplification, etc. The distributed computing framework includes one task allocation node and a plurality of computing nodes, wherein the task allocation node is configured to distribute multi-source data corresponding to each target subregion to the plurality of computing nodes for data preprocessing. As an example: Apache Spark is used as the distributed computing framework to construct a cluster composed of a plurality of computing nodes. The cluster includes one master node (Master Node) for task scheduling and resource management, and a plurality of worker nodes (Worker Node) for actual data processing task execution. The nodes are connected through a high-speed network to ensure the efficiency of data transmission.

[0028] The corresponding data is delivered to the computing nodes according to the computing resource requirement priority of each image processing subtask or each data processing subtask and the current load rate of each computing node, that is, in the present application, on the one hand, the different requirements of each processing task itself for the computing resources of the computing device due to different factors such as computing complexity are considered, for example: image filtering: a complex image bilateral filtering algorithm needs to perform complex weighted calculation on each pixel and its surrounding pixels in the image, involving a large number of mathematical operations. In this case, the computing capacity of the CPU is the key factor limiting the execution speed of the task. If the CPU resource is insufficient, the task execution will be very slow. Simple image enhancement tasks (such as simple contrast, brightness adjustment) mainly perform basic arithmetic operations on image pixels, and the relative computing complexity is low. However, these tasks still need a certain amount of memory space to store intermediate results and original image data when processing image data. Therefore, for this type of task, the amount of free memory is the focus.

[0029] On the other hand, the current load condition of each computing node is considered. The data is delivered by comprehensively considering the above two conditions. The corresponding processing subtasks can be processed in parallel, and the processing subtasks are allocated according to the characteristics of the processing subtasks and the load condition of each computing node, so that the computing resources are reasonably utilized, the overall data processing time is saved, and the computing resources are fully and reasonably utilized.

[0030] The fusion module 130 is configured to perform three-dimensional panoramic space fusion on the to-be-fused image data and the to-be-fused three-dimensional point cloud data, to obtain a three-dimensional panoramic space model corresponding to the target region.

[0031] Specifically, first, geometric fusion is performed: the processed three-dimensional point cloud data is fused with geographic information system (GIS) data of the monitoring area, the point cloud model is accurately mapped to the real geographic coordinate space, and the accurate position and attitude of the point cloud model on the map are determined.

[0032] Based on the three-dimensional point cloud model, the two-dimensional coordinate information of the object in the key frame image is mapped to the three-dimensional space through projection transformation, the geometric correlation between the image and the three-dimensional point cloud is realized, and a three-dimensional panoramic space model with real geometric shape and texture information is constructed.

[0033] Secondly, semantic fusion is carried out. First, a deep learning algorithm (such as a convolutional neural network) is used for semantic segmentation of the key frame image, different object categories (such as people, vehicles, buildings, etc.) in the image are identified, and a semantic label is assigned to each object. The semantic segmentation result is fused with the object geometric information in the three-dimensional point cloud data, and semantic attributes are added to each object or region in the three-dimensional panoramic space, so that the entire scene has rich semantic information, facilitating subsequent alarm analysis and identification.

[0034] The feature acquisition module 140 is configured to acquire behavior feature information of a moving target contained in a target region in a target time period according to the three-dimensional panoramic space model.

[0035] Specifically, first, real-time data updating is performed: new video images and three-dimensional point cloud data are continuously collected, and the new data is fused into the constructed three-dimensional panoramic space model in real time according to the above data processing and space fusion steps, to realize real-time updating of the dynamic monitoring scene. Second, moving target detection and tracking is performed, that is, in the three-dimensional panoramic space, a moving target (such as a pedestrian or a vehicle) is detected from a video image sequence by using an optical flow method, a background difference method, or a target detection algorithm based on deep learning (such as YOLO or FasterR-CNN). Then, the detected moving target is tracked, the position and motion state of the target in consecutive frames are determined through target feature matching (such as color feature, shape feature, motion trajectory feature, etc.) and tracking algorithms (such as Kalman filtering or particle filtering), and a motion trajectory model of the target is established. Finally, behavior feature extraction is performed, that is, for a moving target, its behavior features are extracted. For example, the walking speed, walking direction, and posture change of a pedestrian are analyzed; for a vehicle, the speed, curvature of the driving trajectory, and vehicle type are extracted.

[0036] At the same time, in combination with the data collected by the sensors (such as the target motion speed detected by a motion sensor and the temperature change of the region detected by a temperature sensor), the behavior feature information of the target is further enriched.

[0037] The identification module 150 is configured to obtain an alarm identification result of the target region according to the behavior feature information and a preset alarm identification rule.

[0038] Specifically, the preset alarm identification rule can be that if unknown persons are detected to loiter or gather in a specific region for a long time, and the number of persons exceeds a set threshold, a suspicious gathering alarm is determined; if a vehicle is detected to drive in a prohibited area or the speed is abnormally fast and the trajectory is unstable, a violation driving alarm is determined; if an abnormal temperature rise is detected in a specific region accompanied by smoke and other features, a fire alarm is determined, etc.

[0039] The extracted motion target behavior features and other related data are matched and judged in real time with the set alarm identification rules. Through logical reasoning, data analysis and model calculation, it is determined whether an alarm occurs and the type and severity of the alarm.

[0040] Once the alarm is identified, the early warning mechanism is triggered. According to the type and severity of the alarm, alarm information is sent to the monitoring personnel through various ways (such as sound and light alarm, short message notification, system pop-up window, etc.), and the location and related information of the alarm are highlighted in the three-dimensional panoramic space model, so as to facilitate the monitoring personnel to quickly locate and handle the alarm.

[0041] In summary, the dynamic monitoring alarm identification system based on three-dimensional panoramic space fusion provided in the application has the following advantages: the data (initial image data and initial three-dimensional point cloud data) collected for three-dimensional panoramic space fusion has a large amount and is characterized by multiple sources. In order to save processing time and improve data preprocessing efficiency, a distributed computing framework is used for processing during data preprocessing. The distributed computing framework has a task allocation node and a plurality of computing nodes. The task allocation node is used to allocate all data to a plurality of computing nodes for parallel processing. When allocating data, on the one hand, the different requirements of each processing task for computing resources due to different computing complexities and other factors are considered, and on the other hand, the current load of each computing node is considered. The data is delivered by comprehensively considering the above two situations. The corresponding processing subtasks can be processed in parallel, and the task allocation is also performed according to the characteristics of the processing subtasks and the load of each computing node, so that the computing resources are reasonably utilized, the overall data processing time is saved, and the computing resources are fully and reasonably utilized.

[0042] Finally, based on the preprocessed data, three-dimensional panoramic space fusion is performed for dynamic monitoring alarm. Due to the fast data processing speed, the real-time performance of the entire system can be improved, the monitoring and identification of the alarm can be better achieved, and personal injury and property loss can be avoided.

[0043] In an exemplary embodiment of the application, the preprocessing module includes:

[0044] The subtask acquisition unit is configured to obtain a subtask data list set Z=(Z1, Z2, …, Zn) according to the initial image data and the initial three-dimensional point cloud data; i=1, 2, …, n; wherein n is the number of processing subtasks; n=nimg+ndata; wherein nimg is the number of image processing subtasks; ndata is the number of data processing subtasks; and Zi is the i-th subtask data. i n t s t s i ​​​​​​a set of subtask data lists corresponding to the i-th processing subtask; Z i = (Z i,1 , Z i,2 , …, Z i，j , …, Z i，m ); j = 1, 2, …, m; m is the number of target sub-regions; Z i，j is the image data list or the three-dimensional point cloud data list corresponding to the i-th processing subtask in the j-th target sub-region.

[0045] Specifically, the processing task performed on the initial image data during preprocessing is divided into a plurality of image processing subtasks, and the processing task performed on the initial three-dimensional point cloud data during preprocessing is divided into a plurality of data processing subtasks to obtain a set of subtask data lists. Here, a to-be-processed data list corresponding to each processing subtask in each target sub-region is obtained. As an example: for the processing subtask of point cloud registration task, when the initial image data and the three-dimensional point cloud data collected in the j-th target sub-region are processed, each processing subtask is executed once, but the corresponding basic data (initial image data and three-dimensional point cloud data) may be different when each processing subtask is executed. Therefore, in this application, for a large target region, the target region is divided into a plurality of target sub-regions, and each processing subtask is executed on the data corresponding to each target sub-region, thereby preliminarily reducing the data processing volume.

[0046] a load rate acquisition unit configured to acquire the current load rate of each computing node to obtain a load rate list F = (F1, F2, …, F x , …, F y ); wherein y is the number of computing nodes; F x is the current load rate corresponding to the x-th computing node.

[0047] Specifically, F x satisfies the following conditions:

[0048] F x = α x × Fcpu x + β x × Fmem x

[0049] wherein Fcpu x is the current CPU load rate of the x-th computing node; Fmem x is the current memory load rate of the x-th computing node; α x is the CPU weight of the x-th computing node; β x is the memory weight of the x-th computing node; α x + β x = 1; Fcpu xMeet the following conditions:

[0050] Fcpu x =Ucpu x / Ccpu x

[0051] Among them, Ucpu x is the current CPU usage of the x-th computing node; Ccpu x is the number of cores of the x-th compute node.

[0052] Fmem x Meet the following conditions:

[0053] Fmem x =MU x / MT x

[0054] Among them, MU x is the currently used memory of the x-th computing node; MT x is the total memory of the x-th compute node.

[0055] In this embodiment, the current load rate of each computing node is defined from the two aspects of CPU load rate and memory load rate. This is because computing nodes are mainly composed of two core resources, CPU and memory, which play a key role in the task execution process. The CPU (central processing unit) is the computing core and control core of the computer, responsible for executing program instructions, performing data processing and calculations. The memory (main memory) is used to store running programs and data and is the storage area where the CPU can quickly access data. In a distributed computing environment, the usage of these two resources directly affects the execution efficiency of the task and the overall performance of the node.

[0056] And α x is the CPU weight of the x-th computing node, β x is the memory weight of the x-th computing node, where α x This can be determined based on the hardware configuration of the xth compute node. If the compute node has good CPU performance but relatively poor memory performance, then its corresponding CPU weight is greater. Conversely, if another compute node has better memory performance but poorer CPU performance, then its corresponding memory weight is greater. CPU performance can be influenced by factors such as the number of cores, number of threads, main frequency, cache size, and architecture; while memory can be influenced by factors such as capacity, frequency, type, and timing.

[0057] In addition, it should be noted that in the present embodiment, as an example: the current CPU usage is determined as follows: a single-core CPU has 6 seconds of task execution within 10 seconds, and the CPU usage is 60%. The current CPU load rate is related to the CPU usage and the number of CPU cores, that is, when the CPU usage of two computing nodes is the same, the current CPU load rate of the computing node with more cores is smaller, because the number of cores is larger, and the remaining computing resources are more.

[0058] In summary, the current CPU load rate determined by the present application can make the current CPU load rate more objective and accurate by comprehensively evaluating the CPU and memory load.

[0059] The priority obtaining unit is configured to obtain a computing resource demand priority list Y=(Y1, Y2, …, Y i , …, Y n ) of each processing subtask; Y i is the computing resource demand priority corresponding to the ith processing subtask; the computing resource demand priority includes a first priority, a second priority, and a third priority; the CPU demand degree CPUX1 corresponding to the first priority satisfies the condition: CPUX1≥CPUY; CPUY is a first CPU demand degree threshold; the CPU demand degree CPUX2 corresponding to the second priority satisfies the condition: CPUE

[0060] Specifically, in this embodiment, since each processing subtask has different characteristics and therefore requires different CPU and memory resources from the computing node, this embodiment defines the overall requirements of each processing subtask for the computing node using different priorities. The first priority level is for computationally intensive tasks. For example, a complex bilateral filtering algorithm for an image requires complex weighted calculations for each pixel in the image and its surrounding pixels, involving a large number of mathematical operations. In this case, CPU computing power is the key factor limiting the speed of task execution. If CPU resources are insufficient, task execution will be very slow. Therefore, for first-priority processing subtasks, prioritizing CPU utilization ensures that these tasks are assigned to nodes capable of completing calculations quickly. Second-priority tasks, on the other hand, have certain requirements for both computing and memory resources. The synergy between CPU and memory resources during task execution needs to be considered. Third-priority tasks, such as simple image enhancement tasks (such as simple contrast and brightness adjustments), primarily perform basic arithmetic operations on image pixels and are relatively computationally less complex. However, these tasks still require a certain amount of memory space to store intermediate results and raw image data when processing image data. Therefore, for low-priority tasks, the amount of free memory is a key consideration.

[0061] The allocation unit is used to control the task allocation node to allocate the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-area of ​​n processing subtasks to y computing nodes for distributed data preprocessing according to F and Y, so as to obtain the image data to be fused and the three-dimensional point cloud data to be fused.

[0062] Specifically, in one embodiment, for the first priority, the allocation unit includes:

[0063] The first allocation subunit is used for i is the first priority, then the first target computing node list YJ is obtained according to F i =(YJ i，1 , YJ i，2 ,…,YJ i，a ,…,YJ i，b ); a=1, 2, ..., b; where b is the number of the first target computing nodes; b<y; YJ i，a is the node identifier of the ath first target computing node corresponding to the i-th processing subtask; F i，a <YF; YF is the first load rate threshold; Ccpu i，a >YCcpu i ;YCcpu i Y i The corresponding number of key cores; F i，a The current load rate of the a-th first target computing node corresponding to the i-th processing subtask; Ccpui，a The number of cores of the a-th first target computing node corresponding to the i-th processing subtask.

[0064] The first weight distribution subunit is used to allocate i , get the first node allocation weight list YP i =(YP i，1 , YP i，2 ,…,YP i，a ,…,YP i，b ); among them, YP i，a Assign a weight to the processing subtask of the ath first target computing node corresponding to the i-th processing subtask; YP i，a Meet the following conditions: YP i，a =[(1-F i，a )×Ccpu i，a ] / {Σ b a＝1 [(1-F i，a )×Ccpu i，a ]}.

[0065] The first quantity allocation subunit is used to allocate the quantity according to YP i , get the first node allocation quantity list YS i =(YS i，1 , YS i，2 ,…,YS i，a ,…,YS i，b ); among them, YS i，a The number of target sub-regions to be processed corresponding to the i-th processing sub-task assigned to the a-th first target computing node; YS i，a Meet the following conditions: YS i，a =rounddown(n×YP i，a ); rounddown() is the default rounding function.

[0066] The first processing subunit is used to process the i , distribute the data in the image data list and three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to b first target computing nodes for distributed data preprocessing.

[0067] In this embodiment, when weighting the processing subtasks of the first priority, the first priority is to select the ones with the smaller current load rate (the node load is lighter and the remaining amount of computing resources is larger). Since the processing subtasks of the first priority have higher requirements for the CPU, in this embodiment, the number of CPU cores is greater than Y. iThe corresponding key core number of the computing node is taken as the first target computing node. Here, each processing subtask has a corresponding key core number, which is the core number with higher efficiency in processing the task. As an example, the first load rate threshold can be 0.5. Further, since the computing power of the CPU in the processing subtask corresponding to the first priority is the key factor limiting the execution speed of the task, in this embodiment, only the load rate and the CPU core number of the computing node are considered when the weight is assigned to each computing node, so that the amount of computing tasks obtained by each computing node is the most appropriate when data is allocated to the processing subtask of the first priority. The overall data processing efficiency is improved.

[0068] In one embodiment, for the second priority, the above allocation unit comprises:

[0069] A second allocation subunit is configured to, if Y i For the second priority, a second target computing node list EJ i = (EJ i，1 , EJ i，2 , …, EJ i，c , …, EJ i，d ); c = 1, 2, …, d; wherein d is the number of second target computing nodes; d < y; EJ i，c is the node identifier of the cth second target computing node corresponding to the ith processing subtask; F i，c < EF; EF > YF; EF is the second load rate threshold; F i，c is the current load rate of the cth second target computing node corresponding to the ith processing subtask.

[0070] A second weight allocation subunit is configured to, according to EJ i , obtain a second node allocation weight list EP i = (EP i，1 , EP i，2 , …, EP i，c , …, EP i，d ); wherein EP i，c is the processing subtask allocation weight of the cth second target computing node corresponding to the ith processing subtask; EP i，c satisfies the following condition: EP i，c = [(1-F i，c ) × (Ccpu i，c + MFR i，c )] / {∑ d c＝1 [(1-F i，c ) × (Ccpu i，c + MFR i，c )]}; MFR i，cis the current free memory of the cth second target computing node corresponding to the i-th processing subtask; MFR i，c =MT i，c -MU i，c MT i，c is the current total memory of the cth second target computing node corresponding to the i-th processing subtask; MU i，c is the currently used memory of the cth second target computing node corresponding to the i-th processing subtask;

[0071] The second quantity allocation subunit is used to allocate the quantity according to EP i , get the second node allocation quantity list ES i =(ES i，1 , ES i，2 ,…,ES i，c ,…,ES i，d ); among them, ES i，c The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the c-th second target computing node; ES i，c Meet the following conditions: ES i，c =rounddown(n×EP i，c );

[0072] The second processing subunit is used to i , distribute the data in the image data list and three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to d second target computing nodes for distributed data preprocessing.

[0073] In this embodiment, when weighting the second-priority processing subtasks, the load rate of the selected computing nodes can be lower than that of the first-priority processing subtasks, but it cannot be too low. For example, the first load rate threshold can be 0.7. Furthermore, because the second-priority processing subtasks have certain requirements for both CPU and memory, when weighting each computing node, not only the number of CPU cores but also the impact of free memory on the weight allocation are considered. This ensures that the resulting weight allocation is more consistent with the computing resource requirements of the second-priority processing subtasks.

[0074] In one embodiment, for the third priority, the allocation unit includes: a third allocation subunit, for i For the third priority, the third target computing node list HJ is obtained according to F i =(HJ i，1 , HJ i，2 ,…,HJ i，e ,…,HJ i，f); e = 1, 2, …, f; where, f is the number of third target computing nodes; f < y; HJ i，e is the node identifier of the e-th third target computing node corresponding to the i-th processing subtask; F i，e < HF; HF > EF; HF is a third load rate threshold; F i，e is the current load rate of the e-th third target computing node corresponding to the i-th processing subtask;

[0075] a third weight distribution subunit, configured to obtain a third node distribution weight list HP i according to HJ i = (HP i，1 , HP i，2 , …, HP i，e , …, HP i，f ); where, HP i，e is the processing subtask distribution weight of the e-th third target computing node corresponding to the i-th processing subtask; HP i，e meets the following condition: HP i，e = [(1-F i，e ) x MFR i，e ] / {∑ f e＝1 [(1-F i，e ) x MFR i，e ]}; MFR i，e is the current free memory of the e-th third target computing node corresponding to the i-th processing subtask; MFR i，e = MT i，e -MU i，e ; MT i，e is the current total memory of the e-th third target computing node corresponding to the i-th processing subtask; MU i，e is the current used memory of the e-th third target computing node corresponding to the i-th processing subtask;

[0076] a third quantity distribution subunit, configured to obtain a third node distribution quantity list HS i according to HP i = (HS i，1 , HS i，2 , …, HS i，e , …, HS i，f ); where, HS i，e is the number of target sub-regions to be processed corresponding to the i-th processing subtask distributed to the e-th third target computing node; HS i，e meets the following condition: HS i，e = roundddown(n x HP i，e );

[0077] The third processing subunit is configured to distribute data of the i th processing subtask in the image data list and the three-dimensional point cloud data list corresponding to each target subregion to the f third target computing nodes for distributed data preprocessing. i The third processing subunit is configured to distribute data of the i th processing subtask in the image data list and the three-dimensional point cloud data list corresponding to each target subregion to the f third target computing nodes for distributed data preprocessing.

[0078] In this embodiment, compared with the above two embodiments, the requirement for the number of CPU cores is lower, and there is a certain requirement for the amount of memory, because when the data of each target subregion of the third priority processing subtask is distributed, the influence of the idle memory amount of the computing node is mainly considered. As an example, the third load rate threshold can be 0.9.

[0079] In summary, the present application sets different data distribution methods for different types of processing subtasks, fully considers the demand characteristics of each data subtask for computing resources and the load situation of each computing node itself, and makes different processing for different situations, so that the final data distribution is more reasonable, and the computing efficiency of the distributed computing framework is higher.

[0080] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the system according to the exemplary embodiments of the present application, and are not for limiting purposes. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0081] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units embodied.

[0082] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed by the present application can be easily thought of by those skilled in the art, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion, characterized in that: The system comprises: An acquisition module, configured to acquire initial multi-source data corresponding to each of a plurality of target sub-regions contained in a target region; wherein the initial multi-source data includes initial image data and initial three-dimensional point cloud data; A preprocessing module is configured to perform distributed data preprocessing on the initial image data and the initial three-dimensional point cloud data according to a distributed computing framework to obtain image data to be fused and three-dimensional point cloud data to be fused; wherein the distributed data preprocessing on the initial image data includes a plurality of image processing subtasks; and the distributed data preprocessing on the initial three-dimensional point cloud data includes a plurality of data processing subtasks; the distributed computing framework includes a task allocation node and a plurality of computing nodes; the task allocation node is configured to allocate the image processing subtasks and data processing subtasks contained in the target area to the plurality of computing nodes according to the computing resource requirement priority of each image processing subtask or each data processing subtask and the current load rate corresponding to each computing node; the computing resource requirement priority is determined according to the CPU requirement and the memory requirement; A fusion module, configured to perform 3D panoramic spatial fusion on the image data to be fused and the 3D point cloud data to be fused, to obtain a 3D panoramic spatial model corresponding to the target area; wherein the 3D panoramic spatial fusion includes geometric fusion and semantic fusion; a feature acquisition module, configured to acquire, based on the three-dimensional panoramic space model, behavioral feature information of moving targets contained in a target area within a target time period; The recognition module is used to obtain the alarm situation recognition result of the target area based on the behavioral feature information and the preset alarm situation recognition rules.

2. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 1 is characterized in that: The pre-processing module comprises: The subtask acquisition unit is used to obtain the subtask data list set Z = (Z1, Z2, ..., Z i ,…,Z n ); i = 1, 2, ..., n; where n is the number of processing subtasks; n = n t +n s ; Among them; n t is the number of image processing subtasks; n s is the number of data processing subtasks; Z i is the subtask data list set corresponding to the i-th processing subtask; Z i =(Z i,1 , Z i,2 ,…,Z i，j ,…,Z i，m ); j = 1, 2, ..., m; m is the number of target sub-regions; Z i，j The image data list or 3D point cloud data list corresponding to the i-th processing subtask in the j-th target sub-region; The load rate acquisition unit is used to obtain the current load rate of each computing node to obtain a load rate list F = (F1, F2, ..., F x ,…,F y ), where y is the number of computing nodes; F x is the current load rate corresponding to the x-th computing node; The priority acquisition unit is used to obtain the computing resource requirement priority list Y=(Y1, Y2, ..., Y i ,…,Y n );Y i is the computing resource demand priority corresponding to the i-th processing subtask; the computing resource demand priority includes a first priority, a second priority, and a third priority; wherein, the CPU demand CPUX1 corresponding to the first priority meets the following condition: CPUX1≥CPUY; CPUY is the first CPU demand threshold; the CPU demand CPUX2 corresponding to the second priority meets the following condition: CPUE<CPUX2<CPUY; CPUE is the second CPU demand threshold; and the memory demand MX2 corresponding to the second priority meets the following condition: MX2≥MY; MY is the memory demand threshold; the CPU demand CPUX3 corresponding to the third priority meets the following condition: CPUX3≤CPUE; and the memory demand MX3 corresponding to the third priority meets the following condition: MX3≥MY; The allocation unit is used to control the task allocation node to allocate the data in the image data list and the three-dimensional point cloud data list corresponding to each target sub-area of ​​n processing subtasks to y computing nodes for distributed data preprocessing according to F and Y, so as to obtain the image data to be fused and the three-dimensional point cloud data to be fused.

3. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 2 is characterized in that: F x Meet the following conditions: F x =a x ×Fcpu x +b x ×Fmem x Among them, Fcpu x is the current CPU load rate of the x-th computing node; Fmem x is the current memory load rate of the x-th computing node; α x is the CPU weight of the x-th computing node; β x is the memory weight of the x-th computing node; α x +β x =1.

4. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 3 is characterized in that: Fcpu x Meet the following conditions: Fcpu x =Ucpu x / Ccpu x Among them, Ucpu x is the current CPU usage of the x-th computing node; Ccpu x is the number of cores of the x-th compute node.

5. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 4 is characterized in that: Fmem x Meet the following conditions: Fmem x =MU x / MT x Among them, MU x is the currently used memory of the x-th computing node; MT x is the total memory of the x-th compute node.

6. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 5 is characterized in that: The distribution unit includes: The first allocation subunit is used for i is the first priority, then the first target computing node list YJ is obtained according to F i =(YJ i，1 , YJ i，2 ,…,YJ i，a ,…,YJ i，b ); a=1, 2, ..., b; where b is the number of the first target computing nodes; b<y; YJ i，a is the node identifier of the ath first target computing node corresponding to the i-th processing subtask; F i，a <YF; YF is the first load rate threshold; Ccpu i，a >YCcpu i ;YCcpu i Y i The corresponding number of key cores; F i，a The current load rate of the a-th first target computing node corresponding to the i-th processing subtask; Ccpu i，a The number of cores of the a-th first target computing node corresponding to the i-th processing subtask; The first weight distribution subunit is used to allocate i , get the first node allocation weight list YP i =(YP i，1 , YP i，2 ,…,YP i，a ,…,YP i，b ); among them, YP i，a Assign a weight to the processing subtask of the ath first target computing node corresponding to the i-th processing subtask; YP i，a Meet the following conditions: YP i，a =[(1-F i，a )×Ccpu i，a ] / {Σ b a＝1 [(1-F i，a )×Ccpu i，a ]}; The first quantity allocation subunit is used to allocate the quantity according to YP i , get the first node allocation quantity list YS i =(YS i，1 , YS i，2 ,…,YS i，a ,…,YS i，b ); among them, YS i，a The number of target sub-regions to be processed corresponding to the i-th processing sub-task assigned to the a-th first target computing node; YS i，a Meet the following conditions: YS i，a =rounddown(n×YP i，a ); rounddown() is the default rounding function; The first processing subunit is used to process the i , distribute the data in the image data list and three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to b first target computing nodes for distributed data preprocessing.

7. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 5 is characterized in that: The distribution unit includes: The second allocation subunit is used for i For the second priority, the second target computing node list EJ is obtained according to F i =(EJ i，1 , E.J. i，2 ,…,EJ i，c ,…,EJ i，d ); c = 1, 2, ..., d; where d is the number of the second target computing nodes; d < y; EJ i，c is the node identifier of the cth second target computing node corresponding to the i-th processing subtask; F i，c <EF; EF> YF; EF is the second load rate threshold; F i，c Calculate the current load rate of the node for the cth second target corresponding to the i-th processing subtask; The second weight allocation subunit is used to allocate i , get the second node allocation weight list EP i =(EP i，1 , EP i，2 ,…,EP i，c ,…,EP i，d ); among them, EP i，c Assign a weight to the processing subtask of the cth second target computing node corresponding to the i-th processing subtask; EP i，c Meet the following conditions: EP i，c =[(1-F i，c )×(Ccpu i，c +MFR i，c )] / {Σ d c＝1 [(1-F i，c )×(Ccpu i，c +MFR i，c )]};MFR i，c is the current free memory of the cth second target computing node corresponding to the i-th processing subtask; MFR i，c =MT i，c -MU i，c MT i，c is the current total memory of the cth second target computing node corresponding to the i-th processing subtask; MU i，c is the currently used memory of the cth second target computing node corresponding to the i-th processing subtask; The second quantity allocation subunit is used to allocate the quantity according to EP i , get the second node allocation quantity list ES i =(ES i，1 , ES i，2 ,…,ES i，c ,…,ES i，d ); among them, ES i，c The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the c-th second target computing node; ES i，c Meet the following conditions: ES i，c =rounddown(n×EP i，c ); The second processing subunit is used to i , distribute the data in the image data list and three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to d second target computing nodes for distributed data preprocessing.

8. The dynamic monitoring alarm recognition system based on three-dimensional panoramic space fusion according to claim 5 is characterized in that: The distribution unit includes: The third allocation subunit is used for i For the third priority, the third target computing node list HJ is obtained according to F i =(HJ i，1 , HJ i，2 ,…,HJ i，e ,…,HJ i，f ); e = 1, 2, ..., f; where f is the number of the third target computing nodes; f < y; HJ i，e is the node identifier of the e-th third target computing node corresponding to the i-th processing subtask; F i，e <HF; HF>EF; HF is the third load rate threshold; F i，e Calculate the current load rate of the node for the e-th third target corresponding to the i-th processing subtask; The third weight distribution subunit is used to allocate i , get the third node allocation weight list HP i =(HP i，1 , HP i，2 ,…,HP i，e ,…,HP i，f ); among them, HP i，e Assign a weight to the processing subtask of the eth third target computing node corresponding to the i-th processing subtask; HP i，e Meet the following conditions: HP i，e =[(1-F i，e )×MFR i，e ] / {Σ f e＝1 [(1-F i，e )×MFR i，e ]};MFR i，e is the current free memory of the e-th third target computing node corresponding to the i-th processing subtask; MFR i，e =MT i，e -MU i，e MT i，e is the current total memory of the e-th third target computing node corresponding to the i-th processing subtask; MU i，e is the currently used memory of the e-th third target computing node corresponding to the i-th processing subtask; The third quantity allocation subunit is used to allocate the amount of i , get the third node allocation quantity list HS i =(HS i，1 , HS i，2 ,…,HS i，e ,…,HS i，f ); among them, HS i，e The number of target sub-regions to be processed corresponding to the i-th processing sub-task allocated to the e-th third target computing node; HS i，e Meet the following conditions: HS i，e =rounddown(n×HP i，e ); The third processing subunit is used to process the i , distribute the data in the image data list and three-dimensional point cloud data list corresponding to each target sub-region of the i-th processing subtask to f third target computing nodes for distributed data preprocessing.

Images