AI electronic fence abnormal behavior recognition system for unmanned aerial vehicle video

Abstract

The invention relates to the technical field of electronic fences, in particular to an AI electronic fence abnormal behavior recognition system for unmanned aerial vehicle video, which comprises the steps of: constructing a field simulation model and a planning model according to an initial field picture and a construction planning map, then accurately recognizing a work proceeding area in the planning model according to a construction task, and constructing an AI electronic fence abnormal behavior recognition system for unmanned aerial vehicle video. The method comprises the following steps: setting an electronic fence in an actual scene, determining a real-time task tag, comprehensively collecting behavior information in a whole construction area, identifying a target analyte in the behavior information, determining an electronic fence area where the target analyte is located, and independently extracting a single image for behavior identification processing. Through continuous monitoring of real-time behavior actions of a target analyte and real-time matching with a real-time task label, the system can quickly find abnormal behaviors and timely give out early warning; the method can better adapt to dynamic changes of construction tasks, and the pertinence and effectiveness of electronic fence setting are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic fence, and particularly relates to an AI electronic fence abnormal behavior identification system for UAV video. BACKGROUND

[0002] In many occasions, it is necessary to control the region for the requirements of safety management, control and the like. At present, the means for realizing the region control mainly includes physical fence and electronic fence. The physical fence needs a large amount of construction, and is not flexible in use, and cannot be changed according to the requirements, so the virtual electronic fence has obtained more applications.

[0003] The prior art CN119672880A discloses a kind of operation area electronic fence automatic generation and intrusion alarm method and medium, including obtaining full-scene standard three-dimensional model by the three-dimensional data of substation building and primary equipment and the shooting data of secondary equipment;Digital model is constructed based on the key information of typical work ticket and secondary protection ticket, and the correlation mapping of digital model and full-scene standard three-dimensional model is established by global unique identifier to generate digital expression, the interval and screen cabinet of operation and inspection work are divided into several work areas, and virtual electronic fence is automatically generated based on PRM sampling path planning and collision warning;Different monitoring precision virtual electronic fence is set according to the type of work area for indoor and outdoor environment, the position of irregular obstacle is acquired in real time, and alarm is triggered immediately when intrusion behavior is found.

[0004] But in the traditional construction site supervision mode, manual inspection depends on the subjective judgment and experience of inspection personnel, not only the labor cost is high, and the efficiency is seriously limited. Taking complex construction projects such as large bridges and tunnels as examples, the inspection personnel need to spend a lot of time going back and forth between different work points, and it is difficult to realize high-frequency inspection of the whole region, and in actual construction, the work area and task focus of each stage of the project are constantly changing, such as building construction from foundation excavation to main structure erection, the traditional electronic fence cannot accurately adapt to the phased construction requirements, and it is also difficult to regulate and constrain the specific work behavior in the region. SUMMARY

[0005] The present application relates to the technical field of electronic fence, and particularly relates to an AI electronic fence abnormal behavior identification system for UAV video.

[0006] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme:

[0007] An AI electronic fence abnormal behavior identification system for UAV video, comprising:

[0008] The simulation planning module is configured to set up a field simulation model according to an initial field picture, and to simulate and build the overall construction area and the local planning area in the field simulation model based on a construction planning map to obtain a planning model.

[0009] The task publishing module is configured to determine the construction task of the day according to the construction progress.

[0010] The fence setting module is configured to identify the work area in the planning model according to the construction task of the day, and to set up an electronic fence in the actual scene of the target area according to the area range of the work area, and to determine the real-time task label in each electronic fence according to the construction task.

[0011] The behavior collection module is configured to collect the behavior information in the overall construction area.

[0012] The behavior recognition module is configured to identify the non-stationary object in the behavior information and mark it as a target analysis object, to determine the electronic fence area where the target analysis object is located, to separately extract the target analysis object in the behavior information to obtain a single image, to perform behavior recognition processing on the single image to determine the real-time behavior action of the target analysis object, to perform matching processing on the real-time behavior action and the real-time task label of the corresponding electronic fence area to obtain a behavior matching value, and to determine the abnormal behavior of the target analysis object according to the behavior matching value.

[0013] As a further scheme of the present application, the overall construction area refers to the entire construction range of the engineering project, including all work areas from the early preparation to the final delivery, and the local planning area refers to a subdivided area divided according to the construction stage, functional zoning, professional type or spatial position in the overall construction area.

[0014] As a further scheme of the present application, the determination method of the real-time task label includes:

[0015] The overall construction area is obtained, and a comprehensive boundary line is set based on the area range of the overall construction area.

[0016] Then, the local planning area under construction at the current time is determined based on the received construction task, and this area is marked as a work area, the planning model is obtained, the position of the work area in the planning model is identified, and a work boundary line is set according to the area range of the position, and the corresponding electronic fence is set in the actual scene of the target area according to the work boundary line and the comprehensive boundary line.

[0017] The area surrounded by the comprehensive boundary line is the overall construction area, and the area surrounded by the work boundary line is the work area.

[0018] When the electronic fence is set, the construction task is acquired again, the specific work content in the work area in the construction task is recognized, and the work content is taken as the real-time task label of the work area on the day.

[0019] As a further scheme of the present application, the acquisition device is selected as a UAV device, and the UAV is used to acquire the construction behavior information of the whole construction area, and the behavior information is acquired and transmitted in a video format.

[0020] As a further scheme of the present application, the method for acquiring the single image comprises:

[0021] In the behavior information, the non-stationary object is recognized, when the non-stationary object is recognized, the object is marked as a target analysis object, and then the area position where the target analysis object is located is recognized, and the electronic fence area where the target analysis object is located is determined.

[0022] According to the sampling time, the continuous video in the behavior information is disassembled into single-frame images, and the image position where the target analysis object is located in each single-frame image is recognized, and the target analysis object is extracted separately to obtain a plurality of single images, wherein only the target analysis object exists in the image screen of the single image.

[0023] As a further scheme of the present application, when a plurality of non-stationary objects are recognized in the behavior information, the recognized non-stationary objects are sequentially marked as target analysis objects, and the behavior matching value calculation is sequentially performed on each target analysis object.

[0024] As a further scheme of the present application, the method for determining the abnormal behavior comprises:

[0025] The single images are arranged in time sequence to obtain an image sequence, and the 3D convolutional neural network algorithm is used to extract the features of the single images in the image sequence to obtain the behavior features of the target analysis object.

[0026] Based on the behavior features of the target analysis object, the actual behavior action of the current target analysis object is determined, the electronic fence area where the target analysis object is located is recognized, and the real-time task label of the electronic fence area is acquired, and then the real-time behavior action and the real-time task label are subjected to behavior matching processing to obtain a behavior matching value.

[0027] The behavior matching value of the target analysis object is acquired, the behavior matching value is compared with a matching threshold value, if the behavior matching value is less than the matching threshold value, the actual behavior action of the target analysis object is marked as an abnormal behavior, otherwise, if the behavior matching value is greater than or equal to the matching threshold value, the actual behavior action of the target analysis object is marked as a normal behavior.

[0028] As a further scheme of the present application, the algorithm used by the behavior matching algorithm is a Transformer architecture algorithm, which extracts the spatiotemporal features of real-time behavior actions through 3D CNN, then injects the time sequence information through position encoding to obtain behavior features, converts the real-time task label into a semantic vector through word embedding, and calculates the cross attention between the behavior features and the real-time task label, and after processing through multiple Transformer blocks, uses a fully connected layer to output the behavior matching value.

[0029] As a further scheme of the present application, the initial scene picture is collected by the area collection module and transmitted to the simulation planning module, and the construction planning map is collected by the information collection module and transmitted to the simulation planning module.

[0030] As a further scheme of the present application, it further includes an abnormal behavior display module for receiving abnormal behavior of the target analyte and displaying the abnormal behavior on the terminal display device, and simultaneously, when the abnormal behavior display module receives the abnormal behavior, real-time sound and light reminding information is generated and the management personnel is reminded to reconfirm the abnormal behavior.

[0031] Compared with the prior art, the present application has the following advantages:

[0032] The present application can accurately construct a field simulation model and build a planning model according to the initial scene picture and the construction planning map, then accurately identify the work area in the planning model according to the construction task, set an electronic fence in the actual scene, and determine the real-time task label, through the way of accurate planning and flexible setting of the electronic fence, compared with the traditional fixed area setting of the electronic fence, it can better adapt to the dynamic changes of the construction task, greatly improving the pertinence and effectiveness of the electronic fence setting; then the behavior information in the whole construction area is comprehensively collected, the target analyte in the behavior information is identified and its electronic fence area is determined, the single image is extracted for behavior recognition processing, through the continuous monitoring of the real-time behavior action of the target analyte and the real-time matching with the real-time task label, the system can quickly find abnormal behavior and timely issue a warning, the present application can enable the construction management personnel to real-time understand the behavior dynamics in the construction area, timely find and handle abnormal situations, and avoid construction delay caused by untimely discovery of abnormal behavior; at the same time, the behavior data and analysis results provided by the system can also provide strong support for construction management decision-making, which is helpful for optimizing the construction process and reasonably arranging resources, so as to comprehensively improve the efficiency and quality of construction management. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The figure is a schematic diagram of the system structure of the present application. DETAILED DESCRIPTION

[0034] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments.

[0035] With reference to Figure 1 An AI electronic fence abnormal behavior identification system for UAV video, comprising an information collection module, a region collection module, a simulation planning module, a task publishing module, a fence setting module, a behavior collection module, a behavior identification module and an abnormal display module;

[0036] The information collection module is used for collecting the region information of the construction site, wherein the region information comprises a construction planning map of the construction site, and then the information collection module transmits the region information to the simulation planning module;

[0037] The region collection module collects the initial site picture of the target region based on the intelligent collection device in real time and transmits it to the simulation planning module, further, the initial site picture refers to the site picture of the target region before construction, and the target region refers to the region that needs to be constructed and planned, and when the initial site picture is collected in real time, the collection device is set as a UAV, that is, the UAV collects the video picture of the target region to obtain the initial site picture;

[0038] The simulation planning module sets the site simulation model of the target region based on the received initial site picture, then obtains the construction planning map, identifies the overall construction region and the local planning region in the construction planning map, and simulates and builds the overall construction region and the local planning region in the site simulation model, marks the built site model as a planning model, and transmits it to the fence setting module, wherein the overall construction region refers to the entire construction range of the engineering project, including all work regions from the preliminary preparation to the final delivery, for example, in the construction planning map of a commercial complex, the overall construction region includes all construction sites such as building main bodies, outdoor engineering and underground pipelines within the land red line, and the local planning region refers to the subdivided region according to the construction stage, functional division, professional type or spatial position in the overall construction region, that is, a specific implementation unit, for example, in the commercial complex project, the overall construction region can be divided into local planning regions such as "first-phase residential construction area", "second-phase commercial square construction area" and "underground garage construction area";

[0039] Further, the planning model is a three-dimensional model built by the simulation design software, in this embodiment, the Fusion 20 system is selected as the simulation design software;

[0040] The task publishing module is configured to transmit the daily construction task to the fence setting module. Further, when the target area starts the construction, different construction schedules correspond to different construction times, so the daily construction task is determined based on the construction schedule, and the daily construction task is transmitted to the fence setting module.

[0041] The fence setting module is configured to receive the daily construction task, and set the electronic fence according to the daily construction task. The specific setting method of the electronic fence includes:

[0042] The overall construction area is obtained. First, the comprehensive boundary is set based on the area range of the overall construction area.

[0043] Then, based on the received construction task, the local planning area under the current time is determined, and the area is marked as the working area. The planning model is obtained, the position of the working area in the planning model is identified, and the working boundary is set according to the area range of the position. At the same time, the corresponding electronic fence is set in the actual scene of the target area according to the working boundary and the comprehensive boundary.

[0044] The area surrounded by the comprehensive boundary is the overall construction area, and the area surrounded by the working boundary is the working area.

[0045] When the electronic fence is set, the construction task is obtained again, the specific work content in the working area in the construction task is identified, and the work content is taken as the real-time task label of the working area on the day.

[0046] Then, the one-way communication connection is established between the fence setting module and the behavior recognition module, and the working area and the real-time task label are transmitted to the behavior recognition module.

[0047] The behavior collection module collects the behavior information in the area range of the overall construction area by using the collection device, and transmits the behavior information to the behavior recognition module.

[0048] In this embodiment, the collection device is a drone device, that is, the drone collects the construction behavior information of the overall construction area. The behavior information is collected and transmitted in the form of video. Further, the construction behavior includes basic behavior (such as wearing safety equipment), mechanical operation behavior (such as the operation path of excavators and other earthmoving machinery), feature device behavior (such as whether there is an obstacle within the tower crane rotation radius), and the like.

[0049] The behavior recognition module is configured to receive the behavior information and analyze the behavior information to determine the behavior matching value. Further, the abnormal behavior is identified according to the behavior matching value. The specific determination method of the behavior matching value includes:

[0050] S1: identify the non-stationary object in the behavior information, when the non-stationary object is identified, mark the object as the target analyte, and then identify the area position where the target analyte is located, and determine the electronic fence area where the target analyte is located;

[0051] It should be further pointed out that when a plurality of non-stationary objects are identified in the behavior information, the identified non-stationary objects are sequentially marked as target analytes, and the behavior matching value calculation is sequentially performed on each target analyte. Further, the non-stationary object includes but is not limited to workers, mechanical equipment, and transport vehicles and the like;

[0052] According to the sampling time, the continuous video in the behavior information is disassembled into single-frame images, and the image position of the target analyte in each single-frame image is identified, and the target analyte is extracted separately to obtain a plurality of single images, wherein the image frame of the single image only contains the target analyte, thereby avoiding the interference of other background factors. The sampling time is a threshold value, and the specific value is set by the person skilled in the art according to the big data experience;

[0053] S2: arrange the single images in time sequence to obtain an image sequence, and use a feature recognition algorithm to extract features from the single images in the image sequence to obtain the behavior features of the target analyte. In this embodiment, the feature recognition algorithm is a 3D convolutional neural network algorithm, and the process of the 3D convolutional neural network algorithm for feature extraction is prior art, which will not be described here;

[0054] Based on the behavior features of the target analyte, the actual behavior action of the current target analyte is determined, the electronic fence area where the target analyte is located is identified, and the real-time task label of the electronic fence area is obtained. Then, the real-time behavior action and the real-time task label are subjected to behavior matching processing to obtain a behavior matching value;

[0055] Further, the behavior matching algorithm used in this embodiment is a Transformer architecture algorithm. The spatio-temporal features of the real-time behavior action are extracted by 3D CNN, and then the time sequence information is injected through position coding to obtain the behavior features. The real-time task label is converted into a semantic vector through word embedding, and the cross-attention between the behavior features and the real-time task label is calculated. After processing by multiple Transformer blocks, the behavior matching value is output by using a fully connected layer. The specific processing method of the Transformer architecture algorithm is prior art, which will not be described here;

[0056] S3: obtaining the behavior matching value of the target analyte, comparing the behavior matching value with a matching threshold value, if the behavior matching value is less than the matching threshold value, marking the actual behavior action of the target analyte as abnormal behavior, otherwise, if the behavior matching value is greater than or equal to the matching threshold value, marking the actual behavior action of the target analyte as normal behavior, wherein the specific value of the matching threshold value is obtained by a person skilled in the art through big data operation;

[0057] After the behavior recognition module generates abnormal behavior, the behavior recognition module transmits the abnormal behavior to the abnormal display module;

[0058] The abnormal display module is used for receiving the abnormal behavior of the target analyte and displaying the abnormal behavior on a terminal display device, and when the abnormal display module receives the abnormal behavior, real-time sound and light reminding information is generated and a manager is reminded to reconfirm the abnormal behavior.

[0059] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. An AI electronic fence abnormal behavior recognition system for drone videos, characterized by: include: The simulation planning module is used to set up a site simulation model based on the initial site image. At the same time, based on the construction planning map, the overall construction area and local planning areas are simulated and constructed in the site simulation model to obtain a planning model. The task release module is used to determine the construction tasks of the day according to the construction progress; The fence setting module is used to identify the work area in the planning model based on the construction tasks of the day, and set up electronic fences in the actual scene of the target area based on the regional scope of the work area. At the same time, according to the construction tasks, the real-time task label within each electronic fence is determined; Behavior collection module, used to collect behavior information within the entire construction area; The behavior recognition module is used to identify non-stationary objects in the behavior information and mark them as target analytes. At the same time, it determines the electronic fence area where the target analyte is located, extracts the target analyte from the behavior information separately, obtains a single image, performs behavior recognition processing on the single image, determines the real-time behavior action of the target analyte, matches the real-time behavior action with the real-time task label of the corresponding electronic fence area, obtains a behavior matching value, and determines the abnormal behavior of the target analyte based on the behavior matching value.

2. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 1 is characterized in that: The overall construction area refers to the entire construction scope of the project, including all work areas from preliminary preparation to final delivery. The local planning area refers to the subdivided area within the overall construction area according to the construction stage, functional zoning, professional type or spatial location.

3. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 1 is characterized in that: Methods for determining real-time task labels include: Obtain the overall construction area and first set a comprehensive dividing line based on the area scope of the overall construction area; Then, based on the received construction tasks, the local planned area for construction at the current time is determined and marked as the work area. The planning model is obtained, and the location of the work area is identified in the planning model. Based on the area range of this location, the work boundary is set. At the same time, corresponding electronic fences are set in the actual scene of the target area according to the work boundary and the comprehensive boundary. The area enclosed by the comprehensive demarcation line is the overall construction area, and the area enclosed by the working demarcation line is the work progress area; When the electronic fence is set up, the construction task is obtained again, the specific work content in the work area of ​​the construction task is identified, and this work content is used as the real-time task label of the work area on that day.

4. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 1 is characterized in that: The collection equipment uses drone equipment, which is used to collect construction behavior information of the entire construction area. The behavior information is collected and transmitted in video format.

5. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 1 is characterized in that: The methods for obtaining a single image include: Identifying non-stationary objects in the behavior information, and when a non-stationary object is identified, marking the object as a target analyte, and then identifying the location of the target analyte, and determining the electronic fence area where the target analyte is located; According to the sampling time, the continuous video in the behavioral information is disassembled into single-frame images, and the image position of the target analyte is identified in each single-frame image. At the same time, the target analyte is extracted separately to obtain several single images, in which only the target analyte exists in the image screen of the single image.

6. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 5 is characterized in that: When multiple non-stationary objects are identified in the behavior information, the identified non-stationary objects are marked as target analytes in sequence, and a behavior matching value is calculated for each target analyte in sequence.

7. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 1 is characterized in that: Methods for determining abnormal behavior include: Arrange the individual images in chronological order to obtain an image sequence, and use a 3D convolutional neural network algorithm to extract features from the individual images in the image sequence to obtain the behavioral characteristics of the target analyte; Based on the behavioral characteristics of the target analyte, the actual behavior of the current target analyte is determined, the electronic fence area where the target analyte is located is identified, and the real-time task label of this electronic fence area is obtained. The real-time behavior action is then matched with the real-time task label to obtain a behavior matching value; Obtain the behavior matching value of the target analyte and compare the behavior matching value with the matching threshold. If the behavior matching value is less than the matching threshold, the actual behavior action of the target analyte is marked as abnormal behavior. Conversely, if the behavior matching value is greater than or equal to the matching threshold, the actual behavior action of the target analyte is marked as normal behavior.

8. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 7 is characterized in that: The behavior matching algorithm uses the Transformer architecture algorithm. It extracts the spatiotemporal features of real-time behavior actions through 3D CNN, injects timing information through position encoding, and obtains behavior features. It then converts real-time task labels into semantic vectors through word embedding, calculates the cross-attention between behavior features and real-time task labels, and uses multi-layer Transformer blocks to process and output behavior matching values ​​using a fully connected layer.

9. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 1 is characterized in that: The initial on-site image is collected by the area acquisition module and transmitted to the simulation planning module. The construction planning map is collected by the information acquisition module and transmitted to the simulation planning module.

10. The AI ​​electronic fence abnormal behavior recognition system for drone videos according to claim 1 is characterized in that: It also includes an abnormal display module for receiving abnormal behavior of the target analyte and displaying the abnormal behavior on the terminal display device. At the same time, when the abnormal display module receives abnormal behavior, it generates sound and light reminder information in real time and reminds the management personnel to reconfirm the abnormal behavior.

Images