A dynamic intrusion detection method and system for construction human-machine safety
By identifying construction workers and engineering machinery at the construction site and utilizing target detection and trajectory prediction models, the problem of the limited number of safety officers at the construction site has been solved, enabling high-precision construction safety early warning around the clock and reducing the risk of human-machine collisions.
Patent Information
- Application Number
- CN202311247170.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-25
- Publication Date
- 2025-12-16
- Estimated Expiration
- 2043-09-25
AI Technical Summary
In existing construction organization and management technologies, the number and energy of safety officers at construction sites are limited, making it difficult to effectively identify whether construction workers have entered dangerous areas, resulting in insufficient safety early warning.
The system employs a target detection model to identify construction workers and engineering machinery, uses a tracking algorithm to delineate dangerous areas, and utilizes a construction worker trajectory prediction model for intrusion detection. It combines perspective transformation and ray casting to assess intrusion risk, thereby achieving automated, 24/7 safety early warning.
It improves the safety of construction sites, can provide timely warnings of construction personnel entering dangerous areas, reduces the risk of human-machine collisions, and achieves high-precision safety monitoring around the clock.
Smart Images
Figure CN117152211B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of artificial intelligence, in particular to a dynamic intrusion detection method and system for construction human-machine safety. BACKGROUND
[0002] With the rapid development of artificial intelligence, hardware such as GPU is constantly iterated, and computing power is gradually enhanced. Artificial intelligence technology is widely used in various fields of engineering. In civil engineering construction, there are also a large number of artificial intelligence algorithms, such as convolutional neural networks, recurrent neural networks, etc.
[0003] The safety of construction personnel has always been an important part of construction management. By identifying whether the construction personnel are in the dangerous area of the engineering machinery, timely warning and reducing casualties caused by human-machine collision can be achieved.
[0004] In the existing construction organization and management technology, the construction site safety officer usually controls it, but due to the limited number and energy of the safety officer, it is inevitable that there will be some omissions. Therefore, an automatic 24-hour identification method is needed to timely warn the construction personnel that they have entered the dangerous area of the construction machinery, so as to avoid casualties. SUMMARY
[0005] The purpose of the present application is to provide a dynamic intrusion detection method and system for construction human-machine safety, which significantly improves safety.
[0006] The purpose of the present application can be achieved by the following technical solutions:
[0007] A dynamic intrusion detection method for construction human-machine safety, comprising the following steps:
[0008] Real-time acquisition of construction site image data, wherein the construction site image data includes construction personnel and each type of engineering machinery;
[0009] Inputting the construction site image data into a pre-constructed target detection model to identify the construction personnel and each type of engineering machinery;
[0010] Using a tracking algorithm to track the trajectory of the construction personnel and each type of engineering machinery, and dividing the dangerous area of each type of engineering machinery in real time;
[0011] Inputting the tracking trajectory data of the construction personnel into a pre-constructed construction personnel trajectory prediction model to predict the action trajectory of the construction personnel;
[0012] Based on the predicted action trajectory of the construction personnel and the dangerous area, analysis and comparison are performed to obtain the intrusion detection result.
[0013] Further, the target detection model is trained using YOLO.
[0014] Further, the position of the identified construction personnel and each type of engineering machinery is transformed into a real coordinate system by using a perspective transformation, and trajectory tracking is performed in the real coordinate system.
[0015] Further, the expression of the perspective transformation is:
[0016]
[0017] HX=X'
[0018] In the formula, H is a transformation matrix, X is a coordinate in the camera coordinate system, and X' is a coordinate in the real coordinate system.
[0019] Further, the tracking algorithm is a DeepSort algorithm.
[0020] Further, the construction personnel trajectory prediction model is trained using an LSTM network model.
[0021] Further, the specific steps of obtaining the intrusion detection result include:
[0022] The dangerous area is geometrically approximated, the dangerous area is converted into a polygonal area, and the coordinates of each point of the polygon are recorded to form an ordered detection area;
[0023] The predicted construction personnel movement trajectory is discretized to obtain a to-be-detected trajectory point;
[0024] It is judged whether the to-be-detected trajectory point is in the to-be-detected area, and an intrusion detection result is obtained.
[0025] Further, the ray method is used to judge whether the to-be-detected trajectory point is in the to-be-detected area.
[0026] Further, based on the intrusion detection result, it is judged whether the construction personnel will be in the dangerous area, and if so, a warning is issued, and if not, no warning is issued.
[0027] The present application also provides a detection system based on the above-mentioned construction human-machine safety-oriented dynamic intrusion detection method, comprising:
[0028] A data acquisition module is used to acquire real-time construction site image data, and the construction site image data includes construction personnel and each type of engineering machinery image data;
[0029] An identification module is used to input the construction site image data into a pre-constructed target detection model to identify construction personnel and each type of engineering machinery.
[0030] A tracking module is configured to track the trajectories of the construction personnel and each type of engineering machinery by using a tracking algorithm, and divide the dangerous area of each type of engineering machinery in real time;
[0031] A trajectory prediction module is configured to input the tracking trajectory data of the construction personnel into a pre-constructed construction personnel trajectory prediction model, and predict the action trajectory of the construction personnel;
[0032] An intrusion detection module is configured to analyze and compare the predicted action trajectory of the construction personnel and the dangerous area, and obtain an intrusion detection result.
[0033] Compared with the prior art, the present application has the following beneficial effects:
[0034] (1) The present application uses a target detection model to identify dynamic construction personnel and each type of engineering machinery in the construction site, and automatically divides the dangerous area of the dynamic mechanical target, solving the problem of collision detection between people and machinery in a dynamic situation; through trajectory tracking and prediction, the personnel trajectory is short-term predicted, realizing the early prediction of collision, providing more buffer time for risk avoidance, thereby significantly improving the safety of the construction site.
[0035] (2) The present application can issue a warning to remind the construction personnel to pay attention to safety according to the judgment, further improving the safety of the construction operation.
[0036] (3) The present application does not need to use the traditional manual identification mode, but uses an automatic identification mode, having the advantages of all-weather, 24 hours, and identification precision not being reduced. BRIEF DESCRIPTION OF DRAWINGS
[0037] Figure 1 is a method flowchart of the present application;
[0038] Figure 2 is an engineering machinery dangerous area determination range schematic diagram of the present application;
[0039] Figure 3 is a detection system structure schematic diagram of the present application. DETAILED DESCRIPTION
[0040] The present application will be described in detail below in combination with the drawings and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present application, and gives a detailed implementation mode and specific operation process, but the protection scope of the present application is not limited to the following embodiments.
[0041] The present application provides a dynamic intrusion detection method for construction human-machine safety, as shown in Figure 1 The detection method comprises the following steps:
[0042] Step 1, real-time acquisition of construction site image data, which includes construction personnel and each type of engineering machinery.
[0043] Real-time collection of construction site images by fixed camera, in this embodiment, the camera height is 3-8m higher than the detected plane, and the camera pitch angle should be between ±45°. After fixing, start collecting construction site image data.
[0044] Step 2, input the images of the construction personnel and each type of engineering machinery into the pre-constructed target detection model to detect the construction personnel and engineering machinery in the image.
[0045] In the process of constructing the target detection model, this embodiment uses LabelImg to label 3000 pictures of construction personnel and 1000 pictures of each type of engineering machinery (including excavators, trucks, spreaders, forklifts, and tower cranes). For construction personnel, a rectangular box is used to mark the construction personnel; in order to more accurately identify the position of the main part of each type of key engineering machinery, the anchor box should be determined in the main part of the engineering machinery. For excavators, the anchor box should be determined in the cab and track area of the excavator, excluding the mechanical arm of the excavator; for trucks, the anchor box should be determined in the cab and cargo compartment of the truck; for spreaders, the anchor box should be determined in the cab and track area of the spreader, excluding the mechanical arm of the spreader; for forklifts, the anchor box should be determined in the cab of the forklift, excluding the working device of the forklift; for tower cranes, the anchor box should be determined in the cab of the tower crane, excluding the cantilever of the tower crane. Then the labeled images are trained using the YOLOv5 network model to obtain the final target detection model. In this embodiment, the construction site image data obtained in the previous step is input into the model, and the model can identify the personnel and machinery in the image.
[0046] Step 3, use tracking algorithm to track the trajectory of the construction personnel and each type of engineering machinery, and real-time division of the dangerous area of each type of engineering machinery.
[0047] Perspective transformation: the identified engineering machinery and construction personnel are converted to the real coordinate system through perspective transformation. For the center point of perspective transformation, for construction personnel, the midpoint of the anchor box bottom edge is used as the position point of the person; for engineering machinery, the midpoint of the anchor box bottom edge of the main part of the machinery is used as the position point of the machinery. The specific process of perspective transformation is as follows, where the transformation matrix is HH, the coordinates in the camera coordinate system are X, and the coordinates in the real coordinate system are X'. The transformation process is as follows:
[0048]
[0049] HX=X'
[0050] DeepSort tracking: adopt DeepSort algorithm to track the trajectory of the person, so as to obtain the motion trajectory of the person. Preferably, the motion trajectory of the person adopts the trajectory table of the midpoint of the bottom side of the person anchor frame in each frame of the video.
[0051] Determine the dangerous area of the engineering machinery: determine the corresponding dangerous area of the engineering machinery in the real coordinate system according to the center position point of the engineering machinery. As shown in the figure, wherein the dangerous area of the engineering machinery is determined by the following method: for the excavator, the dangerous area is a circular area with the arm support point as the center and the arm working length as the radius; for the truck, the dangerous area is the blind area of the truck's field of view; for the spreader, the dangerous area is a circular area with the arm support point as the center and the arm working length as the radius; for the forklift, the dangerous area is a fan-shaped area with the front wheel of the forklift as the center, the working device length as the radius, and 60° on the left and right, and a rectangular area behind; for the tower crane, the dangerous area is a circular area with the anchor frame center point as the center and the cantilever length as the radius. Figure 2
[0052] Step 4, input the tracking trajectory data of the construction personnel into the pre-constructed construction personnel trajectory prediction model, and predict the action trajectory of the construction personnel.
[0053] The specific steps of this step for constructing the construction personnel trajectory prediction model are:
[0054] Data collection: shoot the construction personnel's on-site walking video, wherein the camera shooting requirements are the same as those in step 1. Among them, each shooting video should be more than 10s, and not less than 100 videos.
[0055] Data preprocessing: the trajectory data processing steps include perspective transformation, continuous segment cutting, sequence length screening, standardization, and rolling segmentation. Take the midpoint of the bottom side of the pedestrian anchor frame as the pedestrian trajectory, and transform the trajectory to the real coordinate system through perspective transformation to form a plurality of pedestrian trajectory sequences. Further, the position sequence length for training is at least 16, and for each trajectory data, it is normalized by using the maximum and minimum value method. Further, taking 15 as the input sequence length and 1 as the output sequence length, the original data set is divided into training set and test set by rolling segmentation.
[0056] Model training: adopt LSTM network architecture and use the processed data for training.
[0057] After training, the construction personnel trajectory prediction model is constructed,
[0058] According to the above prediction model and the iterative use of the trajectory prediction network, the trajectory of the construction personnel in the next 1-2s is predicted.
[0059] Step 5, based on the predicted construction personnel action trajectory and the dangerous area, analysis comparison is carried out to obtain the intrusion detection result.
[0060] Region approximation: first, in the real coordinate system, if the dangerous area of the construction machinery is a region surrounded by a curve such as a circle or an ellipse, the region is discretized and its edge is approximated to a polygon. If the dangerous area is a polygonal region such as a rectangle or a square, no measures are taken. Further, the coordinates of each point of the polygon are recorded in the form of an ordered point set as the to-be-detected region.
[0061] Personnel trajectory discretization: according to the current position of the detected personnel and the trajectory obtained in the trajectory prediction module for the future 1-2s, 10 trajectory points are extracted therefrom at equal intervals as the to-be-detected points.
[0062] Intrusion detection: for each to-be-detected point and to-be-detected region, the ray method is used to determine whether the point is in the to-be-detected region and to give a warning.
[0063] Embodiment 2
[0064] This embodiment provides a dynamic intrusion detection system for construction human-machine safety, as shown in Figure 3 The system comprises:
[0065] A data acquisition module for acquiring real-time construction site image data, wherein the construction site image data comprises image data of construction personnel and each type of engineering machinery;
[0066] An identification module for inputting the construction site image data into a pre-constructed target detection model to identify the construction personnel and each type of engineering machinery;
[0067] A tracking module for tracking the trajectory of the construction personnel and each type of engineering machinery using a tracking algorithm and dividing the dangerous area of each type of engineering machinery in real time;
[0068] A trajectory prediction module for inputting the tracking trajectory data of the construction personnel into a pre-constructed construction personnel trajectory prediction model to predict the action trajectory of the construction personnel;
[0069] An intrusion detection module for analyzing and comparing based on the predicted action trajectory of the construction personnel and the dangerous area to obtain an intrusion detection result.
[0070] The rest is as shown in Embodiment 1.
[0071] If the above functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or partially contribute to the prior art, or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.
[0072] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.
[0073] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks
[0074] These computer program instructions can also be stored in a computer readable storage medium that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction devices that implement the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks
[0075] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operations steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide the function of realizing the processes specified in the flowcharts Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or steps of the functions specified in the flowcharts
[0076] Although the preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims be construed to include all such modifications and variations as fall within the scope of the application.
[0077] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.
Claims
1. A method for construction-oriented human-machine safety dynamic intrusion detection, characterized in that, Comprise the following steps: Real-time acquisition of construction site image data, the construction site image data including construction personnel and each kind of engineering machinery; The construction site image data is input into the target detection model constructed in advance, and the construction personnel and each kind of engineering machinery are identified; Trajectory tracking of the construction personnel and each kind of engineering machinery is carried out by using a tracking algorithm, and the dangerous area of each kind of engineering machinery is divided in real time, specifically comprising: Perspective transformation: the recognized construction machinery and construction personnel are converted into the real coordinate system through perspective transformation; for the center point of perspective transformation, the midpoint of the bottom edge of the anchor frame is used as the position point of the person; for the construction machinery, the midpoint of the bottom edge of the anchor frame of the main part of the machinery is used as the position point of the machinery; the specific process of perspective transformation is as follows, wherein the transformation matrix is H, the coordinates in the camera coordinate system are X, and the coordinates in the real coordinate system are The transformation process is as follows: DeepSort tracking: the person's trajectory is tracked by using DeepSort algorithm, so as to obtain the person's motion trajectory, wherein the person's motion trajectory adopts the trajectory table of the bottom edge midpoint of the person anchor frame in each frame of video; Determination of engineering machinery dangerous area: the corresponding dangerous area of engineering machinery in the real coordinate system is determined according to the center position point of engineering machinery, wherein the dangerous area of engineering machinery is determined by the following method: for excavator, its dangerous area is a circular area with the mechanical arm fulcrum as the center and the mechanical arm working length as the radius; for truck, its dangerous area is the visual blind area of truck; for cloth machine, its dangerous area is a circular area with the mechanical arm fulcrum as the center and the mechanical arm working length as the radius; for forklift, its dangerous area is a fan-shaped area with the front wheel of forklift as the center, the working device length as the radius, and 60° on the left and right and a rectangular area behind; for tower crane, its dangerous area is a circular area with the anchor frame center point as the center and the cantilever length as the radius; The tracking trajectory data of the construction personnel is input into the construction personnel trajectory prediction model constructed in advance, and the action trajectory of the construction personnel is predicted; Based on the predicted action trajectory of the construction personnel and the dangerous area, analysis and comparison are carried out to obtain intrusion detection results, and the specific steps of obtaining the intrusion detection results comprise: Geometric approximation processing is carried out on the dangerous area, the dangerous area is converted into a polygon area, and the coordinates of each point of the polygon are recorded to form an ordered detection area; The predicted action trajectory of the construction personnel is discretized to obtain a to-be-detected trajectory point; It is judged whether the to-be-detected trajectory point is in the detection area or not to obtain the intrusion detection result.
2. The method of claim 1, wherein the method is a construction human safety-oriented dynamic intrusion detection method. The target detection model is trained by using YOLO.
3. The method of claim 1, wherein the method is a construction human safety-oriented dynamic intrusion detection method. It also includes using perspective transformation to transform the positions of the identified construction personnel and each kind of engineering machinery into a real coordinate system, and tracking the trajectories in the real coordinate system.
4. The method of claim 1, wherein the method is a construction human safety-oriented dynamic intrusion detection method. The construction personnel trajectory prediction model is trained by using LSTM network model.
5. The method of claim 1, wherein the method is a construction human safety-oriented dynamic intrusion detection method. Ray method is used to judge whether the to-be-detected trajectory point is in the detection area or not.
6. The method of claim 1, wherein the method is a construction human safety oriented dynamic intrusion detection method. Based on the intrusion detection result, it is judged whether the construction personnel will be in the dangerous area or not, if yes, a warning is given, and if not, no warning is given.
7. A detection system based on the construction-oriented human safety dynamic intrusion detection method according to any one of claims 1 to 6, characterized in that, Comprise: Data acquisition module: used for real-time acquisition of construction site image data, the construction site image data including construction personnel and each kind of engineering machinery image data; Identification module: for inputting the construction site image data into the target detection model constructed in advance, and identifying the construction personnel and each kind of engineering machinery; Tracking module: for carrying out trajectory tracking of the construction personnel and each kind of engineering machinery by using a tracking algorithm, and dividing the dangerous area of each kind of engineering machinery in real time; The trajectory prediction module is configured to input the tracking trajectory data of the construction personnel into a pre-constructed trajectory prediction model of the construction personnel, and predict the action trajectory of the construction personnel. The intrusion detection module is configured to analyze and compare the predicted action trajectory of the construction personnel and the dangerous area, and obtain an intrusion detection result.