Engineering vehicle and personnel identification system based on edge computing gateway

Abstract

The invention discloses an engineering vehicle and personnel identification system based on an edge computing gateway, which comprises a video acquisition module, an edge computing gateway, an alarm module and a back-end management platform, and is characterized in that the video acquisition module transmits acquired video images to the edge computing gateway, the edge computing gateway is connected with the video acquisition module, and the back-end management platform is connected with the alarm module. The edge computing gateway is transplanted with an engineering vehicle identification algorithm and a personnel identification algorithm, and the alarm module triggers an alarm when the edge computing gateway detects an abnormal condition. Video image processing is carried out in the edge computing gateway, only alarm video images are transmitted to the rear end, the data transmission quantity and delay are greatly reduced, the real-time performance of the system is improved, a large amount of video image data can be rapidly processed by utilizing the strong computing power of the edge box, and the real-time performance of the system is improved. The efficiency of engineering vehicle and personnel identification is improved, the edge computing gateway can not be influenced by factors such as network interruption, and the reliability and stability of the system are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to an engineering vehicle and personnel identification system based on an edge computing gateway. Background Art

[0002] In scenarios such as construction sites, effective identification and monitoring of construction vehicles and personnel are crucial. Traditional monitoring methods often rely on back-end servers to process video images, which have problems such as large data transmission volume, high latency, and high network bandwidth requirements. To this end, we propose an engineering vehicle and personnel identification system based on an edge computing gateway. Summary of the invention

[0003] The purpose of the present invention is to provide an engineering vehicle and personnel identification system based on an edge computing gateway to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an engineering vehicle and personnel identification system based on edge computing gateway, comprising a video acquisition module, an edge computing gateway, an alarm module and a backend management platform;

[0005] The video acquisition module is used to acquire video images of the construction site in real time, and the video acquisition module transmits the acquired video images to the edge-to-edge computing gateway;

[0006] The edge computing gateway is connected to the video acquisition module. The edge computing gateway is transplanted with an engineering vehicle recognition algorithm and a personnel recognition algorithm, which is used to process the received construction site video images and identify the type, location, driving status of the engineering vehicles and the number, location and activity status of the construction personnel in real time;

[0007] The alarm module is connected to the edge computing gateway. When the edge computing gateway detects an abnormal situation, the alarm module triggers an alarm, sends an alarm signal to the on-site personnel, and transmits the video image related to the alarm to the back-end management platform;

[0008] The back-end management platform is used to receive, store and analyze alarm video images from the edge computing gateway. Managers can view on-site conditions and query historical alarm records through the back-end management platform.

[0009] Preferably, the video acquisition module includes at least one high-definition camera for capturing video images of the construction site.

[0010] Preferably, the edge computing gateway is used to process video images in real time on site and supports multi-task parallel computing for the operation of engineering vehicle recognition algorithms and personnel recognition algorithms.

[0011] Preferably, the engineering vehicle identification algorithm includes:

[0012] Object detection uses deep learning models to detect vehicles in video images. With a large amount of engineering vehicle data in the training set, it can identify various types of engineering vehicles, including but not limited to excavators, bulldozers, loaders, and trucks.

[0013] Vehicle classification: classify different types of vehicles. Through feature extraction and pattern recognition, different types of engineering vehicles can be distinguished;

[0014] Vehicle tracking: using multi-target tracking algorithms, the moving trajectory of engineering vehicles can be continuously tracked in the video stream to monitor their driving status, speed and driving path;

[0015] Status analysis is used to analyze the operating status of engineering vehicles, including whether the vehicle is in working condition, whether it is working in the designated area, and whether there are any illegal operations.

[0016] Preferably, the deep learning model works smoothly including:

[0017] Input Image,The object detection model first receives an input image or video frame;

[0018] Feature extraction, the convolution layer processes the image or video to extract low-level features and high-level features. The calculation formula is: Where y(i, j) is the pixel value of the output feature map, x(i+m, j+n) is the pixel value of the input image, w(m, n) is the element of the convolution kernel, and b is the bias term;

[0019] Predict candidate boxes. The model predicts several regions containing objects in the image or video, each of which has a bounding box.

[0020] Classification and bounding box regression. For each predicted region, the model outputs both the object category and the bounding box position correction.

[0021] Post-processing,Through the post-processing step, the model can filter out overlapping,bounding boxes and retain the best detection results.

[0022] Preferably, the personnel identification algorithm includes:

[0023] Human detection, which uses a deep learning-like human detection model to identify people in video images, quickly and accurately detect people on the construction site, and identify their location and outline;

[0024] Posture estimation: Through the posture estimation algorithm, the specific actions and activity status of the personnel are analyzed and identified to determine whether they are in a working state or have abnormal postures;

[0025] Personnel tracking: using multi-target tracking technology, each identified person is numbered and their movement trajectory in the video is tracked, their position changes and activity paths are recorded, and whether the person enters a dangerous area or a violation area is analyzed;

[0026] Density detection counts the number of people on the construction site, analyzes the density of people, determines whether there is excessive gathering of people, and then triggers an alarm.

[0027] Preferably, the alarm module can send out corresponding alarm signals according to different types of abnormal situations, including but not limited to illegal operation of engineering vehicles, entry of personnel into dangerous areas and excessive gathering of personnel.

[0028] Preferably, the alarm signal includes a sound alarm, a light alarm and a smoke alarm, and the alarm video image can be transmitted to a back-end management platform via a network.

[0029] Preferably, the backend management platform has a real-time monitoring function, allowing managers to view the operating status of the construction site in real time through remote devices, and the data transmission between the edge gateway and the backend management platform adopts an encrypted transmission protocol.

[0030] Preferably, the operating method of the system includes:

[0031] Install video acquisition modules and high-definition cameras at key locations on the construction site, covering the activity areas of construction vehicles and personnel;

[0032] Configure the edge computing gateway, transplant the engineering vehicle recognition algorithm and personnel recognition algorithm into the edge computing gateway, perform parameter setting and optimization, and communicate with the video acquisition module and the back-end management platform;

[0033] Connect the alarm module to the edge computing gateway and issue an alarm in time when an abnormal situation is detected;

[0034] Build a backend management platform to receive alarm video images transmitted by the edge computing gateway and store, analyze and manage them;

[0035] System debugging and optimization: debug and optimize the entire system.

[0036] Technical effects and advantages of the present invention:

[0037] (1) The present invention processes video images in the edge computing gateway and only transmits the alarm video images to the back end, which greatly reduces the data transmission volume and delay, improves the real-time performance of the system, and utilizes the powerful computing power of the edge box to quickly process a large amount of video image data, thereby improving the efficiency of engineering vehicle and personnel identification. The edge computing gateway can process video images in an offline state and is not affected by factors such as network interruption, thereby enhancing the reliability and stability of the system.

[0038] (2) The present invention utilizes a back-end management platform to centrally manage and analyze alarm video images, which is convenient for management personnel to timely understand the situation at the construction site and take corresponding measures. At the same time, the system maintenance is also more convenient, and the algorithms in the edge computing gateway can be updated and optimized through remote upgrades and other methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The present invention provides Figure 1 An engineering vehicle and personnel identification system based on an edge computing gateway is shown, comprising a video acquisition module, an edge computing gateway, an alarm module and a backend management platform;

[0042] The video acquisition module is used to collect video images of the construction site in real time. The video acquisition module transmits the collected video images to the edge-to-edge computing gateway;

[0043] The edge computing gateway is connected to the video acquisition module. The edge computing gateway is transplanted with engineering vehicle recognition algorithms and personnel recognition algorithms to process the received construction site video images, and identify the type, location, driving status of engineering vehicles and the number, location and activity status of construction personnel in real time. By processing the video images in the edge computing gateway, only the alarm video images are transmitted to the back end, which greatly reduces the data transmission volume and delay, improves the real-time performance of the system, and utilizes the powerful computing power of the edge box to quickly process a large amount of video image data, improves the efficiency of engineering vehicle and personnel recognition, and the edge computing gateway can process video images offline, which is not affected by factors such as network interruption, and enhances the reliability and stability of the system.

[0044] The alarm module is connected to the edge computing gateway. When the edge computing gateway detects an abnormal situation, the alarm module triggers an alarm, sends an alarm signal to the on-site personnel, and transmits the video images related to the alarm to the back-end management platform;

[0045] The back-end management platform is used to receive, store and analyze the alarm video images from the edge computing gateway. Managers can view the on-site situation and query historical alarm records through the back-end management platform. The back-end management platform can centrally manage and analyze the alarm video images, so that managers can timely understand the situation on the construction site and take corresponding measures. At the same time, the system maintenance is more convenient. The algorithms in the edge computing gateway can be updated and optimized through remote upgrades. In actual applications, the system parameters can be adjusted and optimized according to different construction site requirements. For example, the parameters of the engineering vehicle recognition algorithm and the personnel recognition algorithm can be adjusted to improve the accuracy and efficiency of recognition; the triggering conditions of the alarm module can be adjusted to ensure that the alarm is issued only when a real dangerous situation occurs to avoid false alarms.

[0046] The video acquisition module includes at least one high-definition camera for capturing video images of the construction site.

[0047] Among them, the edge computing gateway is used to process video images in real time on site and supports multi-task parallel computing. It is used for the operation of engineering vehicle recognition algorithms and personnel recognition algorithms. The edge computing gateway is equipped with a high-performance processor.

[0048] Among them, the engineering vehicle recognition algorithm includes:

[0049] Target detection uses deep learning models to detect vehicles in video images. Through the large amount of engineering vehicle data in the training set, it can identify various types of engineering vehicles, including but not limited to excavators, bulldozers, loaders and trucks; vehicle classification classifies different types of vehicles. Through feature extraction and pattern recognition, different types of engineering vehicles can be distinguished; vehicle tracking uses multi-target tracking algorithms to continuously track the movement trajectory of engineering vehicles in video streams, monitor their driving status, speed and driving path; state analysis is used to analyze the operating status of engineering vehicles, including whether the vehicle is in working condition, whether it is working in a designated area, and whether there are any illegal operations.

[0050] The ability to identify various types of engineering vehicles in target detection is directly performed during the target detection process. The main task of the target detection algorithm is to detect the bounding box of the target object from the image or video and identify the type of the target object at the same time. This type recognition is one of the outputs of the target detection network. It means that after the object is detected, the output of the model is directly used to predict that the object belongs to a certain category; while vehicle classification is to further classify these targets after the target is detected. This is usually performed after target detection and can be regarded as a post-processing step. More complex classification models can also be embedded in the target detection model for refined recognition. This classification can be more detailed and can even identify the brand, model or status of the vehicle. The accuracy and refinement of the classification depend on the task requirements; type recognition in target detection is a preliminary recognition process, the purpose of which is to roughly identify the category of the object while detecting the object. For example, when detecting a vehicle, it is marked as a "vehicle" or a more rough "engineering vehicle". Classification in vehicle classification: This is a refined classification step that can be performed after target detection or in combination with target detection to classify the detected objects more accurately and meticulously. For example, distinguish whether the vehicle is a "excavator" or a "bulldozer".

[0051] Among them, the key features of deep learning models include convolutional neural networks, end-to-end training, and nonlinear activation functions. Convolutional neural networks are the basis of target detection models and are particularly good at processing image data. Convolutional neural networks extract spatial features of images through convolutional layers, reduce data dimensions through pooling layers to speed up calculations, and complete classification and regression tasks through fully connected layers. In target detection, convolutional neural networks are responsible for extracting features for classification and positioning from input images. Deep learning models are trained with a large number of annotated images and learn end-to-end how to directly detect and classify the bounding boxes of objects from images. These models use activation functions to enable the network to capture complex nonlinear relationships in images and identify objects of different categories. Deep learning models are trained with large-scale engineering vehicle data sets. The model can automatically learn the detailed features of various vehicles and accurately identify the vehicle type without manually designing features. The model can automatically extract features from images. Deep learning models can effectively detect vehicles in complex construction scenes and handle occlusion and vehicle perspectives at different angles. The smooth working of deep learning models includes:

[0052] Input image, the target detection model first receives the input image or video frame; feature extraction, the convolution layer processes the image or video to extract low-level features (such as edges and colors, etc.) and high-level features (such as the shape and contour of the vehicle). The calculation formula is: Where y(i, j) is the pixel value of the output feature map, x(i+m, j+n) is the pixel value of the input image, w(m, n) is the element of the convolution kernel, and b is the bias term. The convolution moves on the image in a sliding window manner to extract local spatial features; predict candidate boxes. The model predicts several areas containing objects in the image or video, and each area has a bounding box; classification and bounding box regression. For each predicted area, the model will simultaneously output the object category and the correction of the bounding box position; post-processing. Through the post-processing step, the model can filter out overlapping bounding boxes and retain the best detection results. Deep learning models perform well in target detection tasks. They automatically extract image features through large-scale convolutional neural networks, and detect and identify construction vehicles in video images in real time or with high accuracy. Through training, the model can accurately identify different types of vehicles and their activity status in complex construction environments.

[0053] Among them, the personnel recognition algorithm includes:

[0054] Human detection, which uses a deep learning-like human detection model to identify people in video images, quickly and accurately detect people on the construction site, and identify their location and outline;

[0055] Posture estimation: Through the posture estimation algorithm, the specific actions and activity status of the personnel are analyzed and identified to determine whether they are in a working state or have abnormal postures;

[0056] Personnel tracking: using multi-target tracking technology, each identified person is numbered and their movement trajectory in the video is tracked, their position changes and activity paths are recorded, and whether the person enters a dangerous area or a violation area is analyzed;

[0057] Density detection counts the number of people on the construction site, analyzes the density of people, determines whether there is excessive gathering of people, and then triggers an alarm.

[0058] The posture estimation algorithm refers to identifying the position of the joints of the human body through image or video analysis, and then inferring the posture and movement of the human body. It is mainly used to analyze the specific movements and activity states of people, such as standing, walking, falling, etc. These algorithms usually use deep learning models, are trained through a large amount of labeled data, and are combined with image processing technology to locate the key points of the human body. Common posture estimation algorithms include OpenPose, which can detect multiple key points of the human body (such as head, shoulders, elbows, wrists, knees, etc.) from the image, and then reconstruct the 2D or 3D posture of the human body. The core of OpenPose is the joint point detection of the human skeleton model. It predicts the position of each joint point in the image (such as shoulders, knees, etc.). OpenPose introduces the concept of PAF to estimate the association between human joint points. PAF is a two-dimensional vector field that describes the connection relationship between two joint points. Through PAF, the skeleton structure of the human body can be reconstructed. You can also use the posture estimation algorithm of HRNe, which is good at maintaining the high-resolution features of the image, thereby achieving fine joint point detection. Compared with other algorithms that gradually reduce the resolution and then upsample, HRNet maintains high-resolution representation throughout the process and can capture more posture details. HRNet maintains high resolution of the input image and continuously updates representations at different resolutions through multi-scale feature fusion, making joint detection more accurate. HRNet processes feature maps of different resolutions through parallel network branches and fuses these feature maps in each stage to enhance the detection capability of key points of the human body.

[0059] Regardless of the posture estimation algorithm used, the process is generally as follows: input image preprocessing, standardizing and resizing the input image or video frame, and using it as the input of the deep learning model; feature extraction, using convolutional neural networks to extract the spatial features of the image; joint point prediction, predicting the position of the joint points by means of heat maps or vector fields, directly predicting the coordinate position of the joints; posture inference, reconstructing the human skeleton model based on the detected key points, analyzing the specific movements and postures of the person; post-processing, converting the results of posture estimation into motion analysis to determine the activity status of the person. The posture estimation algorithm analyzes the joints of the human body through complex deep learning models to infer the posture and movements of the human body. Different algorithms use different technical means, such as heat maps, regression models, etc., to accurately predict the position of human joints. In the engineering vehicle and personnel recognition system, posture evaluation can be used to monitor the working status of workers on the construction site and detect abnormal postures, providing strong support for improving site safety.

[0060] Among them, the alarm module can send out corresponding alarm signals according to different types of abnormal situations, including but not limited to illegal operation of engineering vehicles, people entering dangerous areas and excessive gathering of people.

[0061] Among them, the alarm signals include sound alarms, light alarms and smoke alarms, and the alarm video images can be transmitted to the back-end management platform through the network.

[0062] Among them, the background management platform has a real-time monitoring function, allowing managers to view the operating status of the construction site in real time through remote devices. The data transmission between the edge gateway and the back-end management platform adopts an encrypted transmission protocol.

[0063] The system operation method includes:

[0064] Install video acquisition modules and high-definition cameras at key locations on the construction site, covering the activity areas of construction vehicles and personnel;

[0065] Configure the edge computing gateway, transplant the engineering vehicle recognition algorithm and personnel recognition algorithm into the edge computing gateway, perform parameter setting and optimization, and communicate with the video acquisition module and the back-end management platform;

[0066] Connect the alarm module to the edge computing gateway and issue an alarm in time when an abnormal situation is detected;

[0067] Build a backend management platform to receive alarm video images transmitted by the edge computing gateway and store, analyze and manage them;

[0068] System debugging and optimization: debug and optimize the entire system.

[0069] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An engineering vehicle and personnel identification system based on edge computing gateway, characterized in that: Includes video acquisition module, edge computing gateway, alarm module and backend management platform; The video acquisition module is used to acquire video images of the construction site in real time, and the video acquisition module transmits the acquired video images to the edge-to-edge computing gateway; The edge computing gateway is connected to the video acquisition module. The edge computing gateway is transplanted with an engineering vehicle recognition algorithm and a personnel recognition algorithm, which is used to process the received construction site video images and identify the type, location, driving status of the engineering vehicles and the number, location and activity status of the construction personnel in real time; The alarm module is connected to the edge computing gateway. When the edge computing gateway detects an abnormal situation, the alarm module triggers an alarm, sends an alarm signal to the on-site personnel, and transmits the video image related to the alarm to the back-end management platform; The back-end management platform is used to receive, store and analyze alarm video images from the edge computing gateway. Managers can view on-site conditions and query historical alarm records through the back-end management platform.

2. According to the edge computing gateway-based engineering vehicle and personnel identification system of claim 1, it is characterized in that: The video acquisition module includes at least one high-definition camera for capturing video images of the construction site.

3. According to the edge computing gateway-based engineering vehicle and personnel identification system of claim 1, it is characterized in that: The edge computing gateway is used to process video images in real time on site and supports multi-task parallel computing for the operation of engineering vehicle recognition algorithms and personnel recognition algorithms.

4. According to claim 3, an engineering vehicle and personnel identification system based on edge computing gateway is characterized in that: The engineering vehicle identification algorithm includes: Object detection uses deep learning models to detect vehicles in video images. With a large amount of engineering vehicle data in the training set, it can identify various types of engineering vehicles, including but not limited to excavators, bulldozers, loaders, and trucks. Vehicle classification: classify different types of vehicles. Through feature extraction and pattern recognition, different types of engineering vehicles can be distinguished; Vehicle tracking: using multi-target tracking algorithms, the moving trajectory of engineering vehicles can be continuously tracked in the video stream to monitor their driving status, speed and driving path; Status analysis is used to analyze the operating status of engineering vehicles, including whether the vehicle is in working condition, whether it is working in the designated area, and whether there are any illegal operations.

5. According to claim 4, an engineering vehicle and personnel identification system based on edge computing gateway is characterized in that: The working flow of the deep learning model includes: Input Image,The object detection model first receives an input image or video frame; Feature extraction, the convolution layer processes the image or video to extract low-level features and high-level features. The calculation formula is: Where y(i, j) is the pixel value of the output feature map, x(i+m, j+n) is the pixel value of the input image, w(m, n) is the element of the convolution kernel, and b is the bias term; Predict candidate boxes. The model predicts several regions containing objects in the image or video, each of which has a bounding box. Classification and bounding box regression. For each predicted region, the model outputs both the object category and the bounding box position correction. Post-processing,Through the post-processing step, the model can filter out overlapping,bounding boxes and retain the best detection results.

6. The engineering vehicle and personnel identification system based on edge computing gateway according to claim 1 is characterized in that: The personnel identification algorithm includes: Human detection, which uses a deep learning-like human detection model to identify people in video images, quickly and accurately detect people on the construction site, and identify their location and outline; Posture estimation: Through the posture estimation algorithm, the specific actions and activity status of the personnel are analyzed and identified to determine whether they are in a working state or have abnormal postures; Personnel tracking: using multi-target tracking technology, each identified person is numbered, and their movement trajectory in the video is tracked, their position changes and activity paths are recorded, and whether the person enters a dangerous area or a violation area is analyzed; Density detection counts the number of people on the construction site, analyzes the density of people, determines whether there is excessive gathering of people, and then triggers an alarm.

7. The engineering vehicle and personnel identification system based on edge computing gateway according to claim 1 is characterized in that: The alarm module can send out corresponding alarm signals according to different types of abnormal situations, including but not limited to illegal operation of engineering vehicles, entry of personnel into dangerous areas and excessive gathering of personnel.

8. The engineering vehicle and personnel identification system based on edge computing gateway according to claim 7 is characterized in that: The alarm signals include sound alarms, light alarms and smoke alarms, and the alarm video images can be transmitted to the back-end management platform through the network.

9. The engineering vehicle and personnel identification system based on edge computing gateway according to claim 1 is characterized in that: The backend management platform has a real-time monitoring function, allowing managers to view the operating status of the construction site in real time through remote devices. The data transmission between the edge gateway and the backend management platform adopts an encrypted transmission protocol.

10. The engineering vehicle and personnel identification system based on edge computing gateway according to claim 1, characterized in that: The method of operating the system includes: Install video acquisition modules and high-definition cameras at key locations on the construction site, covering the activity areas of construction vehicles and personnel; Configure the edge computing gateway, transplant the engineering vehicle recognition algorithm and personnel recognition algorithm into the edge computing gateway, perform parameter setting and optimization, and communicate with the video acquisition module and the back-end management platform; Connect the alarm module to the edge computing gateway and issue an alarm in time when an abnormal situation is detected; Build a backend management platform to receive alarm video images transmitted by the edge computing gateway and store, analyze and manage them; System debugging and optimization: debug and optimize the entire system.

Images