Construction work safety intelligent monitoring system and method based on AI recognition algorithm

Abstract

The application discloses a construction operation safety intelligent monitoring system and method based on an AI recognition algorithm, and relates to the technical field of operation monitoring.The application comprises the following steps: acquiring defect image samples and non-defect image samples, being responsible for defect classification and target feature library training, acquiring original images, carrying out image pre-processing, comparing the feature library, outputting processed images, boxing the detected target with an outer-line rectangle, verifying the identity information of a terminal and an operation worker's work card, verifying the identity information of the operation worker by a camera based on AI recognition analysis, and verifying the operation worker by cross authentication of the terminal and the camera.The application realizes multi-target real-time intelligent analysis and recognition of non-digital vernier calipers, ground resistance oscillation tables, test pens, anti-falling devices and safety cone barrels by fusing computer vision and artificial intelligence, meets the needs of three-dimensional monitoring, operation and service in the industrial field, and improves the intelligent recognition capability of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of operation supervision, in particular to an AI recognition algorithm-based construction operation safety intelligent supervision system and method. BACKGROUND

[0002] With the continuous progress of image and video processing, analysis, transmission technology and artificial intelligence technology, the monitoring system develops from pure analog system to analog-digital combination and pure IP monitoring mode, and constantly moves towards high definition and networking. The intelligent analysis demand of the monitoring system also emerges as the times require, that is, intelligence. The demand of the intelligent recognition market comes from various actual needs of specific industry characteristic monitoring. Each industry monitoring may need several kinds of intelligent monitoring technology. The individualized demand of the subdivided market determines that we must pay close attention to the particularity of different systems, take marketization as the orientation of the whole company, and closely cooperate and cooperate between the sales, market, technology, research and development, product, etc. departments to provide customized products and services that meet the unique needs of customers in various industries, provide full high-definition video stream and intelligent intelligent recognition system and platform.

[0003] The existing intelligent monitoring system mainly has the following two technical schemes:

[0004] 1. Target extraction and event analysis technology based on rules. This scheme extracts and detects the target in the video picture through picture segmentation, foreground extraction and other methods, and then distinguishes different events based on preset rules to realize judgment and alarm linkage. Its principle is to separate the foreground and background through image segmentation algorithm, the foreground is the moving target, and set specific rules such as "crossing" and "regional intrusion" behavior judgment conditions. When the target triggers the rule, the alarm is triggered. It can be applied to early behavior analysis or traffic event detection. This scheme depends on fixed rules, and is effective for single event recognition of simple scene, but has poor flexibility and is difficult to analyze complex environment or multi-feature target;

[0005] 2. Specific object detection technology based on pattern recognition. This scheme uses pattern recognition technology to model specific objects in the picture, trains the model through a large number of samples, and realizes the detection and recognition of specific objects. Its principle is to construct a feature model of a specific object based on machine learning, train the model through a large number of labeled samples, so that the model has the ability to locate and recognize the target from the video or image. It can be applied to personnel detection and specific object defect recognition. This scheme can realize high-precision recognition of specific objects, but relies on a large number of samples for training, has limited generalization ability, and focuses on single object recognition, lacking comprehensive analysis ability for multiple targets and multiple features.

[0006] The prior art scheme has the following disadvantages: 1. mainly using a camera as a collection source, lacking real-time intelligent analysis capability for target features, and lacking real-time and intelligent recognition capability; 2. image recognition technology is usually based on basic management applications, lacking application technology for different industry business scenarios; 3. most of the snake-shaped head edge computing capabilities are simple and single, and need to transmit video streams to a cloud server for centralized processing through a network, lacking division and optimization of data transmission and calculation; 4. in construction operation safety supervision, the identity of the operation personnel is easily disturbed by the environment, single authentication is unreliable, the behavior supervision rule adaptation scene capability is weak and the violation response is lagging, and the risk events are difficult to focus, and the supervision rule is difficult to dynamically optimize. Therefore, in view of the above defects, designing a construction operation safety intelligent supervision system and application method based on an AI recognition algorithm is an urgent problem to be solved by the person skilled in the art. SUMMARY

[0007] In view of the problems of weak intelligent recognition capability, low image detection application level, insufficient fusion of edge computing and Internet of Things in the prior art, the present application aims to provide a construction operation safety intelligent supervision system and method based on an AI recognition algorithm, and the specific purposes include breaking through the limitation of single intelligent recognition type, realizing real-time intelligent analysis of multiple features of people and objects, reducing the dependence on manual work, improving the professionalization and scale of image detection, strengthening the deep application for different industry business scenarios, fully tapping the potential of image resources, integrating edge computing and Internet of Things technology, optimizing data processing flow, reducing transmission pressure, cost and bandwidth demand, and improving the overall efficiency of the system.

[0008] To achieve the above purposes, the present application is realized by the following technical scheme:

[0009] In the first aspect, the present application provides a construction operation safety intelligent supervision method based on an AI recognition algorithm. In a complex scene, target automatic extraction and recognition are particularly important when multiple targets need to be processed in real time. Based on the application of computer vision principles, computer image processing technology is used to track the target in real time, combining high-definition video acquisition, image processing and deep learning to realize the detection and recognition of general targets and customized targets in complex environments.

[0010] The method comprises the following steps:

[0011] Step 1: realized based on a feature training sub-module, the feature training sub-module is integrated in a cloud server, the module acquires defect image samples and non-defect image samples, and is responsible for defect classification and target feature library training;

[0012] Step 2: Based on the real-time analysis sub-module implementation, the feature training sub-module is integrated in the edge computing server. The module obtains the original image, performs image preprocessing, compares the feature library, i.e., the deep learning target detection algorithm, outputs the processed image, and frames the detected target with an external rectangular frame. Users can perform reference size labeling on the original image. According to the image gray point calculation, the position and size information of the external rectangular frame are calculated and displayed in the processed image.

[0013] Step 3: The verification terminal and the operation personnel's work card are networked and the identity information is verified. The camera identifies and analyzes the identity information of the operation personnel based on AI, and the verification terminal and the camera cross-authenticate to confirm the operation personnel.

[0014] Step 4: The supervisor sets the total rule, the cloud server automatically generates sub-rules based on the processed image and distributes them to the edge computing server for execution. The operation personnel triggers the total rule or sub-rule to push the alarm and record the event to the client for the supervisor to check.

[0015] Step 5: The edge computing server predicts the risk of the behavior characteristics of the operation personnel based on the total rule and the sub-rule.

[0016] Further, the defect classification and target feature library training specifically includes the following steps:

[0017] Step 101: The cloud server uses AI intelligent algorithms in the AI algorithm capability core engine to classify defect targets for non-defect image samples and defect image samples. After classification, the next step is executed.

[0018] Step 102: The cloud server uses AI models in the AI algorithm capability core engine to train the curve classification target to obtain a defect classification target feature library. The defect classification target feature library is saved for subsequent repeated retrieval. The next step is executed.

[0019] Step 103: The saved defect classification target feature library is transmitted to the edge computing server, and step 203 is executed for feature library comparison. Step 103 is repeatedly executed until the cloud server obtains new defect image samples and non-defect image samples.

[0020] Further, the camera in the local area network collects the original image and transmits it directly or indirectly to the edge computing server. When directly transmitted, the remote device transmits directly to the edge computing server through the remote network. The remote device is the camera. When indirectly transmitted, the storage and calculation separation technology is adopted, that is, data storage and image processing are performed by different devices. The original image obtained by the camera is saved in the network attached storage, and the hard disk file system built in the network attached storage transmits the original image to the edge computing server through file IO. The network attached storage is the storage end, and the edge computing server is the calculation end. The system IO built in the verification terminal transmits the verification data to the edge computing server through the system serial port or other interface. The verification data is also obtained by the image acquisition module of the edge computing server. The verification data and the original image are bound together for subsequent processing.

[0021] The processing of the original image specifically includes the following steps:

[0022] Step 201: The image acquisition module of the real-time analysis submodule obtains the original image from the remote device, the hard disk file system and the system IO respectively, and transmits the original image to the image pre-processing module for the next step.

[0023] Step 202: The image pre-processing module performs noise reduction and white balance processing on the original image to obtain a pre-processed image, and executes the next step.

[0024] Step 203a: The pre-processed image is compared with the defect classification target feature library output by the feature training submodule to obtain target recognition, and the detected target is boxed with an external rectangle.

[0025] Step 203b: The pre-processed image is annotated with a reference size to obtain target size annotation.

[0026] Step 203c: The pre-processed image is subjected to a clarity judgment, and an image clarity evaluation is output.

[0027] Among them, steps 203a, 203b and 203c are executed in parallel, and after execution is completed, they are merged into a processed image and output uniformly, and the next step is executed.

[0028] Step 204: The real-time analysis submodule outputs the processed image, and sequentially labels the image clarity, defect target size and defect target area, and jumps to step 201 for repeated execution.

[0029] Further, the cross-authentication of the verification terminal and the camera to confirm the work personnel specifically includes the following steps:

[0030] The pre-deployment process is as follows:

[0031] Step 301: Provide each worker with a work card with a built-in Bluetooth module, the Bluetooth module has the work card ID recorded, the work card ID is encrypted using RSA keys, the verification terminal has built-in RSA keys for decryption, the edge computing server has preset the identity information of each worker and establishes a mapping table of identity information and work card ID, and the next step is executed;

[0032] Step 302: Configure the verification terminal as a Bluetooth Mesh gateway and complete initialization, connect each worker's work card Bluetooth module to the verification terminal, complete the pre-deployment process, the Bluetooth modules between the work cards can be used as relays to transmit encrypted data to the verification terminal, each camera is also configured with the same Bluetooth module as the work card to participate in networking, and the Bluetooth module configured by the camera has the device number of each camera recorded, which is used by the verification terminal to determine the specific camera;

[0033] When the worker appears in the camera monitoring picture, the verification operation starts, and the specific steps are as follows:

[0034] Step 303: The camera collects the original image and transmits it to the edge computing server, the edge computing server processes the original image and outputs the processed image, and compares it with the built-in identity information, which includes the worker's name, personal number and personal portrait, the edge computing server uses MSE mean square error and PSNR peak signal-to-noise ratio to calculate the similarity of the processed image and the personal portrait, and outputs the identity information with the highest similarity, and enters the next step;

[0035] Step 304: The verification terminal obtains the device number from the Bluetooth module of the camera that triggers the verification operation, obtains the encrypted data of the Bluetooth module with the strongest connection signal near the camera Bluetooth module, the worker who triggers the verification is closest to the camera, and the Bluetooth connection signal is the strongest, the encrypted data is decrypted using the built-in RSA key to obtain the work card ID, and the next step is entered;

[0036] Step 305: The verification terminal transmits the work card ID to the edge computing server, and the edge computing server finds the corresponding identity information from the mapping table according to the work card ID, and enters the next step;

[0037] Step 306: The edge computing server compares the identity information obtained in steps 303 and 305, if they match, the worker's identity is confirmed, the edge computing server uploads the matching result to the cloud server, if they do not match, the edge computing server retrieves the historical records of the corresponding verification terminal and the corresponding camera from the cloud server, and the one with more successful times is used as the identity information.

[0038] Further, the determination of the total rule and the sub-rule specifically includes the following steps:

[0039] Step 401: The supervisor logs in the system through the client, enters the camera management interface, sets a special tag for each camera according to the monitoring area and the type of work, and the edge computing server classifies all cameras according to the tag, and enters the next step;

[0040] Step 402: The supervisor sets the total rule on the client, the total rule is uploaded to the cloud server through the client, and then synchronized to the edge computing server by the cloud server, and enters the next step;

[0041] Step 403: The edge computing server uploads the processed original image of each camera to the cloud server, and the cloud server analyzes the behavior characteristics of the work personnel through the AI intelligent algorithm of the AI algorithm capability core engine, the behavior characteristics include personnel action and operation process, and enters the next step;

[0042] Step 404: The cloud server calls the AI model to analyze the behavior characteristics of the work personnel, and automatically generates a sub-rule, which is a further constraint condition under the total rule, and the cloud server transmits the sub-rule to the edge computing server through the distribution device for saving and enabling, and enters the next step;

[0043] Step 405: When the original image obtained by the camera appears picture change, the edge computing server extracts the processed image from the original image in real time and matches it with the enabled total rule and sub-rule, if the processed image triggers the total rule or the sub-rule, the next step 406 is performed, if the total rule or the sub-rule is not triggered, no operation is taken, and step 405 is repeated;

[0044] Step 406: When the total rule is triggered, the edge computing server automatically generates an alarm information and pushes the alarm information to the client through the distribution device for real-time viewing by the supervisor, and when the sub-rule is triggered, an event is automatically recorded, and the recording time is pushed to the client in time axis order, for subsequent browsing by the supervisor, and jump to step 405.

[0045] Further, the risk prediction specifically includes the following steps:

[0046] Step 501: The edge computing server calculates the risk value of each camera monitoring area based on the AI intelligent algorithm of the AI algorithm capability core engine according to the enabled sub-rule and the real-time collected work personnel behavior data, generates a risk prediction event, and enters the next step;

[0047] Step 502: The edge computing server classifies and sorts all cameras according to the risk level of the risk prediction event, which is divided into high, medium and low, and preferentially pushes the prediction event with a high risk level and the corresponding camera screenshot to the client of the supervisor through the distribution device, and the client interface preferentially displays high-risk events to facilitate the supervisor to understand the key risk points in time, and enters the next step.

[0048] Step 503: The supervisor checks the pushed risk prediction event on the client, and labels the processing result according to the actual situation, the processing result is divided into handled, unhandled and ignored, handled means having contacted the operation personnel for rectification, unhandled means temporarily unable to contact for subsequent follow-up, ignored means judged as misjudgment or no need to handle, and the processing result is uploaded to the edge computing server through the client, a cumulative threshold number is set, whether the number of processing result uploading exceeds the cumulative threshold number is judged, if not, it jumps to step 501 for repeated execution, if yes, the next step is executed.

[0049] Step 504: The edge computing server counts the labeling number of each processing result, if the labeling number is the most handled, the current setting of the sub-rule is retained, if the labeling number is the most unhandled, the edge computing server will increase the risk level weight of the sub-rule, and the subsequent risk prediction events of this type will be preferentially pushed, if the labeling number is the most ignored, the edge computing server uploads the ignored processing result to the cloud server, and executes step 404 to regenerate the sub-rule.

[0050] In the second aspect, the application provides a construction operation safety intelligent supervision system based on an AI recognition algorithm, which comprises a client, a cloud server, a distribution device, an edge computing server and a plurality of cameras, the distribution device comprises a router and a switch, the cloud server establishes bidirectional communication with the client and the distribution device through the Internet, the distribution device establishes bidirectional communication with the edge computing server and a plurality of n cameras through an internal local area network, and the verification terminal establishes bidirectional communication with the distribution device in a wireless connection or wired connection mode.

[0051] The cloud server internally builds an AI intelligent identification platform, the AI intelligent identification platform includes a basic information management module, an AI video supervision center, a streaming media service platform and an AI service platform, the basic information management module includes perception device basic information management, first log management, camera management and AI intelligent algorithm setting, the AI video supervision center includes AI monitoring alarm, video carousel supervision, offline monitoring analysis and AI statistics, the streaming media service platform includes streaming media basic service, alarm management, camera management, subscription time correction notification, RTSP server management, video acquisition, video distribution, video storage, historical video management, audio and video codec, download management, voice broadcast and voice intercom;

[0052] The AI service platform includes an AI algorithm capability core engine and an AI intelligent service, the AI algorithm capability core engine includes an AI algorithm model optimization, a self-learning engine, an AI algorithm basic framework service, an RTSP video acquisition management, an AI model, an AI intelligent algorithm and an AI core algorithm acceleration, the AI intelligent service includes perception device basic information management, offline data acquisition, second log management, AI identification and retrieval service, business scenario AI service application, message notification, AI video service and AI picture service.

[0053] Further, in the basic information management module, the perception device basic information management is used for entering and updating the basic data of the model, position and access mode of the perception device camera, ensuring device compliance access and state traceability, ensuring unified control of the system on the device, the first log management is used for recording system operation behavior, device running state and abnormal information, providing data basis for fault troubleshooting and responsibility tracing, the camera management is used for configuring the resolution, frame rate and picture angle of the camera, and the AI intelligent algorithm setting is used for adjusting the detection threshold and identification category of the AI algorithm, adapting to different identification scenes of workers, equipment and defects, and the significance is to improve the accuracy of the algorithm in a specific scene;

[0054] In the AI video supervision center, the AI monitoring alarm detects illegal behavior by means of YOLOv12 open source model and triggers alarm through real-time analysis of video stream, the video carousel supervision is used for cyclically switching real-time pictures of multiple cameras, the offline monitoring analysis is used for monitoring the online state of the camera and the edge computing server in real time, analyzing the offline reason, ensuring data acquisition continuity, avoiding supervision vacuum, and the AI statistics is used for summarizing the alarm number, identification accuracy and device online rate.

[0055] Further, in the streaming media service platform, the streaming media basic service is used for processing the RTSP video transmission protocol, ensuring stable transmission of the video stream, connecting the video collection and subsequent analysis and playing links, the alarm management is used for storing, classifying and associating alarm information and corresponding video segments, facilitating the tracing and batch processing of alarm information, the camera management and basic information management module cooperates to supplement the camera coding format and streaming transmission parameter configuration of the streaming media layer, ensuring that the video stream output by the camera is compatible with the streaming media system, the subscription time correction notification is used to push time correction information to each device, ensuring that the device time of the camera and the edge server is synchronized, avoiding video data timestamp deviation, the RTSP server management is used for deploying and maintaining the open source RTSP server, controlling the start and stop of the server and limiting the number of connections, ensuring efficient transmission of the video stream to the AI model and the client through the RTSP protocol, the video collection is used to obtain the original video stream by calling the camera interface, and the original data source is provided for AI analysis, the video distribution is used to distribute the original video stream or the AI processed video stream to the client and the edge computing server, meeting the demand of multiple terminals for simultaneous viewing, reducing the pressure of repeated transmission in the cloud, the video storage is used to store the video data in the network attached storage, realizing long-term storage of historical video, facilitating the review of past accidents, the historical video management is used to provide the search and playback functions of historical video, supporting the tracing of past construction scenes and the investigation of potential problems, the audio and video coding and decoding are used for video compression and audio and video decoding, reducing the video storage space and transmission bandwidth, and ensuring the data transmission efficiency, the download management is used to support users to download video segments, facilitating offline analysis and evidence preservation, the voice broadcast and voice talkback are used for real-time voice interaction between the supervision end and the construction end, timely conveying safety instructions and responding to the inquiries of workers, and improving the communication efficiency.

[0056] Further, in the AI service platform, the AI algorithm model optimization label autonomous learning engine of the AI algorithm capability core engine re-labels the misidentified and missed identified samples, iteratively optimizes the AI model, continuously improves the model identification accuracy, adapts to the changes of the construction scene, the AI algorithm basic framework service provides sample preprocessing, concurrent processing, communication guarantee and video stream and picture identification, provides a basic environment for the operation of the AI algorithm, ensures efficient and stable work of the algorithm, the RTSP video collection management obtains the RTSP video stream and analyzes it into frame data for transmission to the AI model, connects the streaming media collection and AI analysis, ensures the real-time nature of data transmission, the AI model is used for multi-target detection and identification of workers, equipment and defects in the construction scene, replaces manual real-time intelligent analysis, improves supervision efficiency, the AI intelligent algorithm extracts target features, classifies defects, improves the accuracy of target classification and defect identification, reduces misjudgment, the AI core computing power accelerates the inference speed of the AI model, reduces the calculation delay of the edge and the cloud, and adapts to the ARM architecture embedded device;

[0057] The perception device basic information management of the AI intelligent service cooperates with the basic information management module, synchronizes the latest state of the device to the AI video supervision center, ensures that the AI analysis can adjust the identification strategy in combination with the device state, acquires the state data of the device offline for collecting the device context information for the AI model, avoids the misanalysis caused by the device offline, the second log management records the calling record, the identification result and the model running log of the AI service, facilitates the AI service troubleshooting and performance optimization, the AI identification and retrieval service is used for quick retrieval of target identification results and historical identification data, improves the query efficiency of the supervision personnel, quickly locates key information, the business scene AI service application adapts the AI capability to specific construction scenes, realizes the transformation of AI technology from general identification to industry landing, solves the actual construction supervision problem, the message notification is used for pushing the alarm information and the identification result to the supervision personnel through the short message or the system message, ensures that the key information is timely reached, avoids delay disposal, the AI video service provides the enhanced video stream after AI analysis, directly shows the identification result, facilitates the supervision personnel to quickly identify the problem, the AI picture service provides the labeled picture after AI processing, facilitates detail checking, archiving and subsequent analysis, and improves the application efficiency of the image detection.

[0058] The present application has the following beneficial effects:

[0059] 1. By fusing computer vision, artificial intelligence, pattern recognition and other technologies, real-time intelligent analysis and identification of multiple targets such as non-digital vernier caliper, grounding resistance swing meter, electroscope, anti-falling device and safety cone barrel are realized, manual video content identification and judgment are replaced, industrial field stereoscopic supervision, operation and service needs are met, and the intelligent identification capability of the system is improved.

[0060] 2. A customized intelligent application mechanism for industry business scenes is established, key information is quickly extracted from massive videos, image resources are transformed from basic generalization to deep application, system comprehensive efficiency is improved, intelligent and social management needs are met, and the application efficiency of image detection is improved.

[0061] 3. Through the fusion of edge computing and Internet of Things, data real-time processing is realized by using the edge end of the camera, the data transmission amount to the cloud is reduced, the network bandwidth demand and the cloud computing cost are reduced, the data processing speed and the application program efficiency are improved, low-delay, low-cost and high-real-time intelligent identification and management are realized, and the resource utilization rate is improved.

[0062] 4. The success rate of identity recognition is improved through cross authentication, the supervision adaptability is guaranteed by scene-based rules and manual adjustment, the disposal efficiency is improved by risk prediction grading push and sub-rule iteration optimization, and different operation environments and business scenes are deeply adapted.

[0063] Of course, implementing any of the products of the present application does not necessarily require that all of the advantages described above be achieved simultaneously. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the description of the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0065] Figure 1 The block diagram of the construction work safety intelligent monitoring system based on AI recognition algorithm of the present application;

[0066] Figure 2 The execution flow schematic diagram of the feature training submodule and the real-time analysis submodule of the present application;

[0067] Figure 3 The framework schematic diagram of the cloud server AI intelligent recognition platform of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0069] The technical problems to be solved by the present application are as follows: 1. Pre-job safety material carrying missing problem, workers often do not carry key safety materials when entering the work area, and the lack of on-site supervision personnel leads to high missed detection rate, traditional manual point inspection mode cannot achieve full coverage real-time verification, and accident hidden dangers such as electric shock and falling are caused; 2. Safety cone barrel placement deviation problem in operation, relying on on-site supervision patrol is low in efficiency, safety cone barrels are not placed, leading to the mistake of other personnel, and safety risks are caused; 3. Precision instrument manual reading error problem, non-digital display tools rely on manual reading, for example, vernier caliper and ground resistance swing watch, which have defects of low efficiency and high error rate, efficiency is more than 30 seconds per time, error rate is more than 18%, and chain accidents such as part batch scrap and equipment grounding failure are easily caused; 4. Supervision vacuum problem caused by lack of supervision personnel, supervision of high-risk operation surface cannot be fully covered, manual patrol has blind area and time delay, leading to that illegal operation cannot be intercepted in real time, for example, electric pen is not used and anti-falling device is not hung, and the accident prevention mechanism is just a formality; 5. Limitation problem of intelligent identification capability, existing intelligent identification is single, mainly relies on ordinary monitoring cameras, and lacks real-time intelligent analysis capability for multiple characteristics of people and objects, for example, face, biology, physics and defects, intelligent algorithm is concentrated in basic target extraction or specific object detection, and has insufficient robustness and is limited by use environment, mainly relies on manual post-analysis, and cannot meet the needs of stereoscopic supervision, operation and service in the communication field; 6. Low efficiency problem of image detection application, the intelligent identification system is mainly based on basic management application, lacks a professional application mechanism for industry business scenes, and lacks comprehensive intelligent technology support, for example, quickly positioning key images in a large amount of video, the professionalization and scale of image detection are low, image resources only stay at the basic browsing level, potential is not fully tapped, the system comprehensive efficiency cannot be improved, and the intelligent and socialized application management needs cannot be met; 7. Insufficient fusion problem of edge computing and Internet of Things, most cameras on the market have simple and single edge computing capability, data needs to be transmitted to a cloud server for processing through a network, and the deep fusion of edge computing and Internet of Things technology is not realized; data relies on centralized processing of the cloud, leading to large transmission pressure, high delay and high cost, real-time and low-cost processing needs cannot be met, and the overall system efficiency is limited.

[0070] Embodiment one

[0071] Please refer to Figures 1-3 The present application provides a technical solution: an operation safety intelligent supervision method based on an AI identification algorithm, in a complex scene, target automatic extraction and identification are particularly important when multiple targets need to be processed in real time, based on the application of computer vision principle, a computer image processing technology is used to track the target in real time, high-definition video acquisition, image processing and deep learning are combined, and detection and identification of general targets and customized targets in a complex environment are realized.

[0072] The method comprises the following steps:

[0073] Step 1: Based on the feature training sub-module implementation, the feature training sub-module is integrated in the cloud server. The module obtains defect image samples and non-defect image samples from the industry general training data set, is responsible for defect classification and target feature library training;

[0074] Step 2: Based on the real-time analysis sub-module, the feature training sub-module is integrated in the edge computing server. The module obtains the original image from the image acquisition module. The user can perform reference size annotation on the original image, train the model, and perform feature library comparison after image preprocessing. The processed image is output by the deep learning target detection algorithm. The detected target is framed with an outer rectangle. The position and size information of the outer rectangle are calculated according to the image gray point calculation, and are displayed in the processed image. According to the image clarity algorithm, it is automatically judged whether the original image is clear or not, so that the user can adjust the shooting parameters. The clarity and image target detection have a certain relationship. The provided image meets the requirements of target detection;

[0075] It should be noted that the user uses the labelImg tool to mark the position of the target to be recognized in the original image, and generates a.txt annotation file corresponding to each picture. After the annotation is completed, the entire image data set is randomly divided into two parts: a training set and a validation set. The training set is used as the main data for model weight update; the validation set is used to monitor model performance during training to prevent overfitting; select the YOLO model configuration file, modify the key parameters in it, mainly the number of classes, and change it to the number of classes to be recognized; finally, model training, including: select a pre-trained weight file as the initial point, and train it. The training process is an iterative loop. In each iteration through the entire training set, the following operations are performed: forward propagation: input a batch of training images into the model. The model outputs predictions for each grid, including bounding box coordinates, confidence, and class probabilities, based on the current weight parameters. Loss calculation: compare the model's predictions with the true labels generated by labelImg. The loss function calculates an overall error value. Backpropagation and optimization: using the backpropagation algorithm, calculate the gradient of the loss function with respect to each weight of the model (i.e., how much responsibility each weight bears for the total error). The optimizer updates all the model's weights according to the gradients and a certain learning rate, aiming to reduce the loss value in the next iteration. Verification and monitoring: after each iteration through the entire training set, the model runs on the validation set and calculates various performance metrics.

[0076] Step 3: The verification terminal and the worker's ID card are networked and the identity information is verified. The camera recognizes and analyzes the identity information of the operator based on AI. The verification terminal and the camera cross-authenticate to confirm the worker;

[0077] Step 4: The supervisor sets the total rule, the cloud server automatically generates the sub-rule according to the processed image and delivers it to the edge computing server for execution, and the worker triggers the total rule or the sub-rule to push the alarm and record the event to the client for the supervisor to check;

[0078] Step 5: The edge computing server predicts the risk of the worker's behavior characteristics according to the total rule and the sub-rule.

[0079] As shown in Figure 2 , the defect classification and target feature library training specifically includes the following steps:

[0080] Step 101: The cloud server uses AI intelligent algorithms in the AI algorithm capability core engine to classify defect targets for defect-free image samples and defect image samples, and after classification, the next step is executed;

[0081] Step 102: The cloud server uses AI models in the AI algorithm capability core engine to train the curve classification target, obtains a defect classification target feature library, saves the defect classification target feature library, saves the defect classification target feature library for subsequent repeated retrieval, and executes the following step;

[0082] Step 103: The saved defect classification target feature library is transmitted to the edge computing server, and step 203 is executed for feature library comparison, and step 103 is repeatedly executed until the cloud server obtains new defect image samples and non-defect image samples.

[0083] Wherein, the original image collected by the camera in the local area network is directly or indirectly transmitted to the edge computing server, when directly transmitted, the remote device is directly transmitted to the edge computing server through the remote network, the remote device is the camera, when indirectly transmitted, the storage and calculation separation technology is adopted, that is, data storage and image processing are performed by different devices, the original image obtained by the camera is saved in the network attached storage, the hard disk file system built-in the network attached storage transmits the original image to the edge computing server through file IO, the network attached storage is the storage end, the edge computing server is the computing end, the system IO built-in the verification terminal transmits verification data to the edge computing server through the system serial port or other interface, the verification data is also obtained by the image acquisition module of the edge computing server, and the verification data and the original image are bound together for subsequent processing;

[0084] As shown in Figure 2 , the processing of the original image specifically includes the following steps:

[0085] Step 201: The image acquisition module of the real-time analysis submodule obtains the original image from the remote device, the hard disk file system and the system IO respectively, and transmits the original image to the image pre-processing module for the next step;

[0086] Step 202: The image pre-processing module denoises and white balances the original image to obtain a pre-processed image, and the next step is executed;

[0087] Step 203a: The pre-processed image is compared with the defect classification target feature library output by the feature training submodule to obtain target recognition, and the detected target is framed with an outer rectangle;

[0088] Step 203b: The pre-processed image is annotated with a reference size to obtain target size annotation;

[0089] Step 203c: The pre-processed image is subjected to a sharpness judgment, and an image sharpness evaluation is output;

[0090] Steps 203a, 203b, and 203c are executed in parallel, and after execution is completed, they are merged into a processed image and output uniformly, and the next step is executed;

[0091] Step 204: The real-time analysis submodule outputs the processed image, and sequentially annotates the image sharpness, defect target size, and defect target area, and jumps to step 201 for repeated execution.

[0092] The cross-authentication of the verification terminal and the camera to confirm the specific work personnel includes the following steps:

[0093] The pre-deployment process is as follows:

[0094] Step 301: Each work personnel is equipped with a work card with a built-in Bluetooth module, the Bluetooth module is entered with a work card ID, the work card ID is encrypted using an RSA key, the verification terminal is built-in with an RSA key for decryption, the identity information of each work personnel is pre-set in the edge computing server and a mapping table of identity information and work card ID is established, and the next step is executed;

[0095] Step 302: The verification terminal is configured as a Bluetooth Mesh gateway and is initialized, the work card Bluetooth module of each work personnel is connected to the verification terminal, the pre-deployment process is completed, the Bluetooth modules between the work cards can be used as relays to transmit encrypted data to the verification terminal, and each camera is also configured with the same Bluetooth module as the work card to participate in networking, the Bluetooth module configured in the camera is entered with the device number of each camera, which is used by the verification terminal to determine the specific camera;

[0096] When the work personnel appears in the camera monitoring picture, the verification operation is started, and the specific steps are as follows:

[0097] Step 303: The camera collects the original image and transmits it to the edge computing server. The edge computing server processes the original image and outputs a processed image, and compares the processed image with the built-in identity information, including the name, personal number and personal portrait of the worker. The edge computing server uses MSE (Mean Square Error) and PSNR (Peak Signal to Noise Ratio) to calculate the similarity between the processed image and the personal portrait, and the similarity weight calculated by MSE and PSNR is 0.5:0.5. The edge computing server outputs the identity information with the highest similarity, and enters the next step.

[0098] Step 304: The verification terminal obtains the device number from the Bluetooth module of the camera that triggers the verification operation, and obtains the encrypted data of the Bluetooth module with the strongest connection signal near the camera. The worker who triggers the verification is closest to the camera, and the Bluetooth connection signal is the strongest. The built-in RSA key is used to decrypt the encrypted data to obtain the ID of the worker card, and the next step is entered.

[0099] Step 305: The verification terminal transmits the ID of the worker card to the edge computing server, and the edge computing server finds the corresponding identity information from the mapping table according to the ID of the worker card, and enters the next step.

[0100] Step 306: The edge computing server compares the identity information obtained in step 303 and step 305. If the two match, the identity of the worker is confirmed. The edge computing server uploads the matching result to the cloud server. If the two do not match, the edge computing server retrieves the historical records of the corresponding verification terminal and the corresponding camera from the cloud server, and outputs the corresponding identity information according to the number of successful times.

[0101] The determination of the total rule and the sub-rule specifically includes the following steps:

[0102] Step 401: The supervisor logs in to the system through the client and enters the camera management interface. The supervisor sets a special tag for each camera according to the monitoring area and the type of work. The edge computing server classifies all cameras according to the tags, and enters the next step.

[0103] Step 402: The supervisor sets the total rule on the client, such as "high-altitude work requires wearing a fall protection device" and "electrical operation requires using an electrical tester". The total rule is uploaded to the cloud server through the client, and then synchronized to the edge computing server by the cloud server, and enters the next step.

[0104] Step 403: The edge computing server uploads the processed original image of each camera to the cloud server, which analyzes the behavior characteristics of the workers through AI intelligent algorithms of the AI algorithm capability core engine. The behavior characteristics include personnel actions and operation processes, such as "not wearing a fall protector" and "not using an electric pen". The next step is entered;

[0105] Step 404: The cloud server calls AI model to analyze the behavior characteristics of the workers based on YOLOv12 model and supporting behavior analysis model, and automatically generates sub-rules. The sub-rules are further constraint conditions under the total rules, such as "virtual hanging of fall protector triggers alarm". Compared with the total rule "not wearing a fall protector", more detailed operation requirements are made. The cloud server transmits the sub-rules to the edge computing server through the distribution device for saving and enabling, and enters the next step;

[0106] Step 405: When the original image obtained by the camera appears picture variation, the edge computing server extracts and processes the image in real time, and matches it with the enabled total rules and sub-rules. If the processed image triggers the total rules or sub-rules, the next step 406 is performed. If the total rules or sub-rules are not triggered, no operation is taken, and step 405 is repeated;

[0107] Step 406: When the total rules are triggered, the edge computing server automatically generates alarm information and pushes it to the client through the distribution device for real-time viewing by supervisors. When the sub-rules are triggered, the event is automatically recorded and pushed to the client in chronological order for subsequent browsing by supervisors. Go to step 405.

[0108] The risk prediction specifically includes the following steps:

[0109] Step 501: The edge computing server calculates the risk value of each camera monitoring area based on AI intelligent algorithms of the AI algorithm capability core engine, generates risk prediction events, such as "high altitude-01 camera area: 2 times of not wearing a fall protector in 10 minutes, high risk level", and enters the next step, according to the enabled sub-rules and combined with real-time collected worker behavior data;

[0110] Step 502: The edge computing server automatically classifies and sorts all cameras according to the risk level of the risk prediction event, which is divided into high, medium and low. The prediction events with high risk level are pushed to the client of the supervisor through the distribution device together with the corresponding camera screenshot. The client interface displays high-risk events first, making it easy for supervisors to understand key risk points in time, and enters the next step;

[0111] Step 503: The supervisor views the pushed risk prediction event on the client, and labels the processing result according to the actual situation. The processing result is divided into handled, unhandled and ignored. Handled means that the operation personnel have been contacted for rectification, unhandled means that it is temporarily impossible to contact for follow-up, ignored means that it is judged as a false positive or does not need to be processed, and the processing result is uploaded to the edge computing server through the client. Set the cumulative threshold number of times to 15, judge whether the number of times of uploading the processing result exceeds the cumulative threshold number of times, if not, jump to step 501 and repeat, if yes, execute the next step;

[0112] Step 504: The edge computing server counts the number of labels for each processing result. If the most labeled number is handled, the current settings of the sub-rule are retained. If the most labeled number is unhandled, the edge computing server will increase the risk level weight of the sub-rule, and will subsequently prioritize pushing this type of risk prediction event. If the most labeled number is ignored, the edge computing server will upload the ignored processing result to the cloud server, and execute step 404 to regenerate the sub-rule.

[0113] The construction operation safety intelligent supervision system based on AI recognition algorithm, as shown in Figure 1 , includes a client, a cloud server, a distribution device, an edge computing server and n cameras. The distribution device includes a router and a switch. The cloud server establishes bidirectional communication with the client and the distribution device through the Internet. The distribution device establishes bidirectional communication with the edge computing server and the n cameras through an internal local area network. A verification terminal establishes bidirectional communication with the distribution device through wireless connection or wired connection. n is a positive integer.

[0114] As shown in Figure 3 , an AI intelligent recognition platform is built inside the cloud server. The AI intelligent recognition platform includes a basic information management module, an AI video supervision center, a streaming media service platform and an AI service platform. The basic information management module includes perception device basic information management, first log management, camera management and AI intelligent algorithm setting. The AI video supervision center includes AI monitoring and alarming, video carousel supervision, offline monitoring and analysis and AI statistics. The streaming media service platform includes streaming media basic service, alarm management, camera management, subscription time notification, RTSP server management, video acquisition, video distribution, video storage, historical video management, audio and video codec, download management, voice broadcast and voice intercom.

[0115] The AI service platform comprises an AI algorithm capability core engine and an AI intelligent service. The AI algorithm capability core engine comprises an AI algorithm model optimization marking autonomous learning engine, an AI algorithm basic framework service, an RTSP video collection management, an AI model, an AI intelligent algorithm, and an AI core algorithm power acceleration. The AI intelligent service comprises a perception device basic information management, an offline data acquisition, a second log management, an AI recognition and retrieval service, a business scenario AI service application, a message notification, an AI video service, and an AI picture service.

[0116] In the basic information management module, the perception device basic information management is used for inputting and updating the basic data of the model, position, and access mode of the perception device camera, ensuring device compliance access and state traceability, ensuring unified control of the system on the device, the first log management is used for recording system operation behavior, device running state, and abnormal information, providing data basis for fault troubleshooting and responsibility traceability, and the camera management is used for configuring the resolution, frame rate, and picture angle of the camera, ensuring that the camera stably outputs original images or video streams meeting the AI analysis requirements, the AI intelligent algorithm setting is used for adjusting the detection threshold and recognition category of the AI algorithm, adapting to different recognition scenarios of workers, devices, and defects, and the significance is to improve the accuracy of the algorithm in a specific scene, and multi-target detection parameter optimization can be performed based on the open source YOLOv12 algorithm;

[0117] In the AI video monitoring center, the AI monitoring alarm detects illegal behavior by means of the YOLOv12 open source model and triggers an alarm through real-time analysis of the video stream, which can timely warn of safety hazards and avoid accidents, the video carousel monitoring is used for cyclically switching the real-time pictures of multiple cameras, realizing synchronous monitoring of multiple areas of construction work, reducing the monitoring blind area, the offline monitoring analysis is used for real-time monitoring of the online state of the camera and the edge computing server, analyzing the offline reasons, ensuring data collection continuity, and avoiding monitoring vacuum, and the AI statistics are used for summarizing the alarm times, recognition accuracy, and device online rate, providing data support for monitoring strategy optimization and system performance improvement.

[0118] Among them, in the streaming media service platform, the streaming media basic service is used for processing the RTSP video transmission protocol, guaranteeing the stable transmission of the video stream, connecting the video collection and the subsequent analysis and playing links, the alarm management is used for storing, classifying and associating the corresponding video segments of the alarm information, facilitating the tracing and batch processing of the alarm information, the camera management and the basic information management module are coordinated, the camera coding format and the streaming transmission parameter configuration of the streaming media level are supplemented, the compatibility of the video stream output by the camera and the streaming media system is ensured, the time calibration information is pushed to each device through the subscription time correction notification, the device time synchronization of the camera and the edge server is ensured, the video data timestamp deviation is avoided, the RTSP server management is used for deploying and maintaining the open source live555RTSP server, controlling the start and stop of the server, limiting the number of connections, guaranteeing the efficient transmission of the video stream to the AI model and the client through the RTSP protocol, the video collection is used for obtaining the original video stream by calling the camera interface, cooperating with the open source OpenCV library to perform preliminary frame extraction, and providing the original data source for AI analysis, the video distribution is used for distributing the original video stream or the AI processed video stream to the client and the edge computing server, meeting the demand of simultaneous viewing of multiple terminals, reducing the pressure of repeated transmission of the cloud, the video storage is used for storing the video data to the open source MinIO object storage or the network attached storage, realizing the long-term storage of the historical video, facilitating the post-accident review, the historical video management is used for providing the search and playback functions of the historical video, supporting the tracing of the past construction scene and the investigation of potential problems, the audio and video coding and decoding adopts the open source x264 algorithm for video compression and FFmpeg for audio and video decoding, reducing the video storage space and transmission bandwidth, guaranteeing the data transmission efficiency, the download management is used for supporting the user to download the video segment, facilitating the offline analysis and evidence preservation, the voice broadcast and voice talk are based on the open source WebRTC protocol and are used for realizing the real-time voice interaction between the supervision end and the construction end, timely conveying the safety instructions and responding to the inquiries of the operation personnel, and improving the communication efficiency.

[0119] In the AI service platform, the AI algorithm model optimization mark autonomous learning engine of the AI algorithm capability core engine re-labels the mis-identified and missed-identified samples through the open source LabelStudio tool, iteratively optimizes the AI model in combination with the Adam optimizer, continuously improves the model identification accuracy, adapts to the changes of the construction scene, the AI algorithm basic framework service provides sample preprocessing, concurrent processing, communication guarantee and video stream and picture identification, the sample preprocessing is based on OpenCV for image rotation and cropping data enhancement, the concurrent processing is based on the open source TensorFlowServing to realize multi-request scheduling, provides a basic environment for the AI algorithm operation, guarantees the efficient and stable work of the algorithm, the RTSP video acquisition management acquires the RTSP video stream through the live555 server and parses it into frame data transmission to the AI model, connects the streaming media acquisition and AI analysis, ensures the real-time data transmission, the AI model adapts to the open source YOLOv12 model, is used for multi-target detection and identification of workers, equipment and defects in the construction scene, replaces manual real-time intelligent analysis, improves the supervision efficiency, the AI intelligent algorithm extracts target features by using the open source ResNet algorithm, classifies defects by using the support vector machine (SVM) algorithm, improves the accuracy of target classification and defect identification, reduces the misjudgment, the AI core computing power acceleration is based on the OpenCL convolution software acceleration scheme, uses the GPU computing core and LocalMemory to improve the AI model inference speed, reduces the calculation delay of the edge and cloud, adapts to the ARM architecture embedded device;

[0120] The perception device basic information management of the AI intelligent service cooperates with the basic information management module, synchronizes the latest state of the device to the AI video supervision center, ensures that the AI analysis can adjust the identification strategy in combination with the device state, acquires the state data of the device offline for collecting the device context information for the AI model, avoids the misanalysis caused by the device offline, the second log management records the calling record, the identification result and the model running log of the AI service, facilitates the AI service troubleshooting and performance optimization, the AI identification and retrieval service is based on the open source Elasticsearch and is used for realizing the quick retrieval of the target identification result and the historical identification data, improves the query efficiency of the supervision personnel, quickly locates the key information, the AI service application of the business scene adapts the AI capability to the specific construction scene, for example, the state detection of the anti-falling device for high-altitude operation, the compliance inspection of the safety cone barrel laying, the YOLOv12 model is based on the scene optimization, realizes the transformation of the AI technology from general identification to industry landing, solves the actual construction supervision problem, the message notification is used for pushing the alarm information and the identification result to the supervision personnel in the form of a short message or a system message, ensures that the key information is timely reached, avoids the delay of disposal, the AI video service provides the enhanced video stream after the AI analysis, for example, the rectangular box is used for labeling the target and superimposing the defect information, the identification result is directly displayed, and the supervision personnel can quickly identify the problem, the AI picture service provides the labeled picture after the AI processing, for example, the defect area and the size information are labeled, and the details are viewed, archived and analyzed, and the application efficiency of the picture detection is improved.

[0121] Embodiment two

[0122] The application also provides a technical solution: a construction operation safety intelligent supervision system based on an AI identification algorithm, including neural network computing power acceleration and optimization, AI and related modules, non-digital vernier caliper reading AI identification, grounding resistance shaking table reading AI identification, test pen AI identification, anti-falling device AI identification and safety cone barrel AI identification.

[0123] Among them, the neural network computing power acceleration and optimization is based on the convolution software acceleration optimization scheme of OpenCL, OpenCL performs specific convolution optimization on specific network models, including fully utilizing GPU LocalMemory, fully utilizing GPU computing core, GPU has multiple computing cores, each computing core has many work units, and each work unit is equivalent to a thread, fully utilizing the data carried each time: as many points as possible are calculated each time, and the loop is unfolded, such as The for loop is expanded into 36 serial single instructions; GPU SIMD GPU level multi-stage pipeline overlap and optimization technology, the scheme is based on OpenCL and specific convolution optimization for specific network models, has strong applicability, can be widely used in PC and embedded devices supporting OpenCL, especially in embedded devices that do not support the use of Nvidia graphics cards, the scheme has strong practical significance, even if the device supports Nvidia graphics card, considering the price of Nvidia graphics card, this scheme can be used to greatly improve the system computing performance without increasing the hardware cost, at present, terminal widely adopts ARM architecture, and the operating system supports OpenCL, the scheme can be widely applied to the calculation acceleration of terminal device neural network application scene;

[0124] Among them, the AI and related modules include AI algorithm model optimization mark autonomous learning engine and AI algorithm basic framework service, the AI algorithm model optimization mark autonomous learning engine is used for misidentification and missed identification, and the AI algorithm basic framework service is used for sample table preprocessing, concurrent processing, communication guarantee, video stream identification and picture identification;

[0125] Among them, the non-digital vernier caliper is usually used for precise measurement, and the non-digital vernier caliper reading AI recognition automatically identifies the scale of the vernier caliper and reads the accurate value by AI technology, therefore, we need to first collect images containing vernier calipers of different brands and models, and mark the positions of the vernier, the ruler and the scale line in each image, these marked data will be used to train the YoloV12 model to enable it to detect and locate each part of the vernier caliper, in the aspect of image processing, we use data enhancement techniques such as rotation, scaling, cropping, etc. to enhance the robustness of the model, in the model training stage, we use the target detection capability of YoloV12 to detect the outline of the vernier caliper, the specific position of the vernier and the scale line, and through the regression network to predict the distance between the vernier and the reference scale line, and combine these information to calculate the reading, finally, the model will output the accurate reading value, reducing the manual measurement error, improving the work efficiency and accuracy, in order to improve the recognition accuracy of the model, in the future, we can continue to expand the data set and introduce high-resolution training data to further optimize the generalization ability of the model, especially in different light and angle;

[0126] Among them, the ground resistance dial tester is usually used to measure the ground resistance value of electrical equipment, and the AI recognition core of the ground resistance dial tester reading is to automatically read the pointer position and its digital display on the dial tester to obtain the resistance value. First of all, we need to collect images of ground resistance dial testers of different models and brands, and label the dial, pointer and digital area in each image. Image enhancement techniques will also be applied in this stage to adapt to changes in different angles, lighting and dial reflections, etc. In the model training stage, we still use YoloV12 for target detection to identify the positions of the dial, pointer and digital area. Through the regression network, the pointer angle is predicted, and combined with the scale of the dial, the accurate resistance value is finally calculated. At the same time, the digital area will be identified by a classification algorithm to ensure the accuracy of the reading. This model will output the value of the ground resistance in real time. Through model optimization, such as improving data preprocessing and image enhancement techniques, its stability under complex lighting conditions can be further improved, ensuring that it can quickly and accurately complete the measurement in different environments.

[0127] Among them, the ground resistance dial tester is usually used to measure the ground resistance value of electrical equipment, and the AI recognition core of the ground resistance dial tester reading is to automatically read the pointer position and its digital display on the dial tester to obtain the resistance value. First of all, we need to collect images of ground resistance dial testers of different models and brands, and label the dial, pointer and digital area in each image. Image enhancement techniques will also be applied in this stage to adapt to changes in different angles, lighting and dial reflections, etc. In the model training stage, we still use YoloV12 for target detection to identify the positions of the dial, pointer and digital area. Through the regression network, the pointer angle is predicted, and combined with the scale of the dial, the accurate resistance value is finally calculated. At the same time, the digital area will be identified by a classification algorithm to ensure the accuracy of the reading. This model will output the value of the ground resistance in real time. Through model optimization, such as improving data preprocessing and image enhancement techniques, its stability under complex lighting conditions can be further improved, ensuring that it can quickly and accurately complete the measurement in different environments.

[0128] Among them, the fall arrestor is usually used in high-altitude operation, and its main function is to prevent personnel from falling. The fall arrestor AI recognition is used to judge the working state of the fall arrestor, such as whether it is locked or in an active state. The image data of the fall arrestor is collected, and the image contains the fall arrestor in different states to ensure that the data set covers a variety of equipment and scenes. The image annotation will cover the shape characteristics of the fall arrestor and its working state, such as: locking, loosening, etc. In the training process, YoloV12 will be used to detect the position and shape characteristics of the fall arrestor. Next, combined with the appearance changes of the fall arrestor, the model will judge whether it is in normal working state or whether there is a fault. This task requires accurate judgment of the changes of the fall arrestor, especially when the fall arrestor is blocked or in poor lighting conditions. Through optimization of the data set and training strategy, the model's adaptability to these environmental changes can be further improved in the future. The system will provide real-time feedback on the working state of the fall arrestor on the interface, helping workers to discover potential safety hazards in time and avoid accidents.

[0129] Among them, the safety cone barrel is usually used in road construction or safety warning. The safety cone barrel AI recognition judges whether there is a safety cone barrel in the image, and further judges the integrity of the cone barrel, such as whether it is collapsed or damaged. In order to train the model, we need to collect images containing safety cone barrels to ensure that the data set covers safety cone barrels in various environmental conditions, such as daytime, nighttime, different weather, etc. During annotation, in addition to the positioning of the cone barrel, the state information of the cone barrel needs to be annotated, such as intact, inclined or collapsed. In the model training stage, YoloV12 will be used to detect the cone barrel in the image and judge the state of the cone barrel. Since the color of the cone barrel may be affected by the environmental light, we need to introduce diverse lighting conditions in the data set annotation. Through this strategy, the model will be able to accurately identify the state of the cone barrel in different backgrounds and environments. The goal in the training process is to judge the state of the cone barrel through multi-classification, such as intact, collapsed, missing, etc. By strengthening the learning of the collapsed and blocked scenes of the cone barrel, the robustness of the model can be further improved to ensure its stability in complex environments. By combining target detection and segmentation technology, the detail recognition ability of the cone barrel can be further improved to provide more accurate working state feedback.

[0130] The key points that need to be explained in this embodiment are as follows:

[0131] 1、Technical architecture innovation, build the typical architecture of deep integration of edge computing and cloud computing, realize real-time identity detection, recognition and authentication in ubiquitous access scene, solve the problem of lack of edge and cloud cooperation in traditional monitoring system; Ubiquitous monitoring and industry adaptation, for the specific scene of communication industry maintenance work, realize the customization application of "person" ubiquitous monitoring, recognition and access, meet the personalized needs of whole life cycle monitoring of construction work; Intelligent technology fusion application, integrate artificial intelligence, big data, Internet of Things, mobile Internet and other technologies, improve the robustness of intelligent recognition, such as target detection and behavior analysis in complex environment, break through the bottleneck of existing algorithm environment limitation; Full link security and credibility, establish the trusted access, interaction and access mechanism of equipment and target, form the integrated identity authentication system of edge and cloud, protect the security and legality of Internet of Things access; Through the construction of high-precision intelligent service ecological circle, support the whole life cycle monitoring and adaptability management of construction work, improve the level of comprehensive intelligent supervision, operation and service; Whole process safety supervision, from image acquisition, feature extraction, model training to application scene landing, form a complete construction safety supervision closed loop, through real-time monitoring, automatic recording, intelligent early warning and other functions, realize the all-round safety control of construction process, effectively avoid the safety problems caused by improper use of equipment or data measurement error;

[0132] 2、Edge and cloud cooperative identity authentication architecture, protect the cooperative mechanism of edge node real-time detection and cloud deep analysis, including data interaction rules, computing power allocation strategy and authentication process, solve the problem of high delay and large bandwidth pressure of traditional single cloud processing, real-time recognition technology in ubiquitous access scene, protect the multi-modal recognition algorithm for "person" ubiquitous monitoring, such as fusion recognition of personnel biological characteristics and object physical characteristics, and adaptive optimization scheme in complex environment of construction work, industry customized intelligent analysis system, protect the customized function modules for whole life cycle monitoring of construction work, such as real-time processing of full HD video stream, specific behavior intelligent research and judgment, whole link data traceability, etc., device trusted access and safe interaction mechanism, protect the identity authentication protocol, data encryption transmission method and access permission dynamic management strategy of Internet of Things device access, ensure legal access and trusted interaction of device, multi-technology fusion demonstration application mode, protect the integrated application scheme of cloud computing, big data, artificial intelligence and edge computing in construction work monitoring scene, including whole life cycle data linkage and adaptability management decision support model, intelligent algorithm robustness improvement scheme, protect the algorithm optimization technology for complex environment, such as sample enhancement and scene adaptive training model, break through the problem of existing algorithm environment limitation, safety supervision system architecture and process, cover the whole architecture and running process of construction safety supervision system, such as image acquisition device layout, data transmission processing, model deployment application and safety warning mechanism, this system scheme realizes the comprehensive, real-time and intelligent supervision of construction site equipment.

[0133] The above description is only the preferred embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can make equivalent substitutions or changes according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A construction operation safety intelligent monitoring method based on an AI recognition algorithm, characterized by comprising the following steps: Step 1: Obtain defect image samples and non-defect image samples, responsible for defect classification and target feature library training; Step 2: Obtain the original image, perform image preprocessing, compare the feature library, output the processed image, and frame the detected target with an outer rectangle; Step 3: Verify the terminal and the worker's card to form a network and verify the identity information, the camera analyzes the identity information of the operator based on AI recognition, and the terminal and the camera cross-authenticate to confirm the worker, which includes the following steps: The pre-deployment process is as follows: Step 301: Provide each worker with a card with a built-in Bluetooth module, enter the card ID into the Bluetooth module, encrypt the card ID using RSA keys, and the verification terminal is built-in with RSA keys for decryption. The edge computing server has pre-set identity information for each worker and establishes a mapping table between the identity information and the card ID, and the next step is executed; Step 302: Configure the verification terminal as a gateway and complete the initialization, connect each worker's card Bluetooth module to the verification terminal, complete the pre-deployment process, and the Bluetooth modules between the cards act as relays to transmit encrypted data to the verification terminal. Each camera is also configured with the same Bluetooth module as the card to participate in networking, and the camera's Bluetooth module is configured with the device number of each camera; When the worker appears in the camera monitoring screen, the verification operation begins, and the specific steps are as follows: Step 303: The camera captures the original image and transmits it to the edge computing server, which processes the original image and outputs the processed image and compares it with the built-in identity information, which includes the worker's name, personal number, and personal portrait. When comparing, the edge computing server uses MSE mean square error and PSNR peak signal-to-noise ratio to calculate the similarity of the processed image and the personal portrait, and the edge computing server outputs the identity information with the highest similarity, and the next step is entered; Step 304: The verification terminal obtains the device number from the Bluetooth module of the camera that triggered the verification operation, obtains the encrypted data of the Bluetooth module with the strongest connection signal near the camera Bluetooth module, decrypts the encrypted data using the built-in RSA key to obtain the card ID, and enters the next step; Step 305: The verification terminal transmits the card ID to the edge computing server, which finds the corresponding identity information from the mapping table according to the card ID, and enters the next step; Step 306: The edge computing server compares the identity information obtained in steps 303 and 305, and if they match, the worker's identity is confirmed. The edge computing server uploads the matching result to the cloud server, and if they do not match, the edge computing server retrieves the historical records of the successful matching of identity information of the verification terminal and the camera from the cloud service, and the side with more successful times is used as the reference to output the corresponding identity information. ​ Step 4: The supervisor sets the total rule, the cloud server automatically generates the sub-rule based on the processed image and delivers it to the edge computing server for execution, and the worker triggers the total rule or the sub-rule to push the alarm and record the event; Specifically, the following steps are included: Step 401: The supervisor logs in to the system through the client and enters the camera management interface, sets a special tag for each camera according to the monitoring area and the type of work, and the edge computing server classifies all cameras according to the tag, and enters the next step; Step 402: The supervisor sets the total rule on the client, the total rule is uploaded to the cloud server through the client, and then synchronized to the edge computing server, and enters the next step; Step 403: The edge computing server uploads the processed original image of each camera to the cloud server, and the cloud server analyzes the behavior characteristics of the worker through the AI intelligent algorithm of the AI algorithm capability core engine, including personnel action and operation process, and enters the next step; Step 404: The cloud server calls the AI model to analyze the behavior characteristics of the worker, automatically generates the sub-rule, and the sub-rule is a further constraint condition of the total rule, the cloud server transmits the sub-rule to the edge computing server through the distribution device for storage and enablement, and enters the next step; Step 405: When the original image obtained by the camera appears picture change, the edge computing server extracts the processed image from the original image in real time and matches it with the enabled total rule and sub-rule, if the processed image triggers the total rule or the sub-rule, proceed to step 406, if not, do not take any operation, and repeat step 405; Step 406: When the total rule is triggered, the edge computing server automatically generates an alarm message and pushes it to the client through the distribution device for the supervisor to view in real time, when the sub-rule is triggered, a record event is automatically generated, and the record time is pushed to the client in chronological order for the supervisor to browse later, jump to step 405; Step 5: The edge computing server predicts the risk of the worker's behavior characteristics according to the total rule and the sub-rule. 2.The AI identification algorithm-based construction operation safety intelligent supervision method according to claim 1, characterized in that, The defect classification and target feature library training specifically includes the following steps: Step 101: The cloud server uses AI intelligent algorithm to classify defect targets for non-defect image samples and defect image samples, and executes the next step after classification is completed; Step 102: The cloud server uses the AI model in the AI algorithm capability core engine to train the defect classification target, obtains the defect classification target feature library, saves the defect classification target feature library, and executes the next step; Step 103: The saved defect classification target feature library is transmitted to the edge computing server for feature library comparison, and step 103 is repeated until the cloud server obtains new defect image samples and non-defect image samples. 3.The AI identification algorithm-based construction operation safety intelligent supervision method according to claim 1, characterized in that, The camera in the local area network collects original images and directly or indirectly transmits them to the edge computing server. When directly transmitted, the remote device is directly transmitted to the edge computing server through a remote network. The remote device is a camera. When indirectly transmitted, the original images obtained by the camera are saved in a network attached storage, and the original images are transmitted to the edge computing server through file IO by a hard disk file system built in the network attached storage. The system IO built in the verification terminal transmits verification data to the edge computing server through a system serial port or other interfaces. The processing of the original image specifically includes the following steps: Step 201: The image acquisition module of the real-time analysis submodule obtains original images from the remote device, the hard disk file system and the system IO respectively, and transmits the original images to the image pre-processing module for the next step. Step 202: The image pre-processing module performs noise reduction and white balance processing on the original images to obtain pre-processed images, and executes the next step. Step 203a: The pre-processed images are compared with the defect classification target feature library output by the feature training submodule to obtain target recognition, and the detected target is boxed with an external rectangle; Step 203b: The pre-processed images are annotated with reference sizes to obtain target size annotations; Step 203c: The pre-processed images are subjected to clarity judgment, and an image clarity evaluation is output; Steps 203a, 203b and 203c are executed in parallel, and after execution is completed, they are merged into a processed image and output uniformly, and the next step is executed; Step 204: The real-time analysis submodule outputs the processed image, and sequentially labels the image clarity, defect target size and defect target area, and jumps to step 201 for repeated execution. 4.The AI identification algorithm-based construction operation safety intelligent supervision method according to claim 1, characterized in that, The risk prediction specifically includes the following steps: Step 501: The edge computing server calculates the risk value of each camera monitoring area based on the AI intelligent algorithm of the AI algorithm capability core engine according to the enabled sub-rules and in combination with the real-time collected behavior data of the workers, generates a risk prediction event, and enters the next step; Step 502: The edge computing server automatically classifies and sorts all cameras according to the risk level of the risk prediction event, which is divided into high, medium and low, preferentially pushes the prediction event with a high risk level and the corresponding camera picture screenshot to the client of the supervisor through the distribution device, and the client interface preferentially displays the high-risk event, and enters the next step; Step 503: The supervisor checks the pushed risk prediction event on the client, labels the processing result according to the actual situation, the processing result is divided into handled, unhandled and ignored, and the processing result is uploaded to the edge computing server through the client. Set the cumulative threshold number of times, judge whether the number of processing result uploads exceeds the cumulative threshold number of times, if not, jump to step 501 for repeated execution, if yes, execute the next step; Step 504: the edge computing server counts the labeling times of each processing result, if the labeling times are the most, the processed result is retained, if the labeling times are the most, the unprocessed result is retained, the edge computing server will increase the risk level weight corresponding to the sub-rule, and the risk prediction event corresponding to the class with the most unprocessed labeling times will be preferentially pushed in the future, if the labeling times are the most, the ignored processing result is uploaded to the cloud server, and the sub-rule is regenerated.

5. An intelligent supervision system for construction safety based on an AI recognition algorithm, characterized in that The AI recognition algorithm-based construction operation safety intelligent supervision method according to any one of claims 1-4 is implemented, including a client, a cloud server, a distribution device, an edge computing server and a plurality of cameras, the distribution device includes a router and a switch, the cloud server establishes bidirectional communication with the client and the distribution device through the Internet, the distribution device establishes bidirectional communication with the edge computing server and n cameras through an internal local area network, and the verification terminal establishes bidirectional communication with the distribution device through wireless connection or wired connection; An AI intelligent identification platform is built in the cloud server, the AI intelligent identification platform includes a basic information management module, an AI video supervision center, a streaming media service platform and an AI service platform, the basic information management module includes perception device basic information management, first log management, camera management and AI intelligent algorithm setting, the AI video supervision center includes AI monitoring alarm, video carousel supervision, offline monitoring analysis and AI statistics, and the streaming media service platform includes streaming media basic service, alarm management, camera management, subscription time notification, RTSP server management, video acquisition, video distribution, video storage, historical video management, audio and video coding and decoding, download management, voice broadcast and voice intercom. The AI service platform includes an AI algorithm capability core engine and an AI intelligent service, the AI algorithm capability core engine includes an AI algorithm model optimization, a self-learning engine, an AI algorithm basic framework service, an RTSP video acquisition management, an AI model, an AI intelligent algorithm and an AI core algorithm acceleration, and the AI intelligent service includes perception device basic information management, offline data acquisition, second log management, AI identification and retrieval service, business scenario AI service application, message notification, AI video service and AI picture service. 6.The AI identification algorithm-based construction operation safety intelligent monitoring system according to claim 5, characterized in that, In the basic information management module, the perception device basic information management is used for inputting and updating the basic data of the model, position and access mode of the perception device camera, the first log management is used for recording system operation behavior, device running state and abnormal information, the camera management is used for configuring the resolution, frame rate and picture angle of the camera, and the AI intelligent algorithm setting is used for adjusting the detection threshold and identification category of the AI algorithm, and adapting to different identification scenes of construction operation personnel, equipment and defects. In the AI video monitoring center, the AI monitoring alarm analyzes the video stream in real time, detects violations of behavior with the help of the YOLOv12 open source model, and triggers an alarm. The video carousel monitoring is used to cycle through the real-time images of multiple cameras. The offline monitoring and analysis is used to monitor the online status of the camera and the edge computing server in real time, analyze the offline reasons, and the AI statistics are used to summarize the number of alarms, the recognition accuracy and the device online rate. 7.The AI identification algorithm-based construction operation safety intelligent monitoring system according to claim 5, characterized in that, In the streaming media service platform, the streaming media basic service is used to process the RTSP video transmission protocol, the alarm management is used to store, classify and associate the alarm information with the corresponding video segment, the camera management and the basic information management module are used to supplement the camera coding format and the streaming transmission parameter configuration at the streaming media level, the subscription time correction notification is used to push the time correction information to each device, the RTSP server management is used to deploy and maintain the open source live555RTSP server, control the start and stop of the server, limit the number of connections, the video acquisition is used to obtain the original video stream by calling the camera interface, perform preliminary frame extraction, the video distribution is used to distribute the original video stream or the AI processed video stream to the client and the edge computing server, store the video data in the network attached storage, the historical video management is used to provide the search and playback functions of the historical video, the audio and video codec is used for video compression and audio and video decoding, the download management is used to support user download of video segments, and the voice broadcast and voice talkback are used for real-time voice interaction between the supervision end and the construction end. 8.The AI identification algorithm-based construction operation safety intelligent monitoring system according to claim 5, wherein, In the AI service platform, the AI algorithm model optimization and labeling of the AI algorithm capability core engine is learned by the autonomous learning engine, the misidentified and missed identified samples are re-labeled, and the AI model is iteratively optimized. The AI algorithm basic framework service provides sample preprocessing, concurrent processing, communication guarantee, and video stream and picture recognition. The RTSP video acquisition management obtains the RTSP video stream and parses it into frame data for transmission to the AI model. The AI model is used for multi-target detection and identification of workers, equipment and defects in the construction scene. The AI intelligent algorithm extracts target features and classifies defects. The AI core power accelerates the inference speed of the AI model. The perception device basic information management of the AI intelligent service is synchronized with the basic information management module, and the latest state of the device is synchronized to the AI video monitoring center. The offline data acquisition is used to collect the state data of the device when it is offline. The second log management records the call records, recognition results and model running logs of the AI service. The AI recognition and retrieval service is used for fast retrieval of target recognition results and historical recognition data. The business scenario AI service application adapts the AI capability to specific construction scenarios. The message notification is used to push the alarm information and recognition results to the supervisors through SMS or system message. The AI video service provides enhanced video stream after AI analysis, and the AI picture service provides labeled pictures after AI processing.

Images