A construction site safety robot voice alarm linkage system

Abstract

The application discloses a construction site safety side standing robot voice alarm linkage system and relates to intelligent security technology. The system comprises a safety side standing robot, a task and risk rule configuration module, a multi-source sensing and effective monitoring area construction module, a construction object identification and attribute extraction module, a personnel level association, candidate event generation and credible confirmation module, a directional voice alarm scheme generation and execution module, an evidence preservation and platform linkage module, and an improvement review and closed-loop disposal module. The system is implemented in four stages: the first stage is input and sensing; the second stage is identification and judgment; the third stage is alarm linkage; and the fourth stage is review and closed loop. The system has the advantages of reducing missed detection and false association of construction site rule violations, improving the credibility of formal rule violation events, realizing on-site directional voice alarm for target personnel, and forming a traceable improvement closed loop.

Description

Technical Field

[0001] This invention relates to intelligent security technology, and more particularly to a voice alarm linkage system for a safety monitoring robot at a construction site. Background Technology

[0002] Construction sites, especially those involved in substation expansion, power transmission and transformation civil engineering, electrical equipment installation, ground-level work on power lines, and work at heights, are typically characterized by dispersed work sites, overlapping construction procedures, dynamic movement of personnel and machinery, and frequent adjustments to temporary protective facilities. To promptly detect unsafe acts by personnel and risks in the work environment, construction units, supervision units, or contractors usually employ methods such as manual on-site monitoring, fixed video surveillance, intelligent sensing equipment, and mobile inspection equipment for safety management. Existing construction safety on-site monitoring and intelligent alarm technologies mainly include the following categories: 1. Manual on-site monitoring and manual inspection methods Manual on-site supervision typically involves safety management personnel visually inspecting construction workers, protective equipment, the work environment, and the work process near the work site; when non-compliant behaviors are found, they provide on-site reminders or report them. This method allows for judgment based on the current work procedures and site environment, and is often used in conjunction with regular patrols, spot checks, and pre-shift safety briefings.

[0003] 2. Security identification and alarm methods based on fixed video surveillance Fixed cameras can be deployed at construction sites, including entrances, passageways, work platforms, edge openings, temporary protective areas, or high points. Video data is transmitted to edge computing devices or a management platform, and visual recognition algorithms identify targets such as personnel, protective equipment, area intrusions, smoke, and abnormal behavior. When a suspected risk is detected, the system can alert and report it via audio-visual equipment, on-site broadcasts, management personnel's mobile terminals, or platform messages. This system is suitable for continuous monitoring of fixed areas.

[0004] 3. Specific detection methods for specific violations Existing technologies can also identify and issue warnings for behaviors such as crossing guardrails or entering restricted areas, improper work attire, not wearing safety helmets, smoking, and not properly using safety belts when working at heights, using methods such as target detection, personnel tracking, image analysis, electronic fences, or wearable sensors. The identification results are typically used to trigger audible and visual alarms, on-site broadcasts, capture evidence, or report to the platform.

[0005] 4. On-site monitoring methods based on mobile inspection and safety monitoring robots To improve coverage of temporary work areas and dynamic work points, mobile inspection safety monitoring robots can be equipped with video acquisition, positioning, behavior recognition, communication, and voice interaction modules to collect on-site images near the inspection path. After identifying violations or safety hazards, the robots can issue reminders through their speakers, sound and light devices, or remote voice communication, and transmit images, videos, or alarm information.

[0006] 5. Alarm output, information reporting, and handling management methods Existing systems typically output alarms through on-site broadcasts, fixed audible and visual alarms, loudspeakers on safety monitoring robots, notifications from management personnel's mobile terminals, or platform message pushes. The management platform can perform event querying, statistics, retention, and rectification tracking. Thus, the construction site already has the technical foundation for video recognition, mobile inspection, voice reminders, event reporting, and back-end management.

[0007] Based on the aforementioned background technology, although existing construction site safety monitoring, video recognition, mobile inspection, and alarm systems can detect some personnel violations and provide alerts, the following technical problems still exist when the safety monitoring robot implements voice alarm linkage: 1. A lack of unified personnel-level correlation and organization in various types of violations. Existing technologies often set up separate recognition models or alarm rules for single targets such as safety helmets, safety belts, fence crossings, work clothes, or smoking. The results are often presented as single-frame images, snapshots, or independent alarm messages. When multiple people are working, there is obstruction, or the perspective changes, it is difficult to accurately associate protective equipment, area facilities, and the same construction worker. It is also difficult to determine the attribution, priority, and order of handling of the event, which can easily lead to inconsistencies between alarm results and the actual person violating the rules or the actual work status.

[0008] 2. Single-frame or short-time recognition is prone to generating false alarms, missed alarms, or duplicate alarms. Construction sites are constantly changing due to factors such as lighting, dust, rain, fog, vibration, obstruction, camera distance, and temporary facilities. Small objects like safety helmets, safety belt hooks, and cigarette holders are also easily affected by dirt, posture, and background. If alarms are issued based solely on a single frame or short-term recognition results, temporary obstruction or unstable detection may be misjudged as violations, or repeated alarms may be issued for the same persistent violation.

[0009] 3. The on-site voice alarms lack target targeting and adaptability. Fixed broadcast or distributed audio-visual alarms are typically directed towards a specific area, which can easily interfere with unrelated personnel. Furthermore, on-site noise, changes in personnel position, and temporary obstructions may prevent violators from clearly receiving the information. Existing mobile security monitoring robots often use generic logic for voice broadcasts, failing to adequately consider the target personnel's location, the relative position and orientation of the security monitoring robot, ambient noise, the type of violation, and the risk level to determine the appropriate alarm method.

[0010] 4. There is a lack of continuous linkage between alarms, rectification reviews, and platform records. While existing systems can identify anomalies, provide on-site alerts, and report to the platform, they struggle to continuously confirm whether the same person has rectified the issue. Furthermore, they lack logic for providing follow-up alerts, escalation notifications, or manual intervention for unrectified, recurring, or high-risk events, tailored to their risk levels. When the location, task, and communication status of the safety monitoring robot continuously change, the lack of standardized recording of event time, target personnel, on-site location, safety monitoring robot status, and handling status can easily lead to duplicate reporting, delayed alarms, or inconsistencies between local and platform records.

[0011] Therefore, it is necessary to coordinate the handling of personnel-level association, time-series confirmation, targeted voice alarms, event evidence retention, rectification review and escalation of various types of violations in order to improve the accuracy, relevance and traceability of on-site safety alarms. Summary of the Invention

[0012] The purpose of this invention is to provide a voice alarm linkage system for a construction site safety monitoring robot, so as to solve the problems existing in the prior art.

[0013] The construction site safety monitoring robot voice alarm linkage system described in this invention includes a safety monitoring robot, a task and risk rule configuration module, a multi-source perception and effective monitoring area construction module, a construction object identification and attribute extraction module, a personnel-level association, candidate event generation and credible confirmation module, a directional voice alarm scheme generation and execution module, an evidence retention and platform linkage module, and a rectification review and closed-loop handling module. System operation includes the following stages: Phase 1: Input and Sensing; The task and risk rule configuration module provides task, area, risk, and alarm rules; The multi-source sensing and effective monitoring area construction module forms on-site sensing records and effective monitoring areas with unified timestamps, providing consistent input for subsequent modules; Phase Two: Identification and Judgment; The construction object identification and attribute extraction module outputs structured construction object information; The personnel-level association, candidate event generation and credible confirmation module generates formal violation events, target personnel locations, risk levels and related evidence, providing a unified event input for targeted voice alarms; Phase 3: Alarm Linkage; The targeted voice alarm solution generation and execution module transforms formal violation events into on-site handling by the safety monitoring robot; The evidence retention and platform linkage module uniformly retains and interacts with events, execution results, and platform instructions; Phase 4: Review and Closed Loop; The rectification review and closed-loop handling module continuously tracks the rectification status of the target personnel based on the retained event evidence, alarm execution results, and platform linkage status, and completes the event de-archiving, re-alarming, manual review, or escalation handling accordingly.

[0014] The voice alarm linkage system for a construction site safety monitoring robot described in this invention has the advantages of achieving the following specific technical effects: 1) Reduce missed detections and false associations in the identification of violations at construction sites. By effectively constructing monitoring areas, using lightweight multi-task visual networks, and establishing spatiotemporal associations between people, objects, and areas, targets such as safety helmets, work clothes, safety belts, and cigarettes can be associated with their corresponding personnel and work areas, reducing the chances of misidentifying other personnel's protective equipment or background targets as the status of the target personnel.

[0015] 2) Improve the credibility of formal violation events. Candidate violation events are confirmed by continuous frames, duration, recognition confidence, image quality, and scene constraints. Duplicate events are suppressed and merged to avoid duplicate alarms caused by single-frame misidentification, short-term occlusion, or personnel crossing.

[0016] 3) Enable targeted voice alarms for specific personnel. The safety monitoring robot determines the alarm location, broadcast direction, script, volume, and number of broadcasts by combining the location of the target personnel, accessible safe areas, environmental noise, obstruction, and risk level. Compared with fixed broadcast methods, this improves the audibility and relevance of alarms and reduces interference with irrelevant personnel.

[0017] 4) Forming a traceable rectification loop. The safety monitoring robot retains and synchronizes the video before and after the alarm, alarm parameters, execution results, and review results with the platform; it only confirms that the rectification has been completed when the continuous review conditions are met, and implements re-alarm or escalation handling for unrectified, high-risk, or alarm-ineffective events, thereby improving the closed-loop and traceability of on-site safety management. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the operation and closed-loop process of the system described in this invention.

[0019] Figure 2 This is a schematic diagram of the architecture of the construction object identification and attribute extraction module described in this invention.

[0020] Figure 3 This is a flowchart illustrating the personnel-level association, candidate event generation, and trusted confirmation module described in this invention. Detailed Implementation

[0021] The present invention discloses a construction site safety monitoring robot voice alarm linkage system, which includes a safety monitoring robot, a task and risk rule configuration module, a multi-source perception and effective monitoring area construction module, a construction object identification and attribute extraction module, a personnel-level association, candidate event generation and credible confirmation module, a directional voice alarm scheme generation and execution module, an evidence retention and platform linkage module, and a rectification review and closed-loop handling module.

[0022] The system addresses risks such as illegally crossing guardrails or entering restricted areas, not wearing work clothes as required, not wearing safety helmets, smoking, and not using safety belts properly when working at heights. It organizes on-site perception, object recognition, credible event confirmation, directional voice alarms, evidence linkage, and rectification review into a continuous data processing and on-site handling chain.

[0023] like Figure 1 As shown, the system operation flow includes the following steps in sequence: Step 1, task and risk rule configuration; Step 2, multi-source perception and effective monitoring area construction; Step 3, construction object identification and attribute extraction; Step 4, personnel-level association, candidate event generation and credible confirmation; Step 5, directional voice alarm scheme generation and execution; Step 6, evidence retention and platform linkage; Step 7, rectification review and closed-loop handling.

[0024] Phase 1: Input and Perception.

[0025] The input and perception phase is completed collaboratively by the task and risk rule configuration module and the multi-source perception and effective monitoring area construction module: the former provides task, area, risk and alarm rules, and the latter forms on-site perception records and effective monitoring areas with unified timestamps, providing consistent input for subsequent modules.

[0026] Step 1: Configure task and risk rules.

[0027] Before performing an inspection task, the safety monitoring robot obtains the task information from the management platform, local task module, or on-site management terminal and configures a risk map. The monitoring information includes at least the inspection area, start and end times, current task, restricted areas, guardrail locations, high-altitude work areas, and corresponding risk levels. Restricted areas, guardrail locations, and high-altitude work areas are represented by polygonal regions, line segments, or preset spatial coordinate points, and mapped to the on-site coordinate system used by the safety monitoring robot.

[0028] The safety monitoring robot is loaded with risk rules and alarm voice templates corresponding to various violations. The risk rules include criteria for determining violations such as illegally crossing guardrails or entering restricted areas, improper work attire, not wearing a safety helmet, smoking, and improper use of safety belts when working at heights, as well as risk levels, alarm repetition intervals, and escalation conditions. Alarm voice templates correspond to the violation type and risk level.

[0029] When the construction area, guardrail location, work tasks, or risk level are adjusted, the management platform or on-site management terminal sends an update command to the safety monitoring robot. The safety monitoring robot updates its local risk map and rule parameters and continues to perform inspections with the updated configuration.

[0030] Step 2: Construction of multi-source sensing and effective monitoring areas.

[0031] During inspections, the safety monitoring robot collects continuous video frames at a preset sampling frequency and simultaneously acquires its own position, orientation, movement speed, gimbal status, ambient noise, and current task status. The robot's pose is obtained through satellite positioning, inertial measurement, lidar positioning, visual positioning, or any combination thereof. Ambient noise is collected by a microphone and converted into noise level data.

[0032] The safety monitoring robot adds a unified timestamp to video frames and synchronization status data, forming a field perception record that includes image data, acquisition time, robot pose, noise level, and task status. The robot reads this record according to the unified timestamp and determines the current effective monitoring area based on its position, camera orientation, inspection task, and risk map. This effective monitoring area includes at least the general work area, restricted area, area adjacent to guardrails, and work area at height. The effective monitoring area is determined by a preset image area, a risk map mapping area, or the intersection of both; irrelevant background outside the designated area is not considered in the violation determination.

[0033] The safety monitoring robot performs one or more of the following processing on the effective monitoring area: image stabilization, distortion correction, brightness adjustment, noise reduction, size normalization, and local cropping, magnification, or enhancement. This reduces the impact of the robot's movement, lighting changes, dust, occlusion, and complex backgrounds, and improves the recognition of small targets such as safety helmets, cigarettes, and seat belt hooks.

[0034] The safety monitoring robot also features a pre-alarm video loop buffer, which temporarily stores video frames and corresponding status data for a preset time period prior to the current moment. When a formal violation is subsequently confirmed, the pre-alarm video clips are extracted and combined with the post-alarm video clips to form event evidence. This module outputs processed video frames, effective monitoring areas, area type labels, and on-site perception records for use by the construction object identification and attribute extraction module.

[0035] Phase Two: Identification and Judgment.

[0036] The identification and judgment phase is completed sequentially by the construction object identification and attribute extraction module, and the personnel-level association, candidate event generation and credible confirmation module. The former module outputs structured construction object information, and the latter module generates formal violation events, target personnel locations, risk levels and related evidence, providing a unified event input for targeted voice alarms.

[0037] Step 3: Identification and attribute extraction of construction objects.

[0038] like Figure 2 As shown, the construction object identification and attribute extraction module inputs the processed effective monitoring area image output by the multi-source perception and effective monitoring area construction module into a shared convolutional-Transformer multi-task vision network. The Transformer multi-task vision network includes a shared visual feature extraction backbone, multi-scale feature fusion units, personnel and protective equipment detection branches, work area segmentation branches, and human key point and motion feature branches, and outputs a unified structured recognition result at the edge of the safety monitoring robot.

[0039] For the Frame input image The network first extracts edge, texture, color, and local shape information from a lightweight convolutional input layer and a local feature extraction unit to represent detailed targets such as the outline of the safety helmet, reflective strips on work clothes, cigarettes, safety belt hooks, safety ropes, and guardrail boundaries. Then, mid- or deep features are input into the Vision Transformer module, which uses a self-attention mechanism to establish relationships between personnel poses, the positional relationship between personnel and the work area, and the global scene consisting of multiple people, machinery, and construction facilities.

[0040] To reduce the inference overhead at the edge of the safety monitoring robot, the Transformer encoding module is placed in the middle or deep features, or a windowed self-attention approach is used. The shared backbone is represented as: ,in, This indicates a shared convolutional Transformer backbone. , and These are characteristics of the shallow, intermediate, and deep layers, respectively.

[0041] To address the scale difference between small targets such as cigarettes, seatbelt hooks, and long-range safety helmets, and large targets such as people, guardrails, and work areas, the network performs channel transformation, upsampling, downsampling, feature concatenation, or weighted summation on F1, F2, and F3 to obtain multi-scale fused features. The k-th fused feature is represented as: ; Among them, high-resolution fusion features are used for small targets, medium-resolution fusion features are used for tooling, safety belts and key parts of the human body, and low-resolution fusion features are used for personnel, guardrails, work areas and the overall construction scene.

[0042] Multi-scale fusion features are input into the personnel and protective equipment detection branch, the work area segmentation branch, and the human body key point and motion feature branch, respectively. The outputs of each branch together constitute structured construction object information for processing by the personnel-level association, candidate event generation, and reliable confirmation modules.

[0043] The personnel and protective equipment detection branch identifies at least construction workers, safety helmets, work clothes, work uniforms or reflective vests, safety belts, safety ropes and hooks, cigarettes, smoke or fire points, guardrails, warning lines, temporary fencing, and safety anchors or lifelines. Based on the target scale, predictions are made on the corresponding fusion features, and the target category, target location, detection confidence level, and timestamp are output. ; in, Indicates the first The first frame One test result, Indicates the target category. This represents the target detection box or target region. This indicates the confidence level of the detection.

[0044] The work area segmentation branch performs pixel-level classification or boundary extraction of the work areas in the image, outputting at least permitted work areas, prohibited areas, high-altitude work areas, guardrail and warning line boundaries, and hoisting, live-line, and falling object hazard areas. The segmentation results are represented using region masks, boundary lines, or closed polygons, and matched or fused with the risk map provided by the task and risk rule configuration module. When the positional deviation between the two exceeds a preset range, the corresponding area is marked as an area to be updated.

[0045] The human body key point and motion feature branch takes the detected personnel area as input and extracts key human body parts such as the head, mouth, left and right hands, torso, waist, and legs. Key points and motion features are used to determine the head area corresponding to the safety helmet, the torso area corresponding to the work clothes, the shoulder or waist area corresponding to the safety belt, the hand and mouth areas corresponding to the cigarette, and the body position and direction of movement when the person crosses a guardrail. For smoking behavior, it also extracts continuous motion sequences of the hand approaching the mouth, remaining near the mouth, and moving away from the mouth. ; in, This represents the keypoint result of the j-th person in the t-th frame, where M represents the total number of keypoints. This indicates the position of the m-th key point. This indicates the confidence level of its key points.

[0046] Considering the limited number of violation samples at construction sites, and the differences in lighting, personnel clothing, construction facilities, and background environment among different construction sites, a visual model pre-trained on general images, personnel detection, or human behavior data can be selected as the initial model. This model can then be fine-tuned using personnel, protective equipment, work areas, and action samples collected and labeled at the construction site. For scenarios with few samples, such as cigarettes, hooks, and improper use of safety belts during work at heights, image enhancement, category resampling, or category weight adjustment can be employed.

[0047] The Transformer multi-task vision network is trained using a joint loss: ; in, , and These represent the target detection loss, region segmentation loss, and human keypoint or motion feature loss, respectively. , and This indicates the weight of the corresponding task. Through multi-task joint training, basic visual features are shared among personnel, protective equipment, work areas, and body parts.

[0048] To balance visual recognition accuracy and inference speed at the edge of the safety monitoring robot, a teacher-student knowledge distillation approach is used to train the Transformer multi-task visual network in a lightweight manner. The teacher model employs a Vision Transformer or a hybrid convolutional-Transformer model with a relatively large number of parameters. The student model uses a lightweight multi-task network suitable for the edge computing module of the safety monitoring robot. During training, the student model simultaneously learns ground truth annotation information, the class probabilities output by the teacher model, and intermediate features. ; in, This represents the distillation loss between the output probabilities of the teacher model and the student model. This represents the distillation loss between the two intermediate characteristics. and This indicates the corresponding weight.

[0049] After training, the student model is deployed on the edge computing module of the safety monitoring robot for real-time target detection, region segmentation, and human key point recognition. Recognition results with confidence levels below a preset lower limit, or conflicting results across branches, are marked as pending confirmation and do not directly constitute formal violation events. Instead, they are further processed by the personnel-level association, candidate event generation, and credible confirmation modules, combining multi-frame evidence and scene rules.

[0050] For each video frame, a structured recognition result is generated, including at least personnel and protective equipment targets, guardrail and safety anchor point targets, work area masks or boundaries, key human body points and motion features, confidence level of each recognition result, timestamp, and image quality identifier. These results are sent to the personnel-level association, candidate event generation, and trusted confirmation module to assign temporary personnel identifiers to personnel in consecutive video frames and establish associations between personnel and protective equipment, work areas, facilities, and actions.

[0051] Step 4: Personnel-level association, candidate event generation, and credible confirmation.

[0052] Using the aforementioned risk map, effective monitoring areas, and structured results as common inputs, a construction safety scenario relationship graph is constructed. Nodes in the graph include personnel, protective equipment, work facilities, risk areas, and actions. Edges between nodes represent relationships such as wearing, being adjacent to, being located at, crossing, holding, and attaching.

[0053] The safety monitoring robot uses temporary personnel identifiers in continuous video frames as the main guideline to maintain the position, movement trajectory, head area, torso area, key areas of hands and mouth of each construction worker, type of area they are in, and relative position to surrounding protective equipment or facilities. For protective equipment, guardrails, or work facilities that meet preset conditions in terms of distance from personnel, spatial overlap, or maintain a stable correspondence in continuous frames, the safety monitoring robot associates them with the corresponding temporary personnel identifier.

[0054] To quantify the correlation between person i and object j in frame t, the safety monitoring robot calculates the correlation score: ; in, For personnel Key areas For object The area For regional overlap, For normalized distance, Indicates whether the region, action, or connection relationship is satisfied. , and The preset weights are used. When When the association threshold of the corresponding object is not lower than that of the object, the object will be... Associated with personnel .

[0055] Based on personnel-level associations, the safety monitoring robot generates five types of candidate violation events: When a person's trajectory crosses a guardrail, warning line, or restricted area boundary and enters a restricted zone, a candidate event for crossing a guardrail or entering a restricted area is generated. When a person's head or torso area is not associated with a valid safety helmet or workwear, a candidate event for corresponding protective equipment violation is generated. When a cigarette, smoke, or fire point maintains a preset proximity and motion relationship with a person's hands or mouth in consecutive frames, a candidate event for smoking is generated. When a person is in a high-altitude work area and is not effectively associated with a safety belt, safety rope, hook, or safety anchor point, a candidate event for improper use of a safety belt is generated.

[0056] Each candidate violation event includes at least a temporary personnel identifier, violation type, candidate occurrence time, candidate occurrence location, associated object, region type, identification confidence level, and evidence frame index. Candidate events are entered into a multi-frame evidence queue maintained by personnel identifier and violation type, and do not directly trigger voice alarms.

[0057] Each multi-frame evidence queue stores the time series, correlation score, recognition confidence, image quality, personnel location, and evidence frames of candidate events. Only records of the same person, similar violations, and consecutive spatial locations are accumulated. The safety monitoring robot counts the number of consecutively recognized frames, event duration, average recognition confidence, and regional relationship stability within a preset observation time window.

[0058] For violations of type Candidate events, when their multiple frames of evidence meet preset confirmation conditions, are confirmed to form formal violation events: ; in, Type of violation The official confirmation of the incident, To effectively and continuously identify the number of frames, Duration of the event To achieve the average recognition confidence level, To ensure that the relationship between region type, personnel trajectory, and protective equipment meets the scenario constraints for this type of violation, , and Thresholds set according to violation type. Low-quality footage is not considered valid evidence frames.

[0059] Different violation types are configured with different confirmation conditions: For crossing or entering a restricted area, the person must remain on one side of the restricted area for a preset duration. For helmet, workwear, and safety belt incidents, there should be a continuous lack of a valid associated object in consecutive frames. For smoking incidents, the conditions of a cigarette or smoke target, hand-mouth proximity, and continuous action must be met simultaneously. For candidate events with insufficient confidence, mismatched area types, or changes in personnel identification, the safety monitoring robot will either maintain observation or withdraw the candidate event.

[0060] For confirmed formal violations, the safety monitoring robot merges events with the same temporary personnel identifier, the same violation type, and a location change within a preset range within a preset time interval to avoid duplicate alarms. When the same person has multiple formal violation events, the priority events are determined based on risk level, work area type, duration, and previous alarm count.

[0061] For formal violations requiring priority handling, the security monitoring robot calculates a risk ranking value: ; in, The risk ranking value. The basic risk weight for each type of violation. The risk level of the area where the personnel are located. Duration of the event For preset reference duration, This is the number of times the same type of violation is repeated within a preset time period. , , and To adjust the weights, the system... The order of handling is determined from high to low, and the formal violation incident, risk level, location of the target personnel, and related evidence information are sent to the targeted voice alarm scheme generation and execution module.

[0062] like Figure 3 As shown, candidate violation events are generated through a construction safety scenario relationship diagram, and formal violation events are output after multi-frame evidence accumulation, credible confirmation, event merging, and risk ranking.

[0063] Phase 3: Alarm linkage.

[0064] The alarm linkage phase is completed collaboratively by the targeted voice alarm scheme generation and execution module and the evidence retention and platform linkage module: the former transforms formal violation events into on-site handling by the safety monitoring robot, and the latter uniformly retains and interacts with the event, execution results and platform instructions.

[0065] Step 5: Generating and executing a targeted voice alarm scheme.

[0066] The directional voice alarm generation and execution module uses the same formal violation event as the trigger object, combining the safety monitoring robot's movement position, orientation control, and voice output parameters into an alarm execution parameter group. It acquires the target personnel's location, violation type, risk level, and related evidence, and combines this with the safety monitoring robot's current location, passable areas, restricted boundaries, on-site obstacles, environmental noise, and the target personnel's surrounding work status to generate alarm location, planned path, broadcast orientation, script template, broadcast volume, broadcast frequency, and broadcast interval.

[0067] The safety monitoring robot selects candidate alarm locations from accessible safe areas on the risk map. Candidate locations avoid restricted areas, hoisting slewing ranges, vehicle access routes, and high-risk work surfaces, and maintain a preset safe distance from target personnel. If the target personnel are within the current effective broadcast range, their current location is used as a candidate location. For each candidate location q, the alarm location utility value is calculated: ; Where U(q) is the utility value of the candidate alarm location q, A(q) is the voice reachability determined by positional relationship, occlusion, and environmental noise, O(q) is the observability of the target personnel, D(q) is the travel distance of the safety monitoring robot to the candidate location, and R(q) is the passage or operation risk corresponding to the candidate location. , , and The system uses preset weights to select the candidate position with the highest utility value.

[0068] After determining the alarm location, the safety monitoring robot determines the orientation of its body or voice output device based on the directional relationship between the alarm location and the target person's location. When a directional speaker is configured, it faces the target person. When a regular speaker is configured, the robot's body faces the target person. The safety monitoring robot selects the corresponding script template based on the type of violation and risk level, generating an emergency script requiring immediate cessation of related work for high-risk events such as entering restricted areas, illegally crossing guardrails, or failing to properly use safety belts while working at heights.

[0069] The broadcast volume is adaptively determined based on ambient noise and the distance from the alarm location to the target person: ; Where V represents the actual broadcast volume. Where N is the reference volume, and N is the current ambient noise. Let d be the reference noise level, and d be the distance from the alarm location to the target personnel. and These are the minimum and maximum allowed broadcast volumes, respectively. and This is a preset adjustment factor. Volume limiting prevents excessive disturbance in low-noise areas or makes prompts in high-noise areas difficult to hear.

[0070] Before execution, the safety monitoring robot verifies the event status and parameter generation time. Events that have been withdrawn, manually handled, or merged with higher-risk events are canceled from execution, and the remaining events are placed in the alarm task queue. When multiple events are pending handling simultaneously, the safety monitoring robot arranges the execution order according to the risk ranking value output by the personnel-level association, candidate event generation, and trusted confirmation modules. High-risk events will preempt low-risk alarm tasks after the safe parking conditions are met.

[0071] The safety monitoring robot moves to the alarm location according to the planned path, continuously acquiring its own pose, the status of surrounding obstacles, and the latest location of the target personnel. If the path is blocked by temporary material piles, work vehicles, or personnel gatherings, or if the target personnel's location changes beyond a preset range, the safety monitoring robot regenerates the alarm location, plans the path, and announces the direction. If it cannot reach the location within a preset time, it performs a downgraded announcement at the current safe parking location. Upon arrival, the safety monitoring robot executes a directional voice alarm according to the script template, announcement volume, number of announcements, and announcement interval, and continuously monitors the target personnel during the announcement.

[0072] After each broadcast, the security monitoring robot records the actual alarm location, broadcast direction, volume level, script identifier, execution start and end time, target personnel location, and execution result. The execution result includes at least alarm completion, target loss, failure to arrive, task preemption, and equipment malfunction. The result is not directly used as a conclusion of rectification completion, but is sent along with the incident evidence to the evidence retention and platform linkage module.

[0073] Step Six: Evidence Preservation and Platform Collaboration.

[0074] The evidence retention and platform linkage module generates traceable event evidence packages for formal violations and their alarm execution processes. Each event evidence package includes at least the event identifier, temporary personnel identifier, violation type, risk level, location and time of occurrence, alarm location, alarm execution parameters, alarm execution result, video clips before and after the alarm, key evidence frames, and pose and environmental status data of the safety monitoring robot. The video clips before the alarm are obtained through a loop cache from the multi-source perception and effective monitoring area construction module, while the video clips after the alarm are used for subsequent rectification and review.

[0075] The safety monitoring robot sends event evidence packages to the safety management platform using the event identifier as the association key. The platform returns a receipt confirmation message. If no confirmation message is received, the safety monitoring robot retransmits the evidence packages according to the preset number of retransmissions and caches them locally. In the event of a communication interruption, the safety monitoring robot continues to perform on-site alarms and evidence caching, and retransmits the evidence packages according to the event occurrence time and risk level after the network is restored, in order to avoid network conditions affecting on-site safety procedures.

[0076] The security management platform displays the location of the violation, evidence video, and alarm execution status according to the event identifier, and issues instructions for manual confirmation, remote review, alarm suspension, or escalation of handling. The security monitoring robot only updates the task status after receiving a legitimate task instruction and completing security verification. For duplicate reports of the same event, the platform merges them according to the event identifier, thereby maintaining consistency between the security monitoring robot, the platform, and subsequent review records.

[0077] The system sends the retained evidence, platform linkage status, and alarm execution results to the rectification review and closed-loop handling module. The system then continues to rectify and review the target personnel and their associated protection status, and closes the event or triggers escalation handling based on the review results.

[0078] Phase 4: Review and Closed Loop.

[0079] The closed-loop review phase is completed by the rectification review and closed-loop handling module. Based on the retained event evidence, alarm execution results and platform linkage status, it continuously tracks the rectification status of the target personnel and completes the event de-archiving, re-alarming, manual review or escalation handling accordingly.

[0080] Step 7: Rectification, review, and closed-loop management.

[0081] After issuing an on-site alarm, the safety monitoring robot marks the formal violation as pending review and re-acquires video frames, temporary personnel identifiers, location trajectories, and associated protection status of the target personnel's location at a preset review time. The review area prioritizes covering the location of the incident, the alarm location, and the target personnel's movement trajectory after the alarm. If a person temporarily leaves the frame, the personnel identifier cannot be stably matched, or the image quality is insufficient, it is only marked as pending further review or pending manual review; the loss of personnel is not automatically considered as rectification having been completed.

[0082] The safety monitoring robot executes corresponding rectification and review rules according to the type of violation: For violations such as crossing guardrails or entering restricted areas, it confirms that personnel have returned to the permitted work area and remained there for the preset duration. For violations such as not wearing a safety helmet, not wearing work clothes as required, or not using a safety belt properly when working at heights, it confirms that personnel have re-linked effective protective equipment or have left the corresponding work area at heights. For smoking incidents, it confirms that the association between the cigarette, smoke, or fire point and the personnel's hand and mouth areas has continuously disappeared.

[0083] For events with temporary personnel identified as i and violation type k, the safety monitoring robot determines whether rectification has been completed based on continuous valid review frames: ; in, Type of violation The rectification confirmation results To meet the requirements for the number of consecutive valid review frames, To maintain the rectification status for a certain period of time, The average confidence level for identifying the rectification status. This indicates that personnel identification, work area, and associated protection status meet the rectification constraints for this type of violation. , and For the corresponding threshold. Only when If the system determines that the event has been rectified, it will be left as unrectified, pending further review, or pending manual review.

[0084] For incidents that have been rectified, the safety monitoring robot will deactivate any subsequent voice alarms and movement tasks corresponding to the incident, and continue to monitor for a preset observation period to see if a formal incident involving the same person and the same type of violation occurs again. If no further incidents occur during the observation period, the rectification status will be updated to "rectified".

[0085] The system archives rectified events and synchronizes them to the security management platform via the evidence retention and platform linkage module. The archived records include at least the original evidence of violations, alarm execution records, rectification review video clips, rectification confirmation time and location, event status change records, and platform confirmation information to support full-process traceability from violation discovery, on-site alarm, rectification confirmation, and event closure.

[0086] For events that have not been rectified or require further review, the system assesses the appropriate course of action based on risk level, number of alarms, duration of non-rectification, and equipment status. Events that do not meet the escalation criteria are returned to the targeted voice alarm solution generation and execution module, where the alarm location, script, or broadcast parameters are updated before the targeted voice alarm is re-implemented. Events awaiting manual review are synchronized to the platform via the evidence retention and platform linkage module, awaiting legitimate manual confirmation or remote review instructions.

[0087] The system will trigger an escalation process when an incident remains unrectified within a preset review period, the same person repeatedly commits the same type of violation within a preset time period, the incident is classified as high-risk, the safety monitoring robot fails to reach the effective alarm location multiple times, or the voice output device or positioning device malfunctions and affects the alarm effect. The escalation conditions and their thresholds are pre-configured by the task and risk rule configuration module.

[0088] Escalation measures include setting an escalation flag, increasing the event's priority in the alarm task queue, and sending an escalation notification, event evidence package, and real-time location to the safety management platform via the evidence retention and platform linkage module. The platform can also execute repeated broadcasts, remote video verification, or notification of on-site safety management personnel based on platform instructions. For high-risk events such as entering restricted areas, illegally crossing guardrails, and failing to properly use safety belts while working at heights, the platform can directly send escalation notifications to the designated safety personnel.

[0089] Event evidence and status records are continuously retained. Even after subsequent verification confirms rectification, escalated events retain escalation markers and handling records. Thus, the system forms a closed-loop security monitoring system encompassing "violation discovery—targeted alert—rectification review—de-archiving or re-alerting / escalation handling."

[0090] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this invention.

Claims

1. A voice alarm linkage system for a construction site safety monitoring robot, characterized in that, It includes a safety monitoring robot, a task and risk rule configuration module, a multi-source perception and effective monitoring area construction module, a construction object identification and attribute extraction module, a personnel-level association, candidate event generation and credible confirmation module, a directional voice alarm scheme generation and execution module, an evidence retention and platform linkage module, and a rectification review and closed-loop disposal module. System operation includes the following stages: Phase 1: Input and Sensing; The task and risk rule configuration module provides task, area, risk, and alarm rules; The multi-source sensing and effective monitoring area construction module forms on-site sensing records and effective monitoring areas with unified timestamps, providing consistent input for subsequent modules; Phase Two: Identification and Judgment; The construction object identification and attribute extraction module outputs structured construction object information; The personnel-level association, candidate event generation and credible confirmation module generates formal violation events, target personnel locations, risk levels and related evidence, providing a unified event input for targeted voice alarms; Phase 3: Alarm Linkage; The targeted voice alarm solution generation and execution module transforms formal violation events into on-site handling by the safety monitoring robot; The evidence retention and platform linkage module uniformly retains and interacts with events, execution results, and platform instructions; Phase 4: Review and Closed Loop; The rectification review and closed-loop handling module continuously tracks the rectification status of the target personnel based on the retained event evidence, alarm execution results, and platform linkage status, and completes the event de-archiving, re-alarming, manual review, or escalation handling accordingly.

2. The voice alarm linkage system for a construction site safety monitoring robot according to claim 1, characterized in that, The first stage includes the following steps: Step 1: Configure task and risk rules; Before performing the inspection task, the safety monitoring robot obtains the task information from the management platform, local task module or on-site management terminal and configures the risk map; the safety monitoring robot loads the risk rules and alarm voice templates corresponding to various violations; the alarm voice templates correspond to the violation type and risk level; Step 2: Construction of multi-source sensing and effective monitoring areas; During inspections, the safety monitoring robot collects continuous video frames at a preset sampling frequency and simultaneously acquires its own position, orientation, movement speed, gimbal status, environmental noise, and current task status. A unified timestamp is added to the video frames and synchronized status data, forming a field perception record that includes image data, acquisition time, robot pose, noise level, and task status. The field perception record is read according to the unified timestamp, and the current effective monitoring area is determined based on the robot's position, camera orientation, inspection task, and risk map. A pre-alarm video loop buffer is set up to temporarily store video frames and corresponding status data for a preset time period prior to the current moment. When a formal violation is subsequently confirmed, the pre-alarm video clips are extracted and combined with the post-alarm video clips to form event evidence.

3. The voice alarm linkage system for a construction site safety monitoring robot according to claim 2, characterized in that, The second stage includes the following steps: Step 3: Construction object identification and attribute extraction; The processed effective monitoring area image output by the multi-source perception and effective monitoring area construction module is input into the shared convolutional-Transformer multi-task vision network, and a unified structured recognition result is output at the edge of the safety monitoring robot; Step 4: Personnel-level association, candidate event generation and reliable confirmation; The safety monitoring robot uses temporary personnel identifiers in continuous video frames as the main line to maintain the position, movement trajectory, head area, torso area, key areas of hands and mouth of each construction worker, type of area, and relative position with surrounding protective equipment or facilities; For protective equipment, guardrails or work facilities that meet preset conditions in terms of distance from personnel, spatial overlap, or maintain a stable correspondence in continuous frames, the safety monitoring robot will associate them with the corresponding temporary personnel identifier; Based on personnel-level relationships, the security monitoring robot generates candidate violation events. Each candidate violation event includes at least a temporary personnel identifier, violation type, candidate occurrence time, candidate occurrence location, associated object, area type, identification confidence level, and evidence frame index. Candidate events are entered into a multi-frame evidence queue maintained by personnel identifier and violation type, and do not directly trigger voice alarms. Each multi-frame evidence queue stores the time series, correlation score, recognition confidence, image quality, personnel location, and evidence frames of candidate events; only records of the same person, the same type of violation, and continuous spatial location are accumulated; the safety monitoring robot counts the number of consecutive recognition frames, event duration, average recognition confidence, and regional relationship stability within a preset observation time window; For confirmed formal violations, the safety monitoring robot merges events with the same temporary personnel identification, the same violation type, and a location change within a preset range within a preset time interval to avoid duplicate alarms. When the same person has multiple formal violation events, the priority events are determined based on risk level, work area type, duration, and previous alarm count.

4. The voice alarm linkage system for a construction site safety monitoring robot according to claim 3, characterized in that, The third stage includes the following steps: Step 5: Targeted Voice Alarm Scheme Generation and Execution; The targeted voice alarm scheme generation and execution module uses the same formal violation event as the trigger object, combining the safety monitoring robot's movement position, orientation control, and voice output parameters into an alarm execution parameter group; The safety monitoring robot selects candidate alarm locations from the passable safe areas in the risk map; Candidate locations avoid restricted areas, hoisting slewing ranges, vehicle passages, and high-risk work surfaces, and maintain a preset safe distance from the target personnel; When the target personnel are within the current effective broadcast range, the current location is used as the candidate location; After determining the alarm location, the safety monitoring robot determines the broadcast orientation of its body or voice output device based on the positional relationship between the alarm location and the target personnel's location; it selects the corresponding script template according to the violation type and risk level; it verifies the event status and parameter generation time before execution; events that have been withdrawn, manually handled, or merged with higher-risk events are canceled from execution, and the remaining events enter the alarm task queue; when multiple events are pending handling simultaneously, the safety monitoring robot arranges the execution order according to the risk ranking value output by the personnel-level association, candidate event generation, and trusted confirmation modules; it moves to the alarm location according to the planned path and continuously acquires its own pose, the status of surrounding obstacles, and the latest location of the target personnel; it executes directional voice alarms according to the script template, broadcast volume, broadcast frequency, and broadcast interval, and continuously observes the target personnel during the broadcast; After each broadcast, the safety monitoring robot records the actual alarm location, broadcast direction, volume level, script identifier, execution start and end time, target personnel location, and execution result; the execution result includes at least alarm completion, target loss, failure to arrive, task preemption, and equipment malfunction; the result is not directly used as a conclusion of rectification completion, but is sent to the evidence retention and platform linkage module along with the event evidence; Step Six: Evidence Preservation and Platform Linkage; The evidence preservation and platform linkage module generates a traceable event evidence package for formal violations and alarm execution processes; the event evidence package is sent to the security management platform using the event identifier as the association key; the preserved evidence, platform linkage status, and alarm execution results are sent to the rectification review and closed-loop handling module, whereby the system continues to rectify and review the target personnel and related protection status, and closes the event or triggers escalation handling based on the review results.

5. The voice alarm linkage system for a construction site safety monitoring robot according to claim 4, characterized in that, The fourth stage includes the following steps: Step 7: Rectification Review and Closed-Loop Handling; After completing the on-site alarm, the safety monitoring robot will set the formal violation event to a pending review status, and reacquire video frames, temporary personnel identifiers, location trajectories, and associated protection status of the target personnel's location at the preset review time point; and execute the corresponding rectification review rules according to the violation type; For rectified events, the system cancels the corresponding subsequent voice alarms and mobile tasks, and continues to monitor whether a formal event of the same person and the same type of violation occurs again within a preset observation period. If no further occurrence occurs within the observation period, the rectification status is updated to "rectified." The system archives rectified events and synchronizes them to the security management platform via the evidence retention and platform linkage module. For unrectified or pending further review events, the system makes a handling judgment based on the risk level, number of alarms, duration of unrectified events, and device execution status. Events that do not meet the escalation conditions are returned to the targeted voice alarm scheme generation and execution module, where the alarm location, script, or broadcast parameters are updated before the targeted voice alarm is re-implemented. Events awaiting manual review are synchronized to the platform via the evidence retention and platform linkage module, awaiting legal manual confirmation or remote review instructions. When an event is abnormal within the preset review period, the system triggers an escalation process; the escalation conditions and thresholds are pre-configured by the task and risk rule configuration module; event evidence and status records are continuously retained, and escalation events retain escalation marks and processing records even after subsequent reviews confirm that the issues have been rectified.

Images