A virtual fence management method, device and equipment for an overhaul area and a medium

Abstract

The application discloses a virtual fence management and control method, device and equipment for an overhaul area and a medium. The method comprises the following steps: collecting real-time video data of a management and control area boundary in a management and control time period based on obtained management and control configuration data, detecting the real-time video data to obtain a personnel target position track, performing out-of-bound determination based on the personnel target position track to obtain an out-of-bound determination result, detecting the real-time video data to obtain safety event information when the out-of-bound determination result is that the personnel enters the management and control area boundary, generating corresponding management background response instructions, arranging prompt content and delivering the prompt content to a field prompt device, and saving the safety event information. The application arranges prompt content through the management background response instructions, delivers the prompt content to the field prompt device, and saves the calculated safety event information. In this way, the behavior of personnel out-of-bound entry in the overhaul area can be automatically and timely recognized and early warned.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology, and in particular to a method, device, equipment and medium for virtual fence control of maintenance areas. Background Technology

[0002] In the safety management and control scenarios of nuclear power plant maintenance operations, existing technologies generally rely on the deployment of physical fences and manual monitoring, combined with traditional video surveillance for on-site management. However, this type of method lacks the ability to automatically identify and provide immediate warnings for personnel entering the controlled area and violations during the operation process. Usually, the violation can only be dealt with afterward by reviewing the video recording or by the on-site supervisor. This results in a delay in the detection of risk events such as crossing the boundary, thereby increasing the risk of accidental entry and human error. Summary of the Invention

[0003] This invention provides a virtual fence management method, device, equipment, and medium for maintenance areas, aiming to solve the technical problem in the prior art that it is impossible to automatically identify and provide timely warnings of personnel crossing the boundary into maintenance areas.

[0004] In a first aspect, embodiments of the present invention provide a virtual fence management method for a maintenance area, comprising: For each maintenance area, the corresponding control area boundary, deployment time period, and association rule setting information are obtained and integrated to obtain deployment configuration data; Based on the deployment configuration data, real-time video data of the boundary of the controlled area is collected during the deployment period. The target detection algorithm is used to detect the real-time video data to obtain the trajectory of the personnel target position. Based on the trajectory of the personnel target position, boundary crossing is determined to obtain the boundary crossing result. When the boundary crossing determination result indicates that a person has entered the boundary of the controlled area, the real-time video data is detected by combining the area intrusion detection algorithm and the association rules to obtain security event information, and a corresponding management backend response instruction is generated. The management backend responds to the instructions, compiles and distributes the prompts to the on-site prompting devices, and saves the security event information.

[0005] Secondly, embodiments of the present invention provide a virtual fence control device for a maintenance area, comprising: The data acquisition unit is used to acquire the corresponding control area boundary, deployment time period and association rule setting information for the maintenance area, and integrate them to obtain deployment configuration data. The data determination unit is used to collect real-time video data of the boundary of the controlled area during the control period based on the deployment configuration data, use a target detection algorithm to detect the real-time video data to obtain the trajectory of the personnel target, and make a boundary crossing determination based on the trajectory of the personnel target to obtain the boundary crossing determination result. The instruction generation unit is used to detect security event information by combining the area intrusion detection algorithm and the association rules on the real-time video data when the boundary judgment result is that a person has entered the boundary of the controlled area, and to generate a corresponding management backend response instruction. The data output unit is used to compile and distribute the prompt content to the on-site prompting device according to the response instructions from the management backend, and to save the security event information.

[0006] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the virtual fence management method for the maintenance area of ​​the first aspect.

[0007] Fourthly, embodiments of the present invention provide a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the virtual fence management method for the maintenance area of ​​the first aspect.

[0008] This invention provides a virtual fence management method for maintenance areas, including: acquiring the corresponding control area boundary, deployment time period, and association rule settings for the maintenance area, and integrating them to obtain deployment configuration data; based on the deployment configuration data, collecting real-time video data of the control area boundary during the deployment time period, using a target detection algorithm to detect the real-time video data to obtain the trajectory of personnel targets, and performing boundary crossing judgment based on the personnel target trajectory to obtain a boundary crossing judgment result; when the boundary crossing judgment result indicates that personnel have entered the control area boundary, combining an area intrusion detection algorithm and the association rules to detect the real-time video data to obtain security event information, and generating a corresponding management backend response instruction; arranging and sending prompt content according to the management backend response instruction to the on-site prompting device, and saving the security event information. This invention, by arranging and sending prompt content to the on-site prompting device through the management backend response instruction and then saving the calculated security event information, can automatically and in real-time identify personnel crossing the boundary within the maintenance area and provide timely warnings.

[0009] This invention also provides a virtual fence control device, computer equipment, and storage medium for maintenance areas, which have the same beneficial effects as described above. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating a virtual fence management method for a maintenance area provided in an embodiment of the present invention; Figure 2 This is a schematic block diagram of a virtual fence control device for a maintenance area, provided as an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0014] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0015] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0016] Please see below. Figure 1 , Figure 1 The flowchart of a virtual fence management method for an inspection area provided in an embodiment of the present invention specifically includes steps S101 to S104.

[0017] S101. For the maintenance area, obtain the corresponding control area boundary, deployment time period and association rule setting information respectively, and integrate them to obtain deployment configuration data; S102. Based on the deployment configuration data, real-time video data of the boundary of the controlled area is collected during the deployment period. The target detection algorithm is used to detect the real-time video data to obtain the trajectory of the personnel target position. Based on the trajectory of the personnel target position, boundary crossing is determined to obtain the boundary crossing determination result. S103. When the boundary crossing determination result indicates that a person has entered the boundary of the controlled area, the real-time video data is detected by combining the area intrusion detection algorithm and the association rule to obtain security event information, and a corresponding management backend response instruction is generated. S104. Arrange the prompt content according to the management backend response instruction and send it to the on-site prompting device, and save the security event information.

[0018] In step S101, the system receives setting information input for the maintenance area, sets the control area boundary in the video screen, and simultaneously sets the control time period and association rules; then it integrates the control area boundary, control time period and association rules to obtain control configuration data, and stores the control configuration data and the corresponding maintenance area for later retrieval.

[0019] In one embodiment, step S101 includes: The on-site video data of the maintenance area is acquired in advance, and the physical boundary of the maintenance area is identified based on the on-site video data to obtain physical boundary identification data; A virtual electronic fence is drawn for the maintenance area based on the physical boundary identification data, and the boundary of the control area is set according to the virtual electronic fence; Obtain the setting information of the control time period and association rules for the maintenance area, and integrate the control area boundary, control time period and safety rule configuration data to obtain control configuration data.

[0020] In this embodiment, during the on-site deployment phase, mobile AI cameras and smart speakers are first deployed in maintenance areas, such as switchgear areas, to cover each maintenance area and its various locations. The mobile AI camera is a mobile video acquisition device, comprising a high-definition camera and an edge computing module. The high-definition camera captures on-site images of the maintenance area, while the edge computing module is a computing unit integrated on the camera side, used to perform local computational processing on the captured images. The smart speaker is a type of on-site notification device, used to receive data from the management terminal and perform voice broadcasts. To facilitate unified access and configuration of on-site devices, a backend management system is set up. This system supports both APP and Web interfaces. The APP is a management application installed on a mobile terminal, while the Web interface is a management page accessed through a browser. Both are used to present video images and provide configuration entry points for deployment parameters.

[0021] Furthermore, real-time video streams are collected using deployed mobile AI cameras to pre-acquire on-site video data of the maintenance area, and the corresponding video frames are displayed on the APP / Web platform. Within these video frames, the physical boundaries of the maintenance area are identified to obtain physical boundary identification data. The physical boundaries are the visible boundary markers of the maintenance area on-site, which may include the edge contours of equipment architecture, ground markings, etc. The physical boundary identification data is a structured description of the physical boundaries, which may include a set of boundary point coordinates, a set of boundary line segments, or a sequence of boundary polygon vertices. Physical boundary identification can be performed manually or automatically. For manual identification, managers select the edge positions corresponding to the equipment architecture and ground markings in the video frame on the APP / Web platform and connect the points to obtain the physical boundary identification data. For automatic identification, the edge computing module or the backend management system performs edge extraction, line segment detection, or region segmentation processing on the on-site video data, outputting the physical boundary identification data within the video frame coordinate system. The video frame coordinates are used to define the pixel positions within the video frame, mapping the identified boundary positions to drawable coordinate points.

[0022] After obtaining physical boundary identification data, a virtual electronic fence for the maintenance area is drawn on the video screen of the APP / Web terminal, and the control area boundary is set according to the virtual electronic fence. The virtual electronic fence is a fence graphic icon superimposed on the video screen, which corresponds to the boundary range described by the physical boundary identification data. During drawing, the physical boundary identification data is converted into fence drawing parameters, and the fence lines or fence area superimposed display effect is generated on the video screen. At the same time, the boundary coordinate data of the fence is saved as the control area boundary. For scenarios with multiple maintenance areas, corresponding control area boundaries can be set for different maintenance areas. For example, a corresponding virtual electronic fence can be drawn on the video screen for each maintenance area, and each control area boundary can be recorded as a searchable area configuration entry so that the corresponding control area boundary can be called according to the maintenance area during subsequent deployment time periods.

[0023] Furthermore, the system acquires the setting information for the control time period and associated rules for the maintenance area, and integrates the control area boundary, control time period, and safety rule configuration data to obtain control configuration data. The control time period refers to the start and end times for controlling the maintenance area, which can be selected or entered by management personnel via an app / web interface, for example, setting the maintenance time period to 8:30-12:30 or 14:00-17:30. The associated rules are safety rule configuration data related to the maintenance area, used to describe the safety requirements that must be met when entering or being within the control area, such as the mandatory wearing of a safety helmet. During setup, the control area boundary is bound to the control time period, and the safety rule configuration data is bound to the corresponding maintenance area configuration entry. Finally, the area identification information, control area boundary data, control time period data, and safety rule configuration data are summarized into control configuration data, which is then stored for subsequent steps.

[0024] In step S102, during the deployment period, the camera device continuously collects real-time video data corresponding to the boundary of the controlled area, and performs target detection algorithm processing on the real-time video data to output the position information of the personnel target at continuous time. The position information at continuous time is temporally correlated to obtain the position trajectory of the personnel target, and the position relationship between the position trajectory of the personnel target and the boundary of the controlled area is determined to obtain the boundary crossing determination result.

[0025] In one embodiment, step S102 includes: Real-time video data of the maintenance area is collected, and the real-time video data is parsed to obtain real-time video stream data; The system performs real-time motion target detection on the real-time video stream data, outputs human target detection data, and extracts human contour data from the human target detection data. The human body contour data is subjected to multi-target tracking processing, and the trajectory of the continuous frame position results obtained from the tracking is integrated to obtain the trajectory of the human target position. The virtual electronic fence in the deployment configuration data is read, and the coordinate position of the person relative to the virtual electronic fence is calculated based on the trajectory of the person's target position to obtain the relative coordinate position data; Based on the relative coordinate position data, it is determined whether a person has crossed the virtual electronic fence and entered the boundary of the controlled area, thus obtaining a boundary crossing determination result.

[0026] In this embodiment, real-time video data of the maintenance area is acquired, and the real-time video data is parsed to obtain real-time video stream data. Video stream parsing is the process of converting the original video acquisition output into computationally processable streaming data. This may include video frame extraction, timestamp annotation, resolution and frame rate normalization, and decoding and buffer management of the video stream to output real-time video stream data containing continuous video frames and their timing information. This facilitates subsequent algorithms for frame-by-frame detection and tracking of moving targets. Then, real-time moving target detection is performed on the real-time video stream data to output human target detection data, and human contour data is extracted from the human target detection data. The real-time moving target detection involves online detection of moving targets in real-time video stream data. Preferably, it employs deep learning algorithms deployed at the edge, such as the YOLO series or Transformer-based detection models. The YOLO series is an end-to-end target detection model that outputs target category and bounding box information for a single frame. The Transformer-based detection model uses a Transformer structure for feature modeling and target set prediction, outputting target location and category information for a single frame. Using these detection models, human target detection data is output for each frame of video image. This data may include human bounding box coordinates, confidence scores, and identification information associated with the frame number or timestamp. Based on this, human contour data is extracted from the human target detection data. This contour data describes the human body's external shape in the image and can be obtained through edge extraction after bounding box cropping, instance segmentation mask boundary extraction, or keypoint circumscribed contour fitting, thus providing a more stable target representation for subsequent multi-target tracking.

[0027] Furthermore, multi-target tracking processing is performed on the human contour data, and the trajectory of the continuous frame position results obtained from the tracking is integrated to obtain the personnel target position trajectory. The multi-target tracking processing is a process of maintaining the identity and updating the position of the same human target between continuous video frames. Its input is the human contour data of each frame and the corresponding detection information, and the output is the continuous frame position result with the target ID. The continuous frame position result is a set of position coordinates of the human target in each frame recorded in chronological order. The position coordinates can be the center point of the target box, the centroid point of the human contour, or a representative point from the set of human key points. The trajectory integration is to gather the position coordinates of the same target ID in continuous frames in chronological order to obtain the personnel target position trajectory. The personnel target position trajectory is used to describe the movement path and temporal change characteristics of the personnel in the maintenance area. Then, the virtual electronic fence in the deployment configuration data is read, and the coordinate position of the personnel relative to the virtual electronic fence is calculated based on the personnel target position trajectory to obtain the relative coordinate position data. The virtual electronic fence is fence boundary data defined within the coordinate system of the on-site video screen, used to represent the boundary of the controlled area. It can be stored in the deployment configuration data in the form of polygon vertex sequences, line segment sets, or closed regions of pixel coordinates. After reading, the position coordinates in the trajectory of the personnel target are mapped to the same video screen coordinate system, and the spatial relationship between the personnel position and the virtual electronic fence is calculated to obtain relative coordinate position data. This relative coordinate position data may include the personnel's position coordinates, the closest distance to the fence boundary, a determination indicator of whether the personnel's position is inside the fence, and association information with area identifiers. For scenarios with multiple maintenance areas, the virtual electronic fence can correspond to different area entries. For example, for a specific maintenance area, the relative coordinate position data can simultaneously carry the corresponding area entry identifier to facilitate subsequent processing in distinguishing the specific area entered by the personnel.

[0028] Finally, based on the relative coordinate position data, it is determined whether the personnel have crossed the virtual electronic fence and entered the boundary of the controlled area, thus obtaining the boundary crossing determination result. Specifically, based on the inner and outer markers in the relative coordinate position data and their state changes between consecutive frames, it is determined whether the personnel's position has switched from the outside of the fence to the inside of the fence; and combined with the distance change from the location point to the fence boundary and the time sequence consistency check, to eliminate false judgments caused by single-frame jitter or occlusion, thereby outputting the boundary crossing determination result. The boundary crossing determination result includes at least a determination identifier of whether entry has occurred, a timestamp of the trigger time, the corresponding personnel target ID, and the corresponding controlled area boundary marker; when personnel cross the virtual electronic fence and enter the controlled area boundary, the boundary crossing determination result can serve as the trigger input for subsequent area intrusion detection and management terminal handling processes, and can provide data sources for the area direction information of on-site voice prompts, such as location information used to indicate entry into a certain maintenance area.

[0029] In step S103, when the boundary crossing judgment result indicates that personnel have entered the boundary of the controlled area, the area intrusion detection algorithm is invoked to perform intrusion detection on the real-time video data, and the detection result is checked against the association rules to output security event information; the security event information is registered, and a management backend response instruction corresponding to the security event information is generated to indicate the subsequent on-site prompts and the recipients.

[0030] In one embodiment, step S103 includes: The real-time video data is parsed to obtain real-time video footage; The intrusion detection algorithm is used to detect intrusions into personnel within the boundary of the controlled area in the real-time video footage, and the location coordinates of the personnel are obtained. Based on the personnel location coordinates and association rules, the behavior of the personnel is tracked to obtain personnel behavior tags, and the personnel behavior tags are integrated with the real-time video footage and personnel location coordinates to obtain the data stream of the security event information; The data stream of the security incident information is sent to the back-end management system, and the back-end management system is used to perform remote operation compliance confirmation processing to obtain handling judgment data; The disposal judgment data is processed to generate an instruction, which is then used to obtain a response instruction from the management backend.

[0031] In this embodiment, the backend management system is a system platform for centralized management of maintenance area deployment configuration, real-time monitoring, and event handling. It can be deployed as a server-side application and provides an interface for management personnel. To achieve real-time data access, the backend management system establishes a communication connection with the front-end camera via protocols such as MQTT or RTSP. MQTT is a publish-subscribe messaging protocol suitable for pushing lightweight data such as AI analysis results in a topic-based manner; RTSP is a real-time streaming control protocol suitable for session control and transmission of real-time video streams. The data stream uploaded by the front end includes at least real-time video footage and AI analysis result information. The AI ​​analysis results may include personnel location coordinates, personnel behavior tags, etc. Management personnel can perform remote confirmation and command triggering based on the data stream in the backend management system.

[0032] First, the real-time video data is parsed to obtain real-time video frames. Parsing may include decoding, frame serialization, and frame caching of the data stream from the front end to output real-time video frames that can be directly presented and computed by algorithms. The real-time video frames may be a collection of video frames arranged in chronological order, carrying a timestamp and video channel identifier corresponding to each frame for subsequent synchronization and association with personnel location coordinates and personnel behavior tags. Then, an intrusion detection algorithm is used to detect intrusion of personnel within the boundary of the controlled area in the real-time video frames, obtaining the personnel location coordinates. The intrusion detection algorithm is an algorithm that determines whether a personnel target has entered the boundary range based on a pre-set controlled area boundary; its input is the real-time video frames and the corresponding controlled area boundary data, where the controlled area boundary is the boundary description of the virtual electronic fence in the video frame coordinate system. For example, the YOLOv8 target detection algorithm can be used. After the AI ​​camera acquires the real-time video frames, YOLOv8 is used to detect personnel targets in real time and output personnel target boxes, thus obtaining the personnel location coordinates. During execution, the backend management system or edge computing terminal locates the personnel target in the real-time video frame, maps the personnel's position in the frame to coordinate points, and calculates the relationship between the coordinates and the inner and outer sides of the control area boundary to obtain the personnel's position coordinates. The personnel position coordinates are coordinate data representing the personnel's position in the video frame coordinate system. They can be expressed in the form of target box center point coordinates, contour centroid point coordinates, or key point representative coordinates, and are associated with the personnel target ID and timestamp to support continuous frame behavior tracking.

[0033] Furthermore, based on the personnel location coordinates and association rules, the personnel's behavior is tracked to obtain personnel behavior tags. These tags are then integrated with the real-time video footage and personnel location coordinates to obtain the data stream of the safety event information. The association rules are a set of safety rules configured for the maintenance area, specifying compliance requirements and abnormal triggering conditions for operations after entering a specific area. The behavior tracking is a continuous analysis of personnel action states and behavior patterns based on the temporal changes of personnel location coordinates in consecutive frames and the content of real-time video footage. During execution, the personnel location coordinates are input into the behavior analysis module, and the behavior judgment results are checked against the association rules to output personnel behavior tags. These tags characterize the current behavioral state of the personnel, such as distinguishing between normal passage, staying, and crossing fences, and can be categorized according to area entry identifiers. The personnel behavior tags are then packaged and integrated with the real-time video footage and personnel location coordinates at the corresponding time to obtain a data stream of security event information. The data stream of security event information is a structured event stream data for backend processing, which includes at least the time of the event, area identifier, personnel target ID, personnel location coordinates, personnel behavior tags, and corresponding video frame indexes or image references, so that managers can conduct remote verification.

[0034] Furthermore, the data stream of the safety incident information is sent to the backend management system, which then performs remote operation compliance confirmation processing to obtain handling judgment data. During transmission, real-time video footage can be provided for real-time viewing via the RTSP channel, and AI analysis results such as personnel location coordinates and personnel behavior tags can be pushed to the event access module of the backend management system via the MQTT channel, thereby simultaneously presenting the real-time video footage and AI analysis results on the backend management system interface. Based on the backend management system interface, management personnel remotely confirm the personnel entry behavior and operation status indicated by the safety incident information data stream to determine whether they comply with established operation specifications. The remote operation compliance confirmation processing may include verifying whether the entrant is an authorized operator, checking whether their entry area matches the current work area, and verifying whether the status indicated by their behavior tags meets association rules. After confirmation, the backend management system outputs handling judgment data, which characterizes the handling conclusion determined by management personnel or the system, and may include handling types such as release, prompt correction, and triggering an early warning, as well as the corresponding handling level and target area identifier.

[0035] Finally, the disposal judgment data is processed to generate an instruction from the management backend. This instruction generation process converts the disposal judgment data into executable instruction data. It may include mapping the disposal type to an instruction template, filling the template fields with the target area identifier and prompt content, and adding a timestamp and target device identifier to the instruction. The management backend response instruction may include automatically generated prompts or voice instructions manually triggered by management personnel. For example, when management determines that a person has entered the wrong interval or the work area is mismatched, a voice prompt instruction is triggered in the backend management system to generate prompt content, such as "You have entered the wrong interval, please check the work area," etc. This prompt content is then bound to the device identifier of the target on-site prompting device for subsequent steps to send the management backend response instruction to the on-site prompting device for execution.

[0036] In step S104, the management backend response instruction is read, the prompt content is arranged according to the management backend response instruction to obtain the prompt content to be broadcast, and the prompt content to be broadcast is sent to the on-site prompt device for broadcasting; at the same time, the security event information is saved to establish a corresponding record between the management backend response instruction and the security event information for subsequent query and traceability.

[0037] In one embodiment, step S104 includes: The management backend response command is parsed to obtain the command text content data; Based on the instruction text content data, the prompt content is arranged to obtain the arrangement result, and the playback parameters of the local audio file or the synthesis parameters of the real-time synthesized speech are called to synthesize the arrangement result to obtain the speech prompt content arrangement data. The voice prompt content arrangement data is encapsulated for transmission and distributed using a transmission control protocol or Internet Protocol to send the voice prompt content arrangement data to the on-site prompting device.

[0038] In this embodiment, the management backend response command is parsed to obtain command text content data. The management backend response command is command data output by the backend management system, which includes at least the command type, target device identifier, target area identifier, and prompt content field. Command parsing involves decomposing the data structure of the management backend response command and performing format verification. During parsing, the prompt content is extracted from the management backend response command and command text content data is generated. The command text content data is a text data object that can be used for subsequent arrangement and may include the main prompt text, area name or area identifier, job verification prompt field, and broadcast priority field, etc. For example, the command text content data may correspond to text content such as "You have entered the ABC phase area of ​​machine 3, please confirm the job position" or to correct prompt text such as "Please wear your safety helmet correctly". Then, based on the command text content data, the prompt content is arranged to obtain the arrangement result, and the playback parameters of the local audio file or the synthesis parameters of the real-time synthesized speech are called to synthesize the speech of the arrangement result to obtain the speech prompt content arrangement data. The prompt content arrangement involves organizing and splicing instruction text content data according to a preset broadcast format. This broadcast format may include the sequential configuration of title segments, area indication segments, verification prompt segments, and correction prompt segments, and the broadcast queue and insertion strategy can be determined based on the broadcast priority field in the instruction text content data. The arrangement result is the arranged playable content data, which can be either a text sequence or a sequence of selected audio segment identifiers. The speech synthesis stage is executed by the speech function module on the smart speaker side or the backend management system side. This speech function module is a functional unit used to generate or play voice prompts; it can generate speech using one of two paths: first, by calling the playback parameters of a local audio file, mapping the arrangement result to a locally preset audio file, and determining the playback parameters, which may include audio file identifier, volume, number of playbacks, and playback order; second, by calling the synthesis parameters of real-time synthesized speech, synthesizing speech in real-time from the text corresponding to the arrangement result, which may include speech rate, volume, speaker type, etc. Through any of the above paths, audio prompt content arrangement data for broadcasting is generated. The audio prompt content arrangement data is audio data or audio playback command data that can be directly sent to drive the smart speaker to broadcast.

[0039] In actual operation, the backend management system can trigger different prompts for different events. When personnel enter the boundary of the controlled area and need to verify the work area, voice prompt content can be generated and arranged to announce, "You have entered the ABC phase area of ​​machine 3, please confirm your work location." When personnel violate regulations, specifically behaviors that do not meet the definition of the association rules, such as not wearing a safety helmet, the system can automatically trigger a voice warning based on the corresponding handling judgment data and generate voice prompt content arranged to announce, "Please wear your safety helmet correctly." All the above prompts and warnings are completed through the same arrangement and voice generation process, thereby ensuring that the voice content is consistent with the judgment results of the management end and facilitating unified execution on site.

[0040] Finally, the voice prompt content arrangement data is encapsulated for transmission and distributed using a transmission control protocol or Internet Protocol (TCP) to send the voice prompt content arrangement data to the on-site prompting device. The transmission encapsulation involves packaging the voice prompt content arrangement data into a network transmission payload, which may include adding a target device identifier, message header field, length field, checksum field, and timestamp field, and converting the payload into a data packet format compatible with TCP / IP transmission. During distribution, the backend management system establishes a network connection with the smart speaker based on the target device identifier and sends the encapsulated voice prompt content arrangement data to the smart speaker via TCP / IP. After receiving the data, the smart speaker calls the voice function module to parse the received data and executes audio playback according to the local audio file playback parameters or the real-time synthesized speech synthesis parameters to complete the voice reminder and voice warning on-site.

[0041] In one embodiment, the virtual fence control method for the maintenance area further includes: Based on the safety incident information, the time period and personnel identification of the entire operation process are determined to obtain operation identification data; Event data is automatically extracted from the job identification data to obtain an event data set; The event data set is structured and organized to generate a structured job log; The structured log data of the operation is output to the management backend for export and analysis to obtain post-event traceability data.

[0042] In this embodiment, after determining the boundary crossing of the maintenance area and outputting safety event information, the system can proceed to the safety alert and post-event tracing stage. The safety event information refers to event records generated during the control period surrounding personnel entering the boundary of the controlled area and their subsequent actions. It includes at least the event occurrence time, controlled area identifier, personnel identifier, personnel location coordinates, personnel behavior tags, and warning trigger records. The personnel identifier is used to distinguish different personnel targets and can be obtained from the target ID obtained through multi-target tracking or from the authorized personnel identifier verified and bound by the backend. The operation identifier data is a identifiable data object describing the entire process of a single operation by the same person within the same maintenance area, used to group scattered event records into the same operation scope.

[0043] Based on the aforementioned safety event information, the time period and personnel identification of the entire operation process are determined. When obtaining the operation identification data, the safety event information can first be aggregated according to personnel identification and controlled area identification, and arranged in chronological order. In the aggregated event sequence, the time when the personnel first enter the boundary of the controlled area is taken as the operation start time, and the time when the personnel leave the boundary of the controlled area or when no related events occur within a preset idle threshold is taken as the operation end time, thus obtaining the time period of the entire operation process. At the same time, the personnel identification, controlled area identification, and operation start time are combined to generate an operation number, and the operation number, time period, personnel identification, etc. are summarized into operation identification data so that the event data of the entire operation process can be extracted later using the operation number as an index.

[0044] Furthermore, the event data in the job identification data is automatically extracted to obtain an event data set. Event records falling within the specified time period are then filtered from the safety event information according to the job number. The filtered event records undergo field extraction and standardization to generate the event data set. This event data set is a collection of event entries corresponding to the entire job process. Each event entry may include event time, event type, personnel identifier, personnel location coordinates, personnel behavior tags, warning identifiers, and their associated screen indexes. To cover safety reminders, the speakers in the on-site prompting equipment can loop safety reminders during this phase. These safety reminders are audio messages used to remind personnel to pay attention to their work behavior, such as "Please pay attention to your work behavior" and "Use your safety belt correctly." The playback time, content identifier, and target area identifier of each safety reminder can be written into the event data set for comparative analysis with personnel violations and warning records on the same timeline. The event data set is then structured and arranged. When generating a structured job log, the event data set can be rearranged in chronological order, and each event entry can be mapped to a preset set of log fields to generate a searchable structured record. The structured log of the operation is log data that provides a structured description of the entire operation process. It may include at least personnel entry time, personnel exit time, key behavior tag sequence during the personnel's on-site time, violation records, warning records, and safety prompt playback records. Each record may be attached with a control area identifier and a personnel identifier to ensure that the event relationship under the same operation number is traceable and verifiable.

[0045] Furthermore, the structured log data of the operations is output to the management backend for export and analysis. When obtaining post-event traceability data, the system pushes or writes the structured logs to the log management module of the management backend according to the operation number. Managers can retrieve the corresponding structured logs of operations in the management backend by personnel identifier, controlled area identifier, or time period, and perform export operations to obtain log files or datasets that can be analyzed offline. The post-event traceability data is the result data after being exported and analyzed by the management backend. It can be used to review the sequence of personnel entry and exit times, violations, and warning records, and provide verifiable data support for safety management work.

[0046] In one embodiment, the virtual fence control method for the maintenance area further includes: Continuous video capture is performed on the boundary of the controlled area, and the collected real-time video data is parsed to obtain a continuous monitoring video stream; Security event detection is performed on the continuously monitored video stream to obtain real-time behavioral feature data; The real-time behavioral feature data is matched with a preset rule base to obtain violation behavior identification data, and warning trigger data is generated based on the violation behavior identification data. The violation identification data is recorded and processed to generate violation time data and personnel dynamic information data. The warning trigger data, violation time data and personnel dynamic information data are then integrated to obtain safety supervision data.

[0047] In this embodiment, during the safety supervision phase of the operation, an AI camera continuously collects on-site footage corresponding to the boundary of the controlled area to continuously monitor the safety status of the workers and any abnormalities on-site. The continuous video acquisition is the process of continuously outputting real-time video data from the boundary of the same controlled area within the deployment period. The video stream parsing involves decoding, frame extraction, and timestamp annotation of the real-time video data to obtain a continuously monitored video stream that can be processed. This continuously monitored video stream includes at least video frame data arranged in chronological order, along with their frame numbers and timestamp information, thereby providing continuous temporal input for subsequent safety event detection.

[0048] After obtaining the continuous monitoring video stream, safety event detection is performed on the video stream to obtain real-time behavioral feature data. The safety event detection involves identifying and extracting the status of personnel and items within the boundary of the controlled area, covering typical safety events during operations, including but not limited to not wearing a safety helmet and leaving items behind. Not wearing a safety helmet is an event identifying the wearing status of head protective equipment, and leaving items behind is an event identifying items that remain in the controlled area for an extended period and whose movement is inconsistent with that of personnel. The real-time behavioral feature data is a structured representation of the safety event detection results, which may include personnel identification, personnel location coordinates, personnel behavior tags, personal protective equipment status identifiers, item category identifiers, item location coordinates, and feature record entries corresponding to video frame timestamps, enabling continuous comparison of personnel behavior and changes in the on-site status over time.

[0049] Furthermore, the real-time behavioral feature data is matched with a preset rule base to obtain violation identification data, and warning trigger data is generated based on the violation identification data. The preset rule base is a data set used to describe operational safety requirements and violation judgment conditions, and may include rule number, applicable area identifier, applicable time period, trigger condition field, and handling level field. The trigger condition field describes the conditions under which the real-time behavioral feature data is judged as a violation, such as a person's personal protective equipment status indicator indicating that a safety helmet is not being worn, or the location coordinates of an item remaining in the same area within multiple consecutive timestamps and not associated with a person's identifier. During matching, real-time behavioral feature data can be read one by one and compared with the trigger condition field in the preset rule base to output violation identification data; the violation identification data may include a violation type identifier, corresponding person identifier, corresponding area identifier, trigger timestamp, and associated evidence frame index. Based on the violation identification data, early warning trigger data is generated. This early warning trigger data drives on-site alerts and back-end processing, and may include the early warning level, prompt text, target on-site prompt device identifier, and trigger method identifier. The trigger method identifier distinguishes between automatic triggering and back-end emphasis triggering. Back-end emphasis triggering is a method where management personnel issue enhanced prompts for the same violation event on the management terminal. Corresponding to the early warning trigger data, on-site voice speakers can broadcast the prompt upon receipt to provide immediate emphasis and reminders to personnel.

[0050] After outputting the warning trigger data, the violation identification data is recorded and processed to generate violation time data and personnel dynamic information data. The warning trigger data, violation time data, and personnel dynamic information data are then integrated to obtain safety supervision data. The recording process involves continuously writing and archiving the violation identification data. The violation time data records the trigger time, duration, and resolution time of each violation event. The personnel dynamic information data records the sequence of personnel location coordinates, the sequence of personnel behavior tags, and a summary of the movement trajectory corresponding to the personnel identifier before and after the violation event. During integration, the warning trigger data, the corresponding violation time data, and the personnel dynamic information data are associated and summarized according to personnel identifier and timestamp to generate safety supervision data. This safety supervision data can be used as real-time monitoring input for the management backend and as a work process monitoring record for subsequent export and analysis, used to verify the occurrence sequence and handling status of events such as not wearing a safety helmet or leaving items behind throughout the entire work process.

[0051] In summary, this application defines the maintenance area through a virtual electronic fence and combines it with AI recognition and voice confirmation mechanisms, which helps reduce the risk of human error caused by personnel accidentally entering or going to the wrong maintenance area. Regarding deployment and monitoring, compared to methods requiring physical fences on-site and reliance on on-site supervisors, it reduces the time investment in fence installation and removal, and decreases the dependence on the number of on-site supervisors, enabling a centralized management and control model by back-end administrators, thereby reducing labor costs. In terms of safety supervision, it can cover the entire process of entry, operation, and exit, and automatically identify violations such as not wearing a safety helmet. With early warning and identification response time controlled within 2 seconds, the risk of missed inspections due to manual monitoring can be reduced, improving the real-time and comprehensiveness of safety management. In terms of economic benefits, it can reduce the procurement and maintenance costs of physical fences and save on monitoring personnel costs. At the same time, it can shorten the deployment time of maintenance areas from 4 hours per area to 2 minutes per area, thereby reducing the average duration of a single maintenance operation by about 5% and improving on-site maintenance efficiency. In terms of social benefits, it can improve the safety level of maintenance operations, aiming to reduce the human error rate by 90%, and can provide a replicable solution for intelligent management and control of high-risk areas, promoting the safety management and digital transformation of the nuclear power industry.

[0052] Combination Figure 2 As shown, Figure 2 This is a schematic block diagram of a virtual fence control device for a maintenance area provided in an embodiment of the present invention. The virtual fence control device 200 for the maintenance area includes: The data acquisition unit 201 is used to acquire the corresponding control area boundary, deployment time period and association rule setting information for the maintenance area, and integrate them to obtain deployment configuration data. The data determination unit 202 is used to collect real-time video data of the boundary of the controlled area during the control period based on the control configuration data, use a target detection algorithm to detect the real-time video data to obtain the trajectory of the personnel target, and make a boundary crossing determination based on the trajectory of the personnel target to obtain a boundary crossing determination result. The instruction generation unit 203 is used to detect security event information by combining the area intrusion detection algorithm and the association rules on the real-time video data when the boundary judgment result is that a person has entered the boundary of the controlled area, and to generate a corresponding management backend response instruction. The data output unit 204 is used to arrange the prompt content according to the response instructions from the management backend and send it to the on-site prompting device, and save the security event information.

[0053] In this embodiment, the data acquisition unit 201 acquires the corresponding control area boundary, deployment time period, and association rule setting information for the maintenance area, and integrates them to obtain deployment configuration data; the data judgment unit 202, based on the deployment configuration data, collects real-time video data of the control area boundary during the deployment time period, uses a target detection algorithm to detect the real-time video data to obtain the trajectory of the personnel target position, and performs boundary crossing judgment based on the trajectory of the personnel target position to obtain the boundary crossing judgment result; when the boundary crossing judgment result indicates that personnel have entered the control area boundary, the instruction generation unit 203, in conjunction with the area intrusion detection algorithm and the association rules, detects the real-time video data to obtain security event information and generates a corresponding management backend response instruction; the data output unit 204, according to the management backend response instruction, arranges the prompt content and sends it to the on-site prompting device, and saves the security event information.

[0054] In one embodiment, the data acquisition unit 201 is specifically used for: The on-site video data of the maintenance area is acquired in advance, and the physical boundary of the maintenance area is identified based on the on-site video data to obtain physical boundary identification data; A virtual electronic fence is drawn for the maintenance area based on the physical boundary identification data, and the boundary of the control area is set according to the virtual electronic fence; Obtain the setting information of the control time period and association rules for the maintenance area, and integrate the control area boundary, control time period and safety rule configuration data to obtain control configuration data.

[0055] In one embodiment, the data determination unit 202 is specifically used for: Real-time video data of the maintenance area is collected, and the real-time video data is parsed to obtain real-time video stream data; The system performs real-time motion target detection on the real-time video stream data, outputs human target detection data, and extracts human contour data from the human target detection data. The human body contour data is subjected to multi-target tracking processing, and the trajectory of the continuous frame position results obtained from the tracking is integrated to obtain the trajectory of the human target position. The virtual electronic fence in the deployment configuration data is read, and the coordinate position of the person relative to the virtual electronic fence is calculated based on the trajectory of the person's target position to obtain the relative coordinate position data; Based on the relative coordinate position data, it is determined whether a person has crossed the virtual electronic fence and entered the boundary of the controlled area, thus obtaining a boundary crossing determination result.

[0056] In one embodiment, the instruction generation unit 203 is specifically used for: The real-time video data is parsed to obtain real-time video footage; The intrusion detection algorithm is used to detect intrusions into personnel within the boundary of the controlled area in the real-time video footage, and the location coordinates of the personnel are obtained. Based on the personnel location coordinates and association rules, the behavior of the personnel is tracked to obtain personnel behavior tags, and the personnel behavior tags are integrated with the real-time video footage and personnel location coordinates to obtain the data stream of the security event information; The data stream of the security incident information is sent to the back-end management system, and the back-end management system is used to perform remote operation compliance confirmation processing to obtain handling judgment data; The disposal judgment data is processed to generate an instruction, which is then used to obtain a response instruction from the management backend.

[0057] In one embodiment, the data output unit 204 is specifically used for: The management backend response command is parsed to obtain the command text content data; Based on the instruction text content data, the prompt content is arranged to obtain the arrangement result, and the playback parameters of the local audio file or the synthesis parameters of the real-time synthesized speech are called to synthesize the arrangement result to obtain the speech prompt content arrangement data. The voice prompt content arrangement data is encapsulated for transmission and distributed using a transmission control protocol or Internet Protocol to send the voice prompt content arrangement data to the on-site prompting device.

[0058] In one embodiment, the virtual fence control device 200 for the maintenance area is further used for: Based on the safety incident information, the time period and personnel identification of the entire operation process are determined to obtain operation identification data; Event data is automatically extracted from the job identification data to obtain an event data set; The event data set is structured and organized to generate a structured job log; The structured log data of the operation is output to the management backend for export and analysis to obtain post-event traceability data.

[0059] In one embodiment, the virtual fence control device 200 for the maintenance area is further used for: Continuous video capture is performed on the boundary of the controlled area, and the collected real-time video data is parsed to obtain a continuous monitoring video stream; Security event detection is performed on the continuously monitored video stream to obtain real-time behavioral feature data; The real-time behavioral feature data is matched with a preset rule base to obtain violation behavior identification data, and warning trigger data is generated based on the violation behavior identification data. The violation identification data is recorded and processed to generate violation time data and personnel dynamic information data. The warning trigger data, violation time data and personnel dynamic information data are then integrated to obtain safety supervision data.

[0060] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0061] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0062] This invention also provides a computer device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the computer device may also include various network interfaces, a power supply, a graphics card, etc., to utilize the graphics card's performance to operate the model, such as for inference and training.

[0063] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0064] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A virtual fence management method for a maintenance area, characterized in that, include: For each maintenance area, the corresponding control area boundary, deployment time period, and association rule setting information are obtained and integrated to obtain deployment configuration data; Based on the deployment configuration data, real-time video data of the boundary of the controlled area is collected during the deployment period. The target detection algorithm is used to detect the real-time video data to obtain the trajectory of the personnel target position. Based on the trajectory of the personnel target position, boundary crossing is determined to obtain the boundary crossing result. When the boundary crossing determination result indicates that a person has entered the boundary of the controlled area, the real-time video data is detected by combining the area intrusion detection algorithm and the association rules to obtain security event information, and a corresponding management backend response instruction is generated. The management backend responds to the instructions, compiles and distributes the prompts to the on-site prompting devices, and saves the security event information.

2. The virtual fence control method for maintenance areas according to claim 1, characterized in that, Also includes: Based on the safety incident information, the time period and personnel identification of the entire operation process are determined to obtain operation identification data; Event data is automatically extracted from the job identification data to obtain an event data set; The event data set is structured and organized to generate a structured job log; The structured log data of the operation is output to the management backend for export and analysis to obtain post-event traceability data.

3. The virtual fence control method for maintenance areas according to claim 1, characterized in that, The process involves acquiring the corresponding control area boundaries, deployment time periods, and association rule settings, and integrating them to obtain deployment configuration data, including: The on-site video data of the maintenance area is acquired in advance, and the physical boundary of the maintenance area is identified based on the on-site video data to obtain physical boundary identification data; A virtual electronic fence is drawn for the maintenance area based on the physical boundary identification data, and the boundary of the control area is set according to the virtual electronic fence; Obtain the setting information of the control time period and association rules for the maintenance area, and integrate the control area boundary, control time period and safety rule configuration data to obtain control configuration data.

4. The virtual fence control method for maintenance areas according to claim 1, characterized in that, The step of using a target detection algorithm to detect the real-time video data to obtain the trajectory of the person's target position, and then performing a boundary crossing determination based on the trajectory of the person's target position to obtain the boundary crossing determination result, includes: Real-time video data of the maintenance area is collected, and the real-time video data is parsed to obtain real-time video stream data; The system performs real-time motion target detection on the real-time video stream data, outputs human target detection data, and extracts human contour data from the human target detection data. The human body contour data is subjected to multi-target tracking processing, and the trajectory of the continuous frame position results obtained from the tracking is integrated to obtain the trajectory of the human target position. The virtual electronic fence in the deployment configuration data is read, and the coordinate position of the person relative to the virtual electronic fence is calculated based on the trajectory of the person's target position to obtain the relative coordinate position data; Based on the relative coordinate position data, it is determined whether a person has crossed the virtual electronic fence and entered the boundary of the controlled area, thus obtaining a boundary crossing determination result.

5. The virtual fence management method for maintenance areas according to claim 1, characterized in that, The method of combining the regional intrusion detection algorithm and the association rules to detect security events in the real-time video data and generating corresponding management backend response instructions includes: The real-time video data is parsed to obtain real-time video footage; The intrusion detection algorithm is used to detect intrusions into personnel within the boundary of the controlled area in the real-time video footage, and the location coordinates of the personnel are obtained. Based on the personnel location coordinates and association rules, the behavior of the personnel is tracked to obtain personnel behavior tags, and the personnel behavior tags are integrated with the real-time video footage and personnel location coordinates to obtain the data stream of the security event information; The data stream of the security incident information is sent to the back-end management system, and the back-end management system is used to perform remote operation compliance confirmation processing to obtain handling judgment data; The disposal judgment data is processed to generate an instruction, which is then used to obtain a response instruction from the management backend.

6. The virtual fence management method for maintenance areas according to claim 1, characterized in that, The step of arranging and sending the prompt content to the on-site prompting device according to the response command from the management backend includes: The management backend response command is parsed to obtain the command text content data; Based on the instruction text content data, the prompt content is arranged to obtain the arrangement result, and the playback parameters of the local audio file or the synthesis parameters of the real-time synthesized speech are called to synthesize the arrangement result to obtain the speech prompt content arrangement data. The voice prompt content arrangement data is encapsulated for transmission and distributed using a transmission control protocol or Internet Protocol to send the voice prompt content arrangement data to the on-site prompting device.

7. The virtual fence control method for maintenance areas according to claim 1, characterized in that, Also includes: Continuous video capture is performed on the boundary of the controlled area, and the collected real-time video data is parsed to obtain a continuous monitoring video stream; Security event detection is performed on the continuously monitored video stream to obtain real-time behavioral feature data; The real-time behavioral feature data is matched with a preset rule base to obtain violation behavior identification data, and warning trigger data is generated based on the violation behavior identification data. The violation identification data is recorded and processed to generate violation time data and personnel dynamic information data. The warning trigger data, violation time data and personnel dynamic information data are then integrated to obtain safety supervision data.

8. A virtual fence control device for a maintenance area, characterized in that, include: The data acquisition unit is used to acquire the corresponding control area boundary, deployment time period and association rule setting information for the maintenance area, and integrate them to obtain deployment configuration data. The data determination unit is used to collect real-time video data of the boundary of the controlled area during the control period based on the deployment configuration data, use a target detection algorithm to detect the real-time video data to obtain the trajectory of the personnel target, and make a boundary crossing determination based on the trajectory of the personnel target to obtain the boundary crossing determination result. The instruction generation unit is used to detect security event information by combining the area intrusion detection algorithm and the association rules on the real-time video data when the boundary judgment result is that a person has entered the boundary of the controlled area, and to generate a corresponding management backend response instruction. The data output unit is used to compile and distribute the prompt content to the on-site prompting device according to the response instructions from the management backend, and to save the security event information.

9. A computer device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the virtual fence management method for the maintenance area as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the virtual fence control method for the maintenance area as described in any one of claims 1 to 7.

Images