Working site image anomaly detection method and system based on artificial intelligence

Abstract

The invention relates to the technical field of safety protection, in particular to an operation site image anomaly detection method and system based on artificial intelligence, and the method comprises the steps: analyzing a work ticket and an operation scheme text through an NLP large model, and determining a detection range and a key monitoring object; acquiring an operation scene image and a video stream in real time; according to the detection range delimited by the NLP large model, calling a corresponding CV scene classification algorithm and a target detection algorithm, and completing logic classification of indoor services and outdoor services through a CoAtNet model; and inputting a result identified by the CV scene classification algorithm and job text information analyzed by the NLP into a multi-modal large model, filtering error detection results, removing similar defects repeatedly reported in a short time, judging an abnormal level based on a preset threshold value, generating a standardized violation report, and carrying out standard violation analysis. According to the method, hidden risks such as personnel violation, implement violation and environment abnormity are accurately captured, and the scene adaptability and the recognition precision of abnormity detection are improved.

Description

Technical Field

[0001] This invention relates to the field of safety protection technology, specifically to an artificial intelligence-based method and system for detecting anomalies in work site images. Background Technology

[0002] Ultra-high voltage direct current (UHVDC) transmission operations are characterized by their wide coverage, complex and harsh environments, and diverse types of work. During these operations, workers not only face natural environmental risks such as wildfires, lightning, icing, and typhoons, but also engage in numerous high-risk operations including working at heights, equipment maintenance, and live-line work, making safety management extremely challenging. With the widespread application of UHVDC transmission systems in the power industry and the rapid development of artificial intelligence technology, current safety supervision of UHVDC transmission operations mainly relies on manual inspections and post-event verification, which faces numerous technical bottlenecks. There is an urgent need to develop an image anomaly detection solution that deeply integrates business knowledge and artificial intelligence technology to achieve accurate identification, real-time early warning, and full-process control of abnormal behaviors in operational scenarios.

[0003] Traditional methods for detecting anomalies in power operation images have significant technical limitations in ultra-high voltage direct current (UHVDC) transmission operation scenarios, making it difficult to meet the high-precision, high-real-time safety management requirements. First, the algorithm has poor scene adaptability and a high false alarm rate: traditional algorithms do not integrate UHV operation business rules. For example, relying solely on visual recognition to identify that a safety helmet is not being worn, it cannot combine the work order to determine indoor operation scenarios where wearing a helmet is not required. Furthermore, it misclassifies special scenarios such as valve halls and other physical indoor spaces, as well as outdoor business spaces, leading to a large number of false alarms. Second, multimodal data is fragmented and detection dimensions are singular: relying solely on image data without linking it to text information such as work permits, it is impossible to collaboratively judge image behavior and text rules. For example, it is impossible to accurately identify violations of climbing above 2 meters based on the requirement of wearing safety belts for high-altitude operations in the work permit, and relying solely on visual estimation of height has large errors. Third, it lacks real-time performance and redundancy resistance. It does not adopt cloud-edge collaboration and lightweight deployment, and the monitoring of a single operation takes more than 120 minutes, making it difficult to cover more than 10,000 operation points per day; moreover, there is no duplicate alarm filtering mechanism, and the same device reports the same anomaly multiple times in a short period of time, with redundant information consuming monitoring resources. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an artificial intelligence-based method for detecting and monitoring anomalies in work site images. This method deeply integrates the visual features of ultra-high voltage (UHV) work scenarios with textual business information such as work permits and safety regulations, breaks down the barriers of multimodal data fragmentation, accurately captures hidden risks such as personnel violations, equipment violations, and environmental anomalies, and improves the scenario adaptability and recognition accuracy of anomaly detection.

[0005] To achieve one of the above objectives, the present invention provides the following technical solution: An AI-based method for detecting anomalies in work site images is provided, comprising the following steps: Step 1: Analyze the text data including work orders and work plans using an NLP large model, extract core information such as work type, work location, work time period, and safety requirements to determine the detection scenario, and then determine the monitoring object for the detection scenario; Step 2: Real-time acquisition of image data of the work scene; Step 3: Based on the collected image data, call the CV scene classification algorithm containing the CoAtNet model to perform the business logic classification of indoor / outdoor business classification, and call the corresponding target detection algorithm according to the determined detection scenario and monitoring object to identify at least one abnormal operation event among abnormal behavior, illegal equipment or environmental hazards in the image; Step 4: Input the business logic classification, job exception events, and core information parsed by the NLP big model into the multimodal big model, perform cross-modal fusion verification, filter error detection results, and remove the same type of defects that are repeatedly reported in a short period of time, and filter errors and / or repeated job exception events. Step 5: Perform threshold judgment on valid operational anomalies, determine the anomaly level, and generate a standardized violation report. In some implementations, in step 2, terminal devices of surveillance cameras and / or PTZ cameras are deployed at the work site to collect images and video streams of the work scene in real time. The collected image and video stream data are preprocessed, including one or more operations such as image segmentation, mean and variance calculation, and noise filtering.

[0006] In some implementations, in step 5, the preset threshold includes the percentage of similar violations or screen glitches detected N times consecutively within a set time period.

[0007] In some implementations, the set time is 1 to 5 minutes, and N is 2 to 5.

[0008] In some implementations, the set time is 5 minutes and N is 3.

[0009] In some implementations, step 5 also includes closed-loop reporting, which includes: pushing abnormalities for which no violation report is generated to the power grid management platform in real time.

[0010] In some implementations, in step 5, after the data is pushed to the power grid management platform in real time, an alarm notification is triggered simultaneously.

[0011] In some implementations, step 3, the target detection algorithm includes: The following are one or more of the following detection algorithms: a first algorithm for scene classification; a second algorithm for detecting videos not connected according to regulations; a third algorithm for detecting unmanned scenes; a fourth algorithm for detecting the absence of safety helmets; a fifth algorithm for detecting the absence of the person in charge; a sixth algorithm for detecting smoking by personnel; a seventh algorithm for detecting personnel using mobile phones; an eighth algorithm for detecting abnormal falls by personnel; a ninth algorithm for detecting the absence of insulated gloves; a tenth algorithm for detecting the absence of work clothes; an eleventh algorithm for detecting the absence of reflective vests; a twelfth algorithm for detecting the absence of safety belts and related violations; a thirteenth algorithm for detecting personnel staying under crane booms; a fourteenth algorithm for detecting unmanned escalators; a fifteenth algorithm for detecting personnel staying under high-altitude operations; a sixteenth algorithm for detecting the absence of fences and warning signs at construction sites; and a seventeenth algorithm for detecting personnel carrying people while the work platform is moving.

[0012] This invention presents an AI-based anomaly detection method for work site images. It integrates Computer Vision (CV) and Natural Language Processing (NLP) across modalities. By using a large NLP model to parse work orders and safety procedure text information, the detection range is defined, enabling precise identification of the target area and subsequent monitoring of key objects. Combined with a CV scene classification algorithm, accurate scene adaptation is achieved, effectively introducing a multimodal large model for secondary filtering of false alarms. A lightweight CV scene classification algorithm incorporating a CoAtNet model is deployed on the edge to distinguish between indoor and outdoor operations, and then an appropriate target detection algorithm is called for detection, improving detection accuracy. Simultaneously, a duplicate alarm filtering mechanism is constructed, significantly improving detection accuracy and regulatory efficiency. This breaks down the barriers between multimodal data, accurately capturing hidden risks such as personnel violations, equipment misconduct, and environmental anomalies, thus improving the scene adaptability and recognition accuracy of anomaly detection.

[0013] To achieve the second objective mentioned above, the present invention provides the following technical solution: An AI-based ultra-high voltage direct current (UHVDC) transmission line operation image anomaly detection system is provided. This system applies the aforementioned AI-based operation site image anomaly detection method, including: The task scenario and detection range orientation module is used to determine the detection range and key monitoring targets; Image data acquisition is used to acquire images and video streams of the work scene; The CV scene classification algorithm recognition module is used to classify indoor and outdoor business, and call appropriate target detection algorithms to identify one or more problems in the image, such as abnormal behavior, unauthorized equipment, and environmental hazards. The multimodal large model secondary filtering module is used to filter error detection results and remove the same type of defects that are repeatedly reported in a short period of time. The anomaly detection module is used to determine the anomaly level and generate standardized violation reports.

[0014] This invention relates to an AI-based ultra-high voltage direct current (UHVDC) transmission operation image anomaly detection system. The system includes: an operation scene and detection range orientation module to determine the detection range and key monitoring targets, overcoming the problems of poor scene adaptability and high false alarm rate in traditional methods; image data acquisition to collect operation scene images and video streams for subsequent analysis; a CV scene classification algorithm recognition module to classify indoor and outdoor operations and call appropriate target detection algorithms to identify one or more problems in the images, such as abnormal behavior, unauthorized equipment, and environmental hazards, overcoming the problem of single-threaded analysis in traditional methods and improving detection accuracy; and a multimodal large-model secondary filtering module to filter erroneous detection results and remove repeatedly reported similar defects within a short period, further improving detection accuracy.

[0015] An electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the method as described in any one of claims 1 to 8. Attached Figure Description

[0016] Figure 1 This is a flowchart of the AI-based work site image anomaly detection method in the embodiments.

[0017] Figure 2 This is a flowchart of the CoAtNet model diagram in the embodiment.

[0018] Figure 3 This is a logic diagram of the second detection algorithm used in the embodiment to detect videos that have not been accessed in accordance with regulations.

[0019] Figure 4 This is a recognition logic diagram of the third detection algorithm used in the embodiment to detect the absence of people in the scene.

[0020] Figure 5 This is a logic diagram of the fourth detection algorithm used to detect not wearing a safety helmet in the embodiment.

[0021] Figure 6 This is the identification logic diagram of the fifth detection algorithm used in the embodiment to detect the absence of the person in charge.

[0022] Figure 7 This is a logic diagram of the twelfth detection algorithm used in the embodiment to detect not wearing a seatbelt and related violations.

[0023] Figure 8 This is the identification logic diagram of the thirteenth detection algorithm used in the embodiment to detect people standing under the boom. Detailed Implementation

[0024] Preferred embodiments of the invention will now be described in more detail. While preferred embodiments of the invention have been shown, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0025] The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a” and “the” as used in this invention and the appended claims are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0026] It should be understood that although the terms "first," "second," "third," etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, features defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] Traditional methods for detecting anomalies in power operation images have significant technical limitations in ultra-high voltage direct current (UHVDC) transmission operation scenarios, making it difficult to meet the high-precision, high-real-time safety management requirements. Traditional algorithms do not integrate UHV operation rules. For example, they rely solely on visual recognition to identify that no safety helmet is being worn, but cannot combine this with work permits to determine indoor operation scenarios where helmets are not required. Furthermore, they misclassify special scenarios such as valve halls and other physical indoor spaces, as well as outdoor business areas, leading to a large number of false alarms.

[0028] Relying solely on image data without linking it to text information such as work permits makes it impossible to collaboratively judge image behavior and text rules. For example, it cannot accurately identify violations of climbing heights above 2 meters based on the requirement in the work permit that "safety belts must be worn for high-altitude operations," and relying solely on visual estimation of height results in large errors.

[0029] Without cloud-edge collaboration and lightweight deployment, monitoring a single operation takes more than 120 minutes, making it difficult to cover more than 10,000 operation points per day; furthermore, the lack of a duplicate alarm filtering mechanism results in the same device reporting the same type of anomaly multiple times within a short period, consuming redundant monitoring resources. To solve the above technical problems, the following embodiments are disclosed: See Figure 1 , Figure 1 This is a flowchart illustrating the AI-based UHVDC transmission operation image anomaly detection system disclosed in this embodiment. The specific workflow of the AI-based UHVDC transmission operation image anomaly detection system in this embodiment includes: Step 1, Targeting the Work Scenarios and Detection Scope: Using a large NLP model to analyze textual data such as work orders and work plans, core information such as work type, location, time period, and safety requirements is extracted to automatically define the detection scenarios. Then, the key monitoring objects within each scenario are identified. For example, when voltage testing and grounding wire installation / removal are detected, the insulated glove wearing detection module is automatically activated; when high-altitude work is detected, the safety belt wearing and fall-from-height risk detection functions are prioritized.

[0030] Step 2, Image Data Acquisition and Preprocessing: Real-time image data, including image / video streams, is acquired using terminal devices such as surveillance cameras and PTZ cameras deployed at the work site. The acquired image data undergoes preprocessing, including image segmentation, mean and variance calculation, and noise filtering, to provide high-quality data input for subsequent recognition.

[0031] Step 3, Accurate Identification via CV Scene Classification Algorithm: Based on the detection scenarios defined by the NLP large model and the identified key monitoring objects, the corresponding CV scene classification algorithm and target detection algorithm are invoked. First, the CoAtNet model is used to complete the indoor / outdoor business logic classification and physical space decoupling. For example, the valve hall is processed according to the outdoor business logic. Then, specialized detection algorithms are invoked for different scenarios to identify abnormal behavior, unauthorized equipment, environmental hazards, and other issues in the images.

[0032] Step 4, Secondary Filtering of the Multimodal Large Model: The recognition results of the CV scene classification algorithm and the job text information parsed by the NLP large model are input into the multimodal large model for cross-modal fusion verification. This filters out erroneous detection results caused by image blurring, perspective bias, and scene misjudgment, while also removing similar defects that are repeatedly reported within a short period of time, thereby improving recognition accuracy.

[0033] Step 5, Anomaly Judgment and Closed-Loop Reporting: Based on preset thresholds, such as detecting the same type of violation three times consecutively within 5 minutes, or a screen display error exceeding 60%, the anomaly level is determined, and a standardized violation report is generated. For valid anomalies that have not been reported, they are pushed to the power grid management platform in real time, simultaneously triggering alarm notifications, thus achieving closed-loop management of "identification-judgment-reporting-handling".

[0034] To further illustrate the various target detection algorithms, the core algorithm recognition logic of the AI-based work site image anomaly detection method includes: The first detection algorithm for scene classification is built on a function-oriented principle, breaking through the limitations of physical space attributes. It defines "indoor / outdoor" logical labels based on business security supervision needs. For example, although a valve hall is a physically indoor space, it is classified as an outdoor business space because full monitoring of violations is required. (See also...) Figure 2 The CoAtNet model was used to build a multi-scale feature extraction network. After inputting a 224×224 work image, the feature map size was gradually reduced (from 112×112 to 7×7) through the Stem stage (S0) and four repeated feature extraction stages (S1-S4). The Rel-Attention mechanism was integrated to enhance the capture of global scene association features and the deep convolution (DConv3×3) was used to improve the local detail recognition ability. At the same time, the work text information parsed by natural language processing (such as high-altitude work and electrical testing work) was combined to optimize the classification weights. Finally, the scene classification results were output, providing accurate basis for subsequent special detection algorithm calls.

[0035] See Figure 3 This system employs a second detection algorithm to detect videos that are not connected according to regulations. This algorithm identifies invalid images through image block analysis and time-series verification. Its core logic is to quantify image validity and confirm anomalies through multiple frequency detections. The technical solution first divides the input image into 28×28 pixel blocks, calculates the grayscale mean, global mean, and variance of each block, and sets anomaly judgment criteria: no actual image, a single color (black / gray) image, or an image containing a countdown / abnormal icon or a distorted screen with a percentage ≥60%. Then, a time-series verification mechanism is introduced. If the above anomalies are detected three times consecutively within 5 minutes, and no similar issues have been reported within 1 hour, the system automatically determines it as an anomaly of video access not being conducted according to regulations and generates a violation record. This algorithm can effectively filter false alarms caused by transient signal interference.

[0036] See Figure 4This third detection algorithm is used to detect the absence of personnel in the monitoring footage. Based on a human detection model and target validity assessment, it identifies abnormal situations where the camera is not aligned with the workers. The core logic is to exclude invalid human targets and confirm that the monitoring is malfunctioning. Technically, the human detection model scans the pre-processed image, filtering out extremely small human figures (height / width < 24 pixels), those showing only hands and feet, or severely obscured incomplete human targets. If the detection result is "no one present," a time-based verification rule is activated: if no one is detected in the footage three times consecutively within 5 minutes, and no similar violations have been reported within 1 hour, it is considered an abnormality of no one present in the footage, triggering an alarm to prompt adjustment of the camera angle or position. The algorithm can reduce blind spots caused by monitoring angle deviations, effectively improving the effectiveness of video surveillance.

[0037] See Figure 5 This fourth detection algorithm is used to detect those not wearing safety helmets. It combines scene classification with facial clarity filtering, focusing on the safety compliance of outdoor workers. Its core logic is to limit the detection scene and eliminate invalid facial interference. The technical solution first filters outdoor scenes based on scene classification results, initiating helmet detection only for these scenes. Next, it detects the facial region of individuals in the image, using an adaptive thresholding method to filter distant, backlit, or blurry (grayscale variance < 50) facial images, reducing the false alarm rate by 10%. Finally, it performs helmet feature matching (such as the helmet's curved outline and reflective strip features) on clear facial regions. If no helmet features are detected, it is considered abnormal, triggering an alarm. This algorithm maintains high recognition accuracy even in strong light and complex backgrounds.

[0038] See Figure 6 This fifth detection algorithm is used to detect the absence of responsible persons. Based on time window verification, red armband feature recognition, and target size filtering, it ensures that responsible persons are on duty during core operating periods. The core logic is to accurately identify the responsible person's identification and confirm their absence status based on the time dimension. Technically, the detection time window is limited to 9:00 AM to 4:00 PM daily (core operating period). The YOLOv8 model identifies red armband features in images, requiring the armband detection box to be at least 11 pixels wide and long and at least 150 pixels in area, filtering out invalid small targets. If no matching red armband is detected within 30 minutes, and no similar violation has been reported within 90 minutes, it is determined that the responsible person is absent. This algorithm effectively avoids misjudgment of the responsible person due to mixed personnel.

[0039] The sixth detection algorithm for detecting smoking by personnel identifies dangerous behaviors of outdoor workers through target feature matching and behavioral pattern analysis. Its core logic is to identify the "white cigarette" feature and specific smoking posture. The technical solution first filters images of logically outdoor scenes containing people, then uses color space conversion (RGB to HSV) to highlight the white cigarette feature. Next, a target detection model matches the posture features of "holding a cigarette" (hand area containing a thin white target) or "holding a cigarette in the mouth" (lip area containing a small white target). Meeting either condition is considered an abnormal smoking behavior. Simultaneously, the detection weight is enhanced by combining the "no-smoking work area" text information parsed by natural language processing.

[0040] The seventh detection algorithm is used to detect personnel using mobile phones. Based on target features and behavior exclusion rules, it accurately identifies violations of mobile phone use by workers. The core logic is to distinguish between illegal mobile phone use and compliant use scenarios (such as making calls or operating the phone while it is flat). In terms of technical solution, it detects mobile phone targets (rectangular outline, screen reflection features) in logical outdoor scenes, excluding two compliant situations: when the phone is flat, the aspect ratio is ≥2 (such as operating a tablet device), and the mobile phone recognition box and the head recognition box overlap (such as making a call). Other detected mobile phone holding behavior (the hand area contains the mobile phone target) is judged as abnormal. At the same time, it confirms violations through multi-frame continuous detection (capturing the mobile phone holding action 3 times within 5 frames) to reduce instantaneous false judgments.

[0041] The eighth detection algorithm is used to detect abnormal falls. Based on human key point detection and posture angle calculation, it identifies emergency falls. The core logic is to distinguish between falling and normal postures (bending over, squatting). In the technical solution, the OpenPose model is used to detect key nodes such as the head, waist, knees, and feet of the person, and calculate the angle between the human torso and the horizontal direction. An angle <45° is initially judged as a suspected fall. Then, two steps of filtering are used: first, excluding autonomous posture changes such as bending over (the waist key point is higher than the knee key point) and squatting (the knee key point bending angle is >90°); second, filtering low-risk scenarios where there are also standing people. Finally, once an abnormal fall is confirmed, the highest level alarm is triggered immediately.

[0042] The ninth detection algorithm, designed to detect the absence of insulated gloves, integrates text parsing and target matching to detect the wearing of protective equipment for specific job types. Its core logic is to correlate "voltage testing / grounding wire installation / removal" with the requirement for wearing insulated gloves. Technically, a large natural language processing model parses the work order to identify job types requiring insulated gloves, such as "voltage testing" and "grounding wire installation / removal." The target detection model then identifies the areas where insulated tools are held (e.g., voltage testers, grounding clamps) and the characteristic areas of the insulated gloves (rubber material, specific color). If the tool identification box and the insulated glove identification box do not overlap, and the job type falls under the aforementioned specific scenario, then the absence of insulated gloves is determined to be an anomaly.

[0043] The tenth detection algorithm is used to detect individuals not wearing work clothes. Based on a binary classification model and target filtering, this algorithm identifies the compliance of personnel's attire. Its core logic is to focus on upper garment features and exclude invalid detection targets. The technical solution first crops the upper garment recognition bounding box in the image, filtering out invalid boxes with a height / width < 24 pixels. Then, the cropped upper garment image is input into the binary classification model (work clothes / non-work clothes). The model extracts features such as clothing texture and color using a ResNet50 network. When the classification result is "non-work clothes" with a confidence score > 0.8, it is determined to be an anomaly of not wearing work clothes.

[0044] The eleventh detection algorithm is used to detect those not wearing reflective vests. This algorithm combines time-based filtering with feature matching to ensure visibility for workers at night. Its core logic is to limit the detection period to nighttime and focus on reflective features. Technically, the detection period is limited to 20:00-6:00 the next day (nighttime). Image brightness analysis (average brightness < 80) is used to confirm the nighttime scene. The target detection model identifies the intersection between the person's bounding box and reflective stripes / vest features (high brightness, striped texture). If there is no intersection, it is determined that the person is not wearing reflective vests. Multi-frame fusion is introduced (no reflective features detected twice within 3 frames) to reduce false positives due to momentary occlusion.

[0045] See Figure 7 The twelfth detection algorithm is used to detect violations such as not wearing safety belts. This algorithm combines height-based judgment, feature recognition, and attachment point analysis to comprehensively monitor compliance with high-altitude work safety regulations. The core logic is to first confirm the height-based scenario and then detect any violations. In the technical solution, height-based judgment uses differentiated rules: tower / pole climbers are directly judged as climbing at a height > 2 meters; ladder climbers are judged to be working at a height if the actual height is calculated based on their height (assuming 1.6 meters) and is greater than 2 meters; the algorithm detects safety belt features (such as the shoulder straps and waist belt outline of a full-body safety belt), and if these are not detected, it is judged as not wearing a safety belt; simultaneously, the algorithm analyzes the safety belt attachment point position, judging "low attachment, high use" if the attachment point is below the waist, and judging "not wearing a backup safety rope" if the backup safety rope feature is not detected. All three situations trigger an abnormal alarm.

[0046] See Figure 8 The thirteenth detection algorithm is used to detect personnel lingering under a crane boom. Based on dual-target recognition, positional relationship determination, and monocular depth estimation, this algorithm identifies safety hazards in mechanical operations. Its core logic is to confirm the spatial hazard relationship between the crane boom and personnel. Technically, it first identifies the crane boom (outline of a large robotic arm) and personnel targets in the image, confirming that their bounding boxes do not intersect and are horizontally parallel. Then, it generates a pseudo-color depth map using monocular depth technology, converting it to a grayscale image. A grayscale value <10 indicates that the personnel and crane boom are in the same depth plane. Combining the horizontal overlap and the vertical relationship of the personnel under the crane boom, it ultimately determines that the presence of personnel under the crane boom is abnormal.

[0047] The fourteenth detection algorithm for unmanned escalators is based on target intersection analysis to identify violations where the escalator is not being held by someone. Its core logic is to confirm the absence of personnel on the escalator through multi-target association. The technical solution calls a target detection model to identify the bounding boxes of a movable ladder, people on the ladder, people on the escalator (with their hands touching the ladder and in a stable posture), and surrounding pedestrians. A judgment rule is set: if the ladder's bounding box intersects with the bounding box of people on the ladder (confirming someone is on the ladder), but does not intersect with the bounding boxes of people on the escalator or pedestrians (confirming no one is holding the escalator), then the escalator is considered abnormal; otherwise (if people on the escalator or pedestrians are present and holding the escalator), no alarm is triggered.

[0048] The fifteenth detection algorithm is used to detect personnel staying below high-altitude operations. This algorithm identifies unauthorized staying behavior by analyzing the positional relationships of personnel feature boxes. The core logic is to capture the vertical spatial overlap of personnel. Technically, it detects two or more personnel feature boxes in an image, extracting the horizontal coordinate range (x1-x2) and vertical coordinate range (y1-y2) of each box. If the y2 (bottom) of personnel A's box is less than or equal to the y1 (top) of personnel B's box, and the x1-x2 overlap ratio is ≥50%, then it is determined to be an abnormal situation where A is at a higher position and B is below, triggering an alarm to evacuate the personnel below.

[0049] The sixteenth algorithm is used to detect the absence of fences and warning signs at construction sites. Based on scene classification and similar image filtering, this algorithm focuses on safety protection in digging scenarios. Its core logic is to first identify the work scene and then detect the lack of protective facilities. The technical solution identifies the "digging" work scene (features such as irregular depressions in the ground and construction tools) through a scene classification model. For this scene, it detects "fences" (closed outlines such as continuous fences, steel railings, and wire mesh fences) and "warning signs" (text or graphic features such as "High Voltage Danger" and "No Climbing"). If none of these features are detected, it is initially judged as an anomaly. Then, a similar image algorithm (calculating image hash value similarity ≥ 0.8) filters out 80% of duplicate alarms. Finally, after confirming the anomaly, a violation record is generated.

[0050] The seventeenth detection algorithm is used to detect the movement of personnel on the aerial work platform. The sixteenth detection algorithm is based on multi-target recognition and intersection relationship determination to identify the illegal movement of personnel on the aerial work platform. The core logic is to distinguish between personnel carried when the platform is stationary (compliant) and personnel carried when it is moving (illegal). In terms of technical solution, the aerial work platform vehicle (including the lifting arm and the outline of the work platform), personnel on the work platform (human-shaped targets within the platform area), and hydraulic columns (the platform support structure, visible when stationary) are identified in the logical outdoor scene. The judgment rule is set as follows: if the identification box of the work platform vehicle and the identification box of the personnel intersect (confirming that personnel are carried) and do not intersect with the identification box of the hydraulic column (confirming that the platform is not supported and is in a moving state), it is judged as an abnormality of personnel being carried while the work platform is moving; otherwise (if the identification box of the hydraulic column is present), no alarm is triggered.

[0051] This embodiment utilizes a large-scale Natural Language Processing (NLP) model to parse textual data such as work orders and safety procedures, accurately extracting core information such as work type, scene attributes, and safety requirements. It then precisely defines the image detection range and key monitoring dimensions, filtering out irrelevant scene interference at the source, thus initially improving the targeting and efficiency of anomaly detection. Furthermore, it invokes the CoAtNet scene classification model to overcome physical space limitations and complete indoor / outdoor logical label classification based on business regulatory needs. Simultaneously, it matches specialized computer vision (CV) detection algorithms (such as personnel behavior recognition, equipment compliance detection, and environmental hazard identification) to extract multi-dimensional target features from work images. Combining monocular depth estimation and human keypoint detection technologies, it uncovers subtle violation features (such as not wearing a safety belt or backup safety rope, or personnel lingering under the boom) and hidden risks in the images, solving the problems of easily overlooking subtle anomalies and high misjudgment rates in complex scenes in traditional detection methods. Finally, it fuses text parsing results, CV recognition features, and time-series data through a multimodal large-scale model, performing secondary filtering of error detection and duplicate alarms, and determining the anomaly level based on preset thresholds. The result is then a comparative analysis of the anomaly with normal work images and a standardized violation report. This invention realizes intelligent processing of the entire process of image anomaly directional screening, accurate identification, in-depth verification, and closed-loop management in UHVDC transmission operations. It not only fully explores the multi-dimensional details in the images, but also effectively avoids problems such as scene misjudgment and omission of minor anomalies, significantly improving the accuracy of anomaly detection and regulatory efficiency, and meeting the safety management needs of UHV operations at all times and in all scenarios.

[0052] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based method for detecting anomalies in work site images, characterized in that, Includes the following steps: Step 1: Analyze the text data including work orders and work plans using an NLP large model, extract core information such as work type, work location, work time period, and safety requirements to determine the detection scenario, and then determine the monitoring object for the detection scenario; Step 2: Real-time acquisition of image data of the work scene; Step 3: Based on the collected image data, call the CV scene classification algorithm containing the CoAtNet model to perform the business logic classification of indoor / outdoor business classification, and call the corresponding target detection algorithm according to the determined detection scenario and monitoring object to identify at least one abnormal operation event among abnormal behavior, illegal equipment or environmental hazards in the image; Step 4: Input the business logic classification, job exception events, and core information parsed by the NLP big model into the multimodal big model, perform cross-modal fusion verification, filter error detection results, and remove the same type of defects that are repeatedly reported in a short period of time, and filter errors and / or repeated job exception events. Step 5: Perform threshold judgment on valid work anomaly events, determine the anomaly level, and generate a standardized violation report.

2. The method for detecting anomalies in work site images based on artificial intelligence according to claim 1, characterized in that, In step 2, terminal devices of PTZ cameras and / or pan-tilt cameras are deployed at the work site to collect images and video streams of the work scene in real time. The collected image and video stream data are preprocessed, including one or more operations such as image segmentation, mean and variance calculation, and noise filtering.

3. The method for detecting anomalies in work site images based on artificial intelligence according to claim 1, characterized in that, In step 5, the preset threshold includes the percentage of similar violations or screen glitches detected N times consecutively within a set time period.

4. The method for detecting anomalies in work site images based on artificial intelligence according to claim 3, characterized in that, The set time is 1 to 5 minutes, and N is 2 to 5.

5. The artificial intelligence-based anomaly detection method for work site images according to claim 3, characterized in that, The set time is 5 minutes, and N is 3.

6. The method for detecting anomalies in work site images based on artificial intelligence according to claim 1, characterized in that, Step 5 also includes closed-loop reporting, which includes: pushing abnormalities for which no violation report is generated to the power grid management platform in real time.

7. The method for detecting anomalies in work site images based on artificial intelligence according to claim 6, characterized in that, In step 5, after being pushed to the power grid management platform in real time, an alarm notification is triggered simultaneously.

8. The method for detecting anomalies in work site images based on artificial intelligence according to claim 1, characterized in that, In step 3, the target detection algorithm includes: The following are one or more of the following detection algorithms: a first algorithm for scene classification; a second algorithm for detecting videos not connected according to regulations; a third algorithm for detecting unmanned scenes; a fourth algorithm for detecting the absence of safety helmets; a fifth algorithm for detecting the absence of the person in charge; a sixth algorithm for detecting smoking by personnel; a seventh algorithm for detecting personnel using mobile phones; an eighth algorithm for detecting abnormal falls by personnel; a ninth algorithm for detecting the absence of insulated gloves; a tenth algorithm for detecting the absence of work clothes; an eleventh algorithm for detecting the absence of reflective vests; a twelfth algorithm for detecting the absence of safety belts and related violations; a thirteenth algorithm for detecting personnel staying under crane booms; a fourteenth algorithm for detecting unmanned escalators; a fifteenth algorithm for detecting personnel staying under high-altitude operations; a sixteenth algorithm for detecting the absence of fences and warning signs at construction sites; and a seventeenth algorithm for detecting personnel carrying people while the work platform is moving.

9. An artificial intelligence-based image anomaly detection system for ultra-high voltage direct current transmission line operations, characterized in that, The system is applied to the artificial intelligence-based work site image anomaly detection method according to any one of claims 1 to 8, comprising: The task scenario and detection range orientation module is used to determine the detection range and key monitoring targets; Image data acquisition is used to acquire images and video streams of the work scene; The CV scene classification algorithm recognition module is used to classify indoor and outdoor business, and call appropriate target detection algorithms to identify one or more problems in the image, such as abnormal behavior, unauthorized equipment, and environmental hazards. The multimodal large model secondary filtering module is used to filter error detection results and remove the same type of defects that are repeatedly reported in a short period of time. The anomaly detection module is used to determine the anomaly level and generate standardized violation reports.

10. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the method as described in any one of claims 1 to 8.

Images