Pressing plate operation behavior analysis and error prevention method and system based on image recognition and machine learning

Abstract

The invention relates to a pressing plate operation behavior analysis and error prevention method and system based on image recognition and machine learning. The method comprises the following steps: acquiring a pressing plate operation area video stream through image acquisition equipment, and segmenting a video into independent operation event segments based on motion detection and trajectory analysis; extracting key point time sequence data of hands of an operator by using a human body posture estimation model, and constructing a dynamic feature sequence fusing a spatial relationship, kinematics and posture semantics; performing multi-level similarity comparison on the dynamic feature sequence and a standard operation template, and judging operation compliance by a machine learning model in combination with a dynamic threshold value; triggering graded early warning and intervention according to a judgment result; and an incremental learning mechanism is adopted, and the template and the threshold value are adaptively optimized based on historical data. According to the invention, accurate and intelligent analysis and active error prevention of the whole operation process of the pressing plate are realized, and the safety level of electric power operation is effectively improved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of power system safety technology and computer vision, specifically relating to a method and system for analyzing and preventing errors in pressure plate operation behavior based on image recognition and machine learning. Background Technology

[0002] In the operation and maintenance of power systems, the operation of the switch plates on protection cabinets is a frequent and crucial task. The activation and deactivation status of the switch plates directly affects the start and stop of relay protection functions. Incorrect operation can lead to malfunctions or failures to operate the protection devices, or even trigger power grid accidents, causing significant economic losses and social impact. Therefore, ensuring the correctness and standardization of switch plate operation is one of the core aspects of safe power production.

[0003] Currently, on-site monitoring of pressure plate operations mainly relies on traditional manual supervision and operation ticket systems. This model has the following significant limitations: First, the continuity and focus of manual supervision are difficult to guarantee, and it is easily affected by subjective factors, making it impossible to achieve 24 / 7, all-around monitoring; second, although the operation ticket system can provide process guidance, it cannot verify and provide feedback on the operator's specific actions, postures, operation sequence, and operating techniques in real time, making it difficult to prevent unintentional operational errors; in addition, existing technical means, such as anti-misoperation interlocking systems based on radio frequency identification or physical sensors, focus more on detecting the final state of the pressure plate, but cannot perform dynamic analysis and risk warning of the operation process itself, resulting in blind spots in protection.

[0004] In recent years, while image recognition and artificial intelligence technologies have seen some applications in industrial safety, such as identifying helmet wearing status through target detection, their application in the analysis of delicate operational behaviors in the power industry remains in its early stages. Existing solutions are mostly limited to recognizing static scenes or simple actions, lacking the ability to dynamically model and semantically understand the entire process of continuous and complex pressure plate operations. Specifically, this manifests as: difficulty in accurately segmenting effective operational events from continuous video streams; insufficient ability to capture and extract features from delicate hand movements; a lack of intelligent judgment logic deeply integrated with power safety regulations; and a deficiency in the system's adaptive and continuous optimization capabilities.

[0005] Therefore, the power industry urgently needs a solution that can deeply integrate image recognition and machine learning technologies to achieve full-process, refined, and intelligent analysis and proactive error prevention of pressure plate operation behavior, so as to make up for the shortcomings of existing technologies and fundamentally improve the level of operational safety. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system that can dynamically monitor, intelligently analyze, and proactively prevent errors in the entire process of pressure plate operation.

[0007] In a first aspect, embodiments of this application provide a method for analyzing and preventing errors in pressure plate operation behavior based on image recognition and machine learning, the method comprising: Step S1: Acquire the video stream of the pressure plate operation area monitoring through the image acquisition device, analyze the video stream using a motion detection algorithm, and divide the video stream into multiple independent operation event segments based on the start and end of the motion state; and pre-store the standard operation sequence template. Step S2: Using a human posture estimation model, extract key point time-series data of the operator's hand from the operation event segment; based on the key point time-series data, construct a dynamic feature sequence to describe a single pressure plate operation process; Step S3: Compare the real-time generated dynamic feature sequence with the pre-stored standard operation sequence template for similarity; based on the relationship between the comparison result and the preset threshold, the compliance of the current operation is determined by the judgment threshold of the machine learning model. Step S4: Based on the compliance assessment results, trigger different levels of risk warnings or intervention measures; Step S5: Using an incremental learning mechanism, the judgment threshold of the standard operation sequence template or the machine learning model is adaptively optimized by utilizing accumulated historical operation data.

[0008] Secondly, embodiments of this application provide a platen operation behavior analysis and error prevention system based on image recognition and machine learning, applied to the platen operation behavior analysis and error prevention method based on image recognition and machine learning as described in the first aspect, the system comprising: The video event segmentation module is used to acquire video streams of the pressure plate operation area through image acquisition devices, analyze the video streams using motion detection algorithms, and segment the video streams into multiple independent operation event segments based on the start and end of the motion state. The dynamic feature sequence construction module is used to extract key point temporal data of the operator's hand from the operation event segment using a human posture estimation model; and to construct a dynamic feature sequence describing a single pressure plate operation based on the key point temporal data. The operation compliance determination module is used to compare the real-time generated dynamic feature sequence with the pre-stored standard operation sequence template; based on the relationship between the comparison result and the preset threshold, the machine learning model determines the compliance of the current operation. The risk warning and intervention module is used to trigger different levels of risk warnings or intervention measures based on the compliance assessment results. The adaptive optimization module is used to adaptively optimize the judgment threshold of the standard operation sequence template or the machine learning model by using an incremental learning mechanism and accumulating historical operation data.

[0009] Thirdly, embodiments of this application provide an electronic device, including: processor; Memory used to store processor-executable instructions; The processor is configured to implement the platen operation behavior analysis and error prevention method based on image recognition and machine learning as described in the first aspect when executing the instructions.

[0010] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the platen operation behavior analysis and error prevention method based on image recognition and machine learning as described in the first aspect.

[0011] Beneficial effects: 1. It has achieved a leap from state-based error prevention to behavior-based error prevention, enabling dynamic monitoring and analysis of the entire operation process; 2. Through multi-dimensional feature extraction and intelligent comparison, the accuracy and reliability of behavior recognition are improved; 3. Employing a tiered early warning and adaptive optimization mechanism enables the system to continuously improve. 4. It effectively improves the safety level of power system pressure plate operation and makes up for the shortcomings of existing technical means. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of a method for analyzing and preventing errors in platen operation based on image recognition and machine learning, provided in an embodiment of this application.

[0013] Figure 2 The system architecture diagram for platen operation behavior analysis and error prevention based on image recognition and machine learning provided in this application.

[0014] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0016] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0017] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Example 1 Figure 1 This is a schematic flowchart illustrating a method for analyzing and preventing errors in pressure plate operation based on image recognition and machine learning, provided in an embodiment of this application. Figure 1 As shown, a method for analyzing and preventing errors in pressure plate operation based on image recognition and machine learning includes: Step S1 involves acquiring a video stream of the pressure plate operation area using an image acquisition device. A motion detection algorithm is then used to analyze the video stream, dividing it into multiple independent operation event segments based on the start and end points of the motion. Specifically, this step uses a motion detection algorithm to identify valid movements in the video stream caused by the operator's hand movements. Based on the start and end points of the motion, the continuous monitoring video is segmented into discrete, complete operation event segments. This provides a data foundation for subsequent refined analysis of individual operations, is the first step in achieving behavior analysis, and is crucial for extracting valuable information from massive amounts of monitoring video.

[0019] Specifically, in this embodiment, step S1 includes: Step S11 involves using a Gaussian mixture model for background modeling, combined with inter-frame differencing to accurately extract the foreground motion region consisting of the operator's hands and torso from the video stream. Specifically, this step uses a Gaussian mixture model for background modeling to handle slow background changes such as lighting variations, and combines it with inter-frame differencing to quickly capture moving foreground objects. The fusion of these two methods aims to stably and accurately extract the effective motion region consisting of the operator's hands and torso from dynamically changing video footage, while filtering out irrelevant environmental interference (such as flickering light and shadow, movement of other people, etc.) to the greatest extent possible.

[0020] Step S12 involves performing Kalman filtering tracking on the foreground motion region to generate a motion trajectory. When the motion trajectory first enters the preset pressure plate operation interest region and the motion speed is lower than a set dwell speed threshold, it is determined as the starting point of an operation event. This step first uses Kalman filtering tracking to perform smooth and continuous trajectory prediction and updating on the extracted motion region, forming a stable motion trajectory and overcoming brief occlusion or detection jitter. Its core determination logic is: when the trajectory first enters the preset pressure plate operation interest region (i.e., the physical pressure plate position) and the motion speed is lower than the dwell speed threshold, it is determined as the starting point of the operation. This definition combines the simple entry area with the intention to perform the operation, effectively eliminating invalid actions such as the operator's hand quickly swiping or unconsciously approaching.

[0021] Specifically, in this embodiment, determining the starting point of the operation event in step S12 includes: Step S121: Before the movement trajectory enters the preset area of ​​interest for platen operation, a consistency analysis is performed on its movement direction within N consecutive frames. When the angle between the movement direction and the theoretical vector direction from the operator's standing area to the target platen is less than a set angle tolerance, it is determined to be a valid approach with operational intent. This step initiates the analysis before the movement trajectory enters the area of ​​interest, which is a proactive judgment. By analyzing the movement direction of the trajectory within N consecutive frames and comparing it with the theoretical vector direction from the operator's standing area to the target platen, the following steps are taken: Intent recognition: If the movement direction is highly consistent with the theoretical direction (angle less than the tolerance), it indicates that the operator is approaching the target platen purposefully and directly, which is a strong signal of compliant operational intent. Error path interception: If the movement direction is chaotic or deviates too much from the theoretical direction, it may be searching for the platen, heading to another location by mistake, or waving randomly. In this case, it will be regarded as an invalid approach and directly excluded at the starting stage, avoiding unnecessary waste of subsequent analysis resources.

[0022] Step S122: At the moment of determination of the starting point, hand posture recognition is performed on the foreground motion area. If the recognized hand posture is a predefined operation preparation posture, the starting point is finally confirmed as valid; otherwise, it is considered an invalid crossing or accidental touch, and the event is ignored. This step is the final and most refined check after the motion trajectory has met the spatial and speed conditions (i.e., starting point determination). It no longer focuses on where the hand moves, but on how the hand is preparing to move. By recognizing whether the hand posture is an operation preparation posture (such as extending the index finger to prepare to press, rather than clenching the fist, opening the palm, or other random postures), it is confirmed whether the operator has made the physiological preparation for the correct operation. Only when the four conditions of space (entering the area), dynamic (low speed), intention (consistent direction), and posture (readiness) are met simultaneously is the starting point finally confirmed. This can effectively filter out scenarios that seem to meet the preceding conditions but are actually invalid contacts, such as using a fist to support the body near the cabinet or wiping the pressure plate with the back of the hand.

[0023] A simple space-velocity triggering mechanism is upgraded into a multimodal, multi-stage behavioral intent recognition pipeline. Step S121 addresses the question of whether the target is being moved towards the correct objective, focusing on the motion path. Step S122 addresses the question of whether the correct method is being used, focusing on body posture. The combination of these two steps enables a higher level of semantic understanding in determining the start of an operation, significantly improving robustness and reliability in complex, unstructured environments, and ensuring high quality and high relevance of subsequent behavioral analysis data from the outset.

[0024] Step S13: When the motion trajectory completely leaves the region of interest of the pressure plate operation and remains stationary for more than a preset number of frames, it is determined as the end point of the operation event. Based on the start and end points, the corresponding operation event segment is extracted from the continuous video stream. The determination logic of this step is: when the motion trajectory completely leaves the region of interest and remains stationary for more than a preset number of frames. This ensures that the operator's hands have completely completed the operation and withdrawn, and avoids premature interruption of the event due to short pauses or slight tremors during the operation by using the condition of continuous stationaryness. Finally, based on this pair of precise start and end points, the operation event segment corresponding to this complete operation behavior is extracted from the continuous video stream, providing high-quality input data for subsequent analysis.

[0025] Specifically, in this embodiment, determining the endpoint of the operation event in step S13 includes: Step S131: After the motion trajectory leaves the area of ​​interest for the pressure plate operation, continuously monitor the state of the pressure plate itself within that area. If the pressure plate state changes within a certain period after the motion trajectory leaves, the completion moment of the state change is corrected as the endpoint of this operation event. Here, the pressure plate state refers to the actual physical position state of the pressure plate, such as being engaged or disengaged. This is typically obtained through image recognition or a dedicated pressure plate state sensor (such as a microswitch or displacement sensor). The completion moment of the state change refers to the instant the pressure plate switches from one stable state (e.g., engaged) to another stable state (e.g., disengaged). This achieves result-oriented endpoint correction. It ensures that the recorded operation segments completely cover the entire process from the operator's initial operation to the actual effect of the operation. For example, in the uncorrected case: the operator presses down on the pressure plate and quickly removes their hand, but the pressure plate slowly springs back to its final position due to spring force. If the endpoint is determined solely by the operator's hand leaving, the confirmation of the final state change of the pressure plate will be missed. In the corrected case: continuous monitoring continues until the pressure plate clicks into place (as confirmed by sensor signal change or image recognition), and this moment of placement is taken as the true endpoint of the operation. In this way, even if the operator's hands are removed early, the operation segment still includes confirmation of the operation result.

[0026] In step S132, if the motion trajectory continuously passes through or briefly stops at the interest area of ​​multiple adjacent pressure plates in the same operation event segment, the error prevention logic is activated; all pressure plate sets involved in the continuous operation are identified; the pressure plate sets are compared with the operation ticket logic in real time; only when the pressure plate set and operation sequence meet the operation ticket requirements are the entire continuous motion determined as a complete compliant operation event; otherwise, it is determined as a risk event containing misoperation, and a risk warning is immediately triggered in step S4.

[0027] In this context, "single operation event segment" refers to the continuous operation of multiple pressure plates by the hand during a single, continuous movement without complete retraction. The pressure plate set refers to the list of all pressure plates touched or operated by the hand during this continuous operation. The operation ticket logic refers to the list of pressure plates and their strict order that must be operated for a specific task (such as switching a line from operation to maintenance) as stipulated in power safety regulations. This system implements real-time logical error prevention for the high-risk scenario of continuous single-handed operation, moving beyond the perspective of a single pressure plate and performing safety verification from the perspective of the entire transaction. For example, the operation task requires the sequential placement of pressure plates A, B, and C. Compliant operation: The operator's hand continuously and correctly operates A→B→C in that order. The pressure plate set {A,B,C} and its order are identified as consistent with the operation ticket, thus determining it as a complete compliant operation event. Incorrect operation: The operator's hand continuously operates A→C→B (incorrect order). If the order of the pressure plate set {A,C,B} does not match the operation ticket (A→B→C), it is immediately identified as a risk event involving misoperation, and an alarm is triggered on the spot, instead of waiting for all operations to be completed before post-event analysis.

[0028] These two steps, from the dimensions of physical results and logical transactions respectively, greatly enhance the intelligence and security level of event endpoint determination: Step S131 ensures that each operation event is physically complete and valid. Step S132 ensures that a series of operations is logically correct and compliant.

[0029] These interconnected sub-steps collectively enable a refined and intelligent video segmentation process, from pixel-level processing (motion detection) to object-level tracking (trajectory tracing), and then to event-level understanding (start / endpoint semantic determination). In real-world, complex environments, the ability to accurately capture and define each valid pressure plate operation is a crucial first step in the entire behavior analysis chain.

[0030] Step S2 involves extracting key point temporal data of the operator's hand from the operation event segment using a human posture estimation model. Based on this key point temporal data, a dynamic feature sequence describing a single pressing plate operation is constructed. Specifically, this step utilizes a human posture estimation model to quantify and extract spatiotemporal variation data of key parts of the operator's hand (such as the wrist and fingertips) from the operation event segment obtained in Step S1. Furthermore, by integrating multi-dimensional information such as spatial relationships, kinematics, and posture semantics, a dynamic feature sequence capable of comprehensively and accurately describing a single pressing plate operation is constructed. This step is the core step in abstracting specific actions into a feature model.

[0031] Specifically, in this embodiment, step S2 includes: Step S21 employs a human pose estimation model based on a deep convolutional neural network to locate key points of the operator's wrist, palm base, and fingertips in each frame of the image. Optical flow information is then used to correlate and track key points across frames, forming temporal data of these key points. The key points of the wrist, palm base, and fingertips represent the joints of the hand's skeleton and are fundamental to describing hand posture and movement. For example, the wrist point serves as the reference for the arm, and the fingertip point is used to determine the contact position. Optical flow information describes the direction and velocity of pixel movement between consecutive frames, used to correlate the same key point in different frames to form a continuous trajectory. This step stably and accurately acquires the basic spatiotemporal data of the hand. For instance, when processing video clips, in each frame, the human pose estimation model based on a deep convolutional neural network outputs coordinate data similar to {wrist:(x1,y1), fingertip:(x2,y2),...}, where x1 represents the horizontal coordinate of the wrist, y1 represents the vertical coordinate of the wrist, x2 represents the horizontal coordinate of the fingertip, and y2 represents the vertical coordinate of the fingertip. Optical flow ensures that even if the operator's hand is briefly obscured in a frame, the position of the obscured key point can be predicted and tracked based on the motion trend of the previous frame, thus forming a smooth and continuous trajectory line instead of a bunch of jumping points.

[0032] Step S22: Based on the key point time-series data, calculate and fuse the following three types of features to construct the dynamic feature sequence: Spatial relationship features refer to calculating the two-dimensional pixel distance between the fingertip key point and the target pressure plate throwing area (how far the hand is from the pressure plate), and the estimated angle between the hand plane and the cabinet plane (whether the hand is facing the pressure plate directly or at an angle). This answers where the hand is and in which direction, used to determine whether the operation path and position are accurate. For example, compliant operation requires the fingers to press the pressure plate vertically. If the estimated angle is too large, it will be recorded that the hand grazed the pressure plate at an angle instead of pressing it directly.

[0033] Kinematic characteristics: Calculate the instantaneous velocity (how fast the hand moves), acceleration (whether it is uniform, accelerating, or decelerating), and smoothness of the movement direction (whether the path is a smooth straight line / curve or shaky / hesitant). This answers how the hand moves, helping to determine whether the operation is dynamic, stable, and decisive.

[0034] Posture semantic features: Based on the relative positional relationships between different key hand points, hand postures are identified and encoded in real time, such as pointing, pinching, pressing, and rotating. This answers what the hand is doing, converting hand shapes into meaningful operational instructions. For example, the correct posture for operating a pressure plate is pressing. If the system identifies a pinching posture with a rotating motion, even if the position is correct, it will be judged as an incorrect operation because this is not the correct hand posture for operating the pressure plate.

[0035] The raw video data is extracted into a dataset containing location, dynamics, and meaning. Figure 3 Large-scale, machine-processable behavioral language. This is like a translator turning a video of hand movements operating a pressure plate into a series of time-series data that a computer can understand, containing [position coordinates, velocity values, and action labels], thus enabling subsequent intelligent judgment.

[0036] Specifically, the construction of the dynamic feature sequence in step S22 includes: Step S221 converts the two-dimensional pixel distance in the spatial relationship features into an absolute physical distance corresponding to the real-world scale, based on the camera calibration parameters and the known size of the reference object. This eliminates feature deviations caused by the operator's position or different camera perspectives. The two-dimensional pixel distance is the distance, in pixels, between the key point of the hand and the pressure plate in the image captured by the camera. This distance changes depending on the distance between the person and the camera. The absolute physical distance is the actual distance, in centimeters or millimeters, between the hand and the pressure plate in the real world. The camera calibration parameters describe the internal parameters (such as focal length) and external parameters (such as position and angle) of the camera's optical characteristics, used to establish a mathematical relationship between two-dimensional image coordinates and three-dimensional world coordinates. Eliminating feature deviations caused by observation perspective and distance makes the features universal and comparable. For example, a tall operator standing far away may have their hand only 50 pixels away from the pressure plate in the image; while a short operator standing closer may have their hand 100 pixels away from the pressure plate in the image. If pixel distance is used directly, the system might incorrectly assume the latter's operation path is longer or different. Through camera calibration, the system converts both 50 pixels and 100 pixels into the same 20-centimeter real physical distance. This way, regardless of who operates from what location, as long as the real distance is 20 centimeters, the extracted feature values ​​will be consistent. This is a mapping process from two-dimensional image pixel coordinates to three-dimensional physical coordinates, achieved through a camera calibration model, thus eliminating feature bias caused by different shooting angles and distances.

[0037] Step S222: When constructing the dynamic feature sequence, different weights are dynamically assigned to three types of features according to different stages of the operation process: In the approach stage (the stage when the hand moves from the starting position to the point of contact with the pressure plate), the weight of the spatial relationship feature is increased to focus on evaluating the correctness of the operation path; in the execution stage (the moment and process when the hand contacts the pressure plate and changes its state), the weight of the posture semantic feature and the smoothness of the motion direction in the kinematic features is increased to focus on evaluating the standardization and stability of the operation posture; in the departure stage (the stage when the operation is completed and the hand is withdrawn from the pressure plate area), the weight of the instantaneous velocity in the kinematic features is increased to evaluate the decisiveness of the operator's hand withdrawal after the operation is completed. Different assessment focuses are placed on different stages of the operation, making the behavioral analysis more targeted and more in line with human cognition. For example, in the approach stage: the weight of the spatial relationship feature is increased. At this time, the most important thing is to see if the operator has moved to the wrong interval and whether the hand moves directly and accurately towards the target pressure plate. Whether the hand posture is clenched or extended is not important at this time. In the execution stage: the weight of the posture semantic feature and the smoothness of the motion is increased. At this stage, the most important thing is to confirm whether the technique is correct (whether to press with fingertips or pat with the palm) and whether the movement is stable (no shaking or hesitation). Minor changes in position are no longer the focus. In the departure phase: increase the weight of instantaneous speed. At this point, pay attention to whether the operator's withdrawal is decisive. A quick, direct withdrawal indicates confidence and high completion rate; while a slow, hesitant withdrawal may mean the operation is uncertain or incomplete.

[0038] By addressing consistency issues across different individuals and perspectives through feature normalization, and further resolving accuracy issues across different stages and assessment priorities through stage-adaptive weighting, a static feature set is transformed into an intelligent, context-aware, and dynamic discriminator, significantly improving the accuracy and practicality of behavioral analysis.

[0039] Furthermore, when constructing the dynamic feature sequence, context-aware enhanced features are introduced, specifically including: Step S223: Simultaneously extract and monitor the posture and position of the non-operating hand to generate collaborative behavior features. If the non-operating hand is identified as being in a prescribed safe posture, no alarm is generated. If the non-operating hand is identified as being in a dangerous area or exhibiting a prohibited posture, a corresponding risk marker, namely the non-operating hand's collaborative behavior risk marker, is generated and incorporated into the dynamic feature sequence as an enhancement feature. The non-operating hand is the hand not currently used to perform the pressure plate operation. A prescribed safe posture refers to a risk-free posture required by the procedure, such as naturally hanging down, behind the back, or placed at the sides of the body. A dangerous area or prohibited posture refers to the non-operating hand being placed in an inappropriate location (e.g., resting on a cabinet frame, near other pressure plates) or performing a dangerous action (e.g., holding unrelated tools). Monitor and prevent additional risks caused by improper behavior of the non-operating hand to achieve overall behavioral safety. For example, in a compliant scenario: the right hand operates the pressure plate, and the left hand hangs naturally. When the left hand is identified as being in a safe posture, no alarm is triggered. In a non-compliant scenario: the right hand operates pressure plate A, and the left hand, due to loss of balance, instinctively rests on the adjacent pressure plate B. When a left hand is detected in a danger zone, a risk marker is immediately generated and incorporated into a dynamic feature sequence, ultimately triggering an alarm. This prevents unintentional accidental contact.

[0040] Step S224: Receive sensor signal from pressure plate status sensor, and perform timing alignment and logic verification between the signal and the estimated pressure plate action time obtained based on image recognition; when the operation time identified by the image highly matches the change time of the sensor signal, determine that the operation feature is valid, and strengthen the confidence of the feature sequence; when there is a significant deviation between the operation time identified by the image and the change time of the sensor signal, determine that there is interference or misjudgment in the acquired video stream, and trigger the data verification mechanism.

[0041] The pressure plate status sensor refers to a physical sensor (such as a microswitch or displacement sensor) installed on the pressure plate, used to directly and accurately detect the engagement or disengagement status of the pressure plate. The estimated pressure plate action time refers to the time of operation inferred based on image recognition (such as hand contact or pressing actions). The data verification mechanism is a processing flow triggered when data is abnormal, such as re-analyzing the video, prompting manual intervention, or marking the data as low confidence. Cross-validation of visual and physical signals improves confidence and identifies fraudulent or malfunctioning operations that are difficult to detect using purely visual methods. For example, high match (confidence enhancement): The image recognizes a hand pressing action, and almost simultaneously, the pressure plate status sensor signal jumps from 0 (engagement) to 1 (disengagement). This indicates that the operation is genuine and effective, increasing confidence in the operation. Significant deviation (triggering verification): The image recognizes a complete pressing action by the hand, but the pressure plate status sensor signal shows no change. This significant deviation may indicate: Incomplete operation: The hand pressed, but the pressure plate was not properly engaged. Visual misjudgment: Light or angle causes a misinterpretation of an operation. At this point, the data verification mechanism is triggered to perform secondary confirmation or issue an alarm, preventing the overlooking of real potential risks.

[0042] Step S3 involves comparing the real-time generated dynamic feature sequence with a pre-stored standard operation sequence template for similarity. Based on the relationship between the comparison result and a preset threshold, the compliance of the current operation is determined by a judgment threshold set by a machine learning model. Specifically, this step compares the real-time collected and constructed dynamic feature sequence with a pre-stored standard sequence template representing compliant operations. The similarity between the two is calculated using a machine learning model, and a compliance or non-compliance judgment is made based on the machine learning judgment threshold. This is the core decision-making process of the invention, replacing traditional manual supervision and achieving automated and intelligent assessment of operational safety.

[0043] Specifically, in this embodiment, step S3 includes: Step S31: The standard operation sequence template library stores multiple standard templates generated based on the height, arm length, and operating habits of different operators. Before comparison, the most matching standard template is dynamically selected from the library as the comparison benchmark based on the operator's body shape identifier or initial posture. Multiple standard templates refer to multiple standard sequences established for the same standardized operation action, based on the height, arm length, and habitual posture of different operators. For example, for the action of placing the pressure plate, standard templates were established for operators with a height of 1.6 meters and 1.8 meters respectively. Body shape identifier or initial posture refers to information identifying the operator, which can be a work badge (identifier) ​​or the body shape and posture characteristics when the operator is standing statically analyzed through video stream.

[0044] Step S32: The dynamic feature sequence is compared with the selected standard template. A layered weighted fusion strategy is used to calculate the final comprehensive similarity score: Spatial trajectory layer: calculates the similarity of hand movement trajectories and assigns basic weights; Motion dynamics layer: calculates the similarity of movement speed and acceleration curves and assigns core weights; Posture sequence layer: calculates the similarity of the hand posture evolution sequence during the operation and assigns key weights; The similarities of the spatial trajectory layer, motion dynamics layer, and posture sequence layer are weighted and fused to obtain the final comprehensive similarity score. The spatial trajectory layer focuses on whether the hand movement path is correct. The motion dynamics layer focuses on whether the speed and acceleration of hand movements are standardized. The posture sequence layer focuses on whether the evolution process of hand posture (e.g., pointing → approaching → pinching → pressing) is correct. The weights are importance coefficients assigned to each layer of similarity.

[0045] Step S33: The preset threshold is not a fixed value, but is dynamically adjusted according to the complexity and risk level of the operation. For simple, independent pressure plates, the preset threshold uses the standard threshold. For pressure plates involving continuous operation of multiple pressure plates or located in high-risk areas, the preset threshold is adaptively increased, and a stricter high-standard threshold is used for compliance judgment. Here, the preset threshold refers to the critical similarity score for judging whether an operation is compliant or non-compliant. Complexity refers to the difficulty of the operation; for example, operating a single pressure plate is simple, while operating multiple pressure plates continuously is complex. Risk level refers to the severity of the consequences if the operation goes wrong. For example, the risk level of operating a trip pressure plate is much higher than that of operating a signal pressure plate. This achieves risk-oriented differentiated supervision and implements stricter safety reviews for key operations. For example, for operating a common signal reset pressure plate, 85 points is used as the standard threshold. A final comprehensive similarity score of 86 points is considered compliant. For operating a critical main transformer protection trip pressure plate, due to its high risk level, the preset threshold is automatically increased to the high-standard threshold of 95 points. Even if the final overall similarity score reaches 94 points, it will still be judged as non-compliant and an alert will be issued because it does not meet the highest safety standard.

[0046] This transforms a simple comparison and judgment process into a sophisticated analysis engine that varies depending on the individual, the dimension, and the risk. This makes the judgment results no longer rigid, but refined, fair, and closely linked to security risks, thereby improving practicality and acceptability while ensuring security.

[0047] Step S34: Before calculating the final comprehensive similarity score using the hierarchical weighted fusion strategy in step S32, the dynamic time warping algorithm is first used to non-linearly align the dynamic feature sequence with the standard template. After alignment, the local subsequence with the highest deviation from the standard template in the dynamic feature sequence is identified and marked as a high-risk segment. The Dynamic Time Warping (DTW) algorithm is an algorithm capable of optimally aligning two time sequences of different lengths and speeds non-linearly, handling problems of varying operation speeds. Local subsequence: refers to the small continuous frame segment in the aligned real-time sequence that has the worst match with the standard template. 1) Solves the comparison problem of inconsistent operation speeds; 2) Accurately locates the specific time period when violations occur.

[0048] Step S35: Based on the hierarchical weighted fusion strategy in step S32, calculate the intermediate result of the final comprehensive similarity score, and calculate the contribution of the spatial trajectory layer, motion dynamics layer, and posture sequence layer to the final comprehensive similarity score. When an operation is deemed non-compliant, an interpretability report is automatically generated. This report should at least clearly indicate: the main violation type that led to the decrease in the final comprehensive similarity score (such as trajectory deviation, excessive speed, or incorrect posture); the specific time point and duration of the high-risk segment during the operation; and the contribution analysis of each feature layer to indicate the most significant violation factor. The contribution refers to the degree of responsibility of the spatial trajectory, motion dynamics, and posture sequence layers for the decrease in the final comprehensive similarity score in step S32. This transforms the black-box decision-making of the machine learning model into a clear and understandable diagnostic report, increasing trust and guiding rectification.

[0049] Step S36: Maintain a rating file for each operator; for operators with excellent rating files, a positive tolerance coefficient is granted based on the preset threshold mentioned in step S33, appropriately relaxing their judgment threshold; for operators with poor rating files or newly hired operators, a negative strict coefficient is activated to further tighten their judgment threshold.

[0050] Step S4: Based on the compliance assessment results, trigger risk warnings or intervention measures of different levels. Specifically, this step executes preset response strategies based on the compliance assessment results of step S3. These strategies are tiered, ranging from prior visual / audio warnings to mandatory interruption operations during the incident (such as interlocking devices), thereby shifting from post-incident tracing to in-incident intervention and proactively preventing the occurrence of safety incidents.

[0051] Specifically, in this embodiment, step S4, based on the compliance determination result, triggers different levels of risk warnings or intervention measures, including: Step S41, Multi-dimensional Risk Level Assessment. After receiving the compliance determination result from Step S3, the following information is further integrated to conduct a refined risk level assessment to obtain the final risk level: Severity of Operational Violation: The degree of deviation between the final comprehensive similarity score output from Step S3 and the preset threshold. Operational Context Risk: Based on Step S33, identify the type of control panel for the current operation (e.g., trip control panel and outlet control panel are high-risk, signal control panel is medium-risk). Real-time Behavioral Risk Signals: Check for the existence of high-risk segments marked in Step S34, the main violation types analyzed in Step S35, and the risk markers for non-operator collaborative behaviors in Step S223.

[0052] Step S42, Execution of Tiered Response Strategy. Based on the final risk level in Step S41, execute corresponding measures from the predefined response strategy library: Level 1 Warning (Low Risk / Alert): Triggering Conditions: The final comprehensive similarity score is slightly lower than the preset threshold, or there are minor irregularities (such as slight path deviation or slightly faster speed), but the pressure plate type is low-risk and there are no other risk signals. Intervention Measures: On-site Visual Alert: A silent reminder is given on the display terminal next to the operation cabinet with a flashing green border or text prompt (such as "Operation is slightly faster, please maintain a constant speed"). Recording and Marking: Record this operation as needing improvement and include it in the operator's rating file for trend analysis. Level 2 Warning (Medium Risk / Alarm): Triggering Conditions: The final comprehensive similarity score is significantly lower than the preset threshold, or a clear irregular operation is detected (such as incorrect hand posture, non-operating hand entering the danger zone), or the operation object is a medium-risk pressure plate. Intervention Measures: Audible and Visual Alarm: A yellow warning light and a soft but continuous buzzing sound are triggered on-site to attract the operator's attention. Real-time feedback: Clearly displays the type of violation on the display terminal (e.g., error: fingertip not used for pressing). Suspend subsequent operations: Temporarily locks the control panel until the operator manually confirms (e.g., presses the "understood" button). Level 3 warning (high risk / blockage): Triggering conditions: The final comprehensive similarity score is far below the preset threshold, a serious violation is detected (e.g., incorrect operation sequence, serious error in handling high-risk pressure plates), or there is a risk event of misoperation as defined in step S132. Intervention measures: Emergency audible and visual alarm: Triggers a red rotating light and a rapid, high-volume alarm sound. Forced physical lock: Sends a command to the intelligent control panel control system to immediately lock the current operating area or related pressure plates, preventing the execution or continuation of the operation. Remote synchronous alarm: Automatically sends an emergency alarm message to the mobile terminal of the shift leader, safety supervisor, or remote monitoring center, including the operator's ID, screenshot / operation event fragment of the violation, and the interpretability report generated in step S35, requesting immediate intervention.

[0053] Step S43, Intervention Log and Closed-Loop Management. Record complete data for each early warning intervention, including but not limited to: trigger time, operator, target pressure plate, final risk level, specific intervention measures triggered, associated high-risk segments, interpretability reports, and subsequent actions taken by the operator (e.g., confirmation, rectification status). This intervention log is used for post-event auditing, incident tracing, and driving adaptive optimization in step S5.

[0054] Step S5 employs an incremental learning mechanism to adaptively optimize the judgment threshold of the standard operation sequence template or the machine learning model using accumulated historical operation data. Specifically, this step automatically and iteratively optimizes the judgment threshold of the standard operation sequence template or the machine learning model using continuously accumulated historical operation data (including positive and negative examples). This adapts to new operating habits, optimizes judgment criteria, and becomes increasingly intelligent with use, solving the problems of traditional rigidity and difficulty in evolution.

[0055] Specifically, in this embodiment, step S5 employs an incremental learning mechanism, utilizing accumulated historical operational data to continuously evolve learning capabilities. The core is to leverage constantly generated new compliance and violation data to automatically adjust the model and parameters through algorithms, thereby becoming increasingly intelligent with use. This includes: Step S51: Optimize the construction and quality management of the data pool. Automatically collect and filter data for optimization, forming two core datasets: a high-confidence compliance dataset: This dataset includes all operation sequences judged to be compliant, with a confidence level (derived from the sensor signals received from the pressure plate status sensor in step S224, and the timing alignment and logical verification of these signals with the estimated pressure plate action time based on image recognition; and from the responsibility of the three levels of spatial trajectory, motion dynamics, and posture sequence for the reduction in the final comprehensive similarity score in step S32) exceeding a preset threshold (e.g., 98%). The confirmed violation dataset includes all operation sequences that trigger warnings (especially level two and three warnings) and are confirmed as genuine violations in the intervention log of step S43, along with their analysis reports.

[0056] Step S52, Incremental optimization of the standard operation sequence template. Template generalization update: Periodically (e.g., weekly), the high-confidence compliance dataset from step S51 is used as new training data and input into the template generation algorithm. The algorithm uses an incremental clustering method to automatically discover and integrate new, repeatedly validated, and effective operation patterns, thereby enabling the standard template to cover more diverse compliance operation habits, making it more representative and generalizable. Personalized template reinforcement: For specific operators, when their personal high-confidence compliance dataset from step S51 accumulates to a certain scale, a personalized standard template can be automatically generated or fine-tuned for them, allowing the safety and efficiency characteristics in their personal habits to be learned and recognized.

[0057] Step S53: Dynamic adjustment of the judgment threshold of the machine learning model. The judgment threshold refers to the preset threshold used by the model to judge compliance. Global threshold adaptation: Monitor global judgment statistics. When it is found that the overall compliance rate of a certain type of operation (such as placing a pressure plate) is consistently abnormally high (possibly the threshold is too lenient) or low (possibly the threshold is too strict), and a systematic bias occurs in the judgment, the global judgment threshold for this type of operation will be automatically tightened or loosened in small steps (such as 0.5%) to bring it back to a reasonable level that can effectively identify risks. Personalized threshold adjustment (linked with step S36): Quantify the scoring profile in step S36. For operators with excellent scoring profiles, their personal tolerance coefficient will be automatically slightly increased, making their personal threshold more lenient; for those with poor scoring profiles, their personal strictness coefficient will be automatically increased, implementing stricter supervision.

[0058] Step S54 involves feature weight optimization based on adversarial examples. Using the confirmed violation dataset from step S51, the main violation types are analyzed. For example, if a large number of violations are concentrated in the motion dynamics layer (such as excessively fast operations), the weight of the motion dynamics layer (W2) will be appropriately increased in the fusion calculation of step S32, making the model more sensitive to such violations in the future. Learning from discovered vulnerabilities, the attention allocation is dynamically adjusted to achieve targeted reinforcement of emerging or high-risk patterns.

[0059] Step S55: Verification and Rolling Updates of Optimization Effects. Any optimization scheme (new template, new threshold, new weight) generated by steps S52, S53, and S54 must be verified on an independent historical test dataset before formal deployment. Only if the scheme proves to improve the capture rate of historical violations without significantly reducing the pass rate of compliant operations will it be approved and implemented, replacing the old version through rolling updates to ensure that each evolution is robust and positive.

[0060] The overall solution of this invention, through a closed-loop technology system encompassing perception, digitization, decision-making, intervention, and evolution, achieves intelligent management of the entire chain of pressure plate operation behavior, from identification, analysis, and judgment to intervention and self-optimization.

[0061] Example 2 like Figure 2 As shown, this application provides a system architecture diagram for platen operation behavior analysis and error prevention based on image recognition and machine learning, which is applied to the platen operation behavior analysis and error prevention system based on image recognition and machine learning as described in Embodiment 1. It includes: a video event segmentation module 210, a dynamic feature sequence construction module 220, an operation compliance determination module 230, a risk warning and intervention module 240, and an adaptive optimization module 250.

[0062] The video event segmentation module 210 is used to acquire the video stream of the pressure plate operation area through the image acquisition device, analyze the video stream using a motion detection algorithm, and segment the video stream into multiple independent operation event segments based on the start and end of the motion state.

[0063] The dynamic feature sequence construction module 220 is used to extract key point time series data of the operator's hand from the operation event segment using a human posture estimation model; and to construct a dynamic feature sequence to describe a single pressure plate operation process based on the key point time series data.

[0064] The operation compliance determination module 230 is used to compare the real-time generated dynamic feature sequence with the pre-stored standard operation sequence template for similarity; based on the relationship between the comparison result and the preset threshold, the machine learning model determines the compliance of the current operation by the judgment threshold.

[0065] The risk warning and intervention module 240 is used to trigger different levels of risk warnings or intervention measures based on the compliance assessment results.

[0066] The adaptive optimization module 250 is used to adaptively optimize the judgment threshold of the standard operation sequence template or the machine learning model by using an incremental learning mechanism and accumulating historical operation data.

[0067] Figure 3 This is an electronic device provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device includes at least the following components: processor 301 and memory 300, communication interface 303, and bus 302.

[0068] In this embodiment of the application, memory 300 is used to store executable instructions of processor 301, which, when configured to execute instructions, implements the method as described in the first aspect.

[0069] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0070] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0071] It should be noted that a portion of the electronic device described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0072] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.

[0073] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.

[0074] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0075] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for analyzing and preventing errors in pressure plate operation based on image recognition and machine learning, characterized in that, Includes the following steps: Step S1: Acquire the video stream of the pressure plate operation area monitoring through the image acquisition device, analyze the video stream using a motion detection algorithm, and divide the video stream into multiple independent operation event segments based on the start and end of the motion state. And pre-store standard operation sequence templates; Step S2: Using a human posture estimation model, extract key point time-series data of the operator's hand from the operation event segment; based on the key point time-series data, construct a dynamic feature sequence to describe a single pressure plate operation process; Step S3: Compare the real-time generated dynamic feature sequence with the pre-stored standard operation sequence template for similarity; based on the relationship between the comparison result and the preset threshold, the compliance of the current operation is determined by the judgment threshold of the machine learning model. Step S4: Based on the compliance assessment results, trigger different levels of risk warnings or intervention measures; Step S5: Using an incremental learning mechanism, the judgment threshold of the standard operation sequence template or the machine learning model is adaptively optimized by utilizing accumulated historical operation data.

2. The method according to claim 1, characterized in that, Step S1 specifically includes: A Gaussian mixture model is used for background modeling, and combined with the inter-frame difference method, the foreground motion region consisting of the operator's hands and torso is accurately extracted from the video stream. Kalman filtering is applied to the foreground motion region to generate a motion trajectory; when the motion trajectory first enters the preset pressure plate operation interest region and the motion speed is lower than the set dwell speed threshold, it is determined as the starting point of an operation event; When the motion trajectory completely leaves the area of ​​interest of the pressure plate operation and continues for more than a preset number of still frames, it is determined to be the end point of the operation event; based on the start point and the end point, the corresponding operation event segment is extracted from the continuous video stream.

3. The method according to claim 2, characterized in that, The determination of the starting point of an operation event specifically includes: Before the motion trajectory enters the preset pressure plate operation interest area, a consistency analysis is performed on its movement direction within N consecutive frames; when the angle between the movement direction and the theoretical vector direction from the operator's standing area to the target pressure plate is less than the set angle tolerance, it is determined to be a valid approach behavior with operational intent. At the moment the starting point is determined, hand posture recognition is performed on the foreground motion area; if the recognized hand posture is a predefined operation preparation posture, the starting point is finally confirmed to be valid; otherwise, it is regarded as an invalid crossing or accidental touch, and the event is ignored.

4. The method according to claim 2, characterized in that, The determination that the operation event has ended specifically includes: After the motion trajectory leaves the area of ​​interest of the pressure plate operation, the state of the pressure plate itself in that area is continuously monitored; if the state of the pressure plate changes within a period of time after the motion trajectory leaves, the completion time of the state change is corrected as the end point of this operation event. If, within the same operation event segment, the motion trajectory continuously passes through or briefly pauses in the interest area of ​​multiple adjacent pressure plates, then the error prevention logic is activated; all pressure plate sets involved in the continuous operation are identified; the pressure plate sets are compared with the operation ticket logic in real time; only when the pressure plate set and operation sequence comply with the operation ticket requirements are the entire continuous motion determined as a complete and compliant operation event; otherwise, it is determined as a risk event containing erroneous operation, and a risk warning is immediately triggered in step S4.

5. The method according to claim 1, characterized in that, Step S2 specifically includes: A human pose estimation model based on deep convolutional neural networks is used to locate key points of the operator's wrist, palm base, and fingertips in each frame of image; and optical flow information is used to track the key points across frames to form key point temporal data. Based on the key point time series data, the following three types of features are calculated and fused to construct the dynamic feature sequence: Spatial relationship characteristics: Calculate the two-dimensional pixel distance between the fingertip key point and the target pressure plate throwing area, as well as the estimated angle between the hand plane and the cabinet plane; Kinematic characteristics: Calculate the instantaneous velocity, acceleration, and smoothness of motion direction at key hand points; Posture semantic features: Based on the relative positional relationship between different key points of the hand, hand postures are identified and encoded in real time, including pointing, pinching, pressing and rotating.

6. The method according to claim 5, characterized in that, Constructing the dynamic feature sequence specifically includes: The two-dimensional pixel distance in the spatial relationship features is converted into an absolute physical distance corresponding to the real world scale based on the camera calibration parameters and the known size of the reference object, thereby eliminating feature deviations caused by the distance of the operator's position or different camera angles. When constructing the dynamic feature sequence, different weights are dynamically assigned to the three types of features according to different stages of the operation process: During the approach phase, the weight of the spatial relationship features is increased to focus on evaluating the correctness of the operation path; During the execution phase, the weight of motion direction smoothness in the posture semantic features and kinematic features is increased to focus on evaluating the standardization and stability of the operation posture. During the departure phase, the weight of instantaneous velocity in the kinematic features is increased to assess the decisiveness of the operator's hand removal after the operation is completed.

7. The method according to claim 6, characterized in that, When constructing dynamic feature sequences, context-aware enhanced features are further introduced, specifically including: The posture and position of the non-operating hand are extracted and monitored simultaneously to generate collaborative behavior features. If the non-operating hand is identified as being in a specified safe posture, no alarm is generated. If the non-operating hand is identified as being in a dangerous area or in a prohibited posture, a corresponding risk marker is generated and incorporated into the dynamic feature sequence as an enhancement feature. Receive sensor signals from the pressure plate status sensor and perform timing alignment and logic verification with the estimated pressure plate action time obtained based on image recognition; When the operation time detected by the image closely matches the change time of the sensor signal, the operation feature is deemed valid, and the confidence level of the dynamic feature sequence is enhanced. When there is a significant deviation between the operation time detected by the image and the change time of the sensor signal, it is determined that there is interference or misjudgment in the acquired video stream, and the data verification mechanism is triggered.

8. The method according to claim 1, characterized in that, Step S3 specifically includes: The standard operation sequence template library stores a variety of standard templates generated based on the height, arm length and operating habits of different operators. Before comparison, the most matching standard template is dynamically selected from the standard operation sequence template library as the comparison benchmark based on the current operator's body shape or initial posture. The dynamic feature sequence is compared with the selected standard template, and a hierarchical weighted fusion strategy is used to calculate the final comprehensive similarity score, which includes: Spatial trajectory layer: Calculates the similarity of hand movement trajectories and assigns basic weights; Motion Dynamics Layer: Calculates the similarity of motion velocity and acceleration curves and assigns core weights; Posture sequence layer: Calculates the similarity of hand posture evolution sequences during operation and assigns key weights; The similarity scores of the spatial trajectory layer, motion dynamics layer, and attitude sequence layer are weighted and fused to obtain the final comprehensive similarity score. The preset threshold is not a fixed value, but is dynamically adjusted according to the complexity and risk level of the operation, specifically including: For simple, independent pressure plates, the preset threshold uses the standard threshold; for pressure plates involving continuous operation of multiple pressure plates or located in high-risk areas, the preset threshold is adaptively increased, and the preset threshold uses a more stringent high standard threshold.

9. The method according to claim 8, characterized in that, Also includes: Before calculating the final comprehensive similarity score using the hierarchical weighted fusion strategy, a dynamic time warping algorithm is first used to non-linearly align the dynamic feature sequence with the standard template. After alignment, the local subsequence with the highest deviation from the standard template in the dynamic feature sequence is identified and marked as a high-risk segment; The intermediate result of calculating the final comprehensive similarity score is based on the hierarchical weighted fusion strategy, and the contribution of the spatial trajectory layer, motion dynamics layer and posture sequence layer to the final comprehensive similarity score is calculated. When an operation is deemed non-compliant, an explanatory report is automatically generated. A rating file is maintained for each operator. For operators with excellent rating files, a positive tolerance coefficient is granted based on the preset threshold to appropriately relax their judgment threshold. For operators with poor rating files or newly hired operators, a negative strict coefficient is used to further tighten their judgment threshold.

10. A platen operation behavior analysis and error prevention system based on image recognition and machine learning, applied to the method described in any one of claims 1 to 9, characterized in that, The system includes: The video event segmentation module is used to acquire video streams of the pressure plate operation area through image acquisition devices, analyze the video streams using motion detection algorithms, and segment the video streams into multiple independent operation event segments based on the start and end of the motion state. The dynamic feature sequence construction module is used to extract key point temporal data of the operator's hand from the operation event segment using a human posture estimation model; and to construct a dynamic feature sequence describing a single pressure plate operation based on the key point temporal data. The operation compliance determination module is used to compare the real-time generated dynamic feature sequence with the pre-stored standard operation sequence template; based on the relationship between the comparison result and the preset threshold, the machine learning model determines the compliance of the current operation. The risk warning and intervention module is used to trigger different levels of risk warnings or intervention measures based on the compliance assessment results. An adaptive optimization module is used to adaptively optimize the judgment threshold of the standard operation sequence template or the machine learning model by using an incremental learning mechanism and accumulating historical operation data.

Images