Substation key supervision system based on artificial intelligence

Abstract

The invention discloses a transformer substation key supervision system based on artificial intelligence, and relates to the field of transformer substation key supervision, the transformer substation key supervision system comprises a visual module, an analysis module, a display module, a register and a key management module, the key management module performs communication interaction with an RFID tracking tag on a key to obtain worker trajectory information and transmit the worker trajectory information to the analysis module; the visual module comprises two groups of image sensors and transmits image information of facial and eye details and behavior image information to the analysis module, the analysis module analyzes a facial state and a behavior state according to the facial and eye details and the behavior image information, and the analysis module transmits a generated color display signal to the display module; the display module is arranged in a central control room of the transformer substation and is connected with a management personnel system through an internal network, and the display module visually presents the comprehensive state of a worker in a visual color form, so that normal operation of equipment and personnel of the transformer substation can be ensured.

Description

Technical Field

[0001] The present invention relates to the field of substation key supervision, and in particular to a substation key supervision system based on artificial intelligence. Background Art

[0002] With the development of society, the demand for energy is increasing, and electricity supply is an important energy supply method. In the power supply, the safe and stable operation of equipment in substations plays a decisive role in ensuring the normal transmission of electricity. At the same time, with the increasing scale and number of substations, the management of substations faces greater challenges. Therefore, the substation key supervision system based on artificial intelligence has come into being.

[0003] A search of a Chinese patent with publication number CN117576807A discloses a substation smart lock management system, which includes a management workstation for background management; a cloud server for storing information; a communication host for communicating between the management workstation and a computer key; a computer key for reading the smart lock's identity identification code, controlling the opening of the authorized smart lock and automatically recording the unlocking information; a facial recognition device for automatically identifying the identity of on-site personnel and controlling the unlocking of the smart lock; a wireless data terminal for collecting door magnetic signals and transmitting them back to the cloud server, for receiving unlocking instructions from the cloud server and controlling the unlocking of the smart lock; a door magnetic for real-time detection of the door's open or closed status; and a smart lock for unlocking after receiving unlocking commands from the computer key, the facial recognition device, and the wireless data terminal. Although facial recognition can be performed on staff members, comprehensive monitoring and analysis of their facial status and movement trajectory is not possible. The status monitoring dimension is too single, and it is impossible to determine the comprehensive status of the staff members and display and issue corresponding alarms, resulting in frequent safety accidents of substation equipment and personnel.

[0004] In order to solve the above-mentioned defects, a technical solution is now provided. Summary of the Invention

[0005] In order to solve the technical problems raised by the above background technology, the present invention is proposed. The embodiments of the present invention provide a substation key supervision system based on artificial intelligence.

[0006] The objectives of the present invention can be achieved through the following technical solutions: an artificial intelligence-based substation key monitoring system, comprising a visual module, an analysis module, a display module, a register, and a key management module; the key management module communicates and interacts with the RFID tracking tag on the key to obtain the worker's trajectory information and transmit it to the analysis module;

[0007] The vision module includes two sets of image sensors. One set of image sensors obtains image information of facial and eye details of the staff when entering the area in front of the substation key cabinet. The other set of image sensors obtains behavioral image information of the staff entering the area in front of the substation key cabinet and transmits it to the analysis module. The analysis module analyzes the facial state and behavioral state of the staff entering the area in front of the substation key cabinet based on the facial and eye details and behavioral image information.

[0008] The register is used to store all temporary data generated during the execution of the program by the analysis module. The analysis module transmits the generated color display signal to the display module. The display module is set in the substation control room and is connected to the management system through the internal network. The display module visualizes the comprehensive status of the staff in an intuitive color form and issues an alarm to the staff whose display is red, restricting their access rights.

[0009] The temporary data includes original data and analysis data. The original data includes image information, work position information, and substation area information. The analysis data includes monitoring of facial and eye status, monitoring of behavioral status, analysis of trajectory abnormality, and color display signals.

[0010] Furthermore, the method further comprises the following steps:

[0011] Step 1: Facial and eye status monitoring. When a worker enters the area in front of the substation key cabinet, the vision module transmits image information of facial eye details to the analysis module. The analysis module analyzes eye movements and features of the whites of the eyes to obtain the worker's eye movement shape heterogeneity integration value.

[0012] Step 2: Behavior status monitoring. When a worker enters the area in front of the substation key cabinet, the vision module transmits behavioral image information to the analysis module. The analysis module analyzes the hand joint angle, opening degree, and motion trajectory characteristics to obtain the worker's hand fist distortion value.

[0013] Step 3: Trajectory abnormality analysis: The analysis module obtains job position information and substation area information from the register. The analysis module divides the job position and area nodes, matches the business and information interaction relationships, calculates the information linkage value sorting and sets the blue line width. The analysis module obtains the staff trajectory information from the key management module and analyzes it to obtain the total value of the staff trajectory deviation.

[0014] Step 4: Color display setting: The analysis module analyzes the calculated eye movement heterogeneity integration value, hand fist distortion value, and trajectory turbulence standard total value, compares them with the set threshold value, and issues a color display signal;

[0015] Step 5: Color display: the display module receives the color display signal from the analysis module, and visualizes the comprehensive status of the staff in an intuitive color form, and issues an alarm to the staff displayed in red, restricting their access rights.

[0016] Furthermore, the steps for analyzing the heterogeneous integrated value of the staff member's eye movement shape are as follows:

[0017] Step 11: The analysis module calculates the circumscribed polygon of the eye area segmented by the staff, obtains the vertex coordinates of the circumscribed polygon, and calculates the vertex coordinates of the polygon. i , the adjacent vertices are G i+1 and G i-1 , i is the vertex number of the circumscribed polygon, construct the vector and in like Then the vertex is a convex vertex if Then the vertex is a concave vertex, and the convexity and concavity distribution of the vertices of the circumscribed polygon is counted. If all vertices are convex vertices, the polygon is determined to be a convex polygon. If there is a concave vertex, the polygon is determined to be a concave polygon. The number of convexity and concavity changes g of the circumscribed polygon of the staff member's eye white within time a is obtained, and the frequency rate ε of the convexity and concavity changes of the staff member's eye white is calculated to obtain the contour extension rate, airspace ratio rate, and center of mass drift rate of the circumscribed polygon of the eye white area are obtained to calculate the specific value η2 of the staff member's eye white shape movement;

[0018] Step 12: The analysis module records the staff member's eye movement direction change rate η1, eye sclera convexity and concavity abnormality frequency ε, and eye sclera shape abnormality value η2 within a certain period of time as the staff member's eye movement shape abnormality parameter X mn , X mn is the nth index value of the mth data, m is a positive integer, the maximum value is M, n=1,2,3, corresponding to η1,ε,η2, through the formula Get the standard value of the staff's eye movement parameter X* mn , min is the minimum value, max is the maximum value, according to the formula The information entropy en of each eye movement parameter of the staff is calculated according to the formula Get the weights qn of each staff member's eye movement shape parameter, and the standardized eye movement shape parameter standard value X* n ,The data are clustered into three categories using the K-means clustering algorithm. Each category corresponds to a preset level, corresponding to low heterogeneity, medium heterogeneity, and high heterogeneity of eye movement anomalies. The analysis module defines the Gaussian membership function. c1, c2, c3 are the center values ​​of each level, σ1, σ2, σ3 are the standard deviations of the data of each level, for each indicator X* n For all data points m, calculate the mean of their membership to each level and get the matrix R. r ns It represents the membership of the factor eye movement heterogeneity parameter n to the preset level. For example, r12 represents the membership of the eye movement direction change rate to medium heterogeneity. Through matrix fuzzy multiplication B=qn*R, the comprehensive evaluation result vector B is obtained, which reflects the comprehensive membership of the eye movement heterogeneity parameters as a whole to the low heterogeneity, medium heterogeneity and high heterogeneity evaluation levels. The corresponding values ​​of low heterogeneity, medium heterogeneity and high heterogeneity eye movement heterogeneity levels are 1, 2 and 3. Then the comprehensive membership vector B is multiplied and summed with the corresponding values ​​of low, medium and high heterogeneity to obtain the integrated value of the staff's eye movement heterogeneity.

[0019] Furthermore, the step of analyzing the staff member's eye movement direction change rate includes:

[0020] Obtaining raw data, the analysis module obtains image information of the current staff entering the area in front of the substation key cabinet from the visual module;

[0021] The analysis module converts the acquired color image of the eyeball into a grayscale image, uses a threshold segmentation method to separate the white area of ​​the eye from the background and other eye tissues, and fits the white area of ​​the eyeball using an ellipse fitting algorithm to obtain the major and minor axis information of the ellipse. If the major axis deviates from the horizontal direction counterclockwise by more than 5° and the ratio of the major axis to the minor axis is within the range of 1.6-2.5, it is determined that the eyeball is moving to the left. If the major axis deviates from the horizontal direction clockwise by more than 5° and the ratio of the major axis to the minor axis is within the range of 1. If the long axis deviates from the vertical direction by more than 5° counterclockwise and the ratio of the long axis to the short axis is within the range of 1.3-1.7, the eye is judged to be moving upward. If the long axis deviates from the vertical direction by more than 5° clockwise and the ratio of the long axis to the short axis is within the range of 1.3-1.7, the eye is judged to be moving downward. The number of times the staff member's eye movement direction changes c within the time a is obtained, and the eye movement direction change rate η1 of the staff member is calculated.

[0022] Furthermore, the step of analyzing the abnormal value of the hand bending deformity of the staff member includes:

[0023] The analysis module normalizes the staff's hand bending posture deviation value, the staff's suprathreshold fist clenching anomaly value, and the staff's handprint sudden coagulation anomaly value, and analyzes them to obtain the staff's hand fist bending anomaly value.

[0024] Furthermore, the steps of analyzing the deviation value of the staff's hand bending posture, the staff's suprathreshold fist clenching abnormal coagulation value, and the staff's handwriting sudden coagulation abnormality value include:

[0025] The analysis module uses time as the horizontal axis and the hand bending posture value of the staff as the vertical axis to establish a hand bending posture value change curve diagram when the staff enters the substation key cabinet area to take the key, and obtains the hand bending posture value change standard curve. The standard curve is located above, marked as the hand bending posture upper mark line, and the standard curve is located below, marked as the hand bending posture lower mark line. The area above the hand bending posture upper mark line and the area below the hand bending posture lower mark line of the hand bending posture value change curve diagram are obtained, and the sum is taken to obtain the hand bending posture deviation value of the staff. If each hand If the average value of the reduction in the finger joint angle within 1 second exceeds the threshold β1, and the contact area between the fingers is greater than the threshold β2, then the period is determined to be a supra-threshold tight fist clenching period. The total duration of the supra-threshold tight fist clenching period when the staff enters the substation key cabinet area to retrieve the key is counted and multiplied by the correction coefficient to obtain the staff's supra-threshold fist clenching anomaly value. The motion trajectory of the staff's hand in the three-dimensional space when entering the substation key cabinet area to retrieve the key is obtained. The rapid change value, stagnation state value, and speed distortion value in the motion trajectory are obtained, and the normalized processing and summation are performed to obtain the staff's handprint sudden distortion value.

[0026] Furthermore, the step of analyzing the hand bending posture value of the staff member includes:

[0027] The analysis module obtains behavioral image information of the current staff member entering the area in front of the substation key cabinet from the visual module, uses hand key point tracking technology to obtain the joint coordinates between the proximal phalanx, middle phalanx and distal phalanx of the fingers, calculates the sum of the vector angles between adjacent joint points and performs inverse processing to obtain the staff member's joint bending characteristic value, calculates the average distance between the fingertip and the center of the palm, marks it as the finger opening degree, calculates the difference between the joint angle between the middle phalanx and the distal phalanx and the joint angle between the proximal phalanx and the middle phalanx, performs inverse processing, and marks it as the finger near-far angle difference value, performs weighted calculation on the finger opening degree and the finger near-far angle difference value, and multiplies them by the corresponding influencing factor coefficient to obtain the staff member's finger whole extension posture value, divides the staff member's joint bending characteristic value by the staff member's finger whole extension posture value and multiplies it by the correction factor coefficient to obtain the staff member's hand bending characteristic posture value.

[0028] Furthermore, the step of analyzing the total value of the track deviation of the staff member includes:

[0029] The analysis module obtains the communication divergence dimension and average processing time span of each post node and each information interaction connection area node, and normalizes them with the information interaction connection value W. It constructs a regular octahedron as the basic framework with the average processing time span as the side length, generates a rotated bicone with the communication divergence dimension as a parameter, and uses a constraint ball with the information interaction connection value as the radius to clip the intersection of the bicone and the regular octahedron. By calculating the percentage of the difference between the intersection volume and the constraint ball volume to the regular octahedron volume, the information linkage contract value of each post node and each information interaction connection area node is obtained; the analysis module arranges the calculated information linkage contract value in descending order, and matches the blue line connection width of each post node and each information interaction connection area node. The trajectory information of the staff entering each area of ​​the substation is obtained through the RFID tracking tag of the key, and the trajectory turbulence node value of the staff in each area is obtained. The information linkage contract value is when the red line and blue line of the network structure diagram are connected. If the post node and the area node are connected as red connections, and the trajectory turbulence node value is greater than the set threshold When XC1 is reached, a second-level trajectory deviation signal is issued. If the post node and the regional node are connected in red and the trajectory turbulence score is less than or equal to the set threshold XC1, a third-level trajectory deviation signal is issued. If the post node and the regional node are connected in blue, the trajectory turbulence score is greater than or equal to the set threshold XC1 and the information linkage contract value is greater than the set threshold XC2, a first-level trajectory deviation signal is issued. If the post node and the regional node are connected in blue, the trajectory turbulence score is less than or equal to the set threshold XC1 and the information linkage contract value is less than the set threshold XC2, a third-level trajectory deviation signal is issued. In other cases where the post node and the regional node are connected in blue, a second-level trajectory deviation signal is issued. The first-level trajectory deviation signal, the second-level trajectory deviation signal, and the third-level trajectory deviation signal are assigned values ​​g1, g2, and g3, which are 10, 5, and 3 respectively, and are marked as the trajectory deviation standard values ​​of the staff in each area. The trajectory deviation standard values ​​of the staff in each area are added together to obtain the total trajectory deviation standard value of the staff.

[0030] Furthermore, the information interaction connection value analysis step includes:

[0031] The analysis module obtains job information and substation area information from the register. The analysis module divides the staff position nodes into technical support personnel, engineering construction personnel, operation duty personnel, maintenance personnel, safety management personnel, and other personnel, and uses them as a type of node in the network structure diagram, represented by a circle. The substation area nodes are divided into construction area, auxiliary equipment area, main control room, primary equipment area, secondary equipment area, and other areas as nodes in the network structure diagram, and the business process relationship between the position nodes and the area nodes is matched respectively. The technical support personnel are matched with the primary equipment area and the secondary equipment area, and the engineering construction personnel are matched with the construction area. The operation duty personnel are matched to the main control room, the maintenance personnel are matched to the primary equipment area and the secondary equipment area, the safety management personnel are matched to each area, and there is no corresponding matching relationship for other personnel. The business process relationships are connected with red lines, and the connection between the job nodes and other nodes except the matched business process relationships is defined as the information interaction connection, which is connected with blue lines. The information interaction methods correspond to face-to-face communication, video conferencing, voice calls, telephone conferences, emails, written reports and bulletin boards, and the corresponding values ​​are w1, w2, w3, w4, w5, w6 and w7, which are marked as the information interaction connection value W between the job node and the regional node.

[0032] Furthermore, the color display setting analysis steps:

[0033] The analysis module performs signal analysis on the staff's eye movement heterogeneity integration value, the staff's hand fist bending distortion value, and the staff's trajectory turbulence standard total value respectively to obtain the color display signal of the corresponding situation.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The present invention transmits image information of facial and eye details to the analysis module through the visual module when the staff enters the area in front of the substation key cabinet. The analysis module analyzes the eye movement and the white eye area characteristics to obtain the eye movement shape heterogeneous integration value of the staff. When the staff enters the area in front of the substation key cabinet, the visual module transmits behavioral image information to the analysis module. The analysis module analyzes the hand joint angle, opening degree, and movement trajectory characteristics to obtain the hand fist distortion value of the staff. The analysis module obtains job position information and substation area information from the register, divides the job and area nodes, matches the business and information interaction relationship, calculates the information linkage contract value sorting and sets the blue line width, and obtains the staff trajectory information from the key management module. The analysis module analyzes to obtain the total value of the staff trajectory turbulence standard. The facial state and action trajectory of the staff can be comprehensively monitored and analyzed. The state monitoring dimension is more comprehensive and the state of the staff can be analyzed and determined more accurately.

[0036] The present invention uses an analysis module to analyze the calculated eye movement heterogeneity integration value, hand fist distortion value, and trajectory turbulence standard total value of the staff, and compares them with the set threshold value to send a color display signal. The display module receives the color display signal of the analysis module, visualizes the comprehensive status of the staff in an intuitive color form, and issues an alarm reminder to the staff with a red display, restricts access rights, and can judge the comprehensive status of the staff and display and issue corresponding alarms, thereby ensuring the normal operation of substation equipment and personnel, preventing staff from accidentally or intentionally entering unauthorized areas, and also preventing staff with abnormal status from causing damage to substation equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. The following drawings are not intentionally scaled to the actual size, and the focus is on illustrating the main purpose of the present invention.

[0038] Figure 1 is a system block diagram of the present invention;

[0039] Figure 2 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts also fall within the scope of protection of the present invention.

[0041] See also Figure 1-Figure 2 , the present invention provides a technical solution: a substation key supervision system based on artificial intelligence, such as Figure 1 As shown, it includes a visual module, an analysis module, a display module, a register, and a key management module. The key management module communicates with the RFID tracking tag on the key to obtain the worker's trajectory information and transmit it to the analysis module;

[0042] The vision module includes two sets of image sensors. One set of image sensors captures image information of the staff's facial and eye details when entering the area in front of the substation key cabinet. The other set of image sensors captures behavioral image information of the staff entering the area in front of the substation key cabinet and transmits it to the analysis module. The analysis module analyzes the facial and behavioral status of the staff entering the area in front of the substation key cabinet based on the facial and eye details and behavioral image information.

[0043] The register is used to store all temporary data generated during the execution of the program by the analysis module. The analysis module transmits the generated color display signal to the display module. The display module is set in the substation control room and is connected to the management system through the internal network. The display module visualizes the comprehensive status of the staff in an intuitive color form and issues an alarm to the staff whose display is red, restricting their access rights.

[0044] The temporary data includes original data and analysis data. The original data includes image information, work position information, and substation area information. The analysis data includes facial and eye status monitoring, behavioral status monitoring, trajectory abnormality analysis, and color display signals.

[0045] like Figure 2 As shown, the following steps are included:

[0046] Step 1: Facial and eye status monitoring. When a worker enters the area in front of the substation key cabinet, the vision module transmits image information of facial eye details to the analysis module. The analysis module analyzes eye movements and features of the whites of the eyes to obtain the worker's eye movement shape heterogeneity integration value.

[0047] Step 2: Behavior status monitoring. When a worker enters the area in front of the substation key cabinet, the vision module transmits behavioral image information to the analysis module. The analysis module analyzes the hand joint angle, opening degree, and motion trajectory characteristics to obtain the worker's hand fist distortion value.

[0048] Step 3: Trajectory abnormality analysis: The analysis module obtains job position information and substation area information from the register. The analysis module divides the job position and area nodes, matches the business and information interaction relationships, calculates the information linkage value sorting and sets the blue line width. The analysis module obtains the staff trajectory information from the key management module and analyzes it to obtain the total value of the staff trajectory deviation.

[0049] Step 4: Color display setting: The analysis module analyzes the calculated eye movement heterogeneity integration value, hand fist distortion value, and trajectory turbulence standard total value, compares them with the set threshold value, and issues a color display signal;

[0050] Step 5: Color display: The display module receives the color display signal from the analysis module, and visualizes the comprehensive status of the staff in an intuitive color form. It also issues an alarm to the staff whose status is red and restricts their access rights.

[0051] The monitoring of the facial and eye status includes the following steps:

[0052] Step 11: Obtaining raw data, the analysis module obtains image information of the current staff entering the area in front of the substation key cabinet from the visual module;

[0053] Step 12: The analysis module converts the acquired color image of the eyeball into a grayscale image, uses the threshold segmentation method to separate the white area of ​​the eye from the background and other eye tissues, and fits the white area of ​​the eyeball using the ellipse fitting algorithm to obtain the major and minor axis information of the ellipse. If the major axis direction deviates from the horizontal direction counterclockwise by more than 5° and the ratio of the major axis to the minor axis is within the range of 1.6-2.5, it is determined that the eyeball is moving to the left. If the major axis direction deviates from the horizontal direction clockwise by more than 5° and the ratio of the major axis to the minor axis is within the range of 1.6-2.5, it is determined that the eyeball is moving to the left. .5, the eyeball is judged to be moving to the right. If the long axis direction deviates from the vertical direction counterclockwise by more than 5° and the ratio of the long axis to the short axis is within the range of 1.3-1.7, the eyeball is judged to be moving upward. If the long axis direction deviates from the vertical direction clockwise by more than 5° and the ratio of the long axis to the short axis is within the range of 1.3-1.7, the eyeball is judged to be moving downward. The number of times the staff member's eye movement direction changes in time a, c, is obtained, and substituted into the formula η1=c / a to calculate the staff member's eye movement direction change rate η1;

[0054] Step 13: The analysis module calculates the circumscribed polygon of the eye area segmented by the staff, obtains the vertex coordinates of the circumscribed polygon, and calculates the vertex coordinates of the polygon. i , the adjacent vertices are G i+1 and η i-1 , i is the vertex number of the circumscribed polygon, construct the vector and in like Then the vertex is a convex vertex if Then the vertex is a concave vertex, and the convexity and concavity distribution of the vertices of the circumscribed polygon is counted. If all vertices are convex vertices, the polygon is determined to be a convex polygon. If there is a concave vertex, the polygon is determined to be a concave polygon. The number of convexity and concavity changes g of the circumscribed polygon of the staff member's eye white within time a is obtained, and the frequency rate ε of the convexity and concavity changes of the staff member's eye white is calculated to obtain the contour extension rate, airspace ratio rate, and center of mass drift rate of the circumscribed polygon of the eye white area are obtained to calculate the specific value η2 of the staff member's eye white shape movement;

[0055] It should be noted that the contour extension rate of the circumscribed polygon is calculated by using the method of vector calculus to calculate the change value of the length vector of each side of the polygon, and then performing a weighted sum divided by the interval time. The weight is based on the sine value of half the angle between the front and back vectors of each side, and the angle ranges from 0 to 180 degrees. The airspace ratio rate of the circumscribed polygon is the rate of change of the area ratio of the sclera circumscribed polygon in the eye image space over time. The centroid drift rate of the circumscribed polygon is calculated by the ratio of the coordinate change of the polygon's centroid at different time points to the time interval.

[0056] Step 14: The analysis module records the staff member's eye movement direction change rate η1, eye sclera convexity and concavity abnormality frequency ε, and eye sclera shape abnormality value η2 within a certain period of time as the staff member's eye movement shape abnormality parameter X mn , X mn is the nth index value of the mth data, m is a positive integer, the maximum value is M, n=1,2,3, corresponding to η1,ε,η2, through the formula Get the standard value of the staff's eye movement parameter X* mn , min is the minimum value, max is the maximum value, according to the formula The information entropy en of each eye movement parameter of the staff is calculated according to the formula Get the weights qn of each staff member's eye movement shape parameter, and the standardized eye movement shape parameter standard value X* n ,The data are clustered into three categories using the K-means clustering algorithm. Each category corresponds to a preset level, corresponding to low heterogeneity, medium heterogeneity, and high heterogeneity of eye movement anomalies. The analysis module defines the Gaussian membership function. c1, c2, c3 are the center values ​​of each level, σ1, σ2, σ3 are the standard deviations of the data of each level, for each indicator X* n For all data points m, calculate the mean of their membership to each level and get the matrix R. r ns It represents the membership of the factor eye movement anomaly parameter n to the preset level, such as r 12 Represents the heterogeneous membership of the eye movement direction change rate pair, through matrix fuzzy multiplication B = q n *R, we get the comprehensive evaluation result vector B, which reflects the comprehensive membership of the eye movement heterogeneity parameters to the low heterogeneity, medium heterogeneity, and high heterogeneity evaluation levels. The corresponding values ​​of low heterogeneity, medium heterogeneity, and high heterogeneity eye movement heterogeneity are 1, 2, and 3, respectively. Then, we multiply the comprehensive membership vector B with the corresponding values ​​of low, medium, and high heterogeneity and calculate the sum to get the staff member's eye movement heterogeneity integrated value.

[0057] The monitoring of the behavior state includes the following steps:

[0058] Step 21: The analysis module obtains behavioral image information of the current staff member entering the area in front of the substation key cabinet from the visual module, uses hand key point tracking technology to obtain the joint coordinates between the proximal phalanx, middle phalanx and distal phalanx of the fingers, calculates the sum of the vector angles between adjacent joint points and performs inverse processing to obtain the staff member's joint bending characteristic value, calculates the average distance between the fingertip and the center of the palm, and marks it as the finger opening degree, calculates the difference between the joint angle between the middle phalanx and the distal phalanx and the joint angle between the proximal phalanx and the middle phalanx, performs inverse processing, and marks it as the finger near-far angle difference value, normalizes the finger opening degree and the finger near-far angle difference value, and performs weighted calculation, and multiplies them by the corresponding influencing factor coefficient to obtain the staff member's finger whole extension posture value, normalizes the staff member's joint bending characteristic and the staff member's finger whole extension posture value, divides the staff member's joint bending characteristic value by the staff member's finger whole extension posture value, and multiplies it by the correction factor coefficient to obtain the staff member's hand bending characteristic posture value;

[0059] Step 22: The analysis module uses time as the horizontal axis and the hand bending posture value of the staff as the vertical axis to establish a hand bending posture value change curve diagram when the staff enters the substation key cabinet area to take the key, and obtains the hand bending posture value change standard curve. The standard curve is located above, marked as the hand bending posture upper mark line, and the standard curve is located below, marked as the hand bending posture lower mark line. The area above the hand bending posture upper mark line and the area below the hand bending posture lower mark line of the hand bending posture value change curve diagram are obtained, and the sum is taken to obtain the hand bending posture deviation value of the staff. If If the average value of the decrease in the angle of each finger joint within 1 second exceeds the threshold β1, and the contact area between the fingers is greater than the threshold β2, then the period is determined to be a super-threshold tight fist clenching period. The total duration of the super-threshold tight fist clenching period when the staff member enters the substation key cabinet area to retrieve the key is counted and multiplied by the correction coefficient to obtain the super-threshold fist clenching anomaly value of the staff member. The motion trajectory of the staff member's hand in the three-dimensional space during the process of entering the substation key cabinet area to retrieve the key is obtained. The rapid change value, stagnant state value, and speed distortion value in the motion trajectory are obtained, and the normalized processing is performed and summed to obtain the staff member's hand trace sudden distortion value.

[0060] Step 23: The analysis module normalizes the staff member's hand bending posture deviation value, the staff member's suprathreshold fist clenching value, and the staff member's handprint sudden coagulation value, performs weighted calculation, and multiplies them by a corresponding proportional factor coefficient to obtain the staff member's hand fist bending distortion value;

[0061] It should be noted that the sudden change value is the number of times the angle of change in the hand motion direction exceeds the threshold value β3 within 2 seconds; the stagnant state value is the number of times the speed value in the hand motion trajectory is lower than the threshold value β4 within 5 seconds; the speed distortion value is the number of times the absolute value of the acceleration in the hand motion trajectory is greater than the threshold value β5;

[0062] The abnormal trajectory state analysis includes the following steps:

[0063] The analysis module obtains job information and substation area information from the register. The analysis module divides the staff position nodes into technical support personnel, engineering construction personnel, operation duty personnel, maintenance personnel, safety management personnel, and other personnel, and uses them as a type of node in the network structure diagram, represented by a circle. The substation area nodes are divided into construction area, auxiliary equipment area, main control room, primary equipment area, secondary equipment area, and other areas, as nodes of the network structure diagram, and the business process relationship between the position nodes and the area nodes is matched respectively. The technical support personnel are matched with the primary equipment area and the secondary equipment area, the engineering construction personnel are matched with the construction area, and the operation duty personnel are matched with the main Control room, maintenance personnel are matched to the primary equipment area and the secondary equipment area, safety management personnel are matched to each area, and other personnel have no corresponding matching relationship. The business process relationship is connected with red lines. The connection between the job node and other nodes except the matched business process relationship is defined as the information interaction connection, which is connected with blue lines. The information interaction methods correspond to face-to-face communication, video conferencing, voice calls, telephone conferences, emails, written reports and bulletin boards, and the corresponding values ​​are w1, w2, w3, w4, w5, w6 and w7 respectively, among which w1>w2>w3>w4>w5>w6>w7, marked as the information interaction connection value W between the job node and the regional node;

[0064] The analysis module normalizes the communication divergence dimension, average processing time span, and information interaction connection value W between each post node and each information interaction connection area node. A regular octahedron is constructed as the basic framework with the average processing time span as the side length. A rotated bipyramid with the communication divergence dimension as a parameter is generated. A constraint sphere with the information interaction connection value as the radius is used to clip the intersection of the bipyramid and the regular octahedron. The information linkage shape value between each post node and each information interaction connection area node is calculated by calculating the percentage of the difference between the intersection volume and the constraint sphere volume as a percentage of the regular octahedron volume. The analysis module sorts the calculated information linkage shape values ​​in descending order and matches the width of the blue line connecting each post node with each information interaction connection area node. The RFID tracking tag on the key is used to obtain the trajectory information of the staff entering each area of ​​the substation and the turbulence node score of the staff in each area. The information linkage shape value is obtained when the red line and blue line of the network structure diagram are connected. If the post node and the area node are connected as red connections and the trajectory turbulence node score is greater than the set threshold X, the information linkage shape value is obtained. When C1, a second-level trajectory deviation signal is issued. If the post node and the regional node are connected as a red connection, and the trajectory turbulence section score value is less than or equal to the set threshold value XC1, a third-level trajectory deviation signal is issued. If the post node and the regional node are connected as a blue connection, the trajectory turbulence section score value is greater than or equal to the set threshold value XC1 and the information linkage contract value is greater than the set threshold value XC2, a first-level trajectory deviation signal is issued. If the post node and the regional node are connected as a blue connection, the trajectory turbulence section score value is less than or equal to the set threshold value XC1 and the information linkage contract value is less than the set threshold value XC2, a third-level trajectory deviation signal is issued. In other cases where the post node and the regional node are connected as a blue connection, a second-level trajectory deviation signal is issued. The first-level trajectory deviation signal, the second-level trajectory deviation signal, and the third-level trajectory deviation signal are respectively assigned values ​​g1, g2, and g3, specifically 10, 5, and 3, which are marked as the trajectory deviation standard values ​​of the staff in each area, and the trajectory deviation standard values ​​of the staff in each area are added to obtain the total trajectory deviation standard value of the staff;

[0065] It should be noted that the business process relationship matching between job nodes and regional nodes is matched according to the core main business relationship. For example, the operation duty officer mainly monitors the equipment operation and issues operation instructions in the main control room. The communication divergence dimension is the number of processing related to construction planning, equipment operation and maintenance, power grid operation, safety management, and technology updates between job nodes and regional nodes within a certain period of time. The width of the connection line is positively correlated with the information linkage fit value, that is, the higher the information linkage fit value, the wider the width of the corresponding blue line connection line. The trajectory turbulence node score is the sum of the number of turns in the trajectory and the number of branches in the trajectory.

[0066] The color display setting includes the following steps:

[0067] The analysis module analyzes the staff's eye movement heterogeneity integration value, the staff's hand fist distortion value, and the staff's trajectory deviation standard total value respectively. If the staff's eye-hand trajectory distortion value is greater than or equal to the set threshold τ 11 , then the staff member will be sent a signal. If the staff member's eye-hand track distortion value is greater than the threshold τ 21 , then the staff member will be sent a signal. If the staff member's eye-hand track distortion value is greater than the set threshold τ 31 , then the staff member will be given a signal one. If three signal ones appear at the same time, the staff member will be given a red display signal. If there is no signal one, the staff member will be given a green display signal. In other cases, the staff member will be given an orange display signal.

[0068] The above is an illustration of the present invention and should not be considered as limiting thereof. Although several exemplary embodiments of the present invention have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined by the claims. It should be understood that the above is an illustration of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.

Claims

1. An artificial intelligence-based substation key monitoring system, comprising a vision module, an analysis module, a display module, a register, and a key management module. The key management module communicates with RFID tracking tags on keys to obtain worker trajectory information and transmit it to the analysis module. This system is characterized by: The vision module includes two sets of image sensors. One set of image sensors obtains image information of facial and eye details of the staff when entering the area in front of the substation key cabinet. The other set of image sensors obtains behavioral image information of the staff entering the area in front of the substation key cabinet and transmits it to the analysis module. The analysis module analyzes the facial state and behavioral state of the staff entering the area in front of the substation key cabinet based on the facial and eye details and behavioral image information. The register is used to store all temporary data generated during the execution of the program by the analysis module. The analysis module transmits the generated color display signal to the display module. The display module is set in the substation control room and is connected to the management system through the internal network. The display module visualizes the comprehensive status of the staff in an intuitive color form and issues an alarm to the staff whose display is red, restricting their access rights. The temporary data includes original data and analysis data. The original data includes image information, work position information, and substation area information. The analysis data includes monitoring of facial and eye status, monitoring of behavioral status, analysis of trajectory abnormality, and color display signals.

2. The substation key supervision system based on artificial intelligence according to claim 1 is characterized in that: The working method of the system includes the following steps: Step 1: Facial and eye status monitoring. When a worker enters the area in front of the substation key cabinet, the vision module transmits image information of facial eye details to the analysis module. The analysis module analyzes eye movements and features of the whites of the eyes to obtain the worker's eye movement shape heterogeneity integration value. Step 2: Behavior status monitoring. When a worker enters the area in front of the substation key cabinet, the vision module transmits behavioral image information to the analysis module. The analysis module analyzes the hand joint angle, opening degree, and motion trajectory characteristics to obtain the worker's hand fist distortion value. Step 3: Trajectory abnormality analysis: The analysis module obtains job position information and substation area information from the register. The analysis module divides the job position and area nodes, matches the business and information interaction relationships, calculates the information linkage value sorting and sets the blue line width. The analysis module obtains the staff trajectory information from the key management module and analyzes it to obtain the total value of the staff trajectory deviation. Step 4: Color display setting: The analysis module analyzes the calculated eye movement heterogeneity integration value, hand fist distortion value, and trajectory turbulence standard total value, compares them with the set threshold value, and issues a color display signal; Step 5: Color display: the display module receives the color display signal from the analysis module, and visualizes the comprehensive status of the staff in an intuitive color form, and issues an alarm to the staff displayed in red, restricting their access rights.

3. The substation key supervision system based on artificial intelligence according to claim 2 is characterized in that: The heterogeneous integrated value analysis of the staff member's eye movement shapes includes the following sub-steps: Step 11: The analysis module calculates the circumscribed polygon of the eye area segmented by the staff, obtains the vertex coordinates of the circumscribed polygon, and calculates the vertex coordinates of the polygon. i , the adjacent vertices are G i+1 and G i-1 , i is the vertex number of the circumscribed polygon, construct the vector and in like Then the vertex is a convex vertex if Then the vertex is a concave vertex, and the convexity and concavity distribution of the vertices of the circumscribed polygon is counted. If all vertices are convex vertices, the polygon is determined to be a convex polygon. If there is a concave vertex, the polygon is determined to be a concave polygon. The number of convexity and concavity changes g of the circumscribed polygon of the staff member's eye white within time a is obtained, and the frequency rate ε of the convexity and concavity changes of the staff member's eye white is calculated to obtain the contour extension rate, airspace ratio rate, and center of mass drift rate of the circumscribed polygon of the eye white area are obtained to calculate the specific value η2 of the staff member's eye white shape movement; Step 12: The analysis module records the staff member's eye movement direction change rate η1, eye sclera convexity and concavity abnormality frequency ε, and eye sclera shape abnormality value η2 within a certain period of time as the staff member's eye movement shape abnormality parameter X mn , X mn is the nth index value of the mth data, m is a positive integer, the maximum value is M, n=1,2,3, corresponding to η1,ε,η2, through the formula Get the standard value of the staff's eye movement parameter X* mn , min is the minimum value, max is the maximum value, according to the formula, The information entropy en of each eye movement parameter of the staff is calculated according to the formula Get the weights qn of each staff member's eye movement shape parameter, and the standardized eye movement shape parameter standard value X* n ,The data are clustered into three categories using the K-means clustering algorithm. Each category corresponds to a preset level, corresponding to low heterogeneity, medium heterogeneity, and high heterogeneity of eye movement anomalies. The analysis module defines the Gaussian membership function. c1, c2, c3 are the center values ​​of each level, σ1, σ2, σ3 are the standard deviations of the data of each level, for each indicator X* n For all data points m, calculate the mean of their membership to each level and get the matrix R. r ns It represents the membership of the factor eye movement heterogeneity parameter n to the preset level. For example, r12 represents the membership of the eye movement direction change rate to medium heterogeneity. Through matrix fuzzy multiplication B=qn*R, the comprehensive evaluation result vector B is obtained, which reflects the comprehensive membership of the eye movement heterogeneity parameters as a whole to the low heterogeneity, medium heterogeneity and high heterogeneity evaluation levels. The corresponding values ​​of low heterogeneity, medium heterogeneity and high heterogeneity eye movement heterogeneity levels are 1, 2 and 3. Then the comprehensive membership vector B is multiplied and summed with the corresponding values ​​of low, medium and high heterogeneity to obtain the integrated value of the staff's eye movement heterogeneity.

4. The substation key supervision system based on artificial intelligence according to claim 3 is characterized in that: The step of analyzing the staff member's eye movement direction change rate includes: Obtaining raw data, the analysis module obtains image information of the current staff entering the area in front of the substation key cabinet from the visual module; The analysis module converts the acquired color image of the eyeball into a grayscale image, uses a threshold segmentation method to separate the white area of ​​the eye from the background and other eye tissues, and fits the white area of ​​the eyeball using an ellipse fitting algorithm to obtain the major and minor axis information of the ellipse. If the major axis deviates from the horizontal direction counterclockwise by more than 5° and the ratio of the major axis to the minor axis is within the range of 1.6-2.5, it is determined that the eyeball is moving to the left. If the major axis deviates from the horizontal direction clockwise by more than 5° and the ratio of the major axis to the minor axis is within the range of 1. If the long axis deviates from the vertical direction by more than 5° counterclockwise and the ratio of the long axis to the short axis is within the range of 1.3-1.7, the eye is judged to be moving upward. If the long axis deviates from the vertical direction by more than 5° clockwise and the ratio of the long axis to the short axis is within the range of 1.3-1.7, the eye is judged to be moving downward. The number of times the staff member's eye movement direction changes c within the time a is obtained, and the eye movement direction change rate η1 of the staff member is calculated.

5. The substation key supervision system based on artificial intelligence according to claim 2 is characterized in that: The staff member's hand fist deformity abnormal value analysis step includes: The analysis module analyzes the staff's hand bending posture deviation value, the staff's suprathreshold fist clenching anomaly value, and the staff's handprint sudden coagulation anomaly value to obtain the staff's hand fist bending anomaly value.

6. The substation key supervision system based on artificial intelligence according to claim 5 is characterized in that: The steps of analyzing the deviation value of the staff's hand bending posture, the staff's suprathreshold fist clenching anisotropy value, and the staff's handwriting sudden coagulation anomaly value include: The analysis module uses time as the horizontal axis and the hand bending posture value of the staff as the vertical axis to establish a hand bending posture value change curve diagram when the staff enters the substation key cabinet area to take the key, and obtains the hand bending posture value change standard curve. The standard curve is located above, marked as the hand bending posture upper mark line, and the standard curve is located below, marked as the hand bending posture lower mark line. The area above the hand bending posture upper mark line and the area below the hand bending posture lower mark line of the hand bending posture value change curve diagram are obtained, and the sum is taken to obtain the hand bending posture deviation value of the staff. If each hand If the average value of the reduction in the finger joint angle within 1 second exceeds the threshold β1, and the contact area between the fingers is greater than the threshold β2, then the period is determined to be a supra-threshold tight fist clenching period. The total duration of the supra-threshold tight fist clenching period when the staff enters the substation key cabinet area to retrieve the key is counted and multiplied by the correction coefficient to obtain the staff's supra-threshold fist clenching anomaly value. The motion trajectory of the staff's hand in the three-dimensional space when entering the substation key cabinet area to retrieve the key is obtained. The rapid change value, stagnation state value, and speed distortion value in the motion trajectory are obtained, and the normalized processing and summation are performed to obtain the staff's handprint sudden distortion value.

7. The substation key supervision system based on artificial intelligence according to claim 5 is characterized in that: The staff's hand bending posture value analysis step includes: The analysis module obtains behavioral image information of the current staff member entering the area in front of the substation key cabinet from the visual module, uses hand key point tracking technology to obtain the joint coordinates between the proximal phalanx, middle phalanx and distal phalanx of the fingers, calculates the sum of the vector angles between adjacent joint points and performs inverse processing to obtain the staff member's joint bending characteristic value, calculates the average distance between the fingertip and the center of the palm, marks it as the finger opening degree, calculates the difference between the joint angle between the middle phalanx and the distal phalanx and the joint angle between the proximal phalanx and the middle phalanx, performs inverse processing, and marks it as the finger near-far angle difference value, performs weighted calculation on the finger opening degree and the finger near-far angle difference value, and multiplies them by the corresponding influencing factor coefficient to obtain the staff member's finger whole extension posture value, divides the staff member's joint bending characteristic value by the staff member's finger whole extension posture value and multiplies it by the correction factor coefficient to obtain the staff member's hand bending characteristic posture value.

8. The substation key supervision system based on artificial intelligence according to claim 2 is characterized in that: The step of analyzing the total value of the staff member's trajectory deviation includes: The analysis module obtains the communication divergence dimension and average processing time span of each post node and each information interaction connection area node, and normalizes them with the information interaction connection value W. It constructs a regular octahedron as the basic framework with the average processing time span as the side length, generates a rotated bicone with the communication divergence dimension as a parameter, and uses a constraint ball with the information interaction connection value as the radius to clip the intersection of the bicone and the regular octahedron. By calculating the percentage of the difference between the intersection volume and the constraint ball volume to the regular octahedron volume, the information linkage contract value of each post node and each information interaction connection area node is obtained; the analysis module arranges the calculated information linkage contract value in descending order, and matches the blue line connection width of each post node and each information interaction connection area node. The trajectory information of the staff entering each area of ​​the substation is obtained through the RFID tracking tag of the key, and the trajectory turbulence node value of the staff in each area is obtained. The information linkage contract value is when the red line and blue line of the network structure diagram are connected. If the post node and the area node are connected as red connections, and the trajectory turbulence node value is greater than the set threshold When XC1 is reached, a second-level trajectory deviation signal is issued. If the post node and the regional node are connected in red and the trajectory turbulence score is less than or equal to the set threshold XC1, a third-level trajectory deviation signal is issued. If the post node and the regional node are connected in blue, the trajectory turbulence score is greater than or equal to the set threshold XC1 and the information linkage contract value is greater than the set threshold XC2, a first-level trajectory deviation signal is issued. If the post node and the regional node are connected in blue, the trajectory turbulence score is less than or equal to the set threshold XC1 and the information linkage contract value is less than the set threshold XC2, a third-level trajectory deviation signal is issued. In other cases where the post node and the regional node are connected in blue, a second-level trajectory deviation signal is issued. The first-level trajectory deviation signal, the second-level trajectory deviation signal, and the third-level trajectory deviation signal are assigned values ​​g1, g2, and g3, which are 10, 5, and 3 respectively, and are marked as the trajectory deviation standard values ​​of the staff in each area. The trajectory deviation standard values ​​of the staff in each area are added together to obtain the total trajectory deviation standard value of the staff.

9. The artificial intelligence-based substation key supervision system according to claim 8 is characterized in that: The information interaction connection value analysis steps: The analysis module obtains job information and substation area information from the register. The analysis module divides the staff position nodes into technical support personnel, engineering construction personnel, operation duty personnel, maintenance personnel, safety management personnel, and other personnel, and uses them as a type of node in the network structure diagram, represented by a circle. The substation area nodes are divided into construction area, auxiliary equipment area, main control room, primary equipment area, secondary equipment area, and other areas as nodes in the network structure diagram, and the business process relationship between the position nodes and the area nodes is matched respectively. The technical support personnel are matched with the primary equipment area and the secondary equipment area, and the engineering construction personnel are matched with the construction area. The operation duty personnel are matched to the main control room, the maintenance personnel are matched to the primary equipment area and the secondary equipment area, the safety management personnel are matched to each area, and there is no corresponding matching relationship for other personnel. The business process relationships are connected with red lines, and the connection between the job nodes and other nodes except the matched business process relationships is defined as the information interaction connection, which is connected with blue lines. The information interaction methods correspond to face-to-face communication, video conferencing, voice calls, telephone conferences, emails, written reports and bulletin boards, and the corresponding values ​​are w1, w2, w3, w4, w5, w6 and w7, which are marked as the information interaction connection value W between the job node and the regional node.

10. The substation key supervision system based on artificial intelligence according to claim 2 is characterized in that: The color display setting analysis step includes: The analysis module performs signal analysis on the staff's eye movement heterogeneity integration value, the staff's hand fist bending distortion value, and the staff's trajectory turbulence standard total value respectively to obtain the color display signal of the corresponding situation.

Images