A method and device for alerting of a misoperation battery

Abstract

The present application relates to the technical field of city management algorithm, and specifically provides a kind of illegal operation battery alarm method and device, based on machine vision, with the following steps: S1, obtain community monitoring video stream, the monitoring video stream obtained is converted into image frame sequence and pretreated;S2, utilize human posture estimation algorithm, extract human detection frame and key point coordinates from image frame sequence, assess whether user exists the action of bending to disassemble or install battery;S3, integrate battery target detection algorithm to determine the position of battery in frame sequence, determine that the user is illegal to carry battery, S4, introduce battery tracking algorithm to continuously track battery target, confirm that the user carries battery and moves, and issue an alarm.Compared with the prior art, the present application can effectively improve the intelligent level of illegal carrying of battery in the community and alarm, which helps to ensure the standardization and safety of community management.

Description

Technical Field

[0001] The present invention relates to the technical field of urban governance algorithms, and specifically provides a method and device for warning of illegal battery operation. Background Art

[0002] With the advancement of technology and the widespread use of electric vehicles, batteries have become a core component due to their high energy density. However, during use and management, improper battery handling frequently occurs, including unauthorized removal and unauthorized removal. These behaviors can not only damage the equipment but also cause serious safety incidents such as short circuits, fires, and even explosions. Therefore, developing intelligent warning methods to address improper battery handling has become an important direction for improving safety management efficiency.

[0003] Traditional violation detection methods rely primarily on manual oversight and simple monitoring equipment, detecting violations through real-time observation by monitoring personnel. This approach is not only labor-intensive but also lacks the ability to guarantee accurate and timely detection, posing the risk of missed detections and misjudgments. Furthermore, with the widespread adoption of intelligent devices, many scenarios are placing higher demands on the automation, real-time nature, and accuracy of violation detection. Manual oversight struggles to meet the demands for real-time and accurate monitoring of violations in environments with frequent personnel movements and complex equipment. Summary of the Invention

[0004] The present invention aims to address the deficiencies of the above-mentioned prior art and provides a highly practical battery operation warning method.

[0005] A further technical task of the present invention is to provide a battery warning device with reasonable design, safety and applicability for illegal operation.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A battery warning method for illegal operation, based on machine vision, has the following steps:

[0008] S1. Obtain a community surveillance video stream, convert the acquired surveillance video stream into an image frame sequence, and perform preprocessing;

[0009] S2. Utilize a human pose estimation algorithm to extract human body detection frames and key point coordinates from the image frame sequence. By analyzing the bending angle of the human torso and the spatial positional relationship between joints, assess whether the user is bending over to remove or install the battery.

[0010] S3: An integrated battery target detection algorithm uses deep learning technology to identify the unique appearance features of batteries to determine their location within the frame sequence. If a battery is found within the human body detection frame and close to the user's wrist joint, the user is deemed to be illegally carrying a battery.

[0011] S4. A battery tracking algorithm is introduced to continuously track the battery target. If the battery is not lost within a set time period, it can be confirmed that the user has moved with the battery, and an alarm is issued accordingly.

[0012] Furthermore, in step S2, the YOLOv10-PoseNet human pose estimation algorithm is used. First, the YOLOv10 target detection framework is used to quickly locate pedestrians in the frame sequence, generate accurate human detection frames, and then provide input for pose estimation. Subsequently, the PoseNet pose estimation network extracts the coordinates of key points of 17 parts of the human body, thereby establishing a complete joint space relationship and human skeleton structure, calculating the human trunk bending angle and joint space position relationship, and finally realizing the behavior logic judgment of bending over to operate the battery. The following are the basic steps for the behavior logic judgment of bending over to operate the battery:

[0013] (2-1) Extraction and preprocessing of human body key point coordinates;

[0014] (2-2) Calculate the bending angle of the human body trunk;

[0015] (2-3) Calculate the spatial position relationship of the joints.

[0016] Furthermore, in step (2-1), the two-dimensional coordinates of the key points of the head, shoulder, hip and knee joints are extracted, which are:

[0017] (X head 、Y head )、(X shoulder 、Y shoulder )、(X hip 、Y hip )、(X knee 、Y knee );

[0018] In order to ensure the stability of the behavioral logic judgment process, the shoulder center coordinate is the average of the horizontal and vertical coordinates of the left and right shoulders. The calculation formula is as follows:

[0019]

[0020] Among them, (X shoulde_l 、Y shoulder_l )、(X shoulder_r 、Y shoulder_r ) represent the horizontal and vertical coordinates of the left shoulder and right shoulder respectively;

[0021] In step (2-2), the method for obtaining the bending angle of the human body trunk is to construct two vectors from the head to the hip and from the hip to the knee, and calculate the angle between them to determine the curvature of the upper body. When the human body is upright, the head, hip and knee are roughly in a vertical line, and the line from the head to the hip is close to vertical; when bending over, the head leans forward, causing the line from the head to the hip to deviate to the horizontal direction;

[0022] Assumption vector Indicates the direction from head to hip, vector Indicates the direction from the hip to the knee. The angle is calculated as follows:

[0023]

[0024] Where θ represents the bending angle between the upper and lower body. The included angle can be obtained by taking the arc cosine value of cos(θ). If θ is less than the preset bending angle threshold angle_blending_threshold, it is considered that the included angle between the upper and lower body is small, indicating that the posture meets the bending feature.

[0025] In step (2-3), it is determined whether the user has bent over to operate the battery in violation of regulations. In addition to calculating the bending angle of the human body, the spatial position relationship of key parts of the human body is also analyzed to assist in determining bending;

[0026] Specifically, the relative height and horizontal position changes of the head, hips, and knees reveal whether the user's upper body tends to lean forward or press downward. Taking the vertical height difference ratio method as an example, the vertical height difference of the head, hips, and knees is used to determine the degree of bending. When the user is standing normally, the vertical height from the head to the hips should be significantly greater than the vertical height from the hips to the knees. A vertical height ratio is defined as:

[0027]

[0028] Among them, Y hip 、Y knee Respectively represent the center height of the hips and knees. When standing normally, R y Should be greater than the preset vertical height ratio threshold vertical_ratio_threshold. When bending over, the head is close to the hip, resulting in Y head -Y hip Reduce so that R y Decline, if R y If the value is lower than the preset threshold, it is determined to be a bending action;

[0029] Considering that the head will move forward significantly when the user bends over, the horizontal distance method is also used to calculate the horizontal distance increment between the head and hip. The formula for calculating the horizontal distance between the head and hip is as follows:

[0030] D x =|X head -X hip |;

[0031] The formula for calculating shoulder width is as follows:

[0032] W s =|X shoulder_l -X shoulder_r |;

[0033] When the user is standing, D x Smaller, and when bending over, D x Will exceed the normal range if D x If the horizontal distance ratio exceeds the preset threshold horizontal_ratio_threshold, it is considered that the head has moved forward significantly, indicating bending behavior.

[0034] Furthermore, in step S3, the integrated battery target detection algorithm adopts the YOLOv10 battery target detection algorithm to perform the following logical judgment on the illegal carrying of batteries:

[0035] (3-1) Intersection determination between battery target and human body detection frame;

[0036] (3-2) Determination of close interaction between battery target and user;

[0037] (3-3) Continuous assessment of illegal carrying of batteries.

[0038] Furthermore, in step (3-1), assuming that the human body detection frame is a rectangle, the region is demarcated by the coordinates of the upper left corner and the lower right corner, that is, the coordinates of the upper left corner of the human body detection frame are (X left , Y top ), the coordinate of the lower right corner is (X right , Y bottom ), and the coordinate center of the battery is (X battery ,Y battery ); The condition for the battery target to fall into the human body detection frame is that the horizontal and vertical coordinates of the battery center point are both within the range of the human body detection frame, that is:

[0039] X left ≤X battery ≤X right ;

[0040] Y top ≤Y battery ≤Y bottom ;

[0041] If both of the above conditions are met, it means that the center point of the battery falls within the human body detection frame, which serves as the basis for judging that the battery is within the user's activity range;

[0042] In step (3-2), we need to prove that the battery target maintains a close spatial relationship with the user's wrist joint by calculating the distance between the battery center point and the user's wrist joint and determining whether it is lower than the set distance threshold distance_threshold;

[0043] If the distance between the two is small enough, it is considered that they have a close spatial relationship. Assume that the coordinates of the user's left and right wrist joints are (X wirst_l 、Y wirst_l )、(X wirst_r 、Y wirst_r ); Taking the left wrist joint as an example, use the Euclidean distance formula to calculate the distance between the battery center and the left wrist:

[0044]

[0045] Get D l 、D r Then, the shortest distance D between the battery and the wrist is obtained. min :

[0046] D min =min(D l ,D r );

[0047] If D min If the distance is less than or equal to a preset distance threshold distance_threshold, it is considered that the battery and the user's wrist joint are in close spatial relationship. The judgment conditions are as follows:

[0048] D min ≤D threshold ;

[0049] When this condition is met, it means that the battery is within the operating range of the user's wrist, which serves as the basis for judging that the user is operating the battery;

[0050] In step (3-3), to improve the accuracy, D is calculated for multiple frames in the time series. min To determine whether this spatial relationship persists, if D min ≤D threshold It is continuously established and can more stably determine whether the user is performing battery operations. Through this distance calculation and threshold judgment, it can more accurately determine whether the battery target is illegally brought into or out of the community by the user.

[0051] Furthermore, in step S4, the ByteTrack battery tracking algorithm is used. First, the battery target detection algorithm mentioned above is used to identify the battery targets in the frame sequence and obtain the position of each target. Then, the battery tracking algorithm is used to associate the detection results to form a continuous mobile tracking trajectory. Among them, the Kalman filter is designed to predict and update the spatial position and movement speed state of the battery target. The following are the basic calculation steps of the Kalman filter:

[0052] (4-1) Initialization;

[0053] Initialization state estimate X0: initial position state estimate of the battery target;

[0054] Initial error covariance matrix P0: represents the uncertainty of the initial position estimate of the battery target;

[0055] (4-2) prediction;

[0056] State Prediction X k|k-1 : Use dynamic models to predict the state of the battery target at the next moment;

[0057] Error covariance prediction P k|k-1 : Predict the position error covariance of the battery target at the next moment;

[0058] (4-3) Update;

[0059] Calculate the Kalman gain K k : This is a weight used to update the battery target prediction state;

[0060] Update state estimate X k : Combine the predicted state and the new measurement value to update the battery target state estimate;

[0061] Update error covariance P k : Update the estimated uncertainty;

[0062] (4-4) Repeat;

[0063] Repeat the prediction and update steps until the battery target tracking task is completed;

[0064] The calculation formula of the Kalman filter prediction step is as follows:

[0065] X k∣k-1 =F k X k-1∣k-1 +B k U k ;

[0066]

[0067] Among them, Fk 、B k Represent the state transfer matrix, control input matrix, U k represents the control vector, Q k represents the process noise covariance matrix;

[0068] The calculation formula for the Kalman filter update step is as follows:

[0069]

[0070] P k =(IK k H k )P k∣k-1 ;

[0071] Among them, H k 、R k Represent the observation matrix, measurement noise covariance matrix, Z k Represents the actual position measurement value of the battery target at time k;

[0072] Based on the Kalman filter, ByteTrack also introduced the Hungarian algorithm to solve the data association problem. For those battery target detection results that do not match the existing trajectory, ByteTrack regards them as new targets and creates new tracks for them.

[0073] Furthermore, the basic calculation steps of the Hungarian algorithm are:

[0074] (1) Construct a cost matrix;

[0075] (2) Initialization mark;

[0076] (3) Find independent zero elements;

[0077] (4) Adjust the cost matrix;

[0078] (5) Marking the path;

[0079] (6) Repeat steps (3)-(5);

[0080] (7) Output matching results;

[0081] (8) Update trajectory;

[0082] (9) Repeat the above steps.

[0083] Furthermore, in step (1), an association cost is calculated for the battery object in each monitoring video frame and the known trajectory in the previous frame. The association cost is based on a certain metric to determine whether the battery targets in different frames are the same object; the cross-frame spatial distance association cost of the battery target is measured using Euclidean distance, and these association costs constitute a cost matrix C;

[0084] Where rows represent detections in the current frame and columns represent tracks in the previous frame. If the cost of associating the i-th detection with the j-th track is c ij , then C[i][j]=c ij ;

[0085] In step (2), create two Boolean arrays: assigned_row and assigned_col, which respectively indicate whether the battery target in the current frame and the trajectory in the previous frame have matched, and initialize all elements of these two arrays to False;

[0086] In step (3), for each element in the cost matrix, if it is the only zero value in its row and column, that is, an independent zero, it is taken as a matching point and the corresponding row and column are marked. If there is no independent zero, proceed to the next step;

[0087] In step (4), if it is not possible to directly find enough independent zeros to match the battery target with the corresponding trajectory, the cost matrix needs to be adjusted to create more independent zeros;

[0088] Find the uncovered minimum value, then subtract this value from all uncovered rows and add this value from all uncovered columns, creating new zeros at the same time in order not to change the positions of existing zeros, and then continue looking for independent zeros.

[0089] Furthermore, in step (5), if enough independent zeros are still not found after adjustment, it is necessary to mark an augmenting path, starting from an unmarked row, finding a zero in an unmarked column, and then going up along the column to find a zero in another unmarked row, and so on, until a marked row is reached or no more zeros are found. On this path, marks are alternately removed and added to update the matching status of the battery target and the trajectory;

[0090] In step (6), the steps of finding independent zeros, adjusting the matrix, and marking the path are repeated until all rows or columns are marked, which means that a complete match between the battery target and the trajectory is found;

[0091] In step (7), finally, by checking the assigned_row and assigned_col arrays, it is determined that the battery target is matched with the track, and the unmatched detection may be a newly appearing battery target, and the unmatched track means that the corresponding battery target has left the monitoring field of view;

[0092] In step (8), the state information of each track is updated according to the matching result, a new detected battery target starts a new track, and for an old track that loses matching, it is determined to maintain tracking or end tracking according to a preset maximum time threshold max_time_out;

[0093] In step (9), as the monitoring video frame sequence is played, the above process is repeatedly repeated to realize continuous battery target tracking.

[0094] A battery alarm device for illegal operation, comprising: at least one memory and at least one processor;

[0095] The at least one memory is used to store a machine readable program;

[0096] The at least one processor is used to call the machine readable program and execute a battery alarm method for illegal operation.

[0097] Compared with the prior art, the battery alarm method for illegal operation and the device have the following outstanding beneficial effects:

[0098] The present application constructs an efficient, accurate and adaptable intelligent battery alarm method for illegal operation, which not only realizes real-time monitoring and accurate alarm of illegal behavior in complex scenes, but also provides reliable technical support for battery safety management, effectively improves the intelligent level of illegal battery carrying and alarm in the community, and helps to ensure the standardization and safety of community management. BRIEF DESCRIPTION OF DRAWINGS

[0099] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0100] Figure 1 is a flowchart of a battery alarm method for illegal operation; Figure 1 Figure 1 is a flowchart of a battery alarm method for illegal operation;

[0101] Figure 2 is an architecture diagram of a battery target detection algorithm in a battery alarm method for illegal operation; Figure 2

[0102] Figure 2 is an architecture diagram of a battery target detection algorithm in a battery alarm method for illegal operation;​ Figure 3 is a flow chart of a battery tracking algorithm in a method for battery operation violation warning;

[0103] attached Figure 4 is a visualization result of a battery operation violation warning method based on machine vision. DETAILED DESCRIPTION

[0104] In order to make the person in the art better understand the scheme of the present application, the present application will be further described in detail below in combination with specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0105] A preferred embodiment is given below:

[0106] As shown in the figure, the method for battery operation violation warning in the embodiment based on machine vision has the following steps: Figure 1-4

[0107] S1, acquire community monitoring video stream, convert the acquired monitoring video stream into image frame sequence and pre-process;

[0108] S2, use human pose estimation algorithm to extract human detection frame and key point coordinates from the image frame sequence, evaluate whether the user has the action of bending down to disassemble or install the battery by analyzing the bending angle of the human torso and the spatial position relationship between the joints;

[0109] The YOLOv10-PoseNet human pose estimation algorithm is adopted, which combines target detection and human pose estimation technology and aims to identify the key point position and pose feature of the human body in real time. The algorithm first uses the YOLOv10 target detection framework to quickly locate the pedestrians in the frame sequence, generates accurate human detection frame, and then provides reliable input for pose estimation.

[0110] Subsequently, the PoseNet pose estimation network extracts the key point coordinates of 17 parts of the human body, thereby establishing the complete joint spatial relationship and human skeleton structure. In the scene of battery operation violation warning, the head, shoulder, hip and knee key point positions of the human body are obtained through the human pose estimation algorithm, and the bending angle of the human torso and the spatial position relationship between the joints are calculated based on this, and finally the behavior logic judgment of bending down to operate the battery is realized. The following are the basic steps of the behavior logic judgment of bending down to operate the battery:

[0111] (2-1) human key point coordinate extraction and preprocessing; ​

[0112] Extract the two-dimensional coordinates of the key points of the human body. Taking the head, shoulder, hip and knee joints as an example, the coordinates of the key points are: (X head 、Y head )、(X shoulder 、Y shoulder )、(X hip 、Y hip )、(X knee 、Y knee To ensure the stability of the behavioral logic judgment process, the shoulder center coordinates are taken as the average of the horizontal and vertical coordinates of the left and right shoulders, and the calculation formula is as follows:

[0113]

[0114]

[0115] Among them, (X shoulde_l 、Y shoulder_l )、(X shoulder_r 、Y shoulder_r ) represent the horizontal and vertical coordinates of the left and right shoulders, respectively. The calculation formulas for the center coordinates of the hips, knees, etc. are similar and will not be repeated here.

[0116] (2-2) Calculate the bending angle of the human body trunk;

[0117] The method of the present invention to obtain the bending angle of the human body is to construct two vectors from the head to the hip and from the hip to the knee, and calculate the angle between them to determine the curvature of the upper body. When the human body is upright, the head, hip and knee are roughly in a vertical line, and the line from the head to the hip is close to vertical; when bending over, the head leans forward, causing the line from the head to the hip to deviate to the horizontal direction. Assuming the vector Indicates the direction from head to hip, vector Indicates the direction from the hip to the knee. The angle is calculated as follows:

[0118]

[0119] Among them, θ represents the bending angle between the upper body and the lower body. The angle can be obtained by taking the arc cosine value of cos(θ). If θ is less than the system's preset bending angle threshold angle_blending_threshold (taking 150 degrees as an example), it can be considered that the angle between the upper body and the lower body is small, indicating that the posture meets the bending characteristics. This angle reflects whether the upper body is obviously leaning forward. If the threshold is reached, it indicates that the action is likely to involve an illegal bending posture.

[0120] (2-3) Calculate the spatial position relationship of the joints;

[0121] In order to determine whether the user has bent over to operate the battery in violation of regulations, in addition to calculating the bending angle of the human torso, the spatial position relationship of the key parts of the human body can also be analyzed to assist in judging bending. Specifically, the relative height and horizontal position changes of the head, hips and knees can reveal whether the user's upper body has a tendency to lean forward or press down. Taking the vertical height difference ratio method as an example, the vertical height difference of the head, hips and knees is used to determine the degree of bending. When the user is standing normally, the vertical height from the head to the hips should be significantly greater than the vertical height from the hips to the knees. Define a vertical height ratio:

[0122]

[0123] Among them, Y hip 、Y knee Represents the center height of the hips and knees respectively. y It should be greater than the preset vertical height ratio threshold vertical_ratio_threshold (take 1.5 as an example). When bending over, the head is close to the hip, resulting in Y head -Y hip Reduce so that R y Decrease. If R y If the movement is lower than the preset threshold, it can be determined as a bending movement.

[0124] Considering that the head will move forward significantly when the user bends, the horizontal distance method can also be used to calculate the horizontal distance increment between the head and hip. The formula for calculating the horizontal distance between the head and hip is as follows:

[0125] D x =|X head -X hip | (5)

[0126] The formula for calculating shoulder width is as follows:

[0127] W s =|X shoulder_l -X shoulder_r | (6)

[0128] When the user is standing, D x Smaller, and when bending over, D x Will exceed the normal range. x If the horizontal distance ratio exceeds the preset threshold horizontal_ratio_threshold (taking 0.5 times the shoulder width as an example), it can be considered that the head has moved forward significantly, indicating bending behavior.

[0129] In summary, the human posture estimation algorithm not only improves the system's adaptability to complex scenarios through dual analysis of the torso bending angle and the spatial position relationship of the joints, but also effectively filters out misjudgments caused by irrelevant actions, and ultimately achieves accurate logical judgment of illegal behaviors such as bending over to operate the battery.

[0130] S3: An integrated battery target detection algorithm uses deep learning technology to identify the unique appearance features of batteries to determine their location within the frame sequence. If a battery is found within the human body detection frame and close to the user's wrist joint, the user is deemed to be illegally carrying a battery.

[0131] YOLOv10 is used as the target detection algorithm, which abandons the traditional non-maximum suppression (NMS) step and achieves efficient training and detection reasoning through a consistent dual allocation strategy. This strategy enables YOLOv10 to maintain high-precision battery target detection in the illegal battery operation detection task while significantly reducing the computational overhead, providing a solid foundation for subsequent logical judgment of illegal battery carrying behavior, battery tracking and alarming, etc. The YOLOv10 model architecture is as follows Figure 2 As shown in the figure, the YOLOv10 network model mainly consists of the following three parts:

[0132] (a) Backbone (backbone network);

[0133] The Backbone network is the cornerstone of the entire YOLOv10 model, responsible for extracting features from the input surveillance video frame sequence. For battery warning scenarios, the Backbone network effectively captures the battery's basic outline and visual features, such as shape, size, color, and edge contrast, while filtering out irrelevant background, providing visual cues for subsequent steps.

[0134] (b) Neck (neck network);

[0135] The Neck network is located between the Backbone and the Head and plays a role in multi-scale feature fusion. In YOLOv10, the Neck network is typically composed of modules such as the Feature Pyramid Network (FPN) or the Path Aggregation Network (PAN), which are used to combine feature maps of different scales. For battery warning scenarios, the Neck network enables the model to simultaneously focus on the global outline and detailed features of the battery, thereby effectively processing battery objects of different sizes, positions, and angles. Especially in complex scenarios, the multi-scale feature fusion of the Neck network enhances the model's detection capabilities for small batteries and long-distance batteries.

[0136] (c) Head (detection head);

[0137] The head network is the final detection component of YOLOv10, primarily responsible for object classification and bounding box regression. It receives multi-scale features from the neck network and generates the final detection results. In the battery warning scenario, the head network outputs the battery's detection bounding box and category confidence as a specific prediction result, including the bounding box location, battery category, and confidence score.

[0138] The battery target detection algorithm accurately identifies batteries within video frames, ensuring the system can effectively distinguish between batteries and other objects. By training with a large amount of sample data, the deep learning model continuously optimizes its recognition capabilities, thereby improving the accuracy and reliability of battery detection. Once a battery target is captured by the system, its location is compared with the human detection frame and a logical judgment is made regarding illegal battery carrying.

[0139] The following are the basic steps for logically judging illegal battery carrying behavior:

[0140] (3-1) Intersection determination between battery target and human body detection frame;

[0141] Assuming that the human body detection frame is a rectangle, the area is defined by the coordinates of the upper left corner and the lower right corner, that is, the coordinates of the upper left corner of the human body detection frame are (X left , Y top ), the coordinate of the lower right corner is (X right , Y bottom ), and the coordinate center of the battery is (X battery ,Y battery The condition for the battery target to fall into the human body detection frame is that the horizontal and vertical coordinates of the battery center point are both within the range of the human body detection frame, that is:

[0142] X left ≤X battery ≤X right ; (7)

[0143] Y top ≤Y battery ≤Y bottom ; (8)

[0144] If both of the above conditions are met, it means that the center point of the battery falls within the human body detection frame, which can be used as a basis for judging that the battery is within the user's activity range.

[0145] (3-2) Determining the close interaction relationship between the battery target and the user;

[0146] Afterwards, to prove that the battery target and the user's wrist joint maintain a close spatial relationship, we can calculate the distance between the battery center point and the user's wrist joint and determine whether it is less than the set distance threshold distance_threshold. If the distance between the two is small enough, it can be considered that they have a close spatial relationship. Assume that the coordinates of the user's left and right wrist joints are (X wirst_l 、Y wirst_l )、(X wirst_r 、Y wirst_r Taking the left wrist joint as an example, the distance between the battery center and the left wrist can be calculated using the Euclidean distance formula:

[0147]

[0148] Distance D between right wrist and battery r The calculation formula is similar and will not be described here. l 、D r After that, the shortest distance D between the battery and the wrist can be further obtained min :

[0149] D min =min(D l ,D r ) (10)

[0150] If D min If the distance is less than or equal to a preset distance threshold distance_threshold (for example, 0.1 meters), it can be considered that the battery and the user's wrist joint are in a relatively close spatial relationship. The judgment conditions are as follows:

[0151] D min ≤D threshold (11)

[0152] When this condition is met, it means that the battery is within the operable range of the user's wrist, which can be used as a basis for determining that the user is operating the battery.

[0153] (3-3) Continuous determination of illegal battery carrying behavior;

[0154] In addition, to improve the accuracy, D can be calculated for multiple frames in the time series. min To determine whether this spatial relationship persists. min ≤D threshold If this continues to hold, it can more reliably determine whether a user is operating a battery. Through this distance calculation and threshold judgment, it can more accurately determine whether a battery target is illegally brought into or out of the community by a user.

[0155] In summary, the battery target detection algorithm learns the appearance features of the battery, and can accurately identify the battery target in the frame sequence. If the detected battery target falls within the human detection box and is close to the user's wrist joint, it is determined that the user is carrying the battery in violation of the rules.

[0156] S4, introduce the battery tracking algorithm to track the battery target continuously. If the tracking of the battery is not lost within the set time period, it is confirmed that the user has moved with the battery, and an alarm is issued accordingly;

[0157] The battery tracking algorithm uses the ByteTrack battery tracking algorithm, which is different from the traditional algorithm that only focuses on high-confidence detection boxes. ByteTrack tracks by associating each detection box. Such a tracking strategy makes ByteTrack algorithm easy to overcome problems such as mutual occlusion between targets, complex weather conditions, and diverse lighting conditions when facing battery target tracking problems. The battery tracking algorithm process is shown in Figure 3

[0158] The core idea of the battery tracking algorithm is to decompose the battery tracking problem into two stages: battery target detection and target trajectory estimation. First, use the battery target detection algorithm mentioned above to identify the battery target in the frame sequence and get the position of each target. Then, use the battery tracking algorithm to associate these detection results to form a continuous moving tracking trajectory. Among them, the Kalman Filter is designed to predict and update the state of the spatial position and moving speed of the battery target.

[0159] The following are the basic calculation steps of the Kalman Filter:

[0160] (4-1) Initialization;

[0161] Initialize state estimate X0: initial position state estimate of the battery target;

[0162] Initialize error covariance matrix P0: represents the uncertainty of the initial position estimate of the battery target.

[0163] (4-2) Prediction;

[0164] State prediction X k|k-1 : use the dynamic model of the system to predict the state of the battery target at the next time;

[0165] Error covariance prediction P k|k-1 : predict the position error covariance of the battery target at the next time.

[0166] (4-3) Update;

[0167] Calculate Kalman gain K k ​This is a weight used to update the battery target prediction state;

[0168] Update state estimate X k Update the battery target state estimate in combination with the predicted state and new measurements;

[0169] Update error covariance P k Update the uncertainty of the estimate.

[0170] (4-4) Repeat;

[0171] Repeat the prediction and update steps until the battery target tracking task is complete.

[0172] The Kalman filter prediction step calculation formula is as follows:

[0173] X k∣k-1 = F k X k-1∣k-1 + B k U k (12)

[0174]

[0175] Where F k , B k represent the state transition matrix, control input matrix, respectively, U k represents the control vector, and Q k represents the process noise covariance matrix.

[0176] The Kalman filter update step calculation formula is as follows:

[0177]

[0178] P k = (I-K k H k ) P k∣k-1 (15)

[0179] Where H k , R k represent the observation matrix, measurement noise covariance matrix, respectively, and Z k represents the actual position measurement value of the battery target at time k.

[0180] On the basis of the Kalman filter, ByteTrack also introduces the Hungarian algorithm to solve the data association problem. For those battery target detection results that do not match existing trajectories, ByteTrack considers them as new targets and creates new trajectories for them. The following are the basic calculation steps of the Hungarian algorithm:

[0181] (1) Construct the cost matrix;

[0182] A cost of association is computed for each battery object in the current frame and the known tracks in the previous frame. The cost of association is usually based on some metric, such as Euclidean distance, Mahalanobis distance, or a similarity measure of appearance features, to determine whether the battery objects in different frames are the same object. Given that the computation of Euclidean distance is relatively simple and fast, the present invention uses Euclidean distance to measure the spatial distance of battery objects across frames, which is necessary for real-time processing of large amounts of data in battery multi-object tracking algorithms. The method of Euclidean distance computation is shown in equation (9), which is not described here. These costs form a cost matrix C. Where the rows represent detections in the current frame and the columns represent tracks in the previous frame. If the cost of association of the i-th detection and the j-th track is c ij , then C[i][j] = c ij .

[0183] (2) Initialize flags;

[0184] Create two Boolean arrays: assigned_row and assigned_col, which represent whether the battery objects in the current frame and the tracks in the previous frame have been matched, and initialize all elements of these two arrays to False.

[0185] (3) Find independent zeros;

[0186] For each element in the cost matrix, if it is the only zero in its row and column (i.e., an independent zero), it can be taken as a matching point, and the corresponding row and column are marked. If there is no independent zero, go to the next step.

[0187] (4) Adjust the cost matrix;

[0188] If there are not enough independent zeros to match the battery objects and the corresponding tracks directly, the cost matrix needs to be adjusted to create more independent zeros. Find the smallest uncovered value, and subtract this value from all uncovered rows and add this value to all uncovered columns. The purpose of this is to not change the position of existing zeros while creating new zeros. Then continue to find independent zeros.

[0189] (5) Mark a path;

[0190] If there are still not enough independent zeros after adjustment, an augmented path needs to be marked. Start from an unmarked row, find a zero in an unmarked column, then find another zero in an unmarked row along the column, and so on until reaching a marked row or finding no more zeros. Alternately cancel and add marks on this path to update the matching state of battery objects and tracks.

[0191] (6) Repeat steps (c)-(e);

[0192] Repeat the steps of finding independent zeros, adjusting the matrix, and marking the path until all rows or columns are marked, which means that a perfect match between the battery target and the trajectory has been found;

[0193] (7) Output matching results;

[0194] Finally, by checking the assigned_row and assigned_col arrays, we can determine which battery targets are matched with which tracks. Unmatched detections may indicate a newly appeared battery target, while unmatched tracks may mean that the corresponding battery target has left the monitoring field of view.

[0195] (8) Update trajectory;

[0196] The state information of each track is updated based on the matching results. A newly detected battery target can start a new track. For old tracks that have lost matching, the present invention decides whether to maintain or terminate tracking based on a preset maximum time threshold max_time_out (for example, 30 seconds).

[0197] (9) Repeat the above steps;

[0198] As the monitoring video frame sequence is played, the above process is repeated continuously to achieve continuous battery target tracking.

[0199] Furthermore, the battery tracking algorithm employs a delay mechanism during data association, allowing low-confidence detection targets to be reassessed several frames in the future. This means the algorithm can recapture the battery target after it briefly disappears, significantly improving its tracking capabilities for targets obscured by people and vehicles in community scenarios.

[0200] The machine vision-based intelligent alarm system for batteries with illegal operations in the present invention uses a human posture estimation algorithm and a battery target detection algorithm to identify pedestrians and battery targets in the monitoring frame sequence in real time, while the battery tracking algorithm is used to continuously track the above-mentioned detected battery targets and finally issue an alarm. Combining these three technologies, the present invention constructs an efficient intelligent alarm system for batteries with illegal operations. The system can not only judge the compliance of user operation behavior based on battery detection, but also accurately identify long-term illegal behaviors during continuous tracking, greatly improving the reliability and real-time performance of the alarm, and is suitable for places such as communities and factories that need to monitor battery operations. The process of the machine vision-based intelligent alarm system for batteries with illegal operations is as follows: Figure 1 The detailed steps are as follows:

[0201] a) Community surveillance video stream acquisition and preprocessing

[0202] Real-time video streams of community scenes are captured using high-definition cameras. These video frames undergo preprocessing, including resizing, normalization, and possible color correction, to ensure that the input images meet the preset requirements of the human pose estimation algorithm and the battery target detection algorithm.

[0203] b) Human pose estimation

[0204] The system uses a human pose estimation algorithm to identify human detection frames and key point coordinates from a frame sequence. By precisely locating and estimating the skeletal key points of pedestrians in the image, it can accurately identify the pedestrian's body posture and movements. To effectively capture potential bending movements to remove or install batteries, the algorithm comprehensively considers two key indicators: the bending angle of the human torso and the spatial position of the joints. This not only enhances the system's adaptability to complex scenarios, but also effectively filters out misjudgments caused by irrelevant movements, ultimately achieving precise logical judgment of illegal bending over to operate batteries.

[0205] c) Battery target detection

[0206] The trained battery object detection model is used as a single-stage battery object detector to identify and locate battery objects in a frame sequence. The model outputs a bounding box, category label, and confidence score for each battery object. If a battery object is detected within the human body detection frame and in close spatial proximity to the user's wrist joint, the system will determine that the user is illegally carrying a battery.

[0207] d) Battery target tracking

[0208] A battery target tracking algorithm is used to track battery targets across frames. This algorithm combines appearance similarity and motion consistency to maintain target trajectory, ensuring effective tracking even when the battery target is obscured or temporarily out of view. For battery targets that remain undetected for extended periods, tracking is terminated based on a set maximum timeout threshold (max_time_out), preventing the accumulation of invalid tracks.

[0209] e) Analyze illegal battery operation behaviors and issue warnings

[0210] After data collection and tracking is complete, the system analyzes the user's compliance with battery handling based on the battery's location, posture, and behavioral characteristics. If it detects non-compliant behavior, such as battery removal, installation, or carrying violations, the system issues a real-time alert, notifying monitoring personnel to take action. This process enables efficient identification and alerting of illegal battery operations, offering excellent real-time and high-efficiency, making it suitable for battery safety monitoring in a variety of locations, including communities and factories.

[0211] At this point, the machine vision-based illegal operation battery alarm method has been completed.

[0212] Based on the above method, a battery operation violation warning device in this embodiment includes: at least one memory and at least one processor;

[0213] The at least one memory is configured to store a machine-readable program;

[0214] The at least one processor is configured to call the machine-readable program to execute a battery alarm method for illegal operation.

[0215] The model training and experimental evaluation of the present invention integrate the use of two different types of data sets. Among them, in the training and evaluation of the human pose estimation model, the present invention uses the COCO open source human pose estimation image dataset. This dataset is not only large in scale but also rich in diversity. It covers the postures of people of different ages, genders, and body shapes in various daily activities. It also takes into account factors such as occlusion and perspective changes. It is one of the important benchmarks for the research and development of human pose estimation technology. The battery target detection model uses a total of 11,140 image data collected online and recorded on-site. The recording tools include industrial cameras, drones, surveillance cameras, etc., and the recording sites include communities, factories, schools, etc., ensuring the diversification of data sources and application scenarios. Then, the image dataset is divided into training set, validation set, and test set in a ratio of 8:1:1, that is, the training set is 8,912 images, and the validation set and test set are 1,114 images.

[0216] In order to prevent the model from overfitting, the present invention introduces data augmentation technology in the training set design. Data augmentation is a method that expands the size and diversity of the training set through multiple input transformations without changing the output label, and has been widely used in the field of computer vision. As an implicit regularization method, data augmentation technology can effectively improve the generalization ability of deep convolutional neural networks, thereby improving the performance of the model on unknown data. In this experiment, the present invention mainly adopted common data augmentation methods such as horizontal flipping, random angle rotation and random scaling. By introducing these changes, the image representation is enriched and more possible actual scenarios are simulated.

[0217] In this experiment, in order to maximize the utilization of GPU memory resources and optimize the training efficiency, the size of all input images is uniformly adjusted to 640x640 pixels. Through a series of experimental observation and analysis, the present application found that the selection of batchsize has a significant impact on the utilization of GPU memory and training efficiency. Therefore, through multiple iterations and optimization, the optimal batch size is finally determined to be 16. This setting not only makes full use of the memory resources of the GPU, but also achieves a good balance between the speed and stability of model training. In addition, in order to enable the model to fully learn the complex features in the data, the training period is set to 300 rounds. This training strategy ensures that the model can undergo sufficient iterations, thereby improving its adaptability to diverse data.

[0218] In order to evaluate the accuracy and stability of the illegal operation battery alarm model, precision, recall, and mean average precision (mAP) are selected as evaluation indicators. The calculation formulas of each evaluation indicator are as follows:

[0219]

[0220] Among them, True Positive and False Positive represent the number of positive samples and negative samples predicted by the model, respectively, and False Negative represents the number of positive samples predicted by the model. n represents the class of all samples, and in this experiment, the target instances to be detected are divided into two categories, namely battery and pedestrian.

[0221] Table 1 Comparison of illegal operation battery alarm model experiments

[0222]

[0223] Table 1 shows the comparison results of the network model of the present system in the illegal operation battery alarm task with other series of representative models. Specifically, in the battery alarm comparison experiment with RT-DETR (Real-Time Detection Transformer), DINOv2, and ViT-T (Vision Transformer-Transducer), the precision of the network model of the present system is improved by 3.0%, 4.7%, and 1.9%, respectively, the recall is improved by 7.8%, 8.2%, and 3.3%, respectively, and the mean average precision is improved by 3.2%, 7.2%, and 0.5%, respectively. The experimental results show that the network model of the present system has significant advantages in the illegal operation battery alarm task, and its performance surpasses that of existing mainstream detection models.

[0224] To further validate the superiority of this system's network model in the task of warning against illegal battery operation, a random selection of community surveillance videos from the test set were fed into the system's network model for detection and monitoring. Overall, the continuous advancements in deep learning and image processing technologies have revolutionized the implementation of illegal battery operation warning systems. Based on these technologies, the system is able to provide efficient and accurate warnings in complex community surveillance environments, offering real-time monitoring and automated analysis, providing strong technical support for safe battery operation in communities and factories.

[0225] The above-mentioned specific implementation methods are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementation methods. Any technical solutions that conform to the above-mentioned specific implementation methods of the present invention and any appropriate changes or substitutions made thereto by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0226] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A battery alarm method for illegal operation, characterized in that: Based on machine vision, the following steps are involved: S1. Obtain a community surveillance video stream, convert the acquired surveillance video stream into an image frame sequence, and perform preprocessing; S2. Utilize a human pose estimation algorithm to extract human body detection frames and key point coordinates from the image frame sequence. By analyzing the bending angle of the human torso and the spatial positional relationship between joints, assess whether the user is bending over to remove or install the battery. The YOLOv10-PoseNet human pose estimation algorithm is used. First, the YOLOv10 object detection framework is used to quickly locate pedestrians in the frame sequence, generating accurate human detection frames to provide input for pose estimation. Subsequently, the PoseNet pose estimation network extracts the coordinates of key points of 17 human body parts, thereby establishing a complete joint space relationship and human skeleton structure. The bending angle of the human torso and the spatial position relationship of the joints are calculated, ultimately enabling logical judgment of the bending-over battery operation behavior. The following are the basic steps for logical judgment of the bending-over battery operation behavior: (2-1) Extraction and preprocessing of human body key point coordinates; Extract the two-dimensional coordinates of the key points of the head, shoulder, hip and knee joints, which are: (X head ,AND head )、(X shoulder ,AND shoulder )、(X hip ,AND hip )、(X knee ,AND knee ); In order to ensure the stability of the behavioral logic judgment process, the shoulder center coordinate is the average of the horizontal and vertical coordinates of the left and right shoulders. The calculation formula is as follows: Among them, (X shoulde_l 、Y shoulder_l )、(X shoulder_r 、Y shoulder_r ) represent the horizontal and vertical coordinates of the left shoulder and right shoulder respectively; (2-2) Calculate the bending angle of the human body trunk; The method for obtaining the bending angle of the human torso is to construct two vectors from the head to the hip and from the hip to the knee, and calculate the angle between them to determine the curvature of the upper body. When the human body is upright, the head, hip and knee are roughly in a vertical line, and the line from the head to the hip is close to vertical. When bending over, the head leans forward, causing the line from the head to the hip to deviate to the horizontal direction. Assumption vector Indicates the direction from head to hip, vector Indicates the direction from the hip to the knee. The angle is calculated as follows: Where θ represents the bending angle between the upper and lower body. The included angle can be obtained by taking the arc cosine value of cos(θ). If θ is less than the preset bending angle threshold angle_blending_threshold, it is considered that the included angle between the upper and lower body is small, indicating that the posture meets the bending feature. (2-3) Calculate the spatial position relationship of the joints; To determine whether the user is bending over to operate the battery illegally, in addition to calculating the bending angle of the human torso, the spatial position relationship of key parts of the human body is analyzed to assist in the judgment of bending; Specifically, the relative height and horizontal position changes of the head, hips, and knees reveal whether the user's upper body tends to lean forward or press downward. Taking the vertical height difference ratio method as an example, the vertical height difference of the head, hips, and knees is used to determine the degree of bending. When the user is standing normally, the vertical height from the head to the hips should be significantly greater than the vertical height from the hips to the knees. A vertical height ratio is defined as: Among them, Y hip 、Y knee Respectively represent the center height of the hips and knees. When standing normally, R y Should be greater than the preset vertical height ratio threshold vertical_ratio_threshold. When bending over, the head is close to the hip, resulting in Y head -Y hip Reduce so that R y Decline, if R y If the value is lower than the preset threshold, it is determined to be a bending action; Considering that the head will move forward significantly when the user bends over, the horizontal distance method is also used to calculate the horizontal distance increment between the head and hip. The formula for calculating the horizontal distance between the head and hip is as follows: D x =|X head -X hip |; The formula for calculating shoulder width is as follows: W s =|X shoulder_l -X shoulder_r |; When the user is standing, D x Smaller, and when bending over, D x Will exceed the normal range if D x If the horizontal distance ratio exceeds the preset threshold horizontal_ratio_threshold, it is considered that the head has moved forward significantly, indicating bending behavior; S3: An integrated battery target detection algorithm uses deep learning technology to identify the unique appearance features of batteries to determine their location within the frame sequence. If a battery is found within the human body detection frame and close to the user's wrist joint, the user is deemed to be illegally carrying a battery. S4. A battery tracking algorithm is introduced to continuously track the battery target. If the battery is not lost within a set time period, it can be confirmed that the user has moved with the battery, and an alarm is issued accordingly.

2. The battery alarm method for illegal operation according to claim 1, characterized in that: In step S3, the integrated battery target detection algorithm uses the YOLOv10 battery target detection algorithm to perform the following logical judgment on the illegal carrying of batteries: (3-1) Intersection determination between battery target and human body detection frame; (3-2) Determination of close interaction between battery target and user; (3-3) Continuous assessment of illegal carrying of batteries.

3. The battery alarm method for illegal operation according to claim 2, characterized in that: In step (3-1), it is assumed that the human body detection frame is a rectangle, and the area is defined by the coordinates of the upper left corner and the lower right corner. That is, the coordinates of the upper left corner of the human body detection frame are (X left , Y top ), the coordinate of the lower right corner is (X right , Y bottom ), and the coordinate center of the battery is (X battery ,Y battery ); The condition for the battery target to fall into the human body detection frame is that the horizontal and vertical coordinates of the battery center point are both within the range of the human body detection frame, that is: X left ≤X battery ≤X right ; AND top ≤Y battery ≤Y bottom ; If both of the above conditions are met, it means that the center point of the battery falls within the human body detection frame, which serves as the basis for judging that the battery is within the user's activity range; In step (3-2), we need to prove that the battery target maintains a close spatial relationship with the user's wrist joint by calculating the distance between the battery center point and the user's wrist joint and determining whether it is lower than the set distance threshold distance_threshold; If the distance between the two is small enough, it is considered that they have a close spatial relationship. Assume that the coordinates of the user's left and right wrist joints are (X wirst_l 、Y wirst_l )、(X wirst_r 、Y wirst_r ); Taking the left wrist joint as an example, use the Euclidean distance formula to calculate the distance between the battery center and the left wrist: Get D l 、D r Then, the shortest distance D between the battery and the wrist is obtained. min : D min =min( D l , D r ); If D min If the distance is less than or equal to a preset distance threshold distance_threshold, it is considered that the battery and the user's wrist joint are in close spatial relationship. The judgment conditions are as follows: D min ≤D threshold ; When this condition is met, it means that the battery is within the operating range of the user's wrist, which serves as the basis for judging that the user is operating the battery; In step (3-3), to improve the accuracy, D is calculated for multiple frames in the time series. min To determine whether this spatial relationship persists, if D min ≤D threshold It is continuously established and can more stably determine whether the user is performing battery operations. Through this distance calculation and threshold judgment, it can more accurately determine whether the battery target is illegally brought into or out of the community by the user.

4. The battery warning method for illegal operation according to claim 3, characterized in that: In step S4, the ByteTrack battery tracking algorithm is used. First, the battery target detection algorithm mentioned above is used to identify the battery targets in the frame sequence and obtain the position of each target. Then, the battery tracking algorithm is used to associate the detection results to form a continuous mobile tracking trajectory. Among them, the Kalman filter is designed to predict and update the spatial position and movement speed state of the battery target. The following are the basic calculation steps of the Kalman filter: (4-1) Initialization; Initialization state estimate X0: initial position state estimate of the battery target; Initial error covariance matrix P0: represents the uncertainty of the initial position estimate of the battery target; (4-2) prediction; State Prediction X k|k-1 : Use dynamic models to predict the state of the battery target at the next moment; Error covariance prediction P k|k-1 : Predict the position error covariance of the battery target at the next moment; (4-3) Update; Calculate the Kalman gain K k : This is a weight used to update the battery target prediction state; Update state estimate X k : Combine the predicted state and the new measurement value to update the battery target state estimate; Update error covariance P k : Update the estimated uncertainty; (4-4) Repeat; Repeat the prediction and update steps until the battery target tracking task is completed; The calculation formula of the Kalman filter prediction step is as follows: X k∣k-1 =F k X k-1∣k-1 +B k U k ; Among them, F k 、B k Represent the state transfer matrix, control input matrix, U k represents the control vector, Q k represents the process noise covariance matrix; The calculation formula for the Kalman filter update step is as follows: P k =(I-K k H k )P k∣k-1 ; Among them, H k 、R k Represent the observation matrix, measurement noise covariance matrix, Z k Represents the actual position measurement value of the battery target at time k; Based on the Kalman filter, ByteTrack also introduced the Hungarian algorithm to solve the data association problem. For those battery target detection results that do not match the existing trajectory, ByteTrack regards them as new targets and creates new tracks for them.

5. The battery alarm method for illegal operation according to claim 4, characterized in that: The basic calculation steps of the Hungarian algorithm are: (1) Construct a cost matrix; (2) Initialization mark; (3) Find independent zero elements; (4) Adjust the cost matrix; (5) Marking the path; (6) Repeat steps (3)-(5); (7) Output matching results; (8) Update trajectory; (9) Repeat the above steps.

6. The battery warning method for illegal operation according to claim 5, characterized in that: In step (1), an association cost is calculated for each battery object in the monitoring video frame and the known trajectory in the previous frame. The association cost is based on a certain metric to determine whether the battery targets in different frames are the same object; the cross-frame spatial distance association cost of the battery target is measured using Euclidean distance. These association costs constitute a cost matrix C. Where rows represent detections in the current frame and columns represent tracks in the previous frame. If the cost of associating the i-th detection with the j-th track is c ij , then C[i][j]=c ij ; In step (2), create two Boolean arrays: assigned_row and assigned_col, which respectively indicate whether the battery target in the current frame and the trajectory in the previous frame have matched, and initialize all elements of these two arrays to False; In step (3), for each element in the cost matrix, if it is the only zero value in its row and column, that is, an independent zero, it is taken as a matching point and the corresponding row and column are marked. If there is no independent zero, proceed to the next step; In step (4), if it is not possible to directly find enough independent zeros to match the battery target with the corresponding trajectory, the cost matrix needs to be adjusted to create more independent zeros; Find the uncovered minimum value, then subtract this value from all uncovered rows and add this value from all uncovered columns, creating new zeros at the same time in order not to change the positions of existing zeros, and then continue looking for independent zeros.

7. The battery warning method for illegal operation according to claim 6, characterized in that: In step (5), if enough independent zeros are still not found after adjustment, it is necessary to mark an augmenting path, starting from an unmarked row, finding a zero in an unmarked column, and then going up along the column to find a zero in another unmarked row, and so on, until a marked row is reached or no more zeros are found. Alternately cancel and add marks on this path to update the matching status of the battery target and the trajectory; In step (6), the steps of finding independent zeros, adjusting the matrix, and marking the path are repeated until all rows or columns are marked, which means that a complete match between the battery target and the trajectory is found; In step (7), finally, by checking the assigned_row and assigned_col arrays, it is determined that the battery target is matched with the trajectory. The detection without a match may be a new battery target, and the unmatched trajectory means that the corresponding battery target has left the monitoring field of view; In step (8), the status information of each track is updated according to the matching results. The newly detected battery target starts a new track. For the old track that loses the match, the preset maximum time threshold max_time_out is used to decide whether to maintain or end the tracking. In step (9), as the monitoring video frame sequence is played, the above process is repeated continuously to achieve continuous battery target tracking.

8. A battery warning device for illegal operation, characterized in that: include: at least one memory and at least one processor; The at least one memory is configured to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 7.

Images