Method and device for warning illegal operation of storage battery

Abstract

The invention relates to the technical field of urban governance algorithms, and particularly provides an alarm method and device for illegal operation of a storage battery, and the method is based on machine vision, and comprises the following steps: S1, obtaining a community monitoring video stream, converting the obtained monitoring video stream into an image frame sequence, and carrying out the preprocessing of the image frame sequence; s2, extracting a human body detection frame and key point coordinates from the image frame sequence by using a human body posture estimation algorithm, and evaluating whether a user has an action of bending down to disassemble or assemble the storage battery; s3, integrating a storage battery target detection algorithm to determine the position of the storage battery in the frame sequence, and judging that the user carries the storage battery illegally; and S4, introducing a storage battery tracking algorithm to continuously track the storage battery target, and determining that the user carries the storage battery to move, and giving an alarm. Compared with the prior art, the intelligent level of illegal storage battery carrying and alarm in the community can be effectively improved, and standardization and safety of community management can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of urban governance algorithms, and specifically provides a method and device for warning of illegal battery operation. Background Art

[0002] With the development of science and technology and the widespread use of electric vehicles, batteries have become the core components of electric vehicles due to their high energy density. However, during the use and management process, illegal operation of batteries frequently occurs, including arbitrary disassembly and private carrying. These behaviors may not only damage the equipment, but also cause serious safety accidents, such as short circuits, fires, and even explosions. Therefore, the development of intelligent alarm methods for illegal battery operations has become an important direction for improving safety management efficiency.

[0003] Traditional violation detection methods mainly rely on manual supervision and simple monitoring equipment to detect violations through real-time observation by monitoring personnel. This method is not only labor-intensive, but also cannot guarantee the accuracy and timeliness of detection, and there is a risk of missed detection and misjudgment. In addition, with the popularization of intelligent equipment, many scenarios have put forward higher requirements for the automation, real-time and accuracy of violation detection, and manual supervision is difficult to meet the needs of real-time and accurate monitoring of violations in places with frequent personnel flow and complex equipment. Summary of the invention

[0004] The present invention aims at the above-mentioned deficiencies of the prior art and provides a highly practical illegal operation battery alarm method.

[0005] A further technical task of the present invention is to provide a battery warning device for illegal operation that is reasonably designed and safe and applicable.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A battery warning method for illegal operation, based on machine vision, has the following steps:

[0008] S1. Obtain a community surveillance video stream, convert the acquired surveillance video stream into an image frame sequence and perform preprocessing;

[0009] S2. Using a human posture estimation algorithm, extract the human body detection frame and key point coordinates from the image frame sequence, and evaluate whether the user bends down to remove or install the battery by analyzing the bending angle of the human body and the spatial position relationship between the joints;

[0010] S3, integrated battery target detection algorithm, uses deep learning technology to identify the unique appearance features of the battery to determine the battery position in the frame sequence. If the battery is found to appear in the human body detection frame and is close to the user's wrist joint position, it can be determined that the user is carrying a battery illegally;

[0011] S4. Introduce a battery tracking algorithm to continuously track the battery target. If the battery is not lost within a set time period, it can be confirmed that the user has moved with the battery, and an alarm is issued accordingly.

[0012] Furthermore, in step S2, the YOLOv10-PoseNet human posture estimation algorithm is used. First, the YOLOv10 target detection framework is used to quickly locate pedestrians in the frame sequence, generate accurate human detection frames, and then provide input for posture estimation. Subsequently, the PoseNet posture estimation network extracts the coordinates of key points of 17 parts of the human body, thereby establishing a complete joint space relationship and human skeleton structure, calculating the human body trunk bending angle and joint space position relationship, and finally realizing the behavior logic judgment of bending over to operate the battery. The following are the basic steps for the behavior logic judgment of bending over to operate the battery:

[0013] (2-1) Extraction and preprocessing of human body key point coordinates;

[0014] (2-2) Calculate the bending angle of the human body trunk;

[0015] (2-3) Calculate the spatial position relationship of the joints.

[0016] Furthermore, in step (2-1), the two-dimensional coordinates of the key points of the head, shoulder, hip and knee joints are extracted, which are:

[0017] (X head , Y head )、(X shoulder , Y shoulder )、(X hip , Y hip )、(X knee , Y knee );

[0018] In order to ensure the stability of the behavior logic judgment process, the shoulder center coordinates are taken as the average of the horizontal and vertical coordinates of the left and right shoulders. The calculation formula is as follows:

[0019]

[0020] Among them, (X shoulde_l , Y shoulder_l )、(X shoulder_r , Y shoulder_r ) represent the horizontal and vertical coordinates of the left shoulder and the right shoulder respectively;

[0021] In step (2-2), the method for obtaining the bending angle of the human body trunk is to construct two vectors from the head to the hip and from the hip to the knee, and calculate the angle between them to determine the bending degree of the upper body. When the human body is upright, the head, hip and knee are roughly on a vertical line, and the line from the head to the hip is close to vertical; when bending over, the head leans forward, causing the line from the head to the hip to deviate to the horizontal direction;

[0022] Assume that the vector Represents the direction from head to hip, vector Indicates the direction from the hip to the knee. The angle is calculated as follows:

[0023]

[0024] Among them, θ represents the bending angle between the upper body and the lower body. The included angle can be obtained by taking the arc cosine value of cos(θ). If θ is less than the preset bending angle threshold angle_blending_threshold, it is considered that the included angle between the upper body and the lower body is small, indicating that the posture meets the bending feature;

[0025] In step (2-3), it is determined whether the user has bent over to operate the battery in violation of regulations. In addition to calculating the bending angle of the human body, the spatial position relationship of key parts of the human body is also analyzed to assist in determining the bending;

[0026] Specifically, the relative height and horizontal position changes of the head, hips and knees reveal whether the user's upper body has a tendency to lean forward or press down. Taking the vertical height difference ratio method as an example, the vertical height difference of the head, hips and knees is used to judge the degree of bending. When the user stands normally, the vertical height from the head to the hip should be significantly greater than the vertical height from the hip to the knee. A vertical height ratio is defined:

[0027]

[0028] Among them, Y hip , Y knee Respectively represent the center height of the double hips and double knees. When standing normally, R y It should be greater than the preset vertical height ratio threshold vertical_ratio_threshold. When bending over, the head is close to the hip, resulting in Y head -Y hip Reduce so that R y If R y If the value is lower than the preset threshold, it is judged as bending action;

[0029] Considering that the head will move forward significantly when the user bends down, the horizontal distance method is also used to calculate the horizontal distance increment between the head and hip. The calculation formula for the horizontal distance between the head and hip is as follows:

[0030] D x =|X head -X hip |;

[0031] The shoulder width calculation formula is as follows:

[0032] W s =|X shoulder_l -X shoulder_r |;

[0033] When the user is standing, D x Smaller, while when bending over, D x Will exceed the normal range if D x If the horizontal distance ratio exceeds the preset threshold horizontal_ratio_threshold, it is considered that the head has moved forward significantly, indicating bending behavior.

[0034] Furthermore, in step S3, the integrated battery target detection algorithm adopts the YOLOv10 battery target detection algorithm to perform the logic judgment of illegal battery carrying behavior as follows:

[0035] (3-1) Intersection determination between battery target and human body detection frame;

[0036] (3-2) Determination of close interaction between battery target and user;

[0037] (3-3) Continuous assessment of illegal battery carrying.

[0038] Furthermore, in step (3-1), it is assumed that the human body detection frame is a rectangle, and the region is defined by the coordinates of the upper left corner and the lower right corner, that is, the coordinates of the upper left corner of the human body detection frame are (X left , Y top ), the coordinate of the lower right corner is (X right , Y bottom ), and the coordinate center of the battery is (X battery ,Y battery ); The condition for the battery target to fall into the human body detection frame is that the horizontal and vertical coordinates of the center point of the battery are within the range of the human body detection frame, that is:

[0039] X left ≤X battery ≤X right ;

[0040] Y top ≤Y battery ≤Y bottom ;

[0041] If both of the above conditions are met, it means that the center point of the battery falls within the human body detection frame, which serves as the basis for judging that the battery is within the user's activity range;

[0042] In step (3-2), after that, to prove that the battery target maintains a close spatial relationship with the user's wrist joint, the distance between the battery center point and the user's wrist joint is calculated, and it is determined whether it is lower than the set distance threshold distance_threshold;

[0043] If the distance between the two is small enough, it is considered that they have a close spatial relationship. Assume that the coordinates of the left and right wrist joints of the user are (X wirst_l , Y wirst_l )、(X wirst_r , Y wirst_r ); Taking the left wrist joint as an example, the distance between the battery center and the left wrist is calculated using the Euclidean distance formula:

[0044]

[0045] Get D l , D r Then, the shortest distance D between the battery and the wrist is obtained. min :

[0046] D min =min(D l ,D r );

[0047] If D min If the distance is less than or equal to a preset distance threshold distance_threshold, it is considered that the battery and the user's wrist joint are in a close spatial relationship. The judgment conditions are as follows:

[0048] D min ≤D threshold ;

[0049] When this condition is met, it means that the battery is within the operable range of the user's wrist, which serves as the basis for judging that the user is operating the battery;

[0050] In step (3-3), to improve accuracy, D is calculated over multiple frames in the time series. min The value of D is used to determine whether this spatial relationship persists. min ≤D threshold It can be established continuously and more stably determine whether the user is performing battery operations. Through this distance calculation and threshold judgment, it can more accurately determine whether the battery target is illegally carried into or out of the community by the user.

[0051] Furthermore, in step S4, the ByteTrack battery tracking algorithm is used. First, the battery target detection algorithm mentioned above is used to identify the battery targets in the frame sequence to obtain the position of each target. Then, the battery tracking algorithm is used to associate the detection results to form a continuous mobile tracking trajectory. The Kalman filter is designed to predict and update the spatial position and moving speed state of the battery target. The following are the basic calculation steps of the Kalman filter:

[0052] (4-1) Initialization;

[0053] Initialization state estimation X0: initial position state estimation of the battery target;

[0054] Initial error covariance matrix P0: represents the uncertainty of the initial position estimate of the battery target;

[0055] (4-2) prediction;

[0056] Status Prediction X k|k-1 : Use the dynamic model to predict the state of the battery target at the next moment;

[0057] Error covariance prediction P k|k-1 : Predict the position error covariance of the battery target at the next moment;

[0058] (4-3) Update;

[0059] Calculate the Kalman gain K k : This is a weight used to update the battery target prediction state;

[0060] Update state estimate X k : Combine the predicted state and the new measurement value to update the battery target state estimate;

[0061] Update error covariance P k : Update the estimated uncertainty;

[0062] (4-4) Repeat;

[0063] Repeat the prediction and update steps until the battery target tracking task is completed;

[0064] The calculation formula of the Kalman filter prediction step is as follows:

[0065] X k∣k-1 =F k X k-1∣k-1 +B k U k ;

[0066]

[0067] Among them, Fk , B k Respectively represent the state transfer matrix, control input matrix, U k represents the control vector, Q k represents the process noise covariance matrix;

[0068] The calculation formula for the Kalman filter update step is as follows:

[0069]

[0070] P k =(IK k H k ) k∣k-1 ;

[0071] Among them, H k , R k Represent the observation matrix, measurement noise covariance matrix, Z k Represents the actual position measurement value of the battery target at time k;

[0072] Based on the Kalman filter, ByteTrack also introduced the Hungarian algorithm to solve the data association problem. For those battery target detection results that do not match the existing trajectory, ByteTrack regards them as new targets and creates new trajectories for them.

[0073] Furthermore, the basic calculation steps of the Hungarian algorithm are:

[0074] (1) Construct a cost matrix;

[0075] (2) Initialization mark;

[0076] (3) Find independent zero elements;

[0077] (4) Adjust the cost matrix;

[0078] (5) Marking the path;

[0079] (6) Repeat steps (3) to (5);

[0080] (7) Output matching results;

[0081] (8) Update trajectory;

[0082] (9) Repeat the above steps.

[0083] Furthermore, in step (1), an association cost is calculated for each battery object in each monitoring video frame and the known trajectory in the previous frame, wherein the association cost is based on a certain metric to determine whether the battery targets in different frames are the same object; the cross-frame spatial distance association cost of the battery target is measured using the Euclidean distance, and these association costs constitute a cost matrix C;

[0084] Where rows represent detections in the current frame and columns represent tracks in the previous frame. If the cost of associating the i-th detection with the j-th track is c ij , then C[i][j]=c ij ;

[0085] In step (2), create two Boolean arrays: assigned_row and assigned_col, which respectively indicate whether the battery target in the current frame and the trajectory in the previous frame have been matched, and initialize all elements of these two arrays to False;

[0086] In step (3), for each element in the cost matrix, if it is the only zero value in the row and column, that is, an independent zero, it is taken as a matching point and the corresponding row and column are marked. If there is no independent zero, proceed to the next step;

[0087] In step (4), if it is not possible to directly find enough independent zeros to match the battery target with the corresponding trajectory, the cost matrix needs to be adjusted to create more independent zeros;

[0088] Find the smallest uncovered value, then subtract this value from all uncovered rows and add this value from all uncovered columns, creating new zeros at the same time in order not to change the positions of existing zeros, and then continue looking for independent zeros.

[0089] Furthermore, in step (5), if enough independent zeros are still not found after adjustment, it is necessary to mark an augmenting path, starting from an unmarked row, finding a zero in an unmarked column, and then going up along the column to find a zero in another unmarked row, and so on, until a marked row is reached or no more zeros are found, and alternately canceling and adding marks on this path to update the matching status of the battery target and the trajectory;

[0090] In step (6), the steps of finding independent zeros, adjusting the matrix, and marking the path are repeated until all rows or columns are marked, which means that a complete match between the battery target and the trajectory is found;

[0091] In step (7), finally, by checking the assigned_row and assigned_col arrays, it is determined that the battery target is matched with the trajectory. The detection without a match may be a new battery target, and the unmatched trajectory means that the corresponding battery target has left the monitoring field of view;

[0092] In step (8), the status information of each track is updated according to the matching result. The newly detected battery target starts a new track. For the old track that loses the match, it is decided to maintain or end the tracking according to the preset maximum time threshold max_time_out.

[0093] In step (9), as the monitoring video frame sequence is played, the above process is repeated continuously to achieve continuous battery target tracking.

[0094] A battery warning device for illegal operation, comprising: at least one memory and at least one processor;

[0095] The at least one memory is used to store a machine-readable program;

[0096] The at least one processor is used to call the machine-readable program to execute a battery warning method for illegal operation.

[0097] Compared with the prior art, the battery warning method and device for illegal operation of the present invention has the following outstanding beneficial effects:

[0098] The present invention constructs an efficient, accurate and highly adaptable intelligent alarm method for illegal battery operation, which not only realizes real-time monitoring and accurate alarm of illegal behaviors in complex scenarios, but also provides reliable technical guarantee for battery safety management, effectively improves the level of illegal carrying of batteries and alarm intelligence in the community, and helps to ensure the standardization and safety of community management. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0100] Attached Figure 1 It is a flowchart of a battery warning method for illegal operation;

[0101] Attached Figure 2 It is an architecture diagram of a battery target detection algorithm in a battery warning method for illegal operation;

[0102] Attached Figure 3 It is a flow chart of a battery tracking algorithm in a battery warning method for illegal operation;

[0103] Attached Figure 4 The invention is a visualization result diagram of an illegal operation battery warning based on machine vision in an illegal operation battery warning method. DETAILED DESCRIPTION

[0104] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0105] A best embodiment is given below:

[0106] like Figure 1-4 As shown, a battery warning method for illegal operation in this embodiment is based on machine vision and has the following steps:

[0107] S1. Obtain a community surveillance video stream, convert the acquired surveillance video stream into an image frame sequence and perform preprocessing;

[0108] S2. Using a human posture estimation algorithm, extract the human body detection frame and key point coordinates from the image frame sequence, and evaluate whether the user bends down to remove or install the battery by analyzing the bending angle of the human body and the spatial position relationship between the joints;

[0109] The YOLOv10-PoseNet human pose estimation algorithm is used. The YOLOv10-PoseNet human pose estimation algorithm combines target detection and human pose estimation technology to identify the key point positions and pose characteristics of the human body in real time. The algorithm first uses the YOLOv10 target detection framework to quickly locate pedestrians in the frame sequence, generate accurate human detection frames, and then provide reliable input for pose estimation.

[0110] Subsequently, the PoseNet posture estimation network extracts the key point coordinates of 17 parts of the human body, thereby establishing a complete joint space relationship and human skeleton structure. In the scenario of illegal battery operation warning, the human posture estimation algorithm obtains the key point positions of the human head, shoulders, hips and knees, and uses this as a basis to calculate the human torso bending angle and joint space position relationship, and finally realizes the behavior logic judgment of bending over to operate the battery. The following are the basic steps for the behavior logic judgment of bending over to operate the battery:

[0111] (2-1) Extraction and preprocessing of human body key point coordinates;

[0112] Extract the two-dimensional coordinates of the key points of the human body. Taking the head, shoulder, hip and knee joints as examples, the coordinates of the key points are: (X head , Y head )、(X shoulder , Y shoulder )、(X hip , Y hip )、(X knee , Y knee ). In order to ensure the stability of the behavior logic judgment process, the shoulder center coordinates are taken as the average of the horizontal and vertical coordinates of the left and right shoulders, and the calculation formula is as follows:

[0113]

[0114]

[0115] Among them, (X shoulde_l , Y shoulder_l )、(X shoulder_r , Y shoulder_r ) represent the horizontal and vertical coordinates of the left shoulder and the right shoulder respectively. The calculation formulas for the center coordinates of the hip, knee, etc. are similar and will not be repeated here.

[0116] (2-2) Calculate the bending angle of the human body trunk;

[0117] The method of the present invention for obtaining the bending angle of the human body trunk is to construct two vectors from the head to the hip and from the hip to the knee, and calculate the angle between them to determine the bending degree of the upper body. When the human body is upright, the head, hip and knee are roughly on a vertical line, and the line from the head to the hip is close to vertical; when bending over, the head leans forward, causing the line from the head to the hip to deviate to the horizontal direction. Assuming the vector Represents the direction from head to hip, vector Indicates the direction from the hip to the knee. The angle is calculated as follows:

[0118]

[0119] Among them, θ represents the bending angle between the upper body and the lower body. The angle can be obtained by taking the arc cosine value of cos(θ). If θ is less than the system preset bending angle threshold angle_blending_threshold (taking 150 degrees as an example), it can be considered that the angle between the upper body and the lower body is small, indicating that the posture meets the bending characteristics. This angle reflects whether the upper body is obviously leaning forward. If it reaches the threshold, it means that the action is likely to involve an illegal bending posture.

[0120] (2-3) Calculate the spatial position relationship of the joints;

[0121] In order to determine whether the user has bent over to operate the battery in violation of regulations, in addition to calculating the bending angle of the human torso, the spatial position relationship of the key parts of the human body can also be analyzed to assist in judging bending. Specifically, the relative height and horizontal position changes of the head, hips, and knees can reveal whether the user's upper body has a tendency to lean forward or press down. Taking the vertical height difference ratio method as an example, the vertical height difference of the head, hips, and knees is used to determine the degree of bending. When the user is standing normally, the vertical height from the head to the hip should be significantly greater than the vertical height from the hip to the knee. Define a vertical height ratio:

[0122]

[0123] Among them, Y hip , Y knee Represents the center height of both hips and knees. y It should be greater than the preset vertical height ratio threshold vertical_ratio_threshold (take 1.5 as an example). When bending over, the head is close to the hip, resulting in Y head -Y hip Reduce so that R y If R y If it is lower than the preset threshold, it can be determined as a bending action.

[0124] Considering that the head will move forward significantly when the user bends, the horizontal distance method can also be used to calculate the horizontal distance increment between the head and hip. The calculation formula for the horizontal distance between the head and hip is as follows:

[0125] D x =|X head -X hip | (5)

[0126] The shoulder width calculation formula is as follows:

[0127] W s =|X shoulder_l -X shoulder_r | (6)

[0128] When the user is standing, D x Smaller, while when bending over, D x Will exceed the normal range. x If the horizontal distance ratio exceeds the preset threshold horizontal_ratio_threshold (taking 0.5 times of shoulder width as an example), it can be considered that the head has obviously moved forward, indicating bending behavior.

[0129] In summary, the human posture estimation algorithm not only improves the system's adaptability to complex scenes through dual analysis of the torso bending angle and the spatial position relationship of the joints, but also effectively filters out misjudgments caused by irrelevant actions, and ultimately achieves accurate logical judgment of illegal behaviors such as bending over to operate the battery.

[0130] S3, integrated battery target detection algorithm, uses deep learning technology to identify the unique appearance features of the battery to determine the battery position in the frame sequence. If the battery is found to appear in the human body detection frame and is close to the user's wrist joint position, it can be determined that the user is carrying a battery illegally;

[0131] YOLOv10 is used as the target detection algorithm, and the traditional non-maximum suppression (NMS) step is abandoned. Through a consistent dual allocation strategy, efficient training and detection reasoning are achieved. This strategy enables YOLOv10 to maintain high-precision battery target detection in the illegal battery operation detection task while significantly reducing the computational overhead, providing a solid foundation for the subsequent logical judgment of illegal battery carrying behavior, battery tracking alarm, etc. The YOLOv10 model architecture is as follows Figure 2 As shown. The YOLOv10 network model mainly consists of the following three parts:

[0132] (a) Backbone (backbone network);

[0133] The Backbone network is the cornerstone of the entire YOLOv10 model and is responsible for extracting features from the input surveillance video frame sequence. For the battery alarm scenario, the Backbone network can effectively capture the basic outline and visual features of the battery, such as shape, size, color, edge contrast, etc., while filtering out irrelevant background, thereby providing visual clues for subsequent steps.

[0134] (b) Neck (neck network);

[0135] The Neck network is located between the Backbone and the Head, and plays a role in multi-scale feature fusion. In YOLOv10, the Neck network is usually composed of modules such as Feature Pyramid Network (FPN) or Path Aggregation Network (PAN), which are used to combine feature maps of different scales. For battery alarm scenarios, the Neck network enables the model to focus on the global contour and detailed features of the battery at the same time, thereby effectively processing battery objects of different sizes, positions and angles. Especially in complex scenes, the multi-scale feature fusion of the Neck network enhances the model's detection capabilities for small batteries and long-distance batteries.

[0136] (c) Head (detection head);

[0137] The Head network is the network part of YOLOv10 used for final detection, which is mainly responsible for target classification and bounding box regression. It receives multi-scale features from the Neck network and generates the final detection results. In the battery warning scenario, the Head network outputs the battery detection box and category confidence as specific prediction results, including the bounding box position, battery category, and confidence score.

[0138] The battery target detection algorithm can accurately identify battery targets in the video frame sequence, ensuring that the system can effectively distinguish between batteries and other objects. By training a large amount of sample data, the deep learning model can continuously optimize its recognition ability, thereby improving the accuracy and reliability of battery detection. Once the battery target is captured by the system, the system will compare its position with the human body detection frame and perform logical judgment on illegal battery carrying behavior.

[0139] The following are the basic steps for logical judgment of illegal battery carrying behavior:

[0140] (3-1) Intersection determination between battery target and human body detection frame;

[0141] Assuming that the human body detection frame is a rectangle, the area is defined by the coordinates of the upper left corner and the lower right corner, that is, the coordinates of the upper left corner of the human body detection frame are (X left , Y top ), the coordinate of the lower right corner is (X right , Y bottom ), and the coordinate center of the battery is (X battery ,Y battery ). The condition for the battery target to fall into the human body detection frame is that the horizontal and vertical coordinates of the center point of the battery are within the range of the human body detection frame, that is:

[0142] X left ≤X battery ≤X right ; (7)

[0143] Y top ≤Y battery ≤Y bottom ; (8)

[0144] If both of the above conditions are met, it means that the center point of the battery falls within the human body detection frame, which can be used as a basis for judging that the battery is within the user's activity range.

[0145] (3-2) Determination of the close interaction relationship between the battery target and the user;

[0146] After that, to prove that the battery target and the user's wrist joint are in a close spatial relationship, we can calculate the distance between the battery center point and the user's wrist joint and determine whether it is lower than the set distance threshold distance_threshold. If the distance between the two is small enough, it can be considered that they have a close spatial relationship. Assume that the coordinates of the user's left and right wrist joints are (X wirst_l , Y wirst_l )、(X wirst_r , Y wirst_r ). Taking the left wrist joint as an example, the distance between the battery center and the left wrist can be calculated using the Euclidean distance formula:

[0147]

[0148] Distance between right wrist and battery D r The calculation formula is similar to this and will not be described in detail here. l , D r After that, the shortest distance D between the battery and the wrist can be further obtained. min :

[0149] D min =min(D l ,D r ) (10)

[0150] If D min If the distance is less than or equal to a preset distance threshold distance_threshold (for example, 0.1 meters), it can be considered that the battery and the user's wrist joint are in a close spatial relationship. The judgment conditions are as follows:

[0151] D min ≤D threshold (11)

[0152] When this condition is met, it means that the battery is within the operable range of the user's wrist, which can be used as a basis for determining that the user is operating the battery.

[0153] (3-3) Continuous determination of illegal battery carrying behavior;

[0154] In addition, to improve accuracy, D can be calculated for multiple frames in the time series. min to determine whether this spatial relationship persists. min ≤D threshold If the distance calculation and threshold judgment are continued, it can be more stably judged whether the user is performing battery operation. Through this distance calculation and threshold judgment, it can be more accurately judged whether the battery target is illegally brought into or out of the community by the user.

[0155] In summary, the battery target detection algorithm deeply learns the appearance features of the battery and can accurately identify the battery target in the frame sequence. If the battery target is detected to fall into the human body detection frame and maintains a close spatial relationship with the user's wrist joint, it is judged that the user is illegally carrying a battery.

[0156] S4. Introduce a battery tracking algorithm to continuously track the battery target. If the battery is not lost within a set time period, it can be confirmed that the user has moved with the battery, and an alarm is issued accordingly;

[0157] The battery tracking algorithm uses the ByteTrack battery tracking algorithm. Unlike traditional algorithms that only focus on high-confidence detection frames, ByteTrack tracks by associating each detection frame. This tracking strategy makes it easy for the ByteTrack algorithm to overcome problems such as mutual occlusion between targets, complex weather conditions, and diverse lighting conditions when facing battery target tracking. The battery tracking algorithm process is as follows Figure 3 shown.

[0158] The core idea of ​​the battery tracking algorithm is to decompose the battery tracking problem into two stages: battery target detection and target trajectory estimation. First, the battery target detection algorithm mentioned above is used to identify the battery targets in the frame sequence and obtain the position of each target. Then, the battery tracking algorithm is used to associate these detection results to form a continuous mobile tracking trajectory. Among them, the Kalman filter is designed to predict and update the spatial position and movement speed of the battery target.

[0159] The following are the basic calculation steps of Kalman filter:

[0160] (4-1) Initialization;

[0161] Initialization state estimation X0: initial position state estimation of the battery target;

[0162] Initial error covariance matrix P0: represents the uncertainty of the initial position estimate of the battery target.

[0163] (4-2) prediction;

[0164] Status Prediction X k|k-1 : Use the system's dynamic model to predict the state of the battery target at the next moment;

[0165] Error covariance prediction P k|k-1 : Predict the position error covariance of the battery target at the next moment.

[0166] (4-3) Update;

[0167] Calculate the Kalman gain K k: This is a weight used to update the battery target prediction state;

[0168] Update state estimate X k : Combine the predicted state and the new measurement value to update the battery target state estimate;

[0169] Update error covariance P k : Update the estimated uncertainty.

[0170] (4-4) Repeat;

[0171] Repeat the prediction and update steps until the battery target tracking task is completed.

[0172] The calculation formula of the Kalman filter prediction step is as follows:

[0173] X k∣k-1 =F k X k-1∣k-1 +B k U k (12)

[0174]

[0175] Among them, F k , B k Respectively represent the state transfer matrix, control input matrix, U k represents the control vector, Q k represents the process noise covariance matrix.

[0176] The calculation formula for the Kalman filter update step is as follows:

[0177]

[0178] P k =(IK k H k ) k∣k-1 (15)

[0179] Among them, H k , R k Represent the observation matrix, measurement noise covariance matrix, Z k Represents the actual target position measurement value of the battery at time k.

[0180] Based on the Kalman filter, ByteTrack also introduced the Hungarian algorithm to solve the data association problem. For those battery target detection results that do not match the existing track, ByteTrack regards them as new targets and creates new tracks for them. The following are the basic calculation steps of the Hungarian algorithm:

[0181] (1) Construct a cost matrix;

[0182] An association cost is calculated for each battery object in each monitoring video frame and the known trajectory in the previous frame. The association cost is usually based on a certain metric, such as Euclidean distance, Mahalanobis distance, or similarity metric of appearance features, to determine whether the battery targets in different frames are the same object. Considering that the calculation of Euclidean distance is relatively simple and fast, the present invention uses Euclidean distance to measure the cross-frame spatial distance association cost of the battery target, which is necessary for the battery multi-target tracking algorithm that processes a large amount of data in real time. The Euclidean distance calculation method is shown in formula (9) and will not be repeated here. These costs constitute a cost matrix C. Among them, the rows represent the detections in the current frame and the columns represent the trajectories in the previous frame. If the association cost between the i-th detection and the j-th trajectory is c ij , then C[i][j]=c ij .

[0183] (2) Initialization mark;

[0184] Create two Boolean arrays: assigned_row and assigned_col, which respectively indicate whether the battery target in the current frame and the trajectory in the previous frame have been matched, and initialize all elements of these two arrays to False.

[0185] (3) Find independent zero elements;

[0186] For each element in the cost matrix, if it is the only zero value in its row and column (i.e., independent zero), it can be used as a matching point and the corresponding row and column are marked. If there is no independent zero, proceed to the next step.

[0187] (4) Adjust the cost matrix;

[0188] If it is not possible to directly find enough independent zeros to match the battery target with the corresponding trajectory, you need to adjust the cost matrix to create more independent zeros. Find the uncovered minimum value, then subtract this value from all uncovered rows and add this value from all uncovered columns. The purpose of this is to not change the position of existing zeros while creating new zeros. Then continue looking for independent zeros.

[0189] (5) Marking the path;

[0190] If enough independent zeros are still not found after the adjustment, an augmenting path needs to be marked. Start at an unmarked row, find a zero in an unmarked column, then go up that column to find a zero in another unmarked row, and so on, until you reach a marked row or no more zeros are found. Alternately unmark and add marks along this path to update the matching status of the battery target and the trajectory.

[0191] (6) Repeat steps (c) to (e);

[0192] Repeat the steps of finding independent zeros, adjusting the matrix, and marking the path until all rows or columns are marked, which means that a perfect match between the battery target and the trajectory is found;

[0193] (7) Output matching results;

[0194] Finally, by checking the assigned_row and assigned_col arrays, we can determine which battery targets are matched with which tracks. Unmatched detections may be new battery targets, while unmatched tracks may mean that the corresponding battery targets have left the monitoring field of view.

[0195] (8) Update trajectory;

[0196] The state information of each track is updated according to the matching result. A newly detected battery target can start a new track. For an old track that loses matching, the present invention decides to maintain or end tracking according to a preset maximum time threshold max_time_out (taking 30 seconds as an example).

[0197] (9) Repeat the above steps;

[0198] As the monitoring video frame sequence is played, the above process is repeated continuously to achieve continuous battery target tracking.

[0199] In addition, the battery tracking algorithm also uses a delay mechanism during data association, allowing low-confidence detection targets to be re-evaluated in the next few frames. This means that the algorithm can recapture the battery target after it disappears briefly, greatly improving the ability to track targets that are blocked between people and vehicles in community scenarios.

[0200] The machine vision-based intelligent battery warning system for illegal operation in the present invention uses a human posture estimation algorithm and a battery target detection algorithm to identify pedestrians and battery targets in real time in the monitoring frame sequence, while the battery tracking algorithm is used to continuously track the detected battery targets and finally issue an alarm. Combining these three technologies, the present invention constructs an efficient intelligent battery warning system for illegal operation. The system can not only judge the compliance of user operation behavior based on battery detection, but also accurately identify long-term illegal behaviors during continuous tracking, greatly improving the reliability and real-time performance of the alarm, and is suitable for communities, factories and other places where battery operations need to be monitored. The process of the machine vision-based intelligent battery warning system for illegal operation is as follows: Figure 1 The detailed steps are as follows:

[0201] a) Community surveillance video stream acquisition and preprocessing

[0202] The video stream of the community scene is captured in real time by a high-definition camera. These video frames are preprocessed, including resizing, normalization, and possible color correction, to ensure that the input image meets the preset requirements of the human pose estimation algorithm and the battery target detection algorithm.

[0203] b) Human pose estimation

[0204] The system uses a human posture estimation algorithm to identify human detection frames and key point coordinates from a frame sequence. By accurately locating and estimating the skeleton key points of pedestrians in the picture, it can accurately identify the body posture and movements of pedestrians. In order to effectively capture the user's potential bending to remove or install the battery, the algorithm comprehensively considers two key indicators: the bending angle of the human torso and the spatial position relationship of the joints. This not only enhances the system's ability to adapt to complex scenes, but also effectively filters out misjudgments caused by irrelevant actions, and ultimately achieves accurate logical judgment of illegal behaviors of bending over to operate the battery.

[0205] c) Battery target detection

[0206] The trained battery target detection model is used as a single-stage battery target detector to identify and locate battery targets in the frame sequence. The model outputs the position bounding box, category label, and confidence score of each battery target. If the battery target is detected to be within the human body detection frame and maintains a close spatial relationship with the user's wrist joint, the system will determine that the user has violated the regulations by carrying a battery.

[0207] d) Battery target tracking

[0208] The battery target tracking algorithm is used to track the battery target across frames. The present invention combines the appearance similarity and motion consistency of the battery target to maintain the target trajectory, and can maintain a good tracking effect even when the battery target is blocked or temporarily out of the field of view. For battery targets that have not been detected for a long time, the present invention will decide whether to terminate the tracking based on the set maximum time threshold max_time_out, thereby avoiding the accumulation of invalid trajectories.

[0209] e) Analyze illegal battery operation and issue warnings

[0210] After data collection and tracking are completed, the system will analyze the compliance of the user's battery operation based on the battery's location, human posture and behavioral characteristics. Once it detects that the user's behavior is inconsistent with regulations, such as illegal battery operation such as disassembly, installation, and carrying, the system will issue a real-time alarm to notify the monitoring personnel to take measures. Through this process, the system realizes efficient identification and alarm of illegal battery operations, with good real-time and high efficiency, and is suitable for battery safety monitoring in communities, factories and other places.

[0211] At this point, the illegal operation battery warning method based on machine vision has been completed.

[0212] Based on the above method, a battery warning device for illegal operation in this embodiment includes: at least one memory and at least one processor;

[0213] The at least one memory is used to store a machine-readable program;

[0214] The at least one processor is used to call the machine-readable program to execute a battery warning method for illegal operation.

[0215] The model training and experimental evaluation of the present invention integrate the use of two different types of data sets. Among them, in the training and evaluation of the human posture estimation model, the present invention uses the COCO open source human posture estimation image data set. This data set is not only large in scale, but also rich in diversity, covering the postures of people of different ages, genders, and body shapes in various daily activities, while also taking into account factors such as occlusion and perspective changes. It is one of the important benchmarks for researching and developing human posture estimation technology. The battery target detection model uses a total of 11,140 image data collected online and recorded on the spot. The recording tools include industrial cameras, drones, surveillance cameras, etc., and the recording sites include communities, factories, schools, etc., ensuring the diversification of data sources and application scenarios. Then, the image data set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1, that is, 8,912 training sets, and 1,114 validation sets and test sets.

[0216] In order to prevent the model from overfitting, the present invention introduces data augmentation technology in the design of the training set. Data augmentation is a method that expands the size and diversity of the training set through multiple input transformations without changing the output label, and has been widely used in the field of computer vision. As an implicit regularization method, data augmentation technology can effectively improve the generalization ability of deep convolutional neural networks, thereby improving the performance of the model on unknown data. In this experiment, the present invention mainly uses common data augmentation methods such as horizontal flipping, random angle rotation and random scaling. By introducing these changes, the image expression is enriched and more possible actual scenes are simulated.

[0217] In this experiment, in order to maximize the utilization of GPU video memory resources and optimize training efficiency, the present invention uniformly adjusts the size of all input images to 640×640 pixels. After a series of experimental observations and analyses, the present invention found that the choice of batchsize has a significant impact on GPU memory utilization and training efficiency. To this end, through multiple iterations and tuning, it was finally determined that the batch size with the best performance was 16. This setting not only makes full use of the GPU's memory resources, but also achieves a good balance between the speed and stability of model training. In addition, in order to enable the model to fully learn the complex features in the data, the present invention sets the training cycle to 300 rounds. This training strategy ensures that the model can undergo sufficient iterations, thereby improving its adaptability to diverse data.

[0218] In order to evaluate the accuracy and stability of the illegal operation battery warning model, precision, recall and mean average precision (mAP) are selected as evaluation indicators. The calculation formulas of each evaluation indicator are as follows:

[0219]

[0220] Among them, True Positive and False Positive represent the number of positive samples and negative samples predicted by the model, respectively, and False Negative represents the number of positive samples predicted by the model as negative. n represents the category of all samples. In this experiment, the target instances to be detected are divided into two categories, namely batteries and pedestrians.

[0221] Table 1 Experimental comparison of battery warning models for illegal operation

[0222]

[0223] Table 1 shows the experimental comparison results of the network model of this system with other representative models in the battery warning task of illegal operation. Specifically, in the battery warning comparison experiment with RT-DETR (Real-Time Detection Transformer), DINOv2, and ViT-T (Vision Transformer-Transducer), the accuracy of the network model of this system increased by 3.0%, 4.7%, and 1.9%, the recall rate increased by 7.8%, 8.2%, and 3.3%, and the average precision increased by 3.2%, 7.2%, and 0.5%. The experimental results show that the network model of this system has significant advantages in the battery warning task of illegal operation, and its performance exceeds the existing mainstream detection models.

[0224] In order to further verify the superiority of the network model of this system in the task of warning the battery for illegal operation, some community monitoring videos in the test set were randomly selected and input into the network model of this system for detection and monitoring. In general, the continuous advancement of deep learning and image processing technology has brought revolutionary changes to the implementation of the illegal operation battery warning system. Based on these technologies, the system can realize efficient and accurate alarm functions in complex community monitoring environments, provide real-time monitoring and automatic analysis, and provide strong technical guarantees for the safe operation of batteries in communities and factories.

[0225] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any technical solutions that conform to the above-mentioned specific implementations of the present invention and any appropriate changes or substitutions made by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0226] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A battery warning method for illegal operation, characterized in that: Based on machine vision, there are the following steps: S1. Obtain a community surveillance video stream, convert the acquired surveillance video stream into an image frame sequence and perform preprocessing; S2. Using a human posture estimation algorithm, extract the human body detection frame and key point coordinates from the image frame sequence, and evaluate whether the user bends down to remove or install the battery by analyzing the bending angle of the human body and the spatial position relationship between the joints; S3, integrated battery target detection algorithm, uses deep learning technology to identify the unique appearance features of the battery to determine the battery position in the frame sequence. If the battery is found to appear in the human body detection frame and is close to the user's wrist joint position, it can be determined that the user is carrying a battery illegally; S4. Introduce a battery tracking algorithm to continuously track the battery target. If the battery is not lost within a set time period, it can be confirmed that the user has moved with the battery, and an alarm is issued accordingly.

2. The battery warning method for illegal operation according to claim 1, characterized in that: In step S2, the YOLOv10-PoseNet human posture estimation algorithm is used. First, the YOLOv10 target detection framework is used to quickly locate pedestrians in the frame sequence and generate accurate human detection frames to provide input for posture estimation. Subsequently, the PoseNet posture estimation network extracts the coordinates of key points of 17 parts of the human body, thereby establishing a complete joint space relationship and human skeleton structure, calculating the human body torso bending angle and joint space position relationship, and finally realizing the behavior logic judgment of bending over to operate the battery. The following are the basic steps for the behavior logic judgment of bending over to operate the battery: (2-1) Extraction and preprocessing of human body key point coordinates; (2-2) Calculate the bending angle of the human body trunk; (2-3) Calculate the spatial position relationship of the joints.

3. The battery warning method for illegal operation according to claim 2 is characterized in that: In step (2-1), the two-dimensional coordinates of the key points of the head, shoulder, hip and knee joints are extracted, which are: (X head ,AND head )、(X shoulder ,AND shoulder )、(X hip ,AND hip )、(X knee ,AND knee ); In order to ensure the stability of the behavior logic judgment process, the shoulder center coordinates are taken as the average of the horizontal and vertical coordinates of the left and right shoulders. The calculation formula is as follows: Among them, (X shoulde_l , Y shoulder_l )、(X shoulder_r , Y shoulder_r ) represent the horizontal and vertical coordinates of the left shoulder and the right shoulder respectively; In step (2-2), the method for obtaining the bending angle of the human body trunk is to construct two vectors from the head to the hip and from the hip to the knee, and calculate the angle between them to determine the bending degree of the upper body. When the human body is upright, the head, hip and knee are roughly on a vertical line, and the line from the head to the hip is close to vertical; when bending over, the head leans forward, causing the line from the head to the hip to deviate to the horizontal direction; Assume vector Represents the direction from head to hip, vector Indicates the direction from the hip to the knee. The angle is calculated as follows: Among them, θ represents the bending angle between the upper body and the lower body. The included angle can be obtained by taking the arc cosine value of cos(θ). If θ is less than the preset bending angle threshold angle_blending_threshold, it is considered that the included angle between the upper body and the lower body is small, indicating that the posture meets the bending feature; In step (2-3), it is determined whether the user has bent over to operate the battery in violation of regulations. In addition to calculating the bending angle of the human body, the spatial position relationship of key parts of the human body is also analyzed to assist in determining the bending; Specifically, the relative height and horizontal position changes of the head, hips and knees reveal whether the user's upper body has a tendency to lean forward or press down. Taking the vertical height difference ratio method as an example, the vertical height difference of the head, hips and knees is used to judge the degree of bending. When the user stands normally, the vertical height from the head to the hip should be significantly greater than the vertical height from the hip to the knee. A vertical height ratio is defined: Among them, Y hip , Y knee Respectively represent the center height of the double hips and double knees. When standing normally, R y It should be greater than the preset vertical height ratio threshold vertical_ratio_threshold. When bending over, the head is close to the hip, resulting in Y head -Y hip Reduce so that R y If R y If the value is lower than the preset threshold, it is judged as bending action; Considering that the head will move forward significantly when the user bends down, the horizontal distance method is also used to calculate the horizontal distance increment between the head and hip. The calculation formula for the horizontal distance between the head and hip is as follows: D x =|X head -X hip |; The shoulder width calculation formula is as follows: W s =|X shoulder_l -X shoulder_r |; When the user is standing, D x Smaller, while when bending over, D x Will exceed the normal range if D x If the horizontal distance ratio exceeds the preset threshold horizontal_ratio_threshold, it is considered that the head has moved forward significantly, indicating bending behavior.

4. The battery warning method for illegal operation according to claim 3 is characterized in that: In step S3, the integrated battery target detection algorithm adopts the YOLOv10 battery target detection algorithm, and performs the logical judgment of illegal battery carrying behavior as follows: (3-1) Intersection determination between battery target and human body detection frame; (3-2) Determination of close interaction between battery target and user; (3-3) Continuous assessment of illegal battery carrying.

5. The battery warning method for illegal operation according to claim 4 is characterized in that: In step (3-1), it is assumed that the human body detection frame is a rectangle, and the region is defined by the coordinates of the upper left corner and the lower right corner, that is, the coordinates of the upper left corner of the human body detection frame are (X left , Y top ), the coordinate of the lower right corner is (X right , Y bottom ), and the coordinate center of the battery is (X battery ,Y battery ); The condition for the battery target to fall into the human body detection frame is that the horizontal and vertical coordinates of the center point of the battery are within the range of the human body detection frame, that is: X left ≤X battery ≤X right ; AND top ≤And battery ≤And bottom ; If both of the above conditions are met, it means that the center point of the battery falls within the human body detection frame, which serves as the basis for judging that the battery is within the user's activity range; In step (3-2), after that, to prove that the battery target maintains a close spatial relationship with the user's wrist joint, the distance between the battery center point and the user's wrist joint is calculated, and it is determined whether it is lower than the set distance threshold distance_threshold; If the distance between the two is small enough, it is considered that they have a close spatial relationship. Assume that the coordinates of the left and right wrist joints of the user are (X wirst_l , Y wirst_l )、(X wirst_r , Y wirst_r ); Taking the left wrist joint as an example, the distance between the battery center and the left wrist is calculated using the Euclidean distance formula: Get D l , D r Then, the shortest distance D between the battery and the wrist is obtained. min : D min =min(D l ,D r ); If D min If the distance is less than or equal to a preset distance threshold distance_threshold, it is considered that the battery and the user's wrist joint are in a close spatial relationship. The judgment conditions are as follows: D min ≤D threshold ; When this condition is met, it means that the battery is within the operable range of the user's wrist, which serves as the basis for judging that the user is operating the battery; In step (3-3), to improve accuracy, D is calculated over multiple frames in the time series. min The value of D is used to determine whether this spatial relationship persists. min ≤D threshold It can be established continuously and more stably determine whether the user is performing battery operations. Through this distance calculation and threshold judgment, it can more accurately determine whether the battery target is illegally carried into or out of the community by the user.

6. A battery warning method for illegal operation according to claim 5, characterized in that: In step S4, the ByteTrack battery tracking algorithm is used. First, the battery target detection algorithm mentioned above is used to identify the battery targets in the frame sequence to obtain the position of each target. Then, the battery tracking algorithm is used to associate the detection results to form a continuous mobile tracking trajectory. The Kalman filter is designed to predict and update the spatial position and moving speed state of the battery target. The following are the basic calculation steps of the Kalman filter: (4-1) Initialization; Initialization state estimation X0: initial position state estimation of the battery target; Initial error covariance matrix P0: represents the uncertainty of the initial position estimate of the battery target; (4-2) prediction; Status Prediction X k|k-1 : Use the dynamic model to predict the state of the battery target at the next moment; Error covariance prediction P k|k-1 : Predict the position error covariance of the battery target at the next moment; (4-3) Update; Calculate the Kalman gain K k : This is a weight used to update the battery target prediction state; Update state estimate X k : Combine the predicted state and the new measurement value to update the battery target state estimate; Update error covariance P k : Update the estimated uncertainty; (4-4) Repeat; Repeat the prediction and update steps until the battery target tracking task is completed; The calculation formula of the Kalman filter prediction step is as follows: X k∣k-1 =F k X k-1∣k-1 +B k U k ; Among them, F k , B k Respectively represent the state transfer matrix, control input matrix, U k represents the control vector, Q k represents the process noise covariance matrix; The calculation formula for the Kalman filter update step is as follows: P k =(I-K k H k )P k∣k-1 ; Among them, H k , R k Represent the observation matrix, measurement noise covariance matrix, Z k Represents the actual position measurement value of the battery target at time k; Based on the Kalman filter, ByteTrack also introduced the Hungarian algorithm to solve the data association problem. For those battery target detection results that do not match the existing trajectory, ByteTrack regards them as new targets and creates new trajectories for them.

7. The battery warning method for illegal operation according to claim 6 is characterized in that: The basic calculation steps of the Hungarian algorithm are: (1) Construct a cost matrix; (2) Initialization mark; (3) Find independent zero elements; (4) Adjust the cost matrix; (5) Marking the path; (6) Repeat steps (3) to (5); (7) Output matching results; (8) Update trajectory; (9) Repeat the above steps.

8. The battery warning method for illegal operation according to claim 7 is characterized in that: In step (1), an association cost is calculated for each battery object in each monitoring video frame and the known trajectory in the previous frame. The association cost is based on a certain metric to determine whether the battery targets in different frames are the same object; the cross-frame spatial distance association cost of the battery target is measured using Euclidean distance, and these association costs constitute a cost matrix C; Where rows represent detections in the current frame and columns represent tracks in the previous frame. If the cost of associating the i-th detection with the j-th track is c ij , then C[i][j]=c ij ; In step (2), create two Boolean arrays: assigned_row and assigned_col, which respectively indicate whether the battery target in the current frame and the trajectory in the previous frame have been matched, and initialize all elements of these two arrays to False; In step (3), for each element in the cost matrix, if it is the only zero value in the row and column, that is, an independent zero, it is taken as a matching point and the corresponding row and column are marked. If there is no independent zero, proceed to the next step; In step (4), if it is not possible to directly find enough independent zeros to match the battery target with the corresponding trajectory, the cost matrix needs to be adjusted to create more independent zeros; Find the smallest uncovered value, then subtract this value from all uncovered rows and add this value from all uncovered columns, creating new zeros at the same time in order not to change the positions of existing zeros, and then continue looking for independent zeros.

9. The battery warning method for illegal operation according to claim 8, characterized in that: In step (5), if enough independent zeros are still not found after adjustment, it is necessary to mark an augmentation path, starting from an unmarked row, finding a zero in an unmarked column, and then going up along the column to find a zero in another unmarked row, and so on, until a marked row is reached or no more zeros are found. Alternately cancel and add marks on this path to update the matching status of the battery target and the trajectory; In step (6), the steps of finding independent zeros, adjusting the matrix, and marking the path are repeated until all rows or columns are marked, which means that a complete match between the battery target and the trajectory is found; In step (7), finally, by checking the assigned_row and assigned_col arrays, it is determined that the battery target is matched with the trajectory. The detection without a match may be a new battery target, and the unmatched trajectory means that the corresponding battery target has left the monitoring field of view; In step (8), the status information of each track is updated according to the matching result. The newly detected battery target starts a new track. For the old track that loses the match, it is decided to maintain or end the tracking according to the preset maximum time threshold max_time_out. In step (9), as the monitoring video frame sequence is played, the above process is repeated continuously to achieve continuous battery target tracking.

10. A battery warning device for illegal operation, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 9.

Images