Energy feedback elevator car balance state monitoring method based on computer vision

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an energy feedback elevator car balance state monitoring method based on computer vision, which comprises the following steps of: calculating barycentric coordinates and instantaneous speed of each passenger in an elevator car frame by frame through a computer vision technology, and introducing a time decay factor to quantify the behavior of the passenger; obtaining a first offset degree; further, the variance of the first offset degrees of all the passengers is used as a dynamic behavior risk, and the total offset degree representing the overall balance state of the lift car is calculated by integrating the overload risk and the static unbalance loading risk; finally, an activation voltage threshold value of the energy feedback device is dynamically set according to the total offset degree, and energy feedback strategy adjustment with safety and optimal energy efficiency is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a computer vision-based energy feedback elevator car balance state monitoring method. BACKGROUND

[0002] When the elevator is in the light load uplink stage or heavy load downlink stage, the mechanical energy generated is converted into regenerative electric energy by the traction machine inside the elevator, and the regenerative electric energy is stored in the capacitor after being rectified by the frequency converter, resulting in continuous rise of the capacitor voltage. When the capacitor voltage reaches the preset threshold value, the system will start the braking resistor to convert the regenerative electric energy into heat energy, which will cause the temperature of the elevator and other surrounding equipment to rise and the energy consumption to increase. The energy feedback device of the elevator automatically starts when the elevator is in the power generation state, and converts the regenerative electric energy into alternating current with the same frequency and phase as the power grid through inverter technology, and feeds back to the power grid for recycling, which significantly reduces the heating of the braking resistor, thereby reducing the temperature of the elevator and other surrounding equipment and saving energy consumption.

[0003] The energy feedback device of the prior art usually adopts a static threshold triggering mechanism: only when the voltage of the internal capacitor rises to a certain fixed, higher preset value due to the injection of regenerative electric energy, the energy feedback function is started. The defect of this mode is that it cannot predict the upcoming energy impact; when the passengers suddenly move, causing the car balance state to change dramatically, a large amount of regenerative electric energy will be generated instantaneously, and the capacitor voltage may exceed the starting threshold of the energy feedback in a very short time, even triggering the braking resistor at a higher voltage point. At this time, not only a large amount of heat will be generated by the braking resistor, but also the recoverable energy will be wasted, thereby significantly reducing the energy feedback efficiency of the elevator energy feedback system. SUMMARY

[0004] In order to solve the technical problems of the prior art that the energy feedback device for adjusting the elevator energy feedback system cannot predict the upcoming energy impact and is difficult to capture the sudden movement of passengers causing the car balance state to change dramatically, resulting in a significant reduction in the energy feedback efficiency of the elevator energy feedback system, the present application provides a computer vision-based energy feedback elevator car balance state monitoring method.

[0005] In a first aspect, the present application provides a computer vision-based energy feedback elevator car balance state monitoring method, comprising: obtaining the center of gravity coordinate vector of each passenger in each frame of image according to the three-dimensional space coordinate vector of each passenger corresponding key point in each frame of image; obtaining the center of gravity instantaneous speed of each passenger in each frame of image according to the ratio of the Euclidean distance of the center of gravity coordinate vector of the passenger between the current frame and the previous frame to the time interval between the video frames; obtaining the time decay factor according to the difference between the current time and the time when the passenger leaves the elevator car; weighting the difference between the center of gravity instantaneous speed of each passenger in the current frame and the previous frame of image according to the time decay factor to obtain the first offset degree corresponding to each passenger; obtaining the dynamic behavior risk corresponding to each frame of image according to the variance of the first offset degree corresponding to each passenger, combining the overload risk and the static unbalanced load risk corresponding to each frame of image to obtain the total offset degree of the elevator car; obtaining the energy feedback activation voltage threshold to be set by the energy feedback device controller after receiving the current frame of image, so as to dynamically adjust the energy feedback strategy.

[0006] The present application realizes high-precision monitoring of the balance state of the car by analyzing the key point and center of gravity motion state of the passengers in the elevator car in real time through computer vision technology, dynamically evaluating the behavior offset degree of each passenger in combination with the time decay factor, and comprehensively considering the overload risk and the static unbalanced load risk. Furthermore, the activation voltage threshold of the energy feedback device is adaptively adjusted according to the total offset degree, which not only ensures the safety of elevator operation, but also improves the response accuracy and energy saving efficiency of the energy feedback system.

[0007] Preferably, the center of gravity coordinate vector of each passenger in each frame of image is obtained, comprising: ; in the formula, is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the index of the passenger; is the index of the image frame; is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the set of three-dimensional space coordinates of the key point that can be successfully detected in the first frame of image; is the index of a key point in the set of three-dimensional space coordinates is the weight weight corresponding to the body part represented by the key point is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image;

[0008] The method assigns weight weights conforming to the actual mass distribution of different key points of the human body, including but not limited to the head, torso, limbs , and performs a weighted average calculation based on the three-dimensional coordinates of all detectable key points, thereby greatly improving the biomechanical accuracy and physical authenticity of the center of gravity coordinate estimation. Compared with simple arithmetic average, this method can more accurately reflect the true mass center position of the passenger, laying a reliable physical foundation for subsequent accurate calculation of the motion state and overall balance risk of the car. At the same time, this method has fault tolerance for the case where part of the key points are blocked or detection fails. As long as the set is not empty, effective calculation can be performed, ensuring the robustness and usability of the system in complex actual environments.

[0009] Preferably, the obtaining of the first offset degree corresponding to each passenger comprises: ; in the formula, is the first offset degree corresponding to the passenger; is the index of the passenger; is the index of the image frame of the passenger; is the number of image frames collected between the passenger entering the elevator car and leaving the elevator car; is a time decay factor; is the instantaneous velocity of the center of gravity of the passenger in the first image frame; is the instantaneous velocity of the center of gravity of the passenger in the first image frame; is the instantaneous velocity of the center of gravity of the passenger in the first image frame; is the absolute value symbol. The method introduces a time decay factor to perform an exponential weighted summation on the sudden change of the instantaneous velocity of the center of gravity of the passenger between all consecutive frames during the entire boarding process

[0010] , thereby quantifying the first offset degree. This not only captures the entire dynamic history of the passenger's behavior, but also gives higher weight to recent behavior than to long-term behavior, which conforms to the physical intuition and actual situation that "recent actions have a greater impact on the current car balance state". This processing method can effectively distinguish between short-term, occasional small movements and sustained, dangerous actions, significantly improving the perception sensitivity and evaluation accuracy of the system for real risk behaviors, while avoiding the interference of irrelevant past movements on the current state judgment, enhancing the timeliness and reliability of risk assessment, and providing more accurate input for subsequent overall risk aggregation.

[0011] ​​Preferably, the step of obtaining the total offset of the elevator car based on the dynamic behavior risk, overload risk, and static off-center load risk of the elevator car corresponding to each frame of the image includes: a first weight. Second weight Third weight The specific value can be set according to the actual application scenario and needs, and the following conditions must be met: And the first weight Second weight Third weight The range of values ​​is , will the first weight Second weight Third weight Set to respectively Among them, respectively using The overload risk, static off-center load risk, and dynamic behavior risk of the elevator car corresponding to each frame of the image are weighted and summed to obtain the total offset of the elevator car corresponding to each frame of the image.

[0012] This method achieves a comprehensive assessment of the elevator car's balance by weightedly fusing three core indicators: dynamic behavioral risk (variance of individual passenger deviation), static off-center load risk, and overload risk. It not only considers the static factors of passenger number and distribution but also innovatively introduces dynamic risk, representing the consistency of collective passenger behavior. Using variance, it can keenly capture anomalies in the behavior of all passengers in the cabin: even if the absolute value of individual deviation is small, if inconsistent or chaotic passenger behavior shows a trend, the system will identify it as an increased risk, greatly enhancing the early warning capability for dangerous group behaviors such as pushing. Finally, through configurable weights, the system can flexibly adjust the emphasis on each risk factor according to different safety strategies, thereby outputting a total deviation index that reflects instantaneous state and trend changes, providing precise control basis for the energy feedback device.

[0013] Preferably, after receiving the current frame image, the energy feedback device controller sets the energy feedback activation voltage threshold to be set, including: ;in, The energy feedback device controller receives the first After the frame image, the energy feedback activation voltage threshold to be set; It is the standard activation voltage threshold; It is the limiting activation voltage threshold; Based on the natural constant An exponential function with base 0; It is the first The total offset of the elevator car corresponding to the frame image The normalized value can be processed using maximum and minimum value normalization.

[0014] This method uses an exponential function-based mapping model to nonlinearly transform the normalized total offset into an energy feedback activation voltage threshold, achieving optimal coordinated control of safety and energy efficiency: when the car is in good balance, the system automatically sets a lower activation voltage threshold, allowing the energy feedback device to start earlier, maximizing the recovery of braking energy and improving energy efficiency; when the risk of car imbalance increases, the threshold rises exponentially, forcing the elevator to prioritize a smoother, more conservative braking strategy, placing operational safety and equipment protection first. This allows the system to not only cope with sudden imbalance risks and ensure passenger safety, but also tap into the maximum energy-saving potential during daily operation, significantly improving the overall intelligence level of the energy feedback elevator.

[0015] The beneficial effects of this invention are as follows: This invention captures video streams in real time using cameras deployed on the top of the car, and uses a deep learning model to extract key points of each passenger's body in each frame of the image, thereby calculating their three-dimensional spatial center of gravity coordinates; by using the ratio of the Euclidean distance of the passenger's center of gravity coordinates between consecutive frames to the time interval, the instantaneous movement rate of their center of gravity is accurately obtained; to distinguish the impact of recent and early behaviors, a time decay factor is innovatively introduced to weight the sudden change in the passenger's center of gravity rate between the current frame and the previous frame, thereby quantifying the degree of first offset caused by each passenger's movement, pushing, and other behaviors; by calculating the variance of the offset degree of all passengers, the collective consistency of dynamic behavior risks in the car is effectively characterized; in addition, the system synchronously integrates the overload risk based on the number of passengers and the static off-center load risk calculated based on the passenger's position distribution, and finally aggregates them into a total offset degree index that comprehensively reflects the real-time balance state of the car; the energy feedback device controller dynamically sets the optimal activation voltage threshold based on this index: lowering the threshold to maximize energy recovery efficiency when the balance state is good; raising the threshold to prioritize operational safety and equipment life when the risk of imbalance increases. This method represents a leap from passive response to active prediction, significantly improving the adaptive capability and energy-saving benefits of the energy feedback system while ensuring the safe and stable operation of the elevator. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a computer vision-based method for monitoring the balance state of an energy-feedback elevator car according to the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0019] This invention discloses a computer vision-based method for monitoring the balance state of an energy-feedback elevator car, referring to... Figure 1 This includes steps S1 to S4: S1. Preprocess the passenger video stream inside the elevator car to obtain the three-dimensional spatial coordinate vector of the key points of the passenger in each frame; obtain the total weight of the load in each frame based on the gravity sensor.

[0020] It should be noted that this step aims to accurately obtain the dynamic three-dimensional structural information of each passenger in the elevator car from single-view video data, so as to provide high-precision data input for subsequent calculation of the dynamic offset of the center of gravity.

[0021] Specifically, a high-definition camera is installed at any corner inside the elevator car. The camera captures video streams of passengers inside the elevator car at preset time intervals and decomposes these video streams into a series of continuous video frames; wherein the time interval between video frames is... The specific values ​​can be set according to the actual application scenario and requirements, and the time interval between video frames... The range of values ​​is set to .

[0022] It should be noted that if the time interval between video frames is set too large, such as greater than 1 second, the position and posture of the person between adjacent frames will change too much, making it impossible to establish a smooth motion trajectory. This makes it difficult to accurately determine the intention and type of the action, resulting in a significant increase in the false alarm rate and false negative rate. If the time interval between video frames is set too small, it will generate a massive amount of redundant data, increasing computational energy consumption. Therefore, this invention sets the time interval between video frames... Set as .

[0023] Furthermore, each frame of the acquired image is preprocessed. The preprocessing process includes: using a pre-calibrated camera intrinsic parameter matrix to correct the image distortion in order to eliminate geometric distortion caused by the lens; then, using a Gaussian filtering algorithm to smooth the corrected image in order to suppress random noise introduced by the image sensor, and finally generating a clear and distortion-free image as input for subsequent analysis.

[0024] Specifically, obtaining a pre-trained YOLOv5 deep learning object detection network involves: firstly, pre-training on the existing general COCO dataset to enable the model to acquire basic feature extraction and object detection capabilities; then, using a specially collected and finely annotated elevator car scene image dataset for transfer learning and fine-tuning. This dataset covers various elevator models, lighting conditions, passenger density, and postures, such as standing, squatting, facing away, and occlusion, so that the model can adapt well to the complex application scenarios inside the elevator, significantly improving the detection accuracy and robustness, thus obtaining a pre-trained YOLOv5 deep learning object detection network. Furthermore, the pre-processed, clear image is input into the pre-trained YOLOv5 deep learning object detection network. Through forward propagation in the network, an output set containing multiple initial predicted bounding boxes is obtained. Each box includes its normalized center coordinates, width, height, confidence score, and class probability. To eliminate redundant detections of the same passenger, a non-maximum suppression algorithm is applied to filter overlapping redundant detection boxes, suppressing those redundant boxes whose overlap with the highest-scoring box exceeds a set threshold. Finally, a series of accurate and unique passenger bounding boxes are output, and the bounding box region of each passenger is cropped from each frame of the image accordingly, providing high-quality input for subsequent keypoint detection.

[0025] Specifically, within the bounding box region of each passenger in each frame of the image, a set of predefined key points for each passenger in each frame of the image are obtained through the OpenPose human pose estimation algorithm, including but not limited to key points such as head, torso, thigh, calf, and arm.

[0026] Furthermore, taking the center of the elevator car floor as the origin... The floor is The direction perpendicular to the elevator car floor upwards is... Axis, construct a spatial coordinate system; through pre-calibrated camera, obtain the transformation matrix from the camera coordinate system to the spatial coordinate system; through the transformation matrix, map the key points of each passenger to the spatial coordinate system, and thus obtain the three-dimensional spatial coordinate vector of the key points corresponding to each passenger.

[0027] It should be noted that since the mass of the human body is not uniformly distributed, different body parts contribute differently to the overall center of gravity. Therefore, it is necessary to assign weights to key points representing different body parts. For example, the weights for body segments belonging to the head, torso, thigh, calf, and arm are set to 0.06, 0.4, 0.3, 0.2, and 0.04, respectively. It should be clarified that since the thigh, calf, and arm are usually divided into left and right parts, the weight settings here are based on the default that each of the left and right parts corresponds to half of the weight of the entire part. For example, the weights for the left and right arms are 0.02 each.

[0028] After the above processing, for each frame of the video stream, a set of key points for each passenger in the car can be obtained. Each key point contains its three-dimensional spatial coordinate vector in the spatial coordinate system, as well as the weight weight of the body segment it represents. This structured data will be used as input for the subsequent steps to dynamically calculate the overall center of gravity of the car.

[0029] Meanwhile, a gravity sensor is installed at the bottom of the elevator car, setting the total weight of the empty elevator car to 0, and collecting the total weight of the load inside the elevator car in real time for each frame of the image.

[0030] Among them, image distortion correction, Gaussian filtering algorithm, YOLOv5 deep learning object detection network and OpenPose human pose estimation algorithm are all existing technologies, and will not be elaborated here.

[0031] S2. Obtain the centroid coordinate vector based on the three-dimensional spatial coordinate vector of the passenger's key points; obtain the instantaneous centroid velocity based on the Euclidean distance between the centroid coordinate vectors of adjacent frames; and obtain the first offset degree of each passenger by weighting with a time decay factor.

[0032] It should be noted that this step aims to analyze the three-dimensional spatial coordinates of passenger key points output by S1. By accurately calculating the center of gravity trajectory, instantaneous velocity and acceleration changes of individual passengers during the elevator ride, and introducing a time decay factor, unstable behaviors closer to the moment of departure are given higher weights, and finally the first degree of offset is obtained, which is used to quantify the risk of abnormal movement.

[0033] It's important to note that the passenger's center of gravity is the focal point of their mass distribution and the most crucial physical quantity for analyzing their macroscopic motion. However, in a crowded elevator environment, some key body points are often undetectable due to occlusion. Therefore, it's necessary to use the sum of weights for all visible body parts in the current frame. This gives the center of gravity calculation an adaptive capability to occlusion: when a part of the passenger's body is occluded, including but not limited to the legs, the formula automatically normalizes the calculation based on the weights of the remaining visible parts, including but not limited to the head and torso. If no key points of the passenger are detected in the image, the passenger's center of gravity coordinate vector is deemed uncalcifiable, its first offset is set to 0, and it is not included in subsequent dynamic behavior risk calculations. This ensures that even when key points are partially missing, the calculated center of gravity is still an accurate estimate based on the current visible mass distribution, greatly improving the algorithm's robustness in real-world complex scenarios.

[0034] Specifically, passenger information is obtained through step S1. In the In a frame image, the set of three-dimensional spatial coordinates of key points that can be successfully detected is denoted as . ; will passengers Key points In the The three-dimensional spatial coordinate vector in the frame image is denoted as Then calculate the passengers In the The centroid coordinate vector in the frame image is: ; In the formula, Passengers In the The centroid coordinate vector in the frame image; It is an index for passengers; It is the index of the image frame; Passengers In the The set of three-dimensional spatial coordinates of key points that can be successfully detected in a frame image; It is a set of three-dimensional spatial coordinates An index of a key point; This is the key point. The weight weight corresponding to the body part represented; Passengers Key points In the The three-dimensional spatial coordinate vector in the frame image.

[0035] Specifically, to obtain passengers In the Centroid coordinate vector in frame image Then the passenger In the The instantaneous velocity of the centroid in the frame image is: ; In the formula, Passengers In the The instantaneous velocity of the center of gravity in a frame image; It is an index for passengers; Passengers The index of the image frame; It is the time interval between video frames; Passengers In the The centroid coordinate vector in the frame image; Indicates passenger In the Centroid coordinate vector in frame image The Euclidean distance between them.

[0036] Among them, passengers In the The greater the instantaneous velocity of the center of gravity in the frame image, the faster the passenger is moving at that moment; when the passenger is stationary, the passenger's velocity is... In the The instantaneous velocity of the center of gravity in the frame image approaches 0; when passengers move around or make large movements inside the car, the passenger... In the The instantaneous velocity of the center of gravity in the frame image will increase significantly.

[0037] It should be noted that because the elevator pauses noticeably when passengers need to exit the car to reach their target floor, the likelihood of passenger instability is higher at this time, meaning the instantaneous velocity of the passenger's center of gravity is more likely to change between adjacent frames. Conversely, the elevator is more stable when passengers enter the car, making passenger instability less likely, and the instantaneous velocity of the passenger's center of gravity less likely to change between adjacent frames. Therefore, a smaller weighting factor should be assigned to moments further away from exiting the elevator, and a larger weighting factor should be assigned to moments closer to exiting the elevator.

[0038] Furthermore, based on the current time and passengers The difference in the time of exiting the elevator car, and the attenuation coefficient, are used to obtain passenger information. Corresponding time decay factor: ; In the formula, It is the time decay factor; Based on the natural constant An exponential function with base 0; Indicates the current moment; Indicates passenger The moment of leaving the elevator car; It is the attenuation coefficient; It is the current time and passengers The difference in the time of leaving the elevator car.

[0039] Time decay factor In the middle, when The further away At that time, it reflects the difference as soon as the passenger enters the elevator car. The time decay factor is very large, close to 0, indicating that the further away from the moment of leaving the elevator, the less important the change in the instantaneous velocity of the passenger's center of gravity between adjacent frames; when The closer At this time, it indicates that the passenger is about to leave the elevator car, and the difference is... The time decay factor is very small, close to 1, indicating that the closer to the moment of leaving the elevator, the more important the change in the instantaneous velocity of the passenger's center of gravity between two adjacent frames. The YOLOv5 deep learning object detection network can be used to track whether passengers enter or leave the elevator car, so as to obtain the time when passengers enter or leave the elevator car.

[0040] It should be noted that the attenuation coefficient It is a key parameter controlling the rate at which weights decay over time, directly affecting the current time and passenger... Difference in the time of leaving the elevator car Sensitivity, attenuation coefficient The larger the value, the faster the weight decays, and the greater the difference. Very small, the closer to Frames with higher weights are assigned to frames with higher attenuation coefficients. The smaller the value, the slower the weight decay, even with differences. The value is relatively large, and the weight is still retained to a certain extent; the attenuation coefficient The specific value can be set based on historical data statistics of abnormal passenger behaviors, such as the time distribution of staggering, swaying, or falling. This invention uses an attenuation coefficient. Set as This ensures a significant increase in weighting 1 to 2 seconds before leaving the elevator, preventing excessive concentration that could lead to early behaviors being ignored.

[0041] It should be noted that, compared to drastic changes in velocity during uniform motion, such as the high acceleration during a bump, the instability of a passenger's body inside the elevator car, such as staggering, swaying, or falling, is more evident. Therefore, it is necessary to obtain the first degree of offset that can describe the instability of a passenger's body inside the elevator car based on the degree of fluctuation in the instantaneous velocity of the center of gravity.

[0042] Specifically, to obtain passengers The instantaneous velocity of the center of gravity in each frame of the image, according to the passenger The degree of fluctuation in the instantaneous velocity of the center of gravity in consecutive frame images is used to obtain passenger information. The corresponding first offset degree: ; In the formula, Passengers The corresponding first degree of offset reflects the instability of the passenger's body inside the elevator car; It is an index for passengers; Passengers The index of the image frame; From passengers The number of image frames captured between entering and leaving the elevator car; It is the time decay factor; Passengers In the The instantaneous velocity of the center of gravity in a frame image; Passengers In the The instantaneous velocity of the center of gravity in a frame image; Indicates passenger In the Instantaneous velocity of the center of gravity in the frame image and passenger In the The difference in instantaneous velocity between the centers of gravity in the frame image; It is the absolute value symbol.

[0043] in, It measures the smoothness of passenger movement; if the passenger remains stationary or moves at a constant speed, The value will be close to 0; any sudden acceleration, deceleration, or change of direction will cause it to... A peak appears; The higher the value, the more drastic the changes in the passenger's motion state and the higher the instability of the body.

[0044] When passengers In the Instantaneous velocity of the center of gravity in the frame image and passenger In the Difference between instantaneous rates of the center of gravity in frame images A larger value indicates more drastic changes in the passenger's motion state, higher physical instability, and a larger time decay factor. The greater the corresponding first offset, the smaller the offset, and vice versa.

[0045] Furthermore, based on the above operations, the first offset degree corresponding to each passenger is obtained.

[0046] S3. Based on the first degree of offset of each passenger, obtain the dynamic behavior risk corresponding to each frame of the image. Combine the overload risk and static off-center load risk corresponding to each frame of the image to obtain the total degree of offset of the elevator car.

[0047] It should be noted that the total deviation of the elevator car is a multi-dimensional risk assessment result. This invention deconstructs it into three independent risk dimensions, and then performs a weighted fusion. These three independent risk dimensions are overload risk, static off-center load risk, and dynamic behavioral risk. Overload risk reflects the risk that the current load of the elevator car may be overloaded; static off-center load risk reflects the degree of physical imbalance of the car caused by uneven passenger mass distribution; and dynamic behavioral risk reflects the potential risk accumulated from the initial deviation of all passengers, leading to drastic changes in the center of gravity in the future. The greater these three risk dimensions, the greater the probability of abnormal elevator car deviation.

[0048] Specifically, based on the total weight of the load inside the elevator car corresponding to each frame of image obtained in step S1, the first... The total weight of the load inside the elevator car corresponding to the frame image is denoted as: The maximum load capacity inside the elevator car is obtained from the elevator manufacturer's nameplate and denoted as . Then the first The overload risk of the elevator car corresponding to the frame image is: ; in, It is the first The frame image corresponds to the risk of overloading in the elevator car; It is the first The frame image corresponds to the total weight of the load inside the elevator car; It is the maximum load capacity inside the elevator car; when the first The larger the total weight of the load inside the elevator car corresponding to the frame image, the more important it is to consider the number of frames. The higher the frame image, the greater the risk of overload in the elevator car, and vice versa. When the ratio is greater than or equal to 1, it indicates that the elevator is in an overloaded state.

[0049] Specifically, each passenger's information is obtained through step S2 in the first step. The centroid coordinate vector in the frame image, based on the assumption that all passengers have approximately the same weight, is the average of all centroid coordinate vectors as the first... The collective centroid coordinate vector of all passengers in the frame image is denoted as . Project the collective centroid coordinate vector onto From a plane, obtain its modulus in two-dimensional space, as the first... The magnitude of the collective centroid coordinate vector of all passengers in the frame image is denoted as . According to the elevator manufacturer's nameplate, the maximum allowable offset of the elevator car's center of gravity is obtained and denoted as... Then the first The static off-center load risk of the elevator car corresponding to the frame image is: ; in, It is the first The frame image corresponds to the static off-center load risk of the elevator car; It is the first The collective centroid coordinate vector of all passengers in the frame image; It is the coordinate vector of the collective centroid. exist Modulus length in the plane; It represents the maximum allowable deviation of the elevator car's center of gravity, serving a normalization function.

[0050] When all passengers are evenly distributed and the collective center of gravity is located in the center, the first... The frame image corresponds to a static off-center load risk in the elevator car that approaches 0, representing the lowest risk; when all passengers are crowded in one corner, the risk is... The static off-center load risk of the elevator car corresponding to the frame image is close to 1, which is the highest risk.

[0051] Furthermore, the first step is obtained through step S2. Each frame image corresponds to a first offset degree for each passenger. The variance of the first offset degrees for all passengers is then calculated. The normalized value of this variance is used as the dynamic behavior risk of the elevator car corresponding to each frame image. A max-min normalization method can be used to transform it into a dimensionless number. For example, the first frame image... The dynamic behavior risk of the elevator car corresponding to the frame image is denoted as .

[0052] Specifically, based on the overload risk, static off-center load risk, and dynamic behavior risk of the elevator car corresponding to each frame of the image, the total offset degree of the elevator car corresponding to each frame of the image is obtained, and so on. For example, a frame image: ; In the formula, It is the first The frame image corresponds to the total offset of the elevator car; It is the first The frame image corresponds to the risk of overloading in the elevator car; It is the first The frame image corresponds to the static off-center load risk of the elevator car; It is the first The frame images correspond to the dynamic behavioral risks of the elevator car; These are the first weight, the second weight, and the third weight, respectively.

[0053] Among them, the first weight Second weight Third weight The specific value can be set according to the actual application scenario and needs, and the following conditions must be met: And the first weight Second weight Third weight The range of values ​​is , will the first weight Second weight Third weight Set to respectively Used to regulate the first The frame image represents the impact of the elevator car's overload risk, static off-center load risk, and dynamic behavior risk on the total deviation of the elevator car.

[0054] Based on the above operations, the total offset of the elevator car in each frame of the image is obtained.

[0055] S4. Based on the total offset of the elevator car in each frame of the image, obtain the energy feedback activation voltage threshold that the energy feedback device controller will set after receiving the current frame of the image, and thus dynamically adjust the energy feedback strategy.

[0056] It should be noted that the total offset of the elevator car in each frame of the image obtained in step S3 not only reflects the current imbalance of the car, but also predicts the risk of severe shaking in the future. Therefore, the total offset of the elevator car in each frame of the image can be used to dynamically adjust the activation sensitivity of the energy feedback device in real time, specifically to adjust the activation voltage threshold of its energy feedback.

[0057] Specifically, the total offset of the elevator car in each frame of the image, along with the corresponding frame sequence number, is encapsulated into a standard format data frame. This data frame is then transmitted in real time to the energy feedback device controller in the elevator's main control system via the elevator's internal Ethernet bus. The preset control functions within the energy feedback device are as follows: ; in, The energy feedback device controller receives the first After the frame image, the energy feedback activation voltage threshold to be set; It is the standard activation voltage threshold, which is the normal setting value for smooth elevator operation and represents the minimum sensitivity state of the controller. It is the limit activation voltage threshold, which is the lowest safe voltage threshold that can be set when the system detects an extremely high risk. It represents the highest sensitivity state of the system. Based on the natural constant An exponential function with base 0; It is the first The total offset of the elevator car corresponding to the frame image The normalized value can be processed using maximum and minimum value normalization.

[0058] When the The greater the total offset of the elevator car corresponding to the frame image, the more the energy feedback device controller receives the first frame. The smaller the energy feedback activation voltage threshold to be set after the first frame of data, the more sensitive the energy feedback device will be; when the first frame of data is used... The smaller the total offset of the elevator car corresponding to the frame image, the more effective the energy feedback device controller will be when it receives the first frame image. The higher the energy feedback activation voltage threshold is after each frame of data, the more sluggish the energy feedback device will become. This can be addressed by dynamically adjusting the energy feedback device controller upon receiving the first frame of data. After the frame data is set, the energy feedback activation voltage threshold will be set to activate the energy feedback in advance when the risk of car imbalance is high, so as to avoid energy waste or equipment damage due to shaking.

[0059] Specifically, the controller of the energy feedback device will monitor the DC bus voltage of its internal capacitor in real time. Once the DC bus voltage exceeds the activation voltage threshold that has just been dynamically updated, the energy feedback device will immediately start the inverter circuit to convert the regenerated electrical energy stored in the capacitor into AC power that is in phase and frequency with the grid and send it back to the grid.

[0060] Through the above operations, the activation timing of the energy feedback device is no longer fixed, but is intelligently advanced or delayed according to the real-time balance state of the car. When the balance state of the car is about to change drastically, the energy feedback function is activated in advance, thereby effectively avoiding energy waste caused by response delay; it significantly improves the regenerative energy recovery rate of the elevator under various complex working conditions, obtains optimized overall energy feedback efficiency, and at the same time reduces the heating of the braking resistor and improves the ambient temperature of the machine room.

Claims

1. A method for monitoring the balance state of an energy-feedback elevator car based on computer vision, characterized in that, include: Based on the three-dimensional spatial coordinate vector of each passenger's corresponding key point in each frame of the image, the centroid coordinate vector of each passenger in each frame of the image is obtained; the ratio of the Euclidean distance between the current frame and the previous frame of the passenger's corresponding centroid coordinate vector to the time interval between video frames is calculated to obtain the instantaneous velocity of the centroid of each passenger in each frame of the image. Based on the difference between the current time and the time when the passenger leaves the elevator car, a time decay factor is obtained; based on the time decay factor, the difference between the instantaneous velocity of the center of gravity of each passenger in the current frame and the previous frame image is weighted to obtain the first offset degree corresponding to each passenger. Calculate the variance of the first degree of offset for all passengers, use the normalized value of the variance as the dynamic behavior risk for each frame of the image, and combine the overload risk and static off-center load risk of the elevator car for each frame of the image to obtain the total degree of offset of the elevator car. Based on the total offset of the elevator car, the energy feedback device controller obtains the energy feedback activation voltage threshold to be set after receiving the current frame image, thereby dynamically adjusting the energy feedback strategy.

2. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, The step of obtaining the centroid coordinate vector of each passenger in each frame of an image based on the three-dimensional spatial coordinate vector of each passenger's corresponding key point includes: ； In the formula, Passengers In the The centroid coordinate vector in the frame image; It is an index for passengers; It is the index of the image frame; Passengers In the The set of three-dimensional spatial coordinates of key points that can be successfully detected in a frame image; It is a set of three-dimensional spatial coordinates An index of a key point; This is the key point. The weight weight corresponding to the body part represented; Passengers Key points In the The three-dimensional spatial coordinate vector in the frame image.

3. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, The process of obtaining the time decay factor based on the difference between the current time and the time when the passenger leaves the elevator car includes: ； In the formula, It is the time decay factor; Based on the natural constant An exponential function with base 0; Indicates the current moment; Indicates passenger The moment of leaving the elevator car; It is the attenuation coefficient; It is the current time and passengers The difference in the time of leaving the elevator car.

4. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, The step of weighting the difference in the instantaneous velocity of the center of gravity of each passenger in the current frame and the previous frame image according to the time decay factor to obtain the first offset degree corresponding to each passenger includes: ； In the formula, Passengers The corresponding first offset degree; It is an index for passengers; Passengers The index of the image frame; From passengers The number of image frames captured between entering and leaving the elevator car; It is the time decay factor; Passengers In the The instantaneous velocity of the center of gravity in a frame image; Passengers In the The instantaneous velocity of the center of gravity in a frame image; It is the absolute value symbol.

5. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, The method for obtaining the overload risk of the elevator car corresponding to each frame of the image is as follows: The ratio of the total weight of the elevator car corresponding to each frame to the maximum weight of the elevator car is used as the overload risk of the elevator car corresponding to each frame.

6. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, The method for obtaining the static off-center load risk of the elevator car corresponding to each frame image is as follows: The ratio of the magnitude of the collective center of gravity coordinate vector of all passengers in each frame of the image to the maximum allowable offset of the center of gravity of the elevator car is used as the static off-center load risk of the elevator car corresponding to each frame of the image.

7. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, The magnitude of the collective centroid coordinate vector of all passengers in each frame of the image includes: The centroid coordinate vector of each passenger in each frame is used as the average of all centroid coordinate vectors, based on the assumption that all passengers have approximately the same weight. This average is then used as the collective centroid coordinate vector of all passengers in the corresponding frame. The collective centroid coordinate vector is then projected onto... The plane is used to obtain the modulus in its two-dimensional space, which serves as the modulus of the collective centroid coordinate vector of all passengers in each frame of the image.

8. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, The process of obtaining the total offset of the elevator car based on the dynamic behavior risk, overload risk, and static off-center load risk of the elevator car corresponding to each frame of the image includes: First weight Second weight Third weight The specific value can be set according to the actual application scenario and needs, and the following conditions must be met: And the first weight Second weight Third weight The range of values ​​is , will the first weight Second weight Third weight Set to respectively Among them, respectively using The overload risk, static off-center load risk, and dynamic behavior risk of the elevator car corresponding to each frame of the image are weighted and summed to obtain the total offset of the elevator car corresponding to each frame of the image.

9. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, After receiving the current frame image, the energy feedback device controller will set the energy feedback activation voltage threshold, including: ； in, The energy feedback device controller receives the first After the frame image, the energy feedback activation voltage threshold to be set; It is the standard activation voltage threshold; It is the limiting activation voltage threshold; Based on the natural constant An exponential function with base 0; It is the first The total offset of the elevator car corresponding to the frame image The normalized value can be processed using maximum and minimum value normalization.

10. The method for monitoring the balance state of an energy-feedback elevator car based on computer vision according to claim 1, characterized in that, The dynamically adjusted energy feedback strategy includes: The controller of the energy feedback device will monitor the DC bus voltage of its internal capacitor in real time. Once the DC bus voltage exceeds the activation voltage threshold that has just been dynamically updated, the energy feedback device will immediately start the inverter circuit to convert the regenerated electrical energy stored in the capacitor into AC power that is in phase and frequency with the grid and send it back to the grid.

Images