Computer vision-based energy feedback elevator car balance state monitoring method

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a kind of energy feedback elevator car balance state monitoring method based on computer vision, method includes: through computer vision technology, the gravity coordinates of each passenger in elevator car and instantaneous rate are calculated frame by frame, and time attenuation factor is introduced to quantify its behavior, obtain the first offset degree;Further, the variance of the first offset degree of all passengers is taken as the dynamic behavior risk, the overload risk and the static unbalance risk are integrated, and the total offset degree representing the overall balance state of the car is calculated;Finally, according to the total offset degree, the activation voltage threshold of the energy feedback device is dynamically set, and the energy feedback strategy adjustment of safety and energy efficiency optimization is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a computer vision-based energy feedback elevator car balance state monitoring method. BACKGROUND

[0002] When the elevator is in the light load uplink stage or heavy load downlink stage, the mechanical energy generated is converted into regenerative electric energy by the traction machine inside the elevator, and the regenerative electric energy is stored in the capacitor after being rectified by the frequency converter, resulting in continuous rise of the capacitor voltage. When the capacitor voltage reaches the preset threshold value, the system will start the braking resistor to convert the regenerative electric energy into heat energy, which will cause the temperature of the elevator and other surrounding equipment to rise and the energy consumption to increase. The energy feedback device of the elevator automatically starts when the elevator is in the power generation state, and converts the regenerative electric energy into alternating current with the same frequency and phase as the power grid through inverter technology, and feeds back to the power grid for recycling, which significantly reduces the heating of the braking resistor, thereby reducing the temperature of the elevator and other surrounding equipment and saving energy consumption.

[0003] The energy feedback device of the prior art usually adopts a static threshold triggering mechanism: only when the voltage of the internal capacitor rises to a certain fixed, higher preset value due to the injection of regenerative electric energy, the energy feedback function is started. The defect of this mode is that it cannot predict the upcoming energy impact; when the passengers suddenly move, causing the car balance state to change dramatically, a large amount of regenerative electric energy will be generated instantaneously, and the capacitor voltage may exceed the starting threshold of the energy feedback in a very short time, even triggering the braking resistor at a higher voltage point. At this time, not only a large amount of heat will be generated by the braking resistor, but also the recoverable energy will be wasted, thereby significantly reducing the energy feedback efficiency of the elevator energy feedback system. SUMMARY

[0004] In order to solve the technical problems of the prior art that the energy feedback device for adjusting the elevator energy feedback system cannot predict the upcoming energy impact and is difficult to capture the sudden movement of passengers causing the car balance state to change dramatically, resulting in a significant reduction in the energy feedback efficiency of the elevator energy feedback system, the present application provides a computer vision-based energy feedback elevator car balance state monitoring method.

[0005] In a first aspect, the present application provides a computer vision-based energy feedback elevator car balance state monitoring method, comprising: obtaining the center of gravity coordinate vector of each passenger in each frame of image according to the three-dimensional space coordinate vector of each passenger corresponding key point in each frame of image; obtaining the center of gravity instantaneous speed of each passenger in each frame of image according to the ratio of the Euclidean distance of the center of gravity coordinate vector of the passenger between the current frame and the previous frame to the time interval between the video frames; obtaining the time decay factor according to the difference between the current time and the time when the passenger leaves the elevator car; weighting the difference between the center of gravity instantaneous speed of each passenger in the current frame and the previous frame of image according to the time decay factor to obtain the first offset degree corresponding to each passenger; obtaining the dynamic behavior risk corresponding to each frame of image according to the variance of the first offset degree corresponding to each passenger, combining the overload risk and the static unbalanced load risk corresponding to each frame of image to obtain the total offset degree of the elevator car; obtaining the energy feedback activation voltage threshold to be set by the energy feedback device controller after receiving the current frame of image, so as to dynamically adjust the energy feedback strategy.

[0006] The present application realizes high-precision monitoring of the balance state of the car by analyzing the key point and center of gravity motion state of the passengers in the elevator car in real time through computer vision technology, dynamically evaluating the behavior offset degree of each passenger in combination with the time decay factor, and comprehensively considering the overload risk and the static unbalanced load risk. Furthermore, the activation voltage threshold of the energy feedback device is adaptively adjusted according to the total offset degree, which not only ensures the safety of elevator operation, but also improves the response accuracy and energy saving efficiency of the energy feedback system.

[0007] Preferably, the center of gravity coordinate vector of each passenger in each frame of image is obtained, comprising: ; in the formula, is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the index of the passenger; is the index of the image frame; is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the set of three-dimensional space coordinates of the key point that can be successfully detected in the first frame of image; is the index of a key point in the set of three-dimensional space coordinates is the weight weight corresponding to the body part represented by the key point is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the second frame of image; is the three-dimensional space coordinate vector of the key point of the passenger in the first frame of image;

[0008] The method assigns weight weights conforming to the actual mass distribution of different key points of the human body, including but not limited to the head, torso, limbs , and performs a weighted average calculation based on the three-dimensional coordinates of all detectable key points, thereby greatly improving the biomechanical accuracy and physical authenticity of the center of gravity coordinate estimation. Compared with simple arithmetic average, this method can more accurately reflect the true mass center position of the passenger, laying a reliable physical foundation for subsequent accurate calculation of the motion state and overall balance risk of the car. At the same time, this method has fault tolerance for the case where part of the key points are blocked or detection fails. As long as the set is not empty, effective calculation can be performed, ensuring the robustness and usability of the system in complex actual environments.

[0009] Preferably, the obtaining of the first offset degree corresponding to each passenger comprises: ; in the formula, is the first offset degree corresponding to the passenger; is the index of the passenger; is the index of the image frame of the passenger; is the number of image frames collected between the passenger entering the elevator car and leaving the elevator car; is a time decay factor; is the instantaneous velocity of the center of gravity of the passenger in the first image frame; is the instantaneous velocity of the center of gravity of the passenger in the first image frame; is the instantaneous velocity of the center of gravity of the passenger in the first image frame; is the absolute value symbol. The method introduces a time decay factor to perform an exponential weighted summation on the sudden change of the instantaneous velocity of the center of gravity of the passenger between all consecutive frames during the entire boarding process

[0010] , thereby quantifying the first offset degree. This not only captures the entire dynamic history of the passenger's behavior, but also gives higher weight to recent behavior than to long-term behavior, which conforms to the physical intuition and actual situation that "recent actions have a greater impact on the current car balance state". This processing method can effectively distinguish between short-term, occasional small movements and sustained, dangerous actions, significantly improving the perception sensitivity and evaluation accuracy of the system for real risk behaviors, while avoiding the interference of irrelevant past movements on the current state judgment, enhancing the timeliness and reliability of risk assessment, and providing more accurate input for subsequent overall risk aggregation.

[0011] ​​Preferably, the step of obtaining the total offset of the elevator car based on the dynamic behavior risk, overload risk, and static off-center load risk of the elevator car corresponding to each frame of the image includes: a first weight. Second weight Third weight The specific value can be set according to the actual application scenario and needs, and the following conditions must be met: And the first weight Second weight Third weight The range of values ​​is , will the first weight Second weight Third weight Set to respectively Among them, respectively using The overload risk, static off-center load risk, and dynamic behavior risk of the elevator car corresponding to each frame of the image are weighted and summed to obtain the total offset of the elevator car corresponding to each frame of the image.

[0012] This method achieves a comprehensive assessment of the elevator car's balance by weightedly fusing three core indicators: dynamic behavioral risk (variance of individual passenger deviation), static off-center load risk, and overload risk. It not only considers the static factors of passenger number and distribution but also innovatively introduces dynamic risk, representing the consistency of collective passenger behavior. Using variance, it can keenly capture anomalies in the behavior of all passengers in the cabin: even if the absolute value of individual deviation is small, if inconsistent or chaotic passenger behavior shows a trend, the system will identify it as an increased risk, greatly enhancing the early warning capability for dangerous group behaviors such as pushing. Finally, through configurable weights, the system can flexibly adjust the emphasis on each risk factor according to different safety strategies, thereby outputting a total deviation index that reflects instantaneous state and trend changes, providing precise control basis for the energy feedback device.

[0013] Preferably, after receiving the current frame image, the energy feedback device controller sets the energy feedback activation voltage threshold to be set, including: ;in, The energy feedback device controller receives the first After the frame image, the energy feedback activation voltage threshold to be set; It is the standard activation voltage threshold; It is the limiting activation voltage threshold; Based on the natural constant An exponential function with base 0; It is the first The total offset of the elevator car corresponding to the frame image The normalized value can be normalized by using the maximum-minimum value normalization processing.

[0014] The method realizes optimal collaborative control of safety and energy efficiency by a mapping model based on an exponential function, which nonlinearly converts the normalized total deviation degree into an energy feedback activation voltage threshold: when the car balance state is good, the system automatically sets a lower activation voltage threshold, so that the energy feedback device can start earlier, maximize the recovery of braking energy, and improve energy saving efficiency; when the imbalance risk increases, the threshold rapidly increases exponentially, forcing the elevator to preferentially execute a more stable and more conservative braking strategy, giving priority to operation safety and equipment protection, so that the system can calmly cope with sudden imbalance risks and ensure passenger safety, and can also tap the maximum energy saving potential in daily operation, significantly improving the overall intelligent level of the energy feedback elevator.

[0015] The beneficial effects of the present application are that the camera deployed on the top of the car captures the video stream in real time, and uses a deep learning model to extract the human key points of each passenger in each frame of image, and then calculates the three-dimensional spatial barycenter coordinates thereof; the ratio of the Euclidean distance of the passenger barycenter coordinates between consecutive frames to the time interval is accurately obtained; in order to distinguish the influence of recent and early behaviors, a time decay factor is innovatively introduced to weight the mutation of the passenger barycenter speed between the current frame and the previous frame, thereby quantifying the first deviation degree of each passenger due to moving, pushing and other behaviors; by calculating the variance of the deviation degrees of all passengers, the collective consistency of the dynamic behavior risk in the car is effectively represented; in addition, the system synchronously fuses the overload risk based on the number of passengers and the static unbalanced load risk calculated based on the position distribution of the passengers, and finally aggregates into a total deviation degree index that comprehensively reflects the real-time balance state of the car; the energy feedback device controller dynamically sets the optimal activation voltage threshold according to the index: the threshold is lowered when the balance state is good to maximize the energy recovery efficiency; the threshold is raised when the imbalance risk increases to prioritize operation safety and equipment life. The method realizes the leap from passive response to active prediction, significantly improves the adaptive ability and energy saving benefit of the energy feedback system under the premise of ensuring the safe and stable operation of the elevator. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a flowchart schematically showing a computer vision-based energy feedback elevator car balance state monitoring method in the present application. DETAILED DESCRIPTION

[0017] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort are within the scope of the present application.

[0018] The specific implementation of the present application will be described in detail below with reference to the drawings.

[0019] The embodiments of the present application disclose a computer vision-based energy feedback elevator car balance state monitoring method, referring to Figure 1 , comprising steps S1 to S4:

[0020] S1, pre-processing the passenger video stream inside the elevator car to obtain the three-dimensional space coordinate vector of the key points of the passengers in each image; obtaining the total weight of the load in each image according to the gravity sensor.

[0021] It should be noted that this step aims to accurately obtain the dynamic three-dimensional structure information of each passenger inside the elevator car from single-view video data, and to provide high-precision data input for subsequent calculation of the dynamic offset of the center of gravity.

[0022] Specifically, a high-definition camera is installed at any one of the top corners inside the elevator car. The high-definition camera collects the video stream of the passengers inside the elevator car at a preset time interval, and decomposes the video stream into a series of continuous video frames; wherein the time interval between video frames can be set according to the actual application scenario and requirements, and the value range of the time interval between video frames is set to .

[0023] It should be noted that if the time interval of the video frames is set too large, such as more than 1 second, the position and posture of the person between adjacent frames will change too much, and it is difficult to establish a smooth motion trajectory, making it difficult to accurately determine the intention and type of the action, resulting in a significant increase in false positive rate and false negative rate; if the time interval of the video frames is set too small, a large amount of redundant data will be generated, increasing the calculation energy consumption; therefore, the time interval between video frames is set to .

[0024] Further, each obtained frame image is pre-processed, and the pre-processing procedure includes: using a pre-calibrated camera intrinsic matrix to correct the distortion of the image to eliminate the geometric distortion caused by the lens; then, using a Gaussian filtering algorithm to smooth the corrected image to suppress the random noise introduced by the image sensor, and finally generating a clear and distortion-free image as the input for subsequent analysis.

[0025] Specifically, a pre-trained YOLOv5 deep learning target detection network is obtained, including: first, pre-training on the existing general COCO dataset to enable the model to obtain basic feature extraction and target detection capabilities; using a specially collected and finely labeled elevator car scene image dataset for transfer learning and fine tuning, which covers multiple elevator models, lighting conditions, passenger density and posture, such as standing, squatting, turning away, and shielding, so that the model can be well adapted to the complex application scenarios inside the elevator, significantly improving the accuracy and robustness of detection, and obtaining a pre-trained YOLOv5 deep learning target detection network;

[0026] Further, the pre-processed clear image is input into the pre-trained YOLOv5 deep learning target detection network; through forward propagation in the network, an output set containing multiple initial prediction bounding boxes is obtained, each box containing its normalized center coordinates, width, height, confidence score and class probability; to eliminate redundant detection of the same passenger, a non-maximum suppression algorithm is applied to filter overlapping redundant detection boxes, and those redundant boxes with an overlap degree exceeding a set threshold with the highest score box are suppressed; finally, a series of accurate and unique passenger bounding boxes are output, and the bounding box region of each passenger is cropped from each frame image to provide high-quality input for subsequent key point detection.

[0027] Specifically, in the bounding box region of each passenger in each frame image, a set of predefined key points of each passenger in each frame image is obtained through the OpenPose human pose estimation algorithm, including but not limited to head, torso, thigh, calf, arm, etc.

[0028] Further, taking the center of the elevator car floor as the origin , the floor is a plane, and the direction perpendicular to the elevator car floor upward is the axis, a spatial coordinate system is constructed; through pre-camera calibration, a transformation matrix from the camera coordinate system to the spatial coordinate system is obtained; through the transformation matrix, the key points of each passenger are mapped to the spatial coordinate system, and the three-dimensional spatial coordinate vector of each passenger's corresponding key points is obtained.

[0029] It should be noted that, due to the uneven distribution of the mass of the human body, different body parts contribute differently to the overall center of gravity; therefore, the weight weight corresponding to the key points representing different body parts needs to be set, for example, the weight weight corresponding to the body segments belonging to the head, torso, thigh, calf and arm is set to 0.06, 0.4, 0.3, 0.2 and 0.04. It should be noted that, since the thigh, calf and arm are usually divided into left and right parts, the weight setting here is the default weight weight of the left and right parts corresponding to the entire part, for example, the weight weight corresponding to the left and right arms is 0.02 respectively.

[0030] After the above processing, for each frame of image of the video stream, a set of key points of each passenger in the car can be obtained, wherein each key point includes a three-dimensional spatial coordinate vector in the spatial coordinate system and a weight weight of the body segment it represents. This structured data will be used as input for the subsequent step of dynamic calculation of the overall center of gravity of the car.

[0031] At the same time, a gravity sensor is installed at the bottom of the elevator car, and the total weight of the empty elevator car is set to 0, and the total weight of the loaded elevator car corresponding to each frame of image is collected in real time.

[0032] Among them, the distortion correction, Gaussian filtering algorithm, YOLOv5 deep learning target detection network and OpenPose human pose estimation algorithm are prior art, which will not be described here.

[0033] S2, obtaining the center of gravity coordinate vector according to the three-dimensional spatial coordinate vector of the key points of the passengers; obtaining the instantaneous speed of the center of gravity according to the Euclidean distance of the center of gravity coordinate vectors of adjacent frames; obtaining the first degree of deviation of each passenger by weighting through the time decay factor.

[0034] It should be noted that this step aims to analyze the three-dimensional spatial coordinates of the passenger key points output by S1, accurately calculate the center of gravity trajectory, instantaneous speed and acceleration change of the passenger individual during the boarding process, and introduce a time decay factor to give a higher weight to the unstable behavior closer to the off-boarding time. Finally, the first degree of deviation is obtained, which is used to quantify the risk of abnormal motion.

[0035] It's important to note that the passenger's center of gravity is the focal point of their mass distribution and the most crucial physical quantity for analyzing their macroscopic motion. However, in a crowded elevator environment, some key body points are often undetectable due to occlusion. Therefore, it's necessary to use the sum of weights for all visible body parts in the current frame. This gives the center of gravity calculation an adaptive capability to occlusion: when a part of the passenger's body is occluded, including but not limited to the legs, the formula automatically normalizes the calculation based on the weights of the remaining visible parts, including but not limited to the head and torso. If no key points of the passenger are detected in the image, the passenger's center of gravity coordinate vector is deemed uncalcifiable, its first offset is set to 0, and it is not included in subsequent dynamic behavior risk calculations. This ensures that even when key points are partially missing, the calculated center of gravity is still an accurate estimate based on the current visible mass distribution, greatly improving the algorithm's robustness in real-world complex scenarios.

[0036] Specifically, passenger information is obtained through step S1. In the In a frame image, the set of three-dimensional spatial coordinates of key points that can be successfully detected is denoted as . ; will passengers Key points In the The three-dimensional spatial coordinate vector in the frame image is denoted as Then calculate the passengers In the The centroid coordinate vector in the frame image is:

[0037] ;

[0038] In the formula, Passengers In the The centroid coordinate vector in the frame image; It is an index for passengers; It is the index of the image frame; Passengers In the The set of three-dimensional spatial coordinates of key points that can be successfully detected in a frame image; It is a set of three-dimensional spatial coordinates An index of a key point; This is the key point. The weight weight corresponding to the body part represented; Passengers Key points In the The three-dimensional spatial coordinate vector in the frame image.

[0039] Specifically, to obtain passengers In the The center of gravity coordinate vector in the frame image , the passenger In the first The center of gravity instantaneous rate in the frame image is:

[0040] ;

[0041] In the formula, , the passenger In the first The center of gravity instantaneous rate in the frame image; , the index of the passenger; , the passenger , the index of the image frame; , the time interval between video frames; , the center of gravity coordinate vector of the passenger In the first frame image; , the center of gravity coordinate vector of the passenger In the first frame image The Euclidean distance between the center of gravity coordinate vectors of the passenger

[0042] , the greater the center of gravity instantaneous rate of the passenger In the first frame image, the faster the passenger moves at that time; when the passenger is stationary, the center of gravity instantaneous rate of the passenger In the first frame image tends to 0; when the passenger walks or has a large movement in the car, the center of gravity instantaneous rate of the passenger In the first frame image will significantly increase.

[0043] It should be noted that since the elevator will have a significant pause when the passenger needs to exit the car to the target floor, the passenger's body will be more unstable at this time, i.e. the passenger's center of gravity instantaneous rate between adjacent frames will change more likely; when the passenger enters the car, the elevator is relatively stable, at this time the passenger's body is less likely to be unstable, i.e. the passenger's center of gravity instantaneous rate between adjacent frames is less likely to change; therefore, a smaller weight factor needs to be assigned to the time farther away from the time when the passenger leaves the elevator, and a larger weight factor needs to be assigned to the time closer to the time when the passenger leaves the elevator.

[0044] Further, according to the difference between the current time and the time when the passenger leaves the elevator car, and the decay coefficient, the corresponding time decay factor of the passenger is obtained:

[0045] ;

[0046] In the formula, is a time decay factor; is an exponential function with a natural constant as the base; represents the current time; represents the passenger leaving the elevator car; is a decay coefficient; is the difference between the current time and the time when the passenger leaves the elevator car.

[0047] In the time decay factor , when is far away , it reflects that the passenger has just entered the elevator car, at this time the difference is large, and the time decay factor is close to 0, indicating that the farther away from the time when the passenger leaves the elevator, the less important the change of the passenger's center of gravity instantaneous speed between adjacent two frames; when is close , it reflects that the passenger is about to leave the elevator car, at this time the difference is small, and the time decay factor is close to 1, indicating that the closer to the time when the passenger leaves the elevator, the more important the change of the passenger's center of gravity instantaneous speed between adjacent two frames, wherein the time when the passenger enters and exits the elevator car can be obtained according to whether the passenger enters and exits the elevator car tracked by the YOLOv5 deep learning target detection network.

[0048] It should be noted that the decay coefficient is a key parameter for controlling the decay speed of the weight over time, which directly affects the sensitivity to the difference between the current time and the time when the passenger leaves the elevator car , the larger the decay coefficient , the faster the weight decays at this time, the smaller the difference , and the closer to the frame ; when the decay coefficient is smaller, the weight decays slower at this time, even if the difference is large, the weight still has a certain amplitude reservation; the specific value of the decay coefficient can be set according to the time distribution of the passenger's abnormal behavior, such as stumbling, swaying or falling, which is statistically counted from historical data, and the decay coefficient is set to , which can ensure that the weight is significantly improved between 1 to 2 seconds before leaving the elevator, avoiding excessive concentration leading to early behavior being ignored.

[0049] ​​​​​​​​​It should be noted that, compared to drastic changes in velocity during uniform motion, such as the high acceleration during a bump, the instability of a passenger's body inside the elevator car, such as staggering, swaying, or falling, is more evident. Therefore, it is necessary to obtain the first degree of offset that can describe the instability of a passenger's body inside the elevator car based on the degree of fluctuation in the instantaneous velocity of the center of gravity.

[0050] Specifically, to obtain passengers The instantaneous velocity of the center of gravity in each frame of the image, according to the passenger The degree of fluctuation in the instantaneous velocity of the center of gravity in consecutive frame images is used to obtain passenger information. The corresponding first offset degree:

[0051] ;

[0052] In the formula, Passengers The corresponding first degree of offset reflects the instability of the passenger's body inside the elevator car; It is an index for passengers; Passengers The index of the image frame; From passengers The number of image frames captured between entering and leaving the elevator car; It is the time decay factor; Passengers In the The instantaneous velocity of the center of gravity in a frame image; Passengers In the The instantaneous velocity of the center of gravity in a frame image; Indicates passenger In the Instantaneous velocity of the center of gravity in the frame image and passenger In the The difference in instantaneous velocity between the centers of gravity in the frame image; It is the absolute value symbol.

[0053] in, It measures the smoothness of passenger movement; if the passenger remains stationary or moves at a constant speed, The value will be close to 0; any sudden acceleration, deceleration, or change of direction will cause it to... A peak appears; The higher the value, the more drastic the changes in the passenger's motion state and the higher the instability of the body.

[0054] When passengers In the Instantaneous velocity of the center of gravity in the frame image and passenger In the Difference between instantaneous rates of the center of gravity in frame images A larger value indicates more drastic changes in the passenger's motion state, higher physical instability, and a larger time decay factor. The greater the corresponding first offset, the smaller the offset, and vice versa.

[0055] Furthermore, based on the above operations, the first offset degree corresponding to each passenger is obtained.

[0056] S3. Based on the first degree of offset of each passenger, obtain the dynamic behavior risk corresponding to each frame of the image. Combine the overload risk and static off-center load risk corresponding to each frame of the image to obtain the total degree of offset of the elevator car.

[0057] It should be noted that the total deviation of the elevator car is a multi-dimensional risk assessment result. This invention deconstructs it into three independent risk dimensions, and then performs a weighted fusion. These three independent risk dimensions are overload risk, static off-center load risk, and dynamic behavioral risk. Overload risk reflects the risk that the current load of the elevator car may be overloaded; static off-center load risk reflects the degree of physical imbalance of the car caused by uneven passenger mass distribution; and dynamic behavioral risk reflects the potential risk accumulated from the initial deviation of all passengers, leading to drastic changes in the center of gravity in the future. The greater these three risk dimensions, the greater the probability of abnormal elevator car deviation.

[0058] Specifically, based on the total weight of the load inside the elevator car corresponding to each frame of image obtained in step S1, the first... The total weight of the load inside the elevator car corresponding to the frame image is denoted as: The maximum load capacity inside the elevator car is obtained from the elevator manufacturer's nameplate and denoted as . Then the first The risk of overload in the elevator car corresponding to the frame image is:

[0059] ;

[0060] in, It is the first The frame image corresponds to the risk of overloading in the elevator car; It is the first The frame image corresponds to the total weight of the load inside the elevator car; It is the maximum load capacity inside the elevator car; when the first The larger the total weight of the load inside the elevator car corresponding to the frame image, the more important it is to consider the number of frames. The higher the frame image, the greater the risk of overload in the elevator car, and vice versa. When the ratio is greater than or equal to 1, it indicates that the elevator is in an overloaded state.

[0061] Specifically, each passenger's information is obtained through step S2 in the first step. The centroid coordinate vector in the frame image, based on the assumption that all passengers have approximately the same weight, is the average of all centroid coordinate vectors as the first... The collective centroid coordinate vector of all passengers in the frame image is denoted as . Project the collective centroid coordinate vector onto From a plane, obtain its modulus in two-dimensional space, as the first... The magnitude of the collective centroid coordinate vector of all passengers in the frame image is denoted as . According to the elevator manufacturer's nameplate, the maximum allowable offset of the elevator car's center of gravity is obtained and denoted as... Then the first The static off-center load risk of the elevator car corresponding to the frame image is:

[0062] ;

[0063] in, It is the first The frame image corresponds to the static off-center load risk of the elevator car; It is the first The collective centroid coordinate vector of all passengers in the frame image; It is the coordinate vector of the collective centroid. exist Modulus length in the plane; It represents the maximum allowable deviation of the elevator car's center of gravity, serving a normalization function.

[0064] When all passengers are evenly distributed and the collective center of gravity is located in the center, the first... The frame image corresponds to a static off-center load risk in the elevator car that approaches 0, representing the lowest risk; when all passengers are crowded in one corner, the risk is... The static off-center load risk of the elevator car corresponding to the frame image is close to 1, which is the highest risk.

[0065] Furthermore, the first step is obtained through step S2. Each frame image corresponds to a first offset degree for each passenger. The variance of the first offset degrees for all passengers is then calculated. The normalized value of this variance is used as the dynamic behavior risk of the elevator car corresponding to each frame image. A max-min normalization method can be used to transform it into a dimensionless number. For example, the first frame image... The dynamic behavior risk of the elevator car corresponding to the frame image is denoted as .

[0066] Specifically, based on the overload risk, static off-center load risk, and dynamic behavior risk of the elevator car corresponding to each frame of the image, the total offset degree of the elevator car corresponding to each frame of the image is obtained, and so on. For example, a frame image:

[0067] ;

[0068] In the formula, is the total offset degree of the elevator car corresponding to the first frame image; is the overload risk of the elevator car corresponding to the first frame image; is the static unbalance risk of the elevator car corresponding to the first frame image; is the dynamic behavior risk of the elevator car corresponding to the first frame image. is the total offset degree of the elevator car corresponding to the first frame image; is the overload risk of the elevator car corresponding to the first frame image; is the static unbalance risk of the elevator car corresponding to the first frame image; is the dynamic behavior risk of the elevator car corresponding to the first frame image. are respectively a first weight, a second weight and a third weight.

[0069] The specific numerical values of the first weight , the second weight and the third weight may be set according to actual application scenarios and requirements, and need to satisfy the condition: , and the value ranges of the first weight , the second weight and the third weight are all The first weight , the second weight and the third weight are respectively set as , for adjusting the influence degree of the overload risk, the static unbalance risk and the dynamic behavior risk of the elevator car corresponding to the first frame image on the total offset degree of the elevator car. According to the above operation, the total offset degree of the elevator car under each frame image is obtained.

[0070] S4. According to the total offset degree of the elevator car under each frame image, the energy feedback activation voltage threshold to be set by the energy feedback device controller after receiving the current frame image is obtained, so as to dynamically adjust the energy feedback strategy.

[0071] It should be noted that the total offset degree of the elevator car under each frame image obtained in step S3 not only reflects the current unbalance state of the car, but also predicts the risk of future severe shaking, so that the total offset degree of the elevator car under each frame image can be used to dynamically adjust the starting sensitivity of the energy feedback device in real time, and is specifically used for adjusting the activation voltage threshold of the energy feedback thereof.

[0072]

[0073] ​Specifically, the total offset of the elevator car in each frame of the image, along with the corresponding frame sequence number, is encapsulated into a standard format data frame. This data frame is then transmitted in real time to the energy feedback device controller in the elevator's main control system via the elevator's internal Ethernet bus. The preset control functions within the energy feedback device are as follows:

[0074] ;

[0075] in, The energy feedback device controller receives the first After the frame image, the energy feedback activation voltage threshold to be set; It is the standard activation voltage threshold, which is the normal setting value for smooth elevator operation and represents the minimum sensitivity state of the controller. It is the limit activation voltage threshold, which is the lowest safe voltage threshold that can be set when the system detects an extremely high risk. It represents the highest sensitivity state of the system. Based on the natural constant An exponential function with base 0; It is the first The total offset of the elevator car corresponding to the frame image The normalized value can be processed using maximum and minimum value normalization.

[0076] When the The greater the total offset of the elevator car corresponding to the frame image, the more the energy feedback device controller receives the first frame. The smaller the energy feedback activation voltage threshold to be set after the first frame of data, the more sensitive the energy feedback device will be; when the first frame of data is used... The smaller the total offset of the elevator car corresponding to the frame image, the more effective the energy feedback device controller will be when it receives the first frame image. The higher the energy feedback activation voltage threshold is after each frame of data, the more sluggish the energy feedback device will become. This can be addressed by dynamically adjusting the energy feedback device controller upon receiving the first frame of data. After the frame data is set, the energy feedback activation voltage threshold will be set to activate the energy feedback in advance when the risk of car imbalance is high, so as to avoid energy waste or equipment damage due to shaking.

[0077] Specifically, the controller of the energy feedback device will monitor the DC bus voltage of its internal capacitor in real time. Once the DC bus voltage exceeds the activation voltage threshold that has just been dynamically updated, the energy feedback device will immediately start the inverter circuit to convert the regenerated electrical energy stored in the capacitor into AC power that is in phase and frequency with the grid and send it back to the grid.

[0078] Through the above operation, the starting time of the energy feedback device is no longer fixed, but is intelligently advanced or delayed according to the real-time balance state of the car. At the moment when the balance state of the car is about to change sharply, the energy feedback function is activated in advance, thereby effectively avoiding the energy waste caused by response delay. The regeneration electric energy recovery rate of the elevator under various complex working conditions is significantly improved, the overall energy feedback efficiency is optimized, the heating of the brake resistor is reduced, and the environment temperature of the machine room is improved.

Claims

1. A computer vision-based energy feedback elevator car balance condition monitoring method, characterized by, The application relates to an energy feedback device and a method for dynamically adjusting an energy feedback strategy of the energy feedback device. According to three-dimensional space coordinate vectors of each passenger corresponding key point in each frame image, a center of gravity coordinate vector of each passenger in each frame image is obtained; a ratio of an Euclidean distance of the center of gravity coordinate vectors of the passengers between a current frame and a previous frame to a time interval between the video frames is calculated to obtain a center of gravity instantaneous speed of each passenger in each frame image; A time attenuation factor is obtained according to a difference between a current time and a time when the passengers leave the elevator car; a first offset degree corresponding to each passenger is obtained by weighting a difference between the center of gravity instantaneous speeds of each passenger in the current frame and the previous frame image according to the time attenuation factor; A variance of the first offset degrees corresponding to all the passengers is calculated, and a normalized value of the variance is taken as a dynamic behavior risk corresponding to each frame image; and a total offset degree of the elevator car is obtained by combining an overload risk and a static unbalanced load risk of the elevator car corresponding to each frame image. According to the total displacement degree of the elevator car, the energy feedback device controller will set the energy feedback activation voltage threshold after receiving the current frame image, including: ; wherein, is the energy feedback activation voltage threshold to be set by the energy feedback device controller after receiving the first frame image; is the standard activation voltage threshold; is the limit activation voltage threshold; is an exponential function with a natural constant as the base; is the first frame image corresponding to the normalized value of the total displacement degree of the elevator car , using the maximum and minimum value normalization processing, thereby dynamically adjusting the energy feedback strategy.

2. The computer vision-based energy feedback elevator car balance condition monitoring method of claim 1, wherein, The method for obtaining the center of gravity coordinate vector of each passenger in each frame image according to the three-dimensional space coordinate vectors of each passenger corresponding key point in each frame image comprises the following steps: ； In the formula, Passengers In the The centroid coordinate vector in the frame image; It is an index for passengers; It is the index of the image frame; Passengers In the The set of three-dimensional spatial coordinates of key points that can be successfully detected in a frame image; It is a set of three-dimensional spatial coordinates An index of a key point; This is the key point. The weight weight corresponding to the body part represented; Passengers Key points In the The three-dimensional spatial coordinate vector in the frame image.

3. The computer vision-based energy feedback elevator car balance condition monitoring method of claim 1, wherein, The method for obtaining the time attenuation factor according to the difference between the current time and the time when the passengers leave the elevator car comprises the following steps: ； wherein is a time decay factor; is an exponential function with the natural constant as base; denotes the current time; denotes the passenger leaving the elevator car; is a decay coefficient; is the difference between the current time and the time of the passenger leaving the elevator car.

4. The computer vision-based energy feedback elevator car balance condition monitoring method of claim 1, wherein, The method for obtaining the first offset degree corresponding to each passenger by weighting the difference between the center of gravity instantaneous speeds of each passenger in the current frame and the previous frame image according to the time attenuation factor comprises the following steps: ； wherein is the passenger corresponding first offset degree; is the index of the passenger is the passenger the index of the image frame; is the number of image frames taken from the passenger entering the elevator car to leaving the elevator car; is the time decay factor; is the passenger the center of gravity instantaneous velocity in the first frame image; is the passenger the center of gravity instantaneous velocity in the first frame image; is the absolute value sign.

5. The computer vision-based energy feedback elevator car balance condition monitoring method of claim 1, wherein, The method for obtaining the overload risk of the elevator car corresponding to each frame image comprises the following steps: A ratio of a total weight of the loads in the elevator car corresponding to each frame image to a maximum value of the loads in the elevator car is taken as the overload risk of the elevator car corresponding to each frame image.

6. The computer vision-based energy feedback elevator car balance condition monitoring method of claim 1, wherein, The method for obtaining the static unbalanced load risk of the elevator car corresponding to each frame image comprises the following steps: A ratio of a module length of a collective center of gravity coordinate vector of all the passengers in each frame image to a maximum center of gravity offset allowed by the elevator car is taken as the static unbalanced load risk of the elevator car corresponding to each frame image.

7. The computer vision-based energy feedback elevator car balance condition monitoring method of claim 6, wherein, The module length of the collective center of gravity coordinate vector of all the passengers in each frame image comprises the following steps: The gravity center coordinate vector of each passenger in each frame of image, based on the assumption that the weights of all passengers are approximately consistent, takes the average value of all gravity center coordinate vectors as the collective gravity center coordinate vector of all passengers in the corresponding frame of image; projects the collective gravity center coordinate vector to a plane to obtain its length in two-dimensional space as the length of the collective gravity center coordinate vector of all passengers in each frame of image.

8. The computer vision-based energy feedback elevator car balance condition monitoring method of claim 1, wherein, The method for obtaining the total offset degree of the elevator car by combining the dynamic behavior risk, the overload risk and the static unbalanced load risk of the elevator car corresponding to each frame image comprises the following steps: First weight Second weight Third weight The specific value is set according to the actual application scenario and requirements, and must meet the following conditions: And the first weight Second weight Third weight The range of values ​​is , will the first weight Second weight Third weight Set to respectively Among them, respectively using The overload risk, static off-center load risk, and dynamic behavior risk of the elevator car corresponding to each frame of the image are weighted and summed to obtain the total offset of the elevator car corresponding to each frame of the image.

9. The computer vision-based energy feedback elevator car balance condition monitoring method according to claim 1, wherein, The method for dynamically adjusting the energy feedback strategy comprises the following steps: A controller of the energy feedback device monitors a direct-current bus voltage of an internal capacitor in real time, and once the direct-current bus voltage exceeds the activated voltage threshold which is dynamically updated, the energy feedback device immediately starts an inverter circuit to convert the regenerated electric energy stored in the capacitor into alternating current of the same frequency and phase as the power grid and send the alternating current back to the power grid.

Images