An object behavior detection method and device, electronic equipment and storage medium

Abstract

This invention discloses an object behavior detection method, apparatus, electronic device, and storage medium, comprising: acquiring an image of a detection area, including the elevator car, when the elevator car arrives at the destination floor and the elevator door opens; performing object detection and tracking on the image to determine the state information and movement trend of objects in the car; when the object's movement trend is towards the elevator door, determining whether the object's movement trend is a detection interference item based on the state information; if not, determining that the object's behavioral intention is to leave the car. When the detected object's movement trend is towards the elevator door, it indicates that the object may intend to leave the car. To further determine the object's behavioral intention, detection interference items can be eliminated through the object's state information. Determining the object's behavioral intention by eliminating detection interference items can improve the accuracy of detecting the object's intention to leave the car, facilitate elevator control, and improve elevator operating efficiency and safety.

Description

Technical Field

[0001] This invention relates to the field of object behavior detection technology, and in particular to an object behavior detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of intelligent elevators, technologies have emerged that use sensors to sense the movement trends of passengers in the waiting hall and control the elevator doors to close or open based on these trends.

[0003] In existing technologies, after sensors collect multiple frames of images from the waiting hall and the elevator car, the movement speed of passengers in the car is analyzed to predict whether they intend to leave. The elevator doors are then controlled based on this intention. However, passengers in the car may simply be approaching the doors or adjusting their standing position without intending to leave. Judging passenger intentions solely based on movement speed can easily lead to misjudgments. For example, someone pressing the close button on the control panel near the doors may be identified as someone intending to leave the car, resulting in inaccurate prediction of passenger intentions. This is detrimental to elevator door control, causing delayed door closing and reduced elevator door operating efficiency. Summary of the Invention

[0004] This invention provides an object behavior detection method to solve the problems of inaccurate prediction and poor door control when predicting passenger intentions to leave the elevator car by moving speed, which in turn reduces the operating efficiency of the elevator doors.

[0005] In a first aspect, the present invention provides an object behavior detection method, comprising:

[0006] When the elevator car reaches the destination floor and the elevator door opens, an image of the detection area is captured, the detection area including the elevator car;

[0007] Object detection and tracking are performed on the image to determine the state information and motion trend of objects in the elevator car;

[0008] When the object's movement trend is toward the stair door, determine whether the object's movement trend is a detection interference item based on the state information;

[0009] If not, it is determined that the object's behavioral intent is to leave the elevator car.

[0010] In a second aspect, the present invention provides an object behavior detection device, comprising:

[0011] An image acquisition module is used to acquire an image of a detection area, including the elevator car, when the elevator car arrives at the destination floor and the elevator door opens.

[0012] The object information acquisition module is used to perform object detection and tracking on the image to determine the state information and movement trend of the objects in the car;

[0013] The interference detection and judgment module is used to determine whether the motion trend of the object is an interference item based on the state information when the motion trend of the object is to move closer to the door.

[0014] The behavior intent determination module is used to determine that the behavior intent of the object is to leave the car when the object's motion trend is not a detection interference item.

[0015] Thirdly, the present invention provides an electronic device, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the object behavior detection method according to the first aspect of the present invention.

[0019] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the object behavior detection method described in the first aspect of the present invention.

[0020] The object behavior detection method provided in this invention acquires an image of a detection area, including the elevator car, when the elevator car reaches the destination floor and the elevator door opens. Object detection and tracking are performed on the image to determine the state information and movement trend of objects within the car. When the object's movement trend is towards the elevator door, the state information is used to determine whether the object's movement trend is a detection interference item. If not, the object's intention is determined to be to leave the car. Detecting an object's movement trend towards the elevator door indicates that the object may intend to leave the car. To further determine the object's intention, the object's state information can be used to determine whether the object's movement trend is a detection interference item. If so, it indicates that the object is not intending to leave the car; otherwise, it indicates that the object's intention is to leave the car. By eliminating detection interference items to determine whether the object's intention is to leave the car, the accuracy of detecting the object's intention to leave the car can be improved. Furthermore, this facilitates elevator control and improves elevator operating efficiency and safety.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of an object behavior detection method provided in Embodiment 1 of the present invention;

[0024] Figure 2 This is a schematic diagram of the sensor detection range provided in Embodiment 1 of the present invention;

[0025] Figure 3 This is a flowchart of an object behavior detection method provided in Embodiment 2 of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of an object behavior detection device provided in Embodiment 3 of the present invention;

[0027] Figure 5 This is a schematic diagram of the structure of the electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] Example 1

[0030] Figure 1 This is a flowchart of an object behavior detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to object behavior detection. The method can be executed by an object behavior detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the object behavior detection method includes:

[0031] S101. When the elevator car reaches the destination floor and the elevator door opens, an image of the detection area is acquired. The detection area includes the elevator car.

[0032] The elevator can be equipped with sensors for acquiring images of the detection area. There can be at least two sensors, located separately in the hall area and the car area. When there are two sensors, such as... Figure 2 As shown, two sensors 2 can be located above the car door 1, one sensor 2 facing the area outside the hall and the other sensor 2 facing the car area, to collect images of different areas respectively. Simultaneously, when the car door 1 is closed, the angle and distance between the sensor 2 facing the car area and the car door 1 are larger, which can avoid the influence of the high reflectivity of the car door and ensure image quality. Of course, there can also be only one sensor, and the sensor's viewing angle can be determined according to the door's state, such as... Figure 2 As shown, when the car door 1 is in the open state, the sensor 2's viewing angle is towards the area outside the hall; when the car door 1 is in the open state, the sensor 2's viewing angle is towards the car area. This embodiment does not limit the number or arrangement of sensors. It should be noted that the sensor's detection area can include the area between the elevator doors, which includes the elevator hall door and the car door, to collect the status information of objects entering and exiting the car as they pass through the area between the doors.

[0033] S102. Perform object detection and tracking on the image to determine the state information and motion trend of objects in the car.

[0034] The objects are typically passengers, but can also be robots, pets, etc. The object's state information can include at least one of the following: elevator floor, number of objects, amplitude and speed of movement, direction of movement, behavioral information, and location. This state information can be obtained by performing object detection and tracking on the images. In this embodiment, after acquiring the images, an image sequence can be generated according to the image acquisition time. This image sequence is then input into a pre-trained object detection and tracking model to identify each object in the elevator lobby, along with the state information of each object.

[0035] The object detection and tracking model can be a neural network such as RNN, CNN, or DNN. During training, the model can use images labeled with the orientation, position, volume, and speed of objects as training data. The labeled image sequence is input into the model to predict the orientation, position, volume, and speed of each object. The loss rate is then calculated using the predicted and labeled data, and the model parameters are adjusted based on the loss rate until it falls below a preset value, resulting in a well-trained object detection and tracking model. This embodiment does not restrict the model structure or the training method.

[0036] The movement trend of an object is its direction of movement, which can also be detected through object detection and tracking. It should be noted that the direction of movement here refers to the expected direction of movement with a certain degree of confidence, rather than a definite direction of movement.

[0037] S103. When the object's movement trend is toward the door, determine whether the object's movement trend is a detection interference item based on the status information.

[0038] An object's movement trend is towards the elevator door, meaning the object moves from a certain position in the elevator car towards the area where the door is located. If the object wants to leave the elevator car, it will inevitably move towards the area where the door is located. However, conversely, the fact that an object moves towards the area where the door is located does not necessarily mean that its intention is to leave the elevator car. Therefore, an object's movement trend of moving towards the door only indicates that its intention may be to leave the elevator car. For example, the elevator control panel is usually located next to the elevator door (car door). After the elevator reaches its destination floor, some objects inside the car will leave the car. An object located at the rear of the car (usually a passenger) will press the close button in front of the control panel near the door to speed up the closing of the elevator door, but the object's intention is not to leave the car.

[0039] When an object's movement trend is towards the elevator door, in order to further determine whether the object's intention is to leave the elevator car, it is necessary to eliminate detection interference items. Therefore, the object's movement trend can be determined based on the state information to determine whether it is a detection interference item. If not, then execute S104.

[0040] Among them, the detection interference item refers to the tendency of an object to move closer to the elevator door when its intention is not to leave the elevator car. For example, as mentioned above, the tendency of an object to move closer to the elevator door when it goes to the control panel to press the close button; it can also be the tendency of an object to move closer to the elevator door when it moves to improve the comfort of riding the elevator.

[0041] S104. Determine that the object's behavioral intention is to leave the elevator car.

[0042] If the object's movement trend is a detected interference item, then the object's behavioral intention is determined to be not to leave the elevator car, but to remain in the car and continue riding the elevator; if the object's movement trend is not a detected interference item, then the object's behavioral intention is determined to be to leave the elevator car.

[0043] Once the object's behavioral intention is determined, the elevator doors can be controlled accordingly. For example, if the object's behavioral intention is to leave the car, the door opening time can be delayed to ensure that the object can leave the car safely and in a timely manner, avoiding situations such as the object being pinched by the closing door or accidentally missing its destination floor due to the door closing accidentally. If the object's behavioral intention is not to leave the car, the door opening time can be shortened and the door can be closed earlier, which can improve the elevator's operating efficiency.

[0044] The object behavior detection method provided in this invention acquires an image of a detection area, including the elevator car, when the elevator car reaches the destination floor and the elevator door opens. Object detection and tracking are performed on the image to determine the state information and movement trend of objects within the car. When the object's movement trend is towards the elevator door, the state information is used to determine whether the object's movement trend is a detection interference item. If not, the object's intention is determined to be to leave the car. Detecting an object's movement trend towards the elevator door suggests that the object may intend to leave the car. To further determine the object's intention, the object's state information can be used to determine whether its movement trend is a detection interference item. If so, it indicates that the object is not intending to leave the car; otherwise, it indicates whether the object's intention is to leave the car. By eliminating detection interference items to determine the object's intention to leave the car, the accuracy of detecting the object's intention to leave the car can be improved. Furthermore, this facilitates elevator control and improves elevator operating efficiency and safety.

[0045] Example 2

[0046] Figure 3 This is a flowchart of an object behavior detection method provided in Embodiment 2 of the present invention. This embodiment optimizes Embodiment 1 as described above. Figure 3 As shown, the object behavior detection method includes:

[0047] S301. When the elevator car reaches the destination floor and the elevator door opens, an image of the detection area is acquired. The detection area includes the elevator car.

[0048] S302. Perform object detection and tracking on the image to determine the state information and motion trend of objects in the car.

[0049] S301-S302 are similar to S101-S102 in Embodiment 1. For details, please refer to S101-S102 in Embodiment 1. They will not be described here.

[0050] S303. When the object's movement trend is toward the door, determine whether the object's movement trend is a detection interference item based on the status information.

[0051] If not, execute S304.

[0052] Specifically, in an optional embodiment of the present invention, the state information includes the floor level, and determining whether the object's movement trend is a detection interference item based on the state information includes:

[0053] Determine if the elevator floor in the object's status information is the current floor; if so, determine the object's movement trend as a detection interference item.

[0054] A tracking queue can be established for objects entering the elevator until they leave. When an object's movement trend is towards the elevator door, its boarding floor can be identified; this boarding floor is the floor from the lobby to the elevator car. For objects whose movement trend is towards the elevator door, if their boarding floor is the current floor, and they approach the door after entering the car, it can be determined that they are either going to the control panel to press the desired floor button or adjusting their position in the car. In this case, the object's movement trend can be considered a detection interference item. However, if their boarding floor is not the current floor, and they approach the door when it opens, it can be assumed that the object is leaving the car.

[0055] It should be noted that for objects that temporarily leave the elevator to allow other objects to leave the car, their floor number may or may not be updated, and this will not affect the judgment of whether the object's movement trend is a detection interference item.

[0056] When an object's movement trend is toward the stairwell door, comparing the object's current floor with its previous floor in the stairwell can determine whether the movement trend is a distractor. This method is simple and efficient, and can improve the accuracy of judging the object's behavioral intentions.

[0057] In an optional embodiment of the present invention, the state information includes the number of objects, the amplitude of their movements, and their speed. Determining whether the movement trend of the objects is a detection interference item based on the state information includes:

[0058] When an object moves, the amplitude and speed of the object's movement are acquired; a first number of objects in the car before the object moves are determined; a second number of objects in the car after the object moves are determined; when the second number is less than the first number, the amplitude of the movement is less than a preset amplitude threshold, and the speed is less than a preset speed threshold, the movement trend of the object is determined as a detection interference item.

[0059] As a form of public transportation, the number of objects inside an elevator is usually dynamic. When there are many objects (mainly passengers), they will actively reduce their occupied space to allow more passengers to enter the elevator car. When there are few objects, they usually will not reduce or expand their occupied space to ensure a comfortable ride. Therefore, if the number of objects in the elevator car decreases, meaning the usable space per person increases, the objects can adjust their occupied space by moving.

[0060] The range of motion of an object refers to the extent to which its body extends during movement. It can be the ratio of the object's stride length to its height. Generally, a taller object will have a longer stride length under the same conditions. Therefore, the ratio of stride length to height can be used to measure the range of motion. For example, for a passenger moving only inside the elevator car and not leaving, the ratio of stride length to height is smaller, indicating a smaller range of motion. Conversely, for a passenger leaving the car, the ratio is larger, indicating a larger range of motion. The range of motion can also be the combined range of motion of multiple specific parts of the object. When the object is a passenger, specific parts could be the passenger's hands and feet. The combined range of motion can be obtained by combining the ranges of motion of the hands and feet. Specifically, a corner of the elevator car can be used as the origin of a three-dimensional coordinate system. The three-dimensional coordinates of multiple specific parts of the object before and after movement can be recorded. The range of motion of each specific part can be obtained based on these coordinates, leading to the combined range of motion. This combined range of motion can be calculated using methods such as addition or assigning weighting coefficients. If the amplitude of an object's movement is less than a preset amplitude threshold, the object's movement trend is more likely to be a detection interference item; if the amplitude of an object's movement is greater than the preset amplitude threshold, the object's movement trend is less likely to be a detection interference item.

[0061] An object's velocity is its displacement per unit time. The speed of an object leaving the elevator car is generally greater than the speed of an object moving only within the car. Therefore, an object's velocity can be used as one of the factors to determine whether its motion trend is a potential interference. When the velocity is less than a preset velocity threshold, the object's motion trend is more likely to be a potential interference; when the velocity is greater than the preset velocity threshold, the object's motion trend is less likely to be a potential interference.

[0062] When the second quantity is less than the first quantity, the amplitude of the movement is less than a preset amplitude threshold, and the speed is less than a preset speed threshold, the object's motion trend is determined as a detection interference item. When the second quantity is less than the first quantity, it indicates that the object may be moving to improve elevator comfort. Furthermore, if the object's amplitude of movement is less than a preset amplitude threshold and its speed is less than a preset speed threshold, both of these conditions strongly suggest that the object's motion trend is a detection interference item. Therefore, based on the premise that the object may be moving to improve elevator comfort, the object's motion trend can be determined as a detection interference item.

[0063] This embodiment combines the changes in the number of objects inside the car with the range and speed of the objects' movements, which better reflects the actual use of the car and can improve the accuracy of predicting the movement trend of the objects.

[0064] In an optional embodiment of the present invention, the state information includes location points and behavior information. Determining whether the object's movement trend is a detection interference item based on the state information includes:

[0065] When an object moves, its position and behavior information are obtained from the state information. The object's direction of movement is determined based on its position. When the direction of movement is towards the control panel of the car and the behavior information is a hand-raising action, the object's movement trend is determined as a detection interference item.

[0066] The elevator control panel is typically located next to the elevator door (car door), with the door open / close buttons located in a specific position on the control panel. It's important to note that motion trend can represent a broad direction of object movement. When an object moves towards the control panel, since the control panel is next to the elevator door, it can still be considered as moving towards the door. Therefore, this embodiment obtains the object's orientation by further refining the direction of its movement. When an object wants to operate the buttons on the control panel to select a floor or close the door, it typically makes a hand-raising motion, i.e., raising its arm towards the control panel. Therefore, when the object's behavior information indicates a hand-raising motion, the object's motion trend may be a detection interference item.

[0067] Determining the object's orientation based on its location includes:

[0068] Obtain the position of the object in each collection cycle from the status information; calculate the first distance from each position point to the control box of the car; calculate the distance difference between two adjacent first distances; when the distance difference of a first preset number of consecutive times is greater than the preset difference, determine the object's movement orientation as the control box facing the car.

[0069] When an object is about to go to the control panel, it usually walks straight towards it, resulting in a larger difference in the distance between two adjacent first distances. However, when an object is about to go to the elevator door, i.e., leave the car, it does not walk straight towards the control panel, resulting in a smaller difference in the distance between two adjacent first distances. The preset difference can be set according to the position of the control panel in the car, etc., and this embodiment does not impose any restrictions on this.

[0070] Of course, a base point can also be set inside the car, and the direction of the object's movement can be determined based on the angle between the object and the control box and the base point at each position. This invention does not impose any limitations on this.

[0071] This embodiment determines whether the object's movement trend is a detection interference item by combining the object's movement orientation and behavior information, which is more in line with people's usage habits when taking the elevator, and makes the detection results more accurate.

[0072] S304. Determine that the object's behavioral intention is to leave the elevator car.

[0073] When the motion trend of an object is determined to be a detection interference item, the object's behavioral intention is determined not to leave the car. When the motion trend of an object is determined not to be a detection interference item, the object's behavioral intention is determined to leave the car.

[0074] S305. When there are no waiting objects in the elevator lobby, determine whether there is an object in the elevator car whose intention is to leave the elevator car based on the current image.

[0075] When there are no passengers waiting in the elevator lobby, the elevator door status only needs to be controlled based on the passenger status inside the elevator car. For example, during working hours, for tall office buildings, passengers are usually only picked up in the lobby on the first floor and then transported to their destination floors. When the elevator stops at each destination floor, there are usually no passengers waiting. In this case, it is only necessary to monitor whether passengers who intend to leave the elevator car have left.

[0076] When the elevator car arrives at the waiting hall and the elevator door opens, objects intending to leave the car will leave the car one by one. In this step, determining whether there are objects in the car with the intention to leave the car means determining whether all objects in the car with the intention to leave the car have left the car. If yes, execute S306; if no, execute S307.

[0077] Specifically, determining whether there is an object in the elevator car with the intention to leave the car includes: performing target detection and tracking on the object to obtain the current location of the target object; when the location of the object is outside the elevator door, it is determined that there is no target object in the car with the intention to leave the car; when the location of the target object is in the car, it is determined that there is a target object in the car with the intention to leave the car.

[0078] S306. Control the ladder door according to the location of the target object.

[0079] In an optional embodiment, it can be determined whether a target object located in a preset door gap area is detected in the current image; if yes, the elevator door is kept open, a countdown timer is started, and the next frame image is acquired as the current image, returning to the step of determining whether a target object located in the preset door gap area is detected in the current image; if no, it is determined whether a target object located in a preset near-door area on the side of the elevator hall is detected in the current image; when a target object located in the near-door area is detected, a second distance from the target object to the door gap area is acquired; the countdown timer is adjusted according to the second distance, and the elevator door is controlled according to the countdown time; when no target object located in the near-door area is detected, the next frame image is acquired as the current image, and the step of determining whether there is a target object in the car whose behavioral intention is to leave the car is returned based on the current image.

[0080] The area between the doors of a staircase is the area centered on the door, and it can be any one of a square, a sector, or a circle, or a combination of any two shapes. The area near the door is the area adjacent to the door and does not include the area between the doors. The specific size and location of the area near the door and the area between the doors can be determined according to the actual situation.

[0081] Specifically, by adjusting the countdown timer based on the second distance, it can be determined whether the second distance is less than a preset distance threshold. When the second distance is less than or equal to the preset distance threshold, the countdown timer is reduced by a first value (e.g., 1 second). When the second distance is greater than the preset distance threshold, the countdown timer is reduced by a second value (e.g., 2 seconds).

[0082] When controlling the elevator door based on the countdown timer, it can determine if the countdown timer is greater than 0. If so, the elevator door remains open, the next frame is acquired as the current image, and the process returns to determine if a target object within the preset near-door area on the waiting hall side is detected in the current image. If not, the process returns to determining if a target object with the intention to leave the elevator car exists in the current image. In other words, when a target object within the preset near-door area on the waiting hall side is detected, meaning the target object has exited the elevator car, the countdown timer can be reduced. It should be noted that when a new target object exits the elevator car, the countdown timer is restarted and reduced. Ultimately, the reduction is based on the second distance corresponding to the last target object to leave the elevator car.

[0083] S307, Control the elevator door to close.

[0084] When there are no people waiting in the elevator lobby, the elevator door status can be controlled based on the riding status of people in the elevator car. When there are no people in the car who intend to leave, it means that all people who intend to leave have left the car, and the elevator door can be closed without waiting for the preset closing time, thus improving the elevator's operating efficiency. This is especially true during the morning rush hour when there are many floors to stay on, and the control method in this embodiment can significantly improve the elevator's operating efficiency.

[0085] The object behavior detection method provided in this invention acquires an image of a detection area, including the elevator car, when the elevator car reaches the destination floor and the elevator door opens. Object detection and tracking are performed on the image to determine the state information and movement trend of objects within the car. When the object's movement trend is towards the elevator door, the state information is used to determine whether the object's movement trend is a detection interference item. If not, the object's intention is determined to be to leave the car. Detecting an object's movement trend towards the elevator door indicates that the object may intend to leave the car. To further determine the object's intention, the object's state information can be used to determine whether the object's movement trend is a detection interference item. If so, it indicates that the object does not intend to leave the car; otherwise, it indicates that the object's intention is to leave the car. By eliminating detection interference items to determine whether the object's intention is to leave the car, the accuracy of detecting the object's intention to leave the car can be improved. This facilitates elevator control and improves elevator operating efficiency and safety.

[0086] Furthermore, by combining at least one of the following factors—elevator floor, number of objects, range and speed of movement, direction of movement, behavioral information, and location—to determine whether the object's movement trend is a detection interference, the accuracy of detecting the object's behavioral intent can be improved.

[0087] Furthermore, when there are no waiting passengers in the elevator lobby, if the current image indicates that there is a target passenger in the lobby with the intention of leaving the elevator car, a timer is started to count down when the target passenger is in the doorway area. The timer is shortened when the target passenger moves from the doorway area to the near-door area on the side of the elevator lobby. This achieves dynamic adjustment of the timer's timing based on the target passenger's position in the elevator lobby as they move from the elevator car. Finally, when the timer equals 0, the elevator door is controlled to close, which can shorten the door opening time and improve the elevator door's operating efficiency.

[0088] Example 3

[0089] Figure 4 This is a schematic diagram of the structure of an object behavior detection device provided in Embodiment 3 of the present invention. Figure 4 As shown, the object behavior detection device includes:

[0090] The image acquisition module 401 is used to acquire an image of the detection area when the elevator car arrives at the destination floor and the elevator door opens, the detection area including the elevator car;

[0091] The object information acquisition module 402 is used to perform object detection and tracking on the image to determine the state information and movement trend of the objects in the car;

[0092] The interference detection judgment module 403 is used to determine whether the motion trend of the object is an interference detection item based on the state information when the motion trend of the object is to move closer to the door.

[0093] The behavior intent determination module 404 is used to determine the behavior intent of the object as leaving the car when the object's motion trend is not a detection interference item.

[0094] In an optional embodiment of the present invention, the status information includes the floor number on the elevator; the interference detection judgment module 403 includes:

[0095] The elevator floor determination submodule is used to determine whether the elevator floor in the status information of the object is the current floor;

[0096] The first detection interference determination submodule is used to determine the motion trend of the object as a detection interference when the elevator floor in the object's status information is the current floor.

[0097] In an optional embodiment of the present invention, the state information includes the number of objects, the amplitude of their movements, and their speed; the interference detection and judgment module 403 includes:

[0098] The object motion information acquisition submodule is used to acquire the motion amplitude and speed of the object when the object moves;

[0099] A first quantity submodule is used to determine a first quantity of objects in the car before the objects move;

[0100] The second quantity submodule is used to determine a second quantity of objects in the car after the object has moved;

[0101] The second detection interference determination submodule is used to determine the motion trend of the object as a detection interference when the second quantity is less than the first quantity, the motion amplitude is less than a preset amplitude threshold, and the speed is less than a preset speed threshold.

[0102] In an optional embodiment of the present invention, the state information includes location points and behavioral information, and the interference detection judgment module 403 includes:

[0103] The position and behavior information acquisition submodule is used to acquire the position and behavior information of the object from the state information when the object moves;

[0104] The motion orientation determination submodule is used to determine the motion orientation of the object based on the object's position point;

[0105] The third interference detection submodule is used to determine the motion trend of the object as an interference detection item when the motion orientation is towards the operating box of the car and the behavior information is a hand-raising behavior.

[0106] In an optional embodiment of the present invention, the motion orientation determination submodule includes:

[0107] The location point acquisition unit is used to acquire the location points of the object in each acquisition cycle;

[0108] The first distance calculation unit is used to calculate the first distance from each of the said location points to the operation box of the car;

[0109] A distance difference calculation unit is used to calculate the distance difference between two adjacent first distances;

[0110] The motion orientation determination unit is used to determine the motion orientation of the object as the operation box facing the car when the distance difference is greater than a preset first number of consecutive preset distance differences.

[0111] In an optional embodiment of the present invention, the object behavior detection device further includes:

[0112] The target object determination module is used to determine whether there is a target object in the elevator car with the intention of leaving the elevator car when there is no waiting object in the waiting hall; if so, the content executed by the elevator door control module is executed; otherwise, the content executed by the elevator door closing module is executed.

[0113] The ladder door control module is used to control the ladder door according to the location point of the target object;

[0114] The ladder door closing module is used to control the closing of the ladder door.

[0115] In an optional embodiment of the present invention, the stair door control module includes:

[0116] The target object detection submodule is used to determine whether a target object located in a preset door inter-region is detected in the current image; if so, the content executed by the next frame image acquisition submodule is executed; otherwise, the content executed by the near door region detection submodule is executed.

[0117] The next frame image acquisition submodule is used to control the ladder door to remain open, start the countdown timer to count down, acquire the next frame image as the current image, and return to the content executed by the target object detection submodule;

[0118] The near-door area detection submodule is used to determine whether a target object located in a preset near-door area on the side of the waiting hall is detected in the current image;

[0119] The second distance detection submodule is used to obtain the second distance from the target object to the door area when a target object located in the near door area is detected;

[0120] The timing calculation submodule is used to adjust the timing time of the countdown timer according to the second distance, and control the ladder door according to the timing time;

[0121] The next frame image acquisition submodule is used to acquire the next frame image as the current image when no target object is detected in the near door area, and return the content executed by the target object determination module.

[0122] In an optional embodiment of the present invention, the timing calculation submodule includes:

[0123] The second distance determination unit is used to determine whether the second distance is greater than a preset distance threshold.

[0124] The first time reduction unit is used to subtract a preset first value from the countdown timer when the second distance is less than or equal to the preset distance threshold.

[0125] The second time reduction unit is used to subtract a preset second value from the countdown timer when the second distance is greater than the preset distance threshold.

[0126] In an optional embodiment of the present invention, the timing calculation submodule includes:

[0127] The timing judgment unit is used to determine whether the timing time of the countdown timer is greater than 0; if so, the content executed by the first door control unit is executed; otherwise, the content executed by the second door control unit is executed.

[0128] The first stair door control unit is used to control the stair door to remain open, acquire the next frame image as the current image, and return to the step of determining whether a target object located in the preset near-door area on the side of the waiting hall is detected in the current image;

[0129] The second door control unit is used to acquire the next frame image and return the content executed by the target object determination module.

[0130] The object behavior detection device provided in the embodiments of the present invention can execute the object behavior detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0131] Example 4

[0132] Figure 5A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0133] like Figure 5 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded into the RAM 43 from storage unit 48. The RAM 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0134] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as object behavior detection methods.

[0136] In some embodiments, the object behavior detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the object behavior detection method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the object behavior detection method by any other suitable means (e.g., by means of firmware).

[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0139] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0141] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0142] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0143] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0144] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting object behavior, characterized in that, include: When the elevator car reaches the destination floor and the elevator door opens, an image of the detection area is captured, the detection area including the elevator car; Object detection and tracking are performed on the image to determine the state information and movement trend of objects in the elevator car; the state information of the objects includes at least one of the following: floor level, number of objects, amplitude and speed of movement, direction of movement, behavioral information, and location point; When the movement trend of the object is to move closer to the elevator door, it is determined whether the movement trend of the object is a detected interference item based on the state information; the detected interference item includes the movement trend of the object moving closer to the elevator door when it goes to the control box to press the close button, and / or the movement trend of the object moving closer to the elevator door when it moves to improve the comfort of riding the elevator. If not, it is determined that the object's behavioral intent is to leave the elevator car.

2. The method as described in claim 1, characterized in that, The status information includes the floor number on the elevator; determining whether the object's movement trend is a detection interference item based on the status information includes: Determine whether the elevator floor in the object's status information is the current floor; If so, the motion trend of the object is determined as a detection interference item.

3. The method as described in claim 1, characterized in that, The state information includes the number of objects, the amplitude of their movements, and their speed. Based on this state information, it is determined whether the movement trend of the objects constitutes a detection interference item, including: When the object moves, the amplitude and speed of the object's movement are acquired; Determine a first number of objects in the car before the objects move; Determine a second number of objects in the car after the object has moved; When the second quantity is less than the first quantity, the amplitude of the action is less than a preset amplitude threshold, and the speed is less than a preset speed threshold, the motion trend of the object is determined as a detection interference item.

4. The method as described in claim 1, characterized in that, The state information includes location points and behavior information. Determining whether the object's movement trend is a detection interference item based on the state information includes: When the object moves, the object's position and behavior information are obtained from the state information; Determine the direction of motion of the object based on its location point; When the movement direction is toward the control panel of the car and the behavior information is a hand-raising behavior, the movement trend of the object is determined as a detection interference item.

5. The method as described in claim 4, characterized in that, Determining the motion orientation of the object based on its position point includes: Obtain the location points of the object in each acquisition cycle; Calculate the first distance from each of the aforementioned location points to the control panel of the car; Calculate the distance difference between two adjacent first distances; When the distance difference is greater than a preset difference for a first number of consecutive preset number of times, the direction of movement of the object is determined to be towards the control box of the car.

6. The method according to any one of claims 1-5, characterized in that, The status information includes the location point of the object, and after determining that the object's behavioral intention is to leave the elevator car, it also includes: When there are no people waiting in the elevator lobby, determine whether there is a target person in the elevator car whose intention is to leave the elevator car based on the current image; If so, control the ladder door according to the location of the target object; If not, control the elevator door to close.

7. The method as described in claim 6, characterized in that, The step of controlling the stair door based on the location point of the target object includes: Determine whether a target object located in a preset doorway region is detected in the current image; If so, control the ladder door to remain open, start the countdown timer to count down, acquire the next frame image as the current image, and return to the step of determining whether a target object located in the preset door area is detected in the current image; If not, determine whether a target object located in the preset near-door area on the side of the waiting hall is detected in the current image; When a target object is detected within the near-door area, a second distance from the target object to the door area is obtained; The countdown timer is adjusted according to the second distance, and the elevator door is controlled according to the countdown time. If no target object is detected in the near door area, the next frame image is acquired as the current image, and the process of determining whether there is a target object in the car whose behavior is intended to leave the car is returned based on the current image.

8. The method as described in claim 7, characterized in that, The step of adjusting the countdown timer based on the second distance includes: Determine whether the second distance is greater than a preset distance threshold; When the second distance is less than or equal to the preset distance threshold, the countdown timer's timing time is reduced by a preset first value; When the second distance is greater than the preset distance threshold, the countdown timer's timing time is reduced by a preset second value.

9. The method according to claim 7, wherein controlling the elevator door according to the timing comprises: Determine whether the countdown timer's duration is greater than 0; If so, control the elevator door to remain open, acquire the next frame image as the current image, and return to the step of determining whether a target object located in the preset near-door area on the side of the elevator hall is detected in the current image; If not, acquire the next frame image and return to the step of determining whether there is a target object in the car whose behavior intends to exit the car based on the current image.

10. An object behavior detection device, characterized in that, include: An image acquisition module is used to acquire an image of a detection area, including the elevator car, when the elevator car arrives at the destination floor and the elevator door opens. The object information acquisition module is used to perform object detection and tracking on the image to determine the state information and movement trend of the objects in the car; The interference detection and judgment module is used to determine whether the motion trend of the object is an interference item based on the state information when the motion trend of the object is to move closer to the door. The behavior intent determination module is used to determine that the behavior intent of the object is to leave the car when the object's motion trend is not a detection interference item; The object behavior detection device is used to perform the object behavior detection method according to any one of claims 1-9.

11. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the object behavior detection method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the object behavior detection method according to any one of claims 1-9.

Images