A robot lift detection method, device, electronic equipment and storage medium

Abstract

The application discloses a robot elevator riding detection method and device, electronic equipment and a storage medium. The robot elevator riding detection method comprises the following steps: collecting an image of a detection area through a sensor when an elevator door of an elevator is opened; performing target detection and tracking on the image to obtain motion data and behavior detection results of an object in the detection area; calculating a score of the object being a robot according to the motion data and the behavior detection results; and determining that the object is a robot when the score is greater than a score threshold. The motion data of the robot riding the elevator and the behavior are used to identify the robot riding the elevator, the accuracy of identifying the robot is improved, the problem that the robot is misjudged as a child with weak behavior ability and the elevator is temporarily stopped from running and using is avoided, the normal running of the elevator when the robot rides the elevator is ensured, and the consistency of the robot elevator control and the detection system elevator control is ensured.

Description

Technical Field

[0001] This invention relates to the field of elevator control technology, and in particular to a robot elevator detection method, device, electronic device, and storage medium. Background Technology

[0002] With the increasing demand for intelligent elevators, the use of robots to deliver packages, takeout food, and other goods in buildings has become a widely adopted technology.

[0003] In existing elevator waiting technologies, there are solutions that use detection systems to determine whether people entering the elevator are children or adults based on their height and area projection. After a person enters the elevator, the system can further analyze their button presses to determine if they are a child with limited mobility. If a child with limited mobility is identified, the elevator stops running. However, this approach of distinguishing people based solely on height, area projection, and button presses is problematic because service robots in buildings vary greatly in form. Robots of different heights and area projections are constantly emerging, and since they communicate with the elevator car to call the elevator, they do not need to press any buttons after entering. Therefore, when the detection system only identifies objects based on height, area projection, and button presses, it cannot accurately distinguish between people and robots.

[0004] Without a communication connection between the detection system and the robot elevator control, when judging the entry of an object based on height and area projection and button control operation, the robot may be identified as a child with weak behavioral ability, thus causing the elevator to stop operating and causing a logical conflict between the robot elevator control and the detection system elevator control. Summary of the Invention

[0005] This invention provides a robot elevator detection method, device, electronic device, and storage medium to solve the problem in the prior art of misidentifying a robot as a child, causing the elevator to stop operating.

[0006] In a first aspect, the present invention provides a robot elevator detection method, comprising:

[0007] When the elevator door opens, the sensor collects an image of the detection area;

[0008] The image is subjected to target detection and tracking to obtain motion data and behavior detection results of objects within the detection area;

[0009] The score for the object to be a robot is calculated based on the motion data and the behavior detection results;

[0010] When the score is greater than a preset score threshold, the object is determined to be a robot.

[0011] In a second aspect, the present invention provides a robot elevator detection device, comprising:

[0012] The image acquisition module is used to acquire images of the detection area via sensors when the elevator doors open.

[0013] The target tracking module is used to perform target detection and tracking on the image to obtain motion data and behavior detection results of objects inside the car;

[0014] The score calculation module is used to calculate a score for the object as a robot based on the motion data and the behavior detection results;

[0015] The robot identification module is used to identify the object as a robot when the score is greater than a preset score threshold.

[0016] Thirdly, the present invention provides an electronic device, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the robot elevator detection method according to the first aspect of the present invention.

[0020] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the robot elevator detection method described in the first aspect of the present invention.

[0021] This invention, in its embodiment, acquires images of the detection area via sensors when the elevator doors open. Target detection and tracking are performed on the images to obtain motion data and behavior detection results of objects within the detection area. Based on the motion data and behavior detection results, a score is calculated to determine if the object is a robot. If the score exceeds a preset threshold, the object is confirmed as a robot. This embodiment acquires motion data such as the detected object's direction of movement, speed, projected area, and running posture, as well as behavior detection results of button operations after entering the elevator car. It fully utilizes the motion data and behavior of the robot during its elevator ride to determine whether a robot is in the elevator, improving the accuracy of robot identification and avoiding the problem of misjudging a robot as a child with limited behavioral abilities, leading to elevator shutdown. The elevator operates normally when a robot is in the elevator, and the consistency between the robot's elevator control and the detection system's elevator control is ensured.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the sensor installation in the elevator in this embodiment;

[0025] Figure 2 This is a flowchart of a robot elevator detection method provided in Embodiment 1 of the present invention;

[0026] Figure 3 This is a flowchart of a robot elevator detection method provided in Embodiment 2 of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of a robot elevator detection device provided in Embodiment 3 of the present invention;

[0028] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] Example 1

[0031] Figure 2 This is a flowchart of a robot elevator detection method provided in Embodiment 1 of the present invention. This embodiment can be applied to the identification of robots riding elevators. The method can be executed by a robot elevator detection device, which can be implemented in hardware and / or software. The robot elevator detection device can be configured in an electronic device, such as in the elevator controller.

[0032] like Figure 2As shown, the robot's elevator detection method includes:

[0033] S201. When the elevator door opens, the sensor collects an image of the detection area.

[0034] like Figure 1 As shown, in this embodiment, a sensor 3 is installed on the elevator car. The sensor can be a regular camera, such as a black and white camera or an RGB camera, or an active light sensor, such as a TOF (Time of Flight) depth sensor or a structured light sensor, which is a sensor that senses depth by emitting and receiving light. Of course, the sensor can also be a radar, etc. This embodiment does not limit the type of sensor.

[0035] In one example, the sensor in this embodiment can be an active light sensor 3, which can be installed on the car 1 so that the detection area of ​​the sensor 3 covers the area inside the car 1 and the area of ​​the waiting hall 2. Optionally, the sensor 3 can be installed on the lintel of the car door of the car 1, for example, in the middle position of the lintel of the car door, so that the detection area of ​​the sensor 3 covers the area inside the car 1 and the area of ​​the waiting hall 2.

[0036] In one example, the number of sensors 3 can be one, that is, the sensor 3 can be an active light sensor with a large field of view. Furthermore, when the number of sensors 3 is one, the angle of the sensor 3 can be adjusted or fixed, or, that is, the angle between the light emission axis of the sensor 3 and the vertical direction is variable, that is, the sensor 3 can adjust the angle so that the detection area of ​​the sensor 3 can be expanded or reduced.

[0037] like Figure 1 As shown, in another example, there can be two sensors 3. One sensor 3 has its light emission axis facing the inside of the car 1 to capture images of the area inside the car 1, while the other sensor 3 has its light emission axis facing the waiting hall 2 to capture images of the area in the waiting hall 2. Optionally, the detection ranges of the two sensors 3 overlap to ensure that images of the entire area from the car 1 to the waiting hall 2 can be captured, while avoiding overexposure problems that occur after the elevator doors close when the sensors are active light sensors with a single vertical light emission axis and simultaneously capturing images of the entire area from the car 1 to the waiting hall 2. Exemplarily, the angle between the light emission axes of the two sensors 3 and the vertical direction can be adjusted to adjust the detection area of ​​the two sensors 3. Of course, the angle between the light emission axes of the two sensors 3 and the vertical direction can also remain fixed.

[0038] In practical applications, sensor 3 can collect depth images at a preset frame rate and send the depth images to the elevator controller. The elevator usually opens the elevator door when it reaches a certain target floor. At this time, sensor 3 collects depth images of the area inside the car 1 and the area of ​​the waiting hall 2.

[0039] S202. Perform target detection and tracking on the image to obtain motion data and behavior detection results of objects within the detection area.

[0040] In this embodiment, motion data includes at least one of the following: the object's movement direction, movement speed, projected area, and running posture. The behavior detection result can be whether the object operates the buttons inside the elevator car. The movement direction refers to the angle between the direction in which the detected object moves toward the elevator door and the plane where the elevator door is located. The projected area is the projection of the outer contour of the detected object in the direction perpendicular to the bottom of the elevator car. The movement speed can be the speed at which the detected object enters the elevator car from the waiting hall. The running posture can be the object's height change, the amplitude of the outer contour swing, etc. Operating the buttons inside the elevator car can be selecting the button corresponding to the target floor or opening / closing the elevator door.

[0041] This embodiment can pre-train a target detection and tracking model, which can identify objects and track them, obtaining at least one of the object's movement direction, movement speed, projected area, and running posture, as well as identifying whether the object has operated the buttons inside the car.

[0042] S203. Calculate the score of the robot based on motion data and behavior detection results.

[0043] In this embodiment, a score calculation method can be set based on the characteristics of the robot entering the elevator car from the waiting hall. For example, when the robot rides the elevator, it moves perpendicularly from the waiting hall to the elevator door into the car. During this process, the robot's posture, speed, and height are relatively stable. Furthermore, compared to a human, the robot does not swing its arms or make angular movements, and the robot's projected area on the car floor remains relatively fixed. Moreover, after entering the car, the robot does not need to operate the floor selection buttons inside. Based on these characteristics, a score calculation method can be set. In one example, the rate of change of speed, the rate of change of area, and the rate of change of posture can be calculated using the movement speed, projected area, and running posture detected in two adjacent frames. A positive value is assigned when the detected object operates any buttons inside the car, and a negative value is assigned otherwise. Therefore, the score for the robot can be set to be positively correlated with the direction of movement and negatively correlated with the average rate of change of speed, the average rate of change of projected area, the average rate of change of posture, and the assigned value of the behavior detection result. Of course, weights can also be assigned to each data point, and a weighted sum can be calculated to obtain the score for the robot as the object.

[0044] The higher the score, the greater the probability that the object is a robot; conversely, the lower the score, the lower the probability that the object is a robot.

[0045] S204. When the score is greater than the preset score threshold, the object is determined to be a robot.

[0046] This embodiment can set a score threshold for an object to be a robot. When the score of an object is greater than the score threshold, the object is determined to be a robot. The elevator door can be controlled to close according to the elevator door control strategy for objects to be robots. For example, when only a robot is riding in the elevator car, the elevator car can be controlled to go directly to the floor that the robot needs to reach.

[0047] This invention, in its embodiment, acquires images of the detection area via sensors when the elevator doors open. Target detection and tracking are performed on the images to obtain motion data and behavior detection results of objects within the detection area. Based on the motion data and behavior detection results, a score is calculated to determine if the object is a robot. If the score exceeds a preset threshold, the object is confirmed as a robot. This embodiment acquires motion data such as the detected object's direction of movement, speed, projected area, and running posture, as well as behavior detection results of button operations after entering the elevator car. It fully utilizes the motion data and behavior of the robot during its elevator ride to determine whether a robot is in the elevator, improving the accuracy of robot identification and avoiding the problem of misjudging a robot as a child with limited behavioral abilities, leading to elevator shutdown. The elevator operates normally when a robot is in the elevator, and the consistency between the robot's elevator control and the detection system's elevator control is ensured.

[0048] Example 2

[0049] Figure 3 This is a flowchart of a robot elevator detection method provided in Embodiment 2 of the present invention. This embodiment of the present invention is an optimization based on Embodiment 1 described above, such as... Figure 3 As shown, the robot's elevator detection method includes:

[0050] S301. When the elevator door opens, the control sensor acquires multiple frames of images of the detection area according to a preset frame rate to obtain an image sequence.

[0051] In practical applications, the elevator car stops when it reaches the waiting hall of the target floor during its upward or downward movement, and controls the elevator doors to open so that passengers in the car can exit the elevator, and objects in the waiting hall who need to take the elevator can enter the car. These objects can be people, pets, robots, or other objects that need to take the elevator.

[0052] When the elevator doors open, the sensors can acquire depth images at a preset frame rate and send the depth images to the elevator controller, such as... Figure 1As shown, when there is one sensor 3, the sensor 3 can be controlled to collect multiple frames of images of the detection area, including the waiting hall 2 and the car 1, when the elevator door starts to open. When there are two sensors 3, the sensor facing the car 1 can be controlled to collect multiple frames of images of the area of ​​the car 1, and the sensor facing the waiting hall 2 can be controlled to collect multiple frames of images of the area of ​​the waiting hall. It should be noted that the sensor 3 can collect images from the opening of the elevator door to the closing of the elevator door. In addition, the frame rate of the sensor images can be a fixed frame rate or a dynamically adjustable frame rate.

[0053] After the sensor acquires each frame of image, it can perform preprocessing on the image, such as denoising and binarization, to generate an image sequence.

[0054] S302. Input the image sequence into the target detection and tracking model to obtain at least one of the following as the motion data of the object in the detection area: the moving direction, moving speed, projected area, and running posture; and identify whether the object has pressed any buttons inside the car after entering the car as the object's behavior detection result.

[0055] In this embodiment, the target detection and tracking model can be a neural network such as RNN, CNN, or DNN. During training, the target detection and tracking model can use images labeled with the moving direction, moving speed, projected area, running posture, and behavior detection results of objects in the image as training data. The labeled image sequence is input into the target detection and tracking model to predict the moving direction, moving speed, projected area, running posture, and behavior detection results of each object in the image sequence. The loss rate is calculated using the predicted data and the labeled data. The model parameters are adjusted based on the loss rate until the loss rate is less than a preset value, at which point a trained target detection and tracking model is obtained. In this embodiment, there are no restrictions on the model structure or the training method.

[0056] Of course, after identifying the object, the object's direction of movement, projected area, position, and motion posture can be calculated by the sensor's installation location, imaging principle, sensor's internal and external parameters, etc., and the movement speed can be calculated by the object's position difference and frame rate in two frames. This embodiment does not limit the method of obtaining the object's motion data.

[0057] S303. Calculate the rate of change of the moving speed, projected area, and running posture of the object detected in two adjacent frames, and obtain the rate of change of speed, the rate of change of projected area, and the rate of change of posture.

[0058] Specifically, this implementation inputs the image sequence into the target detection and tracking model to obtain the movement direction, movement speed, projected area, and running posture of the object detected in each frame of the image sequence. For the same object, the rate of change of speed, rate of change of projected area, and rate of change of posture can be calculated from the movement speed, projected area, and running posture detected in two adjacent frames. The rate of change is obtained by calculating the difference between the speed, area, and posture detected in two adjacent frames, and then calculating the ratio of this difference to the value detected in the previous frame of the two adjacent frames.

[0059] In an optional embodiment, the posture data may include the height and outer contour data of the object. The height difference detected in two adjacent frames can be calculated, and the ratio of this height difference to the height detected in the previous frame of the two adjacent frames is used to obtain the height change rate. The contour difference of the outer contour data detected in two adjacent frames is calculated, and the ratio of this contour difference to the outer contour of the object detected in the previous frame of the two adjacent frames is used to obtain the outer contour change rate. The average of the height change rate and the outer contour change rate is calculated to obtain the posture change rate of the object detected in the two adjacent frames. This embodiment measures the object's posture using its height and outer contour data. The calculated posture change rate better reflects the characteristics of small height changes and small outer contour sway during robot movement, making the score calculated using the posture change rate more accurate in determining the robot's posture.

[0060] S304. Calculate the mean values ​​of velocity change rate, projected area change rate, and attitude change rate respectively to obtain the mean values ​​of velocity change rate, projected area change rate, and attitude change rate.

[0061] This embodiment performs target detection and tracking through image sequences. It can obtain various data of the object from each frame of the image, and calculate the rate of change of each data through two adjacent frames of the image. Then, it calculates the mean of the rate of change of each data and performs smoothing filtering on the rate of change of each data to avoid the problem of inaccurate score calculation caused by abnormal rate of change. This improves the anti-interference performance of the method and makes the score calculation more accurate.

[0062] S305. Obtain the behavior value corresponding to the behavior detection result. The behavior value is negative when the detection result is that the button inside the car is operated, and positive when the detection result is that the button inside the car is not operated.

[0063] For example, when the detection result indicates that the object operates on the floor selection button inside the car, the corresponding behavior value of the detection result is equal to -1, indicating that the object is likely not a robot. When the detection result indicates that the object does not operate on the floor selection button inside the car, the corresponding behavior value of the detection result is equal to 1, indicating that the object is likely a robot.

[0064] S306. The robot's score is calculated using the mean of the movement direction, the mean of the velocity change rate, the mean of the projected area change rate, the mean of the posture change rate, and the behavior value. The score is positively correlated with the movement direction and negatively correlated with the mean of the velocity change rate, the mean of the projected area change rate, the mean of the posture change rate, and the behavior value.

[0065] For example, the formula for calculating the score of a robot is as follows:

[0066]

[0067] In the above formula, A represents the direction of movement, and B, C, D, and E are the mean rate of change of velocity, the mean rate of change of projected area, the mean rate of change of attitude, and the behavior value, respectively. In another example, the parameters A, B, C, D, and E in the formula can be normalized values. Of course, weights can also be assigned to the parameters A, B, C, D, and E. This embodiment does not restrict the method of calculating the score.

[0068] S307. When the score is greater than the preset score threshold, the object is identified as a robot.

[0069] In this embodiment, a score threshold can be set for the object to be a robot. When the score S of the object is greater than the score threshold, the object is determined to be a robot, and the elevator door can be controlled to close according to the elevator door control strategy for objects to be robots.

[0070] S308. After detecting that the robot has entered the elevator car, use image detection to check whether there are still passengers in the waiting hall.

[0071] This embodiment tracks each object in the elevator waiting hall using an image sequence. For example, an object sequence can be established, and each object in the object sequence can maintain a position parameter. When an object belonging to the robot is determined to have entered the elevator car through the position parameter, the image can be used to detect whether there are still passengers in the waiting hall.

[0072] In one example, the position, facial orientation, volume, and movement speed of an object in the elevator lobby can be obtained from an image. Different weights are assigned to each of these factors, and the weighted sum of these factors is calculated as the object's behavior score. If the behavior score is greater than a preset score, it is determined that the object intends to take the elevator. The position of the object with the intention to take the elevator is then tracked. Once all objects with the intention to take the elevator in the elevator lobby are inside the elevator car, it is determined that there are no objects in the elevator lobby, and S310 can be executed. Otherwise, S309 is executed to continue waiting for all objects with the intention to take the elevator to enter the elevator car.

[0073] S309. After detecting that all passengers in the waiting hall have entered the elevator car, control the elevator doors to close.

[0074] This means that once all passengers are inside the elevator car, the elevator doors can be closed automatically without waiting for the closing time, thus improving elevator operating efficiency.

[0075] S310, Control the elevator door to close.

[0076] Once the robot enters the elevator car and all those in the waiting area who need to take the elevator have entered the car, the elevator doors can be closed automatically without waiting for the closing time to arrive, thus improving the elevator's operating efficiency.

[0077] In another alternative embodiment, after detecting that the robot has entered the elevator car and the elevator door has closed, the system detects whether there are other passengers inside the car using image analysis. If not, it obtains the target floor called by the robot and controls the car to proceed directly to the target floor. Specifically, the area inside the car can be identified using images. If there are no other passengers inside the car, it indicates that only the robot is using the elevator, and the elevator car can be controlled to proceed directly to the target floor that the robot needs to reach, thereby improving the elevator's operating efficiency.

[0078] In this embodiment, when the elevator door opens, the control sensor collects multiple frames of images of the detection area at a preset frame rate to obtain an image sequence. The image sequence is then input into the target detection and tracking model to obtain at least one of the following motion data of the object in the detection area: movement direction, movement speed, projected area, and running posture. The model also detects whether the object operates any buttons inside the elevator car after entering, which is considered the object's behavior detection result. This allows for the calculation of a score based on the object's movement direction, movement speed, projected area, running posture, and behavior detection results. When the score exceeds a preset threshold, the object is identified as a robot. This fully utilizes the motion and behavior data of the robot during its elevator ride to determine whether a robot is in the elevator, improving the accuracy of robot identification and avoiding the problem of misidentifying a robot as a child with limited abilities, which could lead to the elevator being suspended. The elevator operates normally when a robot is in the elevator, and the consistency between the robot's elevator control and the detection system's elevator control is ensured.

[0079] Furthermore, on the one hand, the rate of change of various data is smoothed and filtered to avoid inaccurate score calculations caused by abnormal rates of change, thus improving the anti-interference performance of the method and making the score calculation more accurate. On the other hand, the score is positively correlated with the direction of movement and negatively correlated with the mean rate of change of velocity, the mean rate of change of projected area, the mean rate of change of posture, and the behavior value. This score truly reflects the operating and behavioral characteristics of the robot riding the elevator, and the accuracy of the score is high.

[0080] Example 3

[0081] Figure 4 This is a schematic diagram of a robot elevator detection device provided in Embodiment 3 of the present invention. Figure 4 As shown, the robot elevator detection device includes:

[0082] Image acquisition module 401 is used to acquire images of the detection area through sensors when the elevator door opens;

[0083] The target tracking module 402 is used to perform target detection and tracking on the image to obtain motion data and behavior detection results of objects inside the car;

[0084] The score calculation module 403 is used to calculate a score for the object as a robot based on the motion data and the behavior detection results;

[0085] The robot identification module 404 is used to identify the object as a robot when the score is greater than a preset score threshold.

[0086] Optionally, the image acquisition module 401 includes:

[0087] The image sequence generation unit is used to control the sensor to acquire multiple frames of images of the detection area at a preset frame rate when the elevator door opens, so as to obtain an image sequence.

[0088] Optionally, the target tracking module 402 includes:

[0089] The model input unit is used to input the image sequence into the target detection and tracking model to obtain at least one of the following as motion data of the object in the detection area: movement direction, movement speed, projected area, and running posture; and to identify whether the object has pressed any buttons inside the car after entering the car as the behavior detection result of the object.

[0090] Optionally, the motion data includes at least one of the following: the object's direction of movement, speed of movement, projected area at the bottom of the car, and running posture in each frame of the image; the behavior detection result includes whether the object has operated any buttons inside the car; and the score calculation module 403 includes:

[0091] The rate of change calculation unit is used to calculate the rate of change of the moving speed, projected area, and running posture of the object detected in two adjacent frames, and to obtain the rate of change of speed, the rate of change of projected area, and the rate of change of posture.

[0092] The mean value calculation unit is used to calculate the mean values ​​of the velocity change rate, the projected area change rate, and the attitude change rate, respectively, to obtain the mean values ​​of the velocity change rate, the projected area change rate, and the attitude change rate.

[0093] A behavior value acquisition unit is used to acquire the behavior value corresponding to the behavior detection result. The behavior value is negative when the detection result is that the button inside the car is operated, and positive when the detection result is that the button inside the car is not operated.

[0094] The score calculation unit is used to calculate a score for the object as a robot using the movement direction, the average rate of change of velocity, the average rate of change of projected area, the average rate of change of posture, and the behavior value. The score is positively correlated with the movement direction and negatively correlated with the average rate of change of velocity, the average rate of change of projected area, the average rate of change of posture, and the behavior value.

[0095] Optionally, the rate of change calculation unit includes:

[0096] The height difference calculation subunit is used to calculate the difference in height of the object detected in two adjacent frames of images;

[0097] The height change rate calculation subunit is used to calculate the ratio of the height difference to the height of the object detected in the previous frame of two adjacent frames to obtain the height change rate.

[0098] The contour difference calculation subunit is used to calculate the contour difference of the outer contour of the object detected in two adjacent frames;

[0099] The contour change rate calculation subunit is used to calculate the ratio of the contour difference to the outer contour of the object detected in the previous frame image in two adjacent frames, and obtain the outer contour change rate.

[0100] The pose change rate calculation subunit is used to calculate the average of the height change rate and the outer contour change rate to obtain the pose change rate of the object detected in the two adjacent frames.

[0101] Optionally, it also includes:

[0102] The elevator waiting area object detection module is used to detect whether there are still objects in the waiting area after the robot enters the elevator car by using the image.

[0103] The first elevator door control module is used to control the elevator door to close after detecting that all passengers in the waiting hall have entered the elevator car;

[0104] The first elevator door control module is used to control the closing of the elevator door.

[0105] Optionally, it also includes:

[0106] The elevator car object detection module is used to detect whether there are other elevator passengers in the car after the robot enters the car by using the image.

[0107] The target floor acquisition module is used to acquire the target floor summoned by the robot;

[0108] The direct-drive control module is used to control the car to drive directly to the target floor.

[0109] The robot elevator detection device provided in this embodiment of the invention can execute the robot elevator detection method provided in Embodiment 1 and Embodiment 2 of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0110] Example 4

[0111] Figure 5 A schematic diagram of an electronic device 50 that can be used to implement embodiments of the present invention is shown. The electronic device 50 is intended to represent various forms of digital computers, such as desktop computers, workbenches, servers, blade servers, mainframe computers, etc. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0112] like Figure 5 As shown, the electronic device 50 includes at least one processor 51 and a memory, such as a read-only memory (ROM) 52 and a random access memory (RAM) 53, communicatively connected to the at least one processor 51. The memory stores computer programs executable by the at least one processor. The processor 51 can perform various appropriate actions and processes based on the computer program stored in the ROM 52 or loaded into the RAM 53 from storage unit 58. The RAM 53 can also store various programs and data required for the operation of the electronic device 50. The processor 51, ROM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0113] Multiple components in electronic device 50 are connected to I / O interface 55, including: input unit 56, such as keyboard, mouse, sensor, etc.; output unit 57, such as various types of display, speaker, etc.; storage unit 58, such as disk, optical disk, etc.; and communication unit 59, such as network card, modem, wireless transceiver, etc. Communication unit 59 allows electronic device 50 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0114] Processor 51 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 51 performs the various methods and processes described above, such as the robot elevator detection method.

[0115] In some embodiments, the robot elevator detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 50 via ROM 52 and / or communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the robot elevator detection method described above may be performed. Alternatively, in other embodiments, processor 51 may be configured to perform the robot elevator detection method by any other suitable means (e.g., by means of firmware).

[0116] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0118] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0120] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0121] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0122] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0123] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting elevator use by a robot, characterized in that, include: When the elevator door opens, the sensor collects an image of the detection area; The image is subjected to target detection and tracking to obtain motion data and behavior detection results of objects within the detection area; The score for the object to be a robot is calculated based on the motion data and the behavior detection results; When the score is greater than a preset score threshold, the object is determined to be a robot; The motion data includes at least one of the following: the object's direction of movement, speed of movement, projected area at the bottom of the car, and running posture in each frame of the image. The behavior detection result includes whether the object operated any buttons inside the car. Calculating the object's score as a robot based on the motion data and the behavior detection result includes: Calculate the rate of change of the moving speed, projected area, and running posture of the object detected in two adjacent frames to obtain the rate of change of speed, the rate of change of projected area, and the rate of change of posture; Calculate the mean values ​​of the velocity change rate, the projected area change rate, and the attitude change rate, respectively, to obtain the mean values ​​of the velocity change rate, the projected area change rate, and the attitude change rate. Obtain the behavior value corresponding to the behavior detection result. The behavior value is negative when the detection result is that the button inside the car is operated, and positive when the detection result is that the button inside the car is not operated. The object is scored as a robot using the movement direction, the mean rate of change of velocity, the mean rate of change of projected area, the mean rate of change of posture, and the behavior value. The score is positively correlated with the movement direction and negatively correlated with the mean rate of change of velocity, the mean rate of change of projected area, the mean rate of change of posture, and the behavior value.

2. The method as described in claim 1, characterized in that, The process of acquiring images of the detection area via sensors when the elevator doors open includes: When the elevator door opens, the control sensor collects multiple frames of images of the detection area according to a preset frame rate, thus obtaining an image sequence.

3. The method as described in claim 1, characterized in that, The step of performing target detection and tracking on the image to obtain motion data and behavior detection results of objects within the detection area includes: The image sequence is input into the target detection and tracking model to obtain at least one of the following as motion data of the object in the detection area: movement direction, movement speed, projected area, and running posture. The model also identifies whether the object performs button operations inside the car after entering the car as the behavior detection result of the object.

4. The method as described in claim 1, characterized in that, The running posture includes the height of the object and the outer contour of the object, and the calculation of the posture change rate includes: Calculate the difference in height of the object detected in two adjacent frames; The height change rate is obtained by calculating the ratio of the height difference to the height of the object detected in the previous frame of two adjacent frames. Calculate the contour difference between the outer contours of the object detected in two adjacent frames; The ratio of the contour difference to the outer contour of the object detected in the previous frame of two adjacent frames is calculated to obtain the outer contour change rate. The mean of the height change rate and the outer contour change rate is calculated to obtain the pose change rate of the object detected in the two adjacent frames.

5. The method according to any one of claims 1-3, characterized in that, Also includes: After detecting that the robot has entered the elevator car, the system uses the image to detect whether there are still passengers in the waiting hall. If so, after detecting that all passengers in the waiting hall have entered the elevator car, the elevator door will be closed. If not, control the elevator door to close.

6. The method as described in claim 5, characterized in that, Also includes: After detecting that the robot has entered the elevator car, the system uses the image to detect whether there are other passengers inside the elevator car. If not, obtain the target floor summoned by the robot; Control the car to drive directly to the target floor.

7. A robot elevator detection device, characterized in that, include: The image acquisition module is used to acquire images of the detection area via sensors when the elevator doors open. The target tracking module is used to perform target detection and tracking on the image to obtain motion data and behavior detection results of objects within the detection area; The score calculation module is used to calculate a score for the object as a robot based on the motion data and the behavior detection results; A robot identification module is used to identify the object as a robot when the score is greater than a preset score threshold. The motion data includes at least one of the following: the object's direction of movement, speed of movement, projected area at the bottom of the car, and running posture in each frame image. The behavior detection result includes whether the object operated any buttons inside the car. The score calculation module includes: The rate of change calculation unit is used to calculate the rate of change of the moving speed, projected area, and running posture of the object detected in two adjacent frames, and to obtain the rate of change of speed, the rate of change of projected area, and the rate of change of posture. The mean value calculation unit is used to calculate the mean values ​​of the velocity change rate, the projected area change rate, and the attitude change rate, respectively, to obtain the mean values ​​of the velocity change rate, the projected area change rate, and the attitude change rate. A behavior value acquisition unit is used to acquire the behavior value corresponding to the behavior detection result. The behavior value is negative when the detection result is that the button inside the car is operated, and positive when the detection result is that the button inside the car is not operated. The score calculation unit is used to calculate a score for the object as a robot using the movement direction, the average rate of change of velocity, the average rate of change of projected area, the average rate of change of posture, and the behavior value. The score is positively correlated with the movement direction and negatively correlated with the average rate of change of velocity, the average rate of change of projected area, the average rate of change of posture, and the behavior value.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the robot elevator detection method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the robot elevator detection method according to any one of claims 1-6.

Images