System for robot boarding of elevator, and control method therefor

Abstract

The present invention relates to a system for robot boarding of an elevator, and a control method therefor. The system according to one embodiment of the present invention comprises: an elevator server for controlling an elevator in which a depth camera is provided; and a robot server which enables communication with the elevator server and which controls a robot that is intended to board the elevator, wherein the elevator server calculates the internal congestion of the elevator on the basis of an image received from the depth camera and transmits the calculated internal congestion to the robot server, and the robot server controls the robot on the basis of the internal congestion such that the robot boards the elevator.

Description

System for robots to ride in an elevator and its control method

[0001] The present invention relates to a system for a robot to ride an elevator and a control method thereof.

[0002] With technological advancements, robots are playing a crucial role as tools that replace humans in various tasks. Robots are primarily used in manufacturing production lines to automate various tasks, including assembly, welding, and painting, replacing human arms, thereby contributing to increased productivity.

[0003] Autonomous robots are robots that can independently navigate their surroundings, detect obstacles, and use wheels or legs to find the optimal route to their destination. They are being developed and utilized in various fields, including autonomous vehicles, logistics, hotel services, and robot vacuum cleaners.

[0004] Robots used to provide services within a building may need to board an elevator (elevator car) installed within the building to provide services on a specific floor of the building.

[0005] However, when a robot boards an elevator used by people, collisions or interference between the robot and the person often occur, or there is not enough space inside the elevator for the robot to board.

[0006] Accordingly, there is a need for an elevator server and a system including the same that can determine whether there is sufficient space in an elevator for a robot to board, and determine an elevator suitable for boarding the robot and control the robot to board the elevator, upon a request from a robot or a robot server.

[0007] One object of the present invention is to provide a system and a control method thereof capable of determining the level of congestion inside an elevator in an optimized manner and controlling a robot to board the elevator based on the determined level of congestion.

[0008] According to one embodiment of the present invention for achieving the above object, a system includes an elevator server for controlling an elevator in which a depth camera is installed, and a robot server configured to communicate with the elevator server and for controlling a robot that wants to board the elevator, wherein the elevator server calculates an internal congestion level of the elevator based on an image received from the depth camera and transmits the calculated internal congestion level to the robot server, and the robot server controls the robot to board the elevator based on the internal congestion level.

[0009] In an embodiment, the elevator server includes a congestion calculation module that calculates the internal congestion of the elevator using two weighting functions.

[0010] In an embodiment, the congestion calculation module includes an image receiving unit that receives a depth information image captured by the depth camera, an image correction unit that preprocesses the depth information image, a congestion measurement unit that calculates the internal congestion of the elevator by applying the two types of weighting functions to the preprocessed depth information image, and a communication unit that transmits the calculated internal congestion to the elevator server.

[0011] In an embodiment, each pixel of the depth information image includes a depth value, the depth camera is installed at the upper center of the elevator, and the two types of weighting functions include a first weighting function that assigns a weight in proportion to the depth value of the pixel so as to give weight to the congestion of the floor surface of the elevator, and a second weighting function that assigns a weight in inverse proportion to the pixel area ratio from the center of the image to the edge so as to compensate for the fact that the object is photographed in a smaller area the closer it is to the center of the elevator.

[0012] In an embodiment, the first weight function is characterized in that it is formed to assign weights such that a greater depth value among the depth values ​​of each pixel in the depth information image is assigned a greater weight, so that the congestion level is measured based on the floor surface of the elevator.

[0013] In an embodiment, the second weighting function is characterized in that it is formed to apply weights so that the area proportion decreases from the center to the edge, in order to compensate for the fact that when an object is photographed by a depth camera installed at the upper center of the elevator, an object located at the center of the elevator is photographed vertically and thus has a small pixel area proportion measured, and an object located at the edge of the elevator is photographed obliquely and thus has a large pixel area proportion measured.

[0014] In an embodiment, the congestion calculation module is characterized in that it measures congestion by applying the two types of weighting functions to the preprocessed depth information image, and calculates an internal congestion that represents the proportion of an object relative to the area of ​​the elevator by combining the two types of measured congestion.

[0015] In an embodiment, the robot server is characterized in that, when the robot arrives at a position where it can board the elevator, it requests information on the internal congestion level of the elevator from the elevator server.

[0016] In an embodiment, the elevator server is characterized in that, when the elevator arrives at the floor where the robot is located, the elevator server calculates the internal congestion level of the elevator and transmits the calculated internal congestion level to the robot server.

[0017] In an embodiment, the robot server is characterized in that, when the internal congestion of the elevator is lower than a reference value, the robot is controlled to allow the robot to board the elevator.

[0018] In an embodiment, the robot server is characterized in that, when the internal congestion of the elevator is higher than a reference value, the robot server does not allow the robot to ride and requests the elevator server to re-call the elevator in the forward direction in which the robot wants to move.

[0019] In an embodiment, the robot server is characterized in that, when the internal congestion of the elevator is higher than a reference value and the number of times the robot has not boarded the elevator reaches a preset number of times, the robot server requests the elevator server to re-call the elevator in the opposite direction to the direction in which the robot intends to move.

[0020] In an embodiment, the robot server is characterized in that, if the number of times the robot has not boarded the elevator reaches a certain number even when the elevator has arrived in the reverse direction, the robot determines that the robot cannot board the elevator.

[0021] According to embodiments of the present invention, by measuring internal congestion while taking into account the influence of distortion, noise, and field of view of a depth camera image, the accuracy of internal congestion can be improved in a low-performance equipment environment.

[0022] The present invention can determine the level of congestion inside an elevator, including not only passengers but also general objects such as boxes and carts.

[0023] The present invention can minimize the situation in which a robot collides with passengers and cargo when boarding an elevator by determining the level of congestion inside the elevator when the elevator arrives at the floor on which the robot wishes to board and then boarding the elevator when a space for boarding the robot is secured.

[0024] The present invention can provide a countermeasure for frequent boarding failures due to many passengers trying to board in a specific direction by re-calling the same elevator or the direction of the corresponding elevator in case boarding space is not secured, and can increase the robot's elevator boarding rate.

[0025] FIG. 1 is a conceptual diagram illustrating a system for a robot to ride an elevator according to one embodiment of the present invention.

[0026] FIG. 2 is a conceptual diagram illustrating a congestion calculation module according to one embodiment of the present invention.

[0027] Figures 3 and 4 are diagrams for explaining two types of weighting functions of the present invention.

[0028] Figure 5 is a flowchart for explaining a method for calculating the congestion inside an elevator by applying the weighting function of the present invention.

[0029] Figure 6 is a flowchart for explaining a method for boarding a robot in an elevator based on the internal congestion of the elevator of the present invention.

[0030] FIG. 7 is a drawing for explaining the result of calculating the congestion level inside an elevator according to one embodiment of the present invention.

[0031] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0032] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0033] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0034] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0035] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0036] The robot control device described in this specification may be a control unit provided in the robot, or may be an independent control module separated from the robot and connected to the robot so as to be able to communicate with the robot and control the robot.

[0037] FIG. 1 is a conceptual diagram illustrating a system for a robot to ride an elevator according to one embodiment of the present invention.

[0038] Referring to FIG. 1, a system for a robot to ride an elevator according to an embodiment of the present invention may include an elevator server (300) that controls an elevator (200) in which a depth camera (100) is installed, a robot server (400) that is configured to be able to communicate with the elevator server (300) and that controls a robot (500) that wants to ride the elevator (200).

[0039] A depth camera (or depth detection camera) (100) captures a depth information image that extracts depth information up to the floor of the elevator, and can be attached to the center of the ceiling of the elevator (200).

[0040] The congestion calculation module (310) can calculate the congestion inside an elevator based on an image capturing depth information (depth information image). In addition, the congestion calculation module (310) can communicate and transmit the calculated internal congestion to the elevator server (300).

[0041] The elevator server (300) directly controls the elevator (200) and may provide a communication function for remote control purposes to enable external control of the elevator. In addition, the elevator server (300) may receive internal congestion calculation results from the congestion calculation module (310).

[0042] The robot server (400) can send control commands to the robot (500) (autonomous driving, elevator boarding / disembarking commands, etc.). In addition, the robot server (400) can communicate with the elevator server (300) to transmit elevator control requests and receive the results.

[0043] The robot (500) can be configured to board an elevator using autonomous / manual driving functions. The robot (500) can communicate with the robot server (400) to transmit elevator control requests and receive the results.

[0044] The elevator (200) can be used by people and / or robots together, and can communicate with the elevator server (300) to receive control requests and transmit the results.

[0045] In FIG. 1, an elevator (200) installed in a building configured to allow a person and a robot (500) to ride, an elevator server (300) controlling the elevator (200), and a robot server (400) controlling the robot (500) are illustrated.

[0046] The robot (500) may be a service robot for providing service on at least one floor within a building.

[0047] An elevator (200) is a device that moves between floors within a building. To provide a service, a robot (500) may be mounted thereon and the robot (500) may be moved between floors within the building. The elevator (200) may be an elevator that allows both people and robots (500) to ride thereon. In other words, the elevator (200) may be distinct from a robot-only elevator that only allows robots (500) to ride thereon.

[0048] Although not illustrated, a building in which an elevator (200) is installed may have multiple elevators, and the illustrated elevator (200) may represent any one of these multiple elevators. Furthermore, the robot (500) may represent any one of the illustrated multiple robots, which calls an elevator to provide service and wishes to board the elevator (200).

[0049] The elevator (200) can be controlled by the elevator server (300) (or based on a signal generated by the elevator server (300). For example, the elevator server (300) can select an elevator (200) among a plurality of elevators in which the robot (500) can board (e.g., a space in which the robot (500) can board is secured) based on a call from a service requester or a robot (500) (a robot server (400) that controls the robot (500)), and move the selected elevator (200) to the location of the robot (500) (i.e., the floor on which the robot (500) is located).

[0050] When the elevator (200) arrives at the location of the robot (500), the elevator server (300) can control the door of the elevator (200) to allow the robot (500) to board the elevator (200).

[0051] Movement of the robot (500) and boarding or disembarking of the elevator (200) of the robot (500) can be controlled by the robot server (400).

[0052] A camera may be installed inside the elevator (200), and the interior of the elevator (200) may be photographed by the camera. Here, the camera installed in the elevator (200) may be, for example, a depth camera (100) capable of acquiring depth information or a depth value.

[0053] Such a depth camera (100) can be installed in the center of the ceiling of the elevator (200).

[0054] The installed camera may be configured to photograph the floor of the elevator (200) (vertically or as vertically as possible).

[0055] As shown in Fig. 1, the depth camera (100) may be installed in the center of the upper part (ceiling) of the elevator (200), but is not limited thereto.

[0056] As illustrated in FIG. 1, the depth camera (100) may be installed to vertically photograph the floor, and the depth camera (100) may have an angle of view that enables it to photograph the entire floor. Unlike what is illustrated, the camera may also be installed on the wall of the elevator (200).

[0057] In an embodiment, an internal congestion level may be calculated based on an image of the interior of an elevator (200) captured from a depth camera installed inside the elevator (200) and the degree to which objects (e.g., robots, people, or other objects) riding in the elevator (200) occupy the internal space of the elevator (200) (i.e., the degree of space occupancy or space occupancy information).

[0058] For example, the elevator server (300) can receive an image of the interior of the elevator (200) from a depth camera installed inside the elevator (200) and can analyze the received image to calculate the internal congestion level. When the elevator (200) arrives at the location of the robot (500), the elevator server (200) can determine whether to allow the robot (500) to board the elevator (200) based on the internal congestion level calculated based on the image captured by the depth camera. The robot server (400) can control the robot (500) to board the elevator (200) when the door of the elevator (200) opens if the internal congestion level satisfies a standard.

[0059] The operation of controlling the elevator (200) by the elevator server (200) and the process of calculating the internal congestion can be performed after the elevator (200) arrives at the floor where the robot is located at the request of the robot server (400).

[0060] That is, the present invention does not calculate the internal congestion level when there is an elevator call from a robot (500) or a robot server (400), but is performed after the elevator (200) arrives at the floor where the robot (500) is located, which means that the internal congestion level is measured after the elevator arrives at the boarding floor and the passengers inside have gotten on and off.

[0061] Through this, when a request to call an elevator is received from a robot (100) or a robot server (400), the elevator server (300) calculates the internal congestion level for each elevator and sends an elevator with secured space. In contrast to the prior art, the present invention can solve the problem of the prior art in which a robot cannot board an elevator if many passengers board before the robot at the boarding floor.

[0062] Alternatively, the elevator server (300) may be configured to analyze the image inside the elevator (200) in real time or periodically to calculate the internal congestion level, regardless of a request from the robot (500) or the robot server (400).

[0063] "Internal congestion" is a parameter indicating the extent to which objects boarding the elevator (200) occupy the internal space of the elevator (200), and may be related to space occupancy, for example. For example, the internal congestion may be calculated based on the difference between the area of ​​the floor of the elevator (200) and the area occupied by objects boarding the elevator (200) on the floor, or the ratio of the area occupied by objects to the area of ​​the floor.

[0064] The elevator server (300) can determine whether a robot (500) can board the elevator (200) by calculating the internal congestion level of the elevator (200).

[0065] Accordingly, in an embodiment, the internal congestion level of an elevator (200) can be calculated by analyzing an image of the interior of the elevator (200) without having to determine whether the elevator (200) is full by a weight sensor included in the elevator (200). In addition, in an embodiment, various sensors may not be installed inside the elevator (200) to calculate the internal congestion level of the elevator (200).

[0066] Referring to FIG. 1, an elevator server (300) can control an elevator (200) in which a depth camera (100) is installed. For example, the elevator server (300) can move the elevator (200) to a desired floor and perform control to open or close the door of the elevator (200).

[0067] The robot server (400) may be configured to be able to communicate with the elevator server (300). Here, the robot server (400) may perform wired / wireless communication with the elevator server (300) and transmit and receive data or control signals.

[0068] The robot server (400) is configured to be able to communicate with the robot (500) and control the robot (500). For example, the robot server (400) can use location information received from the robot (500) to move the robot (500) or control the movement of the robot (500).

[0069] The robot server (400) can control the robot (500) to board the elevator (200).

[0070] The elevator server (300) can calculate the internal congestion level of the elevator based on the image received from the depth camera (100) and transmit the calculated internal congestion level to the robot server (400).

[0071] The robot server (400) can control the robot (500) to board the elevator based on the internal congestion level. The robot boarding control method of the robot server (400) will be examined in more detail with reference to FIG. 6.

[0072] The elevator server (300) may include a congestion calculation module (310) that calculates the internal congestion of the elevator using two types of weighting functions.

[0073] As illustrated in FIG. 1, the congestion calculation module (310) may be a separate module separated from the elevator server (300) or may be a module included in the elevator server (300). If the congestion calculation module (310) is formed as an independent module separated from the elevator server (300), the congestion calculation module (310) may be formed to be capable of communicating with the elevator server (300) or the robot server (400).

[0074] FIG. 2 is a conceptual diagram illustrating a congestion calculation module according to one embodiment of the present invention.

[0075] Referring to FIG. 2, the congestion calculation module (310) may include an image receiving unit (311) that receives a depth information image captured by a depth camera (100), an image correction unit (312) that preprocesses (corrects) the depth information image, a congestion measurement unit (313) that calculates the internal congestion of an elevator by applying two types of weighting functions to the preprocessed depth information image, and a communication unit (314) that transmits the calculated internal congestion to an elevator server (300).

[0076] The depth information image captured by the depth camera (100) may include a depth value or depth information.

[0077] Specifically, each pixel of a depth information image may contain a depth value.

[0078] Here, a large depth value means that the object is located far from the depth camera, and a small depth value means that the object is located close to the depth camera.

[0079] The depth camera (100) may be installed, for example, at the upper center of the elevator as shown in FIG. 1, to photograph the inside of the elevator from top to bottom.

[0080] Accordingly, the depth value will be small in pixels corresponding to the location where the object is on board, and the depth value will be large in pixels corresponding to the location where the object is not on board (for example, a value indicating the depth from the depth camera to the floor).

[0081] Figures 3 and 4 are diagrams for explaining two types of weighting functions of the present invention.

[0082] The system of the present invention can apply two types of weighting functions to a depth information image when calculating internal congestion.

[0083] Here, the two types of weighting functions may include a first weighting function that assigns weights in proportion to the depth value of a pixel to give weight to the congestion of the floor surface of the elevator, as illustrated in (a) of FIG. 3, and a second weighting function that assigns weights inversely proportional to the pixel area ratio from the center of the reverse image to the edge to compensate for the object being photographed in a smaller area the closer it is to the center of the elevator, as illustrated in (b) of FIG. 3.

[0084] Pixels in the image that are close to the camera have small depth values ​​(red (or light color) in Fig. 4), and pixels in the image that are far from the camera have large depth values ​​(blue (or dark color) in Fig. 4).

[0085] Referring to (a) of Fig. 3, the first weight function (w1(x)) is such that the greater the depth value of each pixel in the depth information image (the greater the x), the greater the weight (j) so that the congestion is measured based on the floor surface of the elevator. <k)가 더 부여되도록 가중치를 부여하도록 형성될 수 있다.

[0086] That is, referring to (a) of Fig. 4, the first weighting function can be applied so as to be proportional to the depth value of the pixel.

[0087] Since the elevator congestion must be measured based on the area of ​​the elevator floor, the congestion calculation module (310) can apply a first weighting function to reduce the proportion of the depth value in the image as the depth value of the pixel is closer to the camera (smaller).

[0088] In addition, the congestion calculation module (310) can compensate for the distortion phenomenon that occurs when the upper body area appears larger than the floor area due to the camera viewing angle by applying the first weighting function, thereby reducing the upper body proportion in the depth information image and reflecting the lower body proportion to a greater extent in the elevator floor area.

[0089] Referring to (b) of FIG. 3, the second weighting function (w2(y)) can be formed to apply weights so that the area proportion decreases from the center to the edge, in order to compensate for the fact that when an object is photographed with a depth camera installed at the upper center of the elevator, an object located at the center of the elevator is photographed vertically and thus has a small pixel area proportion measured, and an object located at the edge of the elevator is photographed obliquely and thus has a large pixel area proportion measured.

[0090] Referring to (b) of Fig. 4, the second weight function (w2(y)) can adjust the pixel area proportion by applying a weight function inversely proportional to the distance from the center of the image to the edge (or each pixel).

[0091] Referring to (b) of FIG. 4, as the depth camera (100) is installed to take images vertically downward from the center of the upper part of the elevator (200), depending on the location of the person / object in the elevator, the person (or object) (410) at the edge is displayed from the face to the feet, taking up a large area, and the person (420) in the middle is displayed only from the head / shoulders, taking up a small area.

[0092] The congestion calculation module (310) reduces the proportion of the area occupied by objects at the edges farther from the center by applying a second weighting function, thereby reducing the proportion of pixels closer to the edge of the elevator (the farther away from the center of the image) relative to the elevator floor area. Accordingly, the effect of compensating for the area proportion to be similar to that of people / objects located at the center can be achieved.

[0093] Figure 5 is a flowchart for explaining a method for calculating the congestion inside an elevator by applying the weighting function of the present invention.

[0094] When a robot (500) sends a request to the robot server (400) to board an elevator (200), the robot server (400) can request the elevator server (300) to move the elevator (200) to the floor where the robot is located.

[0095] When the elevator (200) arrives at the floor where the robot (500) is located, the elevator server (300) can take pictures of the inside of the elevator through the depth camera (100) (S510).

[0096] Thereafter, the elevator server (300) can transmit the image captured by the depth camera (100) to the congestion calculation module (310).

[0097] The congestion calculation module (310) can perform image preprocessing to remove noise from an image after receiving the image (S520).

[0098] The congestion calculation module (310) can measure congestion by applying a first weight function that proportionally applies depth value-based weights (S530).

[0099] In addition, the congestion calculation module (310) can measure congestion by applying a second weighting function that applies weights inversely based on the distance from the center of the image to the edge (or the distance to each pixel) (S540).

[0100] Thereafter, the congestion calculation module (310) can measure (calculate) the internal congestion based on the proportion of the weight function relative to the area of ​​the congestion measurement space (S550).

[0101] Specifically, the congestion calculation module (310) can measure congestion by applying the two types of weighting functions to the preprocessed depth information image, and can calculate the internal congestion, which represents the proportion of objects relative to the area of ​​the elevator, by combining the two types of measured congestion.

[0102] Specifically, the congestion calculation module (310) can measure congestion as a ratio of the result of combining two weighting functions on an input image to the area of ​​the measurement space. Here, the first weighting function can measure congestion by applying a depth value-based weight, and the second weighting function can measure congestion by applying a weight that is inversely proportional to the distance from the center of the image.

[0103] Figure 6 is a flowchart for explaining a method for boarding a robot in an elevator based on the internal congestion of the elevator of the present invention.

[0104] The present invention calculates the internal congestion level of an elevator after the elevator arrives at the floor where the robot is located, rather than calculating the internal congestion level when an elevator boarding request is received from a robot.

[0105] Specifically, when the robot (500) arrives at a position where it can board the elevator (200), the robot server (400) can request the elevator server (300) to call the elevator (200) and request information on the internal congestion of the elevator.

[0106] When the elevator (200) arrives at the floor where the robot (500) is located (S602), the elevator server (300) can acquire a depth information image (S604), calculate the internal congestion of the elevator, and transmit the calculated internal congestion to the robot server (400).

[0107] Here, the internal congestion of the elevator can be calculated by applying the two types of weighting functions described above.

[0108] The robot server (400) can control the robot (500) to board the elevator (200) when the internal congestion of the elevator is lower than a reference value (S606, S608).

[0109] Meanwhile, if the internal congestion of the elevator is higher than a reference value, the robot server (400) may request the elevator server to not allow the robot to ride and to re-call the elevator in the forward direction in which the robot wants to move.

[0110] Specifically, the robot server (400) can determine that the robot has not boarded the elevator and has failed to board if the internal congestion of the elevator is higher than (or equal to) a reference value (S606, S610).

[0111] If the number of failed boarding attempts is less than a preset number (N times) (S612), the robot server (400) can request the elevator server to re-call the elevator in the forward direction in which the robot wants to move (S614).

[0112] Here, forward direction means that the elevator is called according to the direction in which the robot wants to move (whether to go up or down). If the robot wants to go up, the elevator is called in the direction of going up, and if the robot wants to go down, the elevator is called in the direction of going down.

[0113] That is, the forward direction means that it is set downward when moving from a high floor to a low floor, and it is set upward when moving from a low floor to a high floor.

[0114] Meanwhile, if the internal congestion of the elevator is higher than the reference value and the number of times the robot has not boarded the elevator reaches a preset number of times (N times), the robot server (400) can request the elevator server (300) to re-call the elevator in the opposite direction of the direction in which the robot intends to move (S616).

[0115] Thereafter, when the elevator arrives to move in the reverse direction (S618), the robot server (400) requests information on the internal congestion level from the elevator server (300), and the elevator server (300) obtains depth image information through the depth camera based on the request, applies two types of weighting functions to calculate the internal congestion level, and transmits the result to the robot server (400).

[0116] Here, reverse direction can mean calling an elevator in the downward direction when the robot wants to go up, or calling an elevator in the upward direction when the robot wants to go down.

[0117] In other words, reverse direction means calling the elevator by setting it upward when moving from a high floor to a low floor, or by setting it downward when moving from a low floor to a high floor.

[0118] If the transmitted internal congestion is lower than the reference value (S620), the robot server (400) can board the robot (500) in the elevator (200) (S608).

[0119] Meanwhile, if the internal congestion of the elevator is higher than the reference value (S620), the robot server (400) may determine that boarding has failed and may not allow the robot to board (S622).

[0120] The robot server (400) can attempt to board the elevator in the reverse direction through internal congestion until a certain number of times (M times) has elapsed even after the elevator has arrived at the floor where the robot is located (S624), and can re-call the elevator in the reverse direction until boarding is successful before the certain number of times has elapsed (S16).

[0121] Meanwhile, if the number of times the robot has failed to board the elevator despite it arriving in the opposite direction reaches a certain number (M times), the robot server (400) may determine that the robot cannot board the elevator (S626). Thereafter, the robot server (400) may cause the robot to attempt to board the elevator again after a certain amount of time has passed, or may move the robot to attempt to board a different elevator.

[0122] FIG. 7 is a drawing for explaining the result of calculating the congestion level inside an elevator according to one embodiment of the present invention.

[0123] As illustrated in FIG. 7, the present invention can calculate the internal congestion of an elevator by applying two types of weighting functions, and can determine whether to allow a robot to board the elevator based on the calculated internal congestion after the elevator arrives by calculating the internal congestion.

[0124] According to embodiments of the present invention, by measuring internal congestion while taking into account the influence of distortion, noise, and field of view of a depth camera image, the accuracy of internal congestion can be improved in a low-performance equipment environment.

[0125] The present invention can determine the level of congestion inside an elevator, including not only passengers but also general objects such as boxes and carts.

[0126] The present invention can minimize the situation in which a robot collides with passengers and cargo when boarding an elevator by determining the level of congestion inside the elevator when the elevator arrives at the floor on which the robot wishes to board and then boarding the elevator when a space for boarding the robot is secured.

[0127] The present invention can provide a countermeasure for frequent boarding failures due to many passengers trying to board in a specific direction by re-calling the same elevator or the direction of the corresponding elevator in case boarding space is not secured, and can increase the robot's elevator boarding rate.

[0128] Although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Furthermore, such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. An elevator server that controls an elevator with a depth camera installed; and It is formed to be able to communicate with the above elevator server and includes a robot server that controls a robot that wants to board the elevator. The above elevator server, Based on the image received from the depth camera, the internal congestion level of the elevator is calculated and the calculated internal congestion level is transmitted to the robot server. The above robot server, A system for controlling the robot to board the elevator based on the internal congestion level.

2. In paragraph 1, The above elevator server, A system including a congestion calculation module that calculates the internal congestion of the elevator using two types of weighting functions.

3. In paragraph 2, The above congestion calculation module is, An image receiving unit that receives a depth information image captured by the depth camera; An image correction unit for preprocessing the above depth information image; A congestion measurement unit that calculates the internal congestion of the elevator by applying the two types of weighting functions to the preprocessed depth information image; and A system including a communication unit that transmits the calculated internal congestion level to the elevator server.

4. In paragraph 3, Each pixel of the above depth information image contains a depth value, The above depth camera is installed at the top center of the elevator, The above two types of weighting functions are, A first weighting function that assigns weights in proportion to the depth value of the pixel so as to give weight to the congestion of the floor surface of the elevator; and A system including a second weighting function that inversely weights the pixel area proportion from the center of the image to the edge to compensate for the fact that the closer an object is to the center of the elevator, the smaller the area is captured.

5. In paragraph 4, The above first weighting function is, A system characterized in that the depth information image is formed so that a greater depth value is assigned to each pixel in order to measure the congestion level based on the floor surface of the elevator, and weights are assigned accordingly.

6. In paragraph 4, The second weighting function is, A system characterized in that, when an object is photographed with a depth camera installed at the upper center of the elevator, an object located at the center of the elevator is photographed vertically and thus has a small pixel area proportion measured, and an object located at the edge of the elevator is photographed obliquely and thus has a large pixel area proportion measured, so that weights are applied so that the area proportion decreases from the center to the edge.

7. In paragraph 3, The above congestion calculation module is, A system characterized in that the congestion is measured by applying the two types of weighting functions to the preprocessed depth information image, and the internal congestion is calculated by combining the two types of measured congestion to indicate the proportion of the area occupied by objects in the elevator.

8. In paragraph 1, The above robot server, A system characterized in that when the robot arrives at a position where it can board the elevator, it requests information on the internal congestion level of the elevator from the elevator server.

9. In paragraph 8, The above elevator server, A system characterized in that when the elevator arrives at the floor where the robot is located, the internal congestion level of the elevator is calculated and the calculated internal congestion level is transmitted to the robot server.

10. In paragraph 8, The above robot server, A system characterized in that, when the internal congestion of the elevator is lower than a reference value, the robot is controlled to board the elevator.

11. In paragraph 8, The above robot server, A system characterized in that, when the internal congestion of the elevator is higher than a reference value, the robot is not allowed to ride and the elevator server is requested to re-call the elevator in the forward direction in which the robot wants to move.

12. In paragraph 11, The above robot server, A system characterized in that, when the internal congestion of the elevator is higher than a reference value and the number of times the robot has not boarded the elevator reaches a preset number of times, the system requests the elevator server to re-call the elevator in the opposite direction to the direction in which the robot intends to move.

13. In paragraph 12, The above robot server, A system characterized in that when the number of times the elevator has arrived in the reverse direction and the number of times the user has not boarded it reaches a certain number, the robot determines that the user cannot board the elevator.

Images