Elevator System

Abstract

To provide an elevator system that accurately detects a state in which a user's hands are occupied with luggage or the like and enables the user to smoothly register a destination floor.SOLUTION: An elevator system according to one embodiment includes a camera installed in an elevator car, user detection means for detecting a user who has entered the elevator car based on multiple images continuously taken by the camera, feature point detection means for detecting the coordinates of feature points corresponding to each part of the user's body for each image, state determination means for evaluating the state of the user's arms based on the coordinates of each feature point of the user and determining whether the user's hands are occupied, and registration control means for registering a destination floor by voice if the user's hands are occupied.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE An embodiment of the present invention relates to an elevator system. [Background technology]

[0002] Normally, a passenger who gets into a car presses a destination floor button installed inside the car to register the destination floor. However, for example, if a user who is carrying luggage in both hands gets into a car, the user needs to place the luggage on the floor in order to press the destination floor button, which makes it difficult to register the destination floor smoothly.

[0003] In recent years, systems have become known that can detect wheelchair users, elderly people, and users with luggage in both hands (people carrying luggage), and register the destination floor by voice. By using this system, even if the user's hands are full of luggage, the destination floor can be smoothly registered without placing the luggage on the floor. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5447659 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned system needs to detect a wide variety of users with different attributes, such as not only people carrying baggage but also wheelchair users and the elderly, and therefore has problems such as being unable to accurately detect users who are carrying baggage in both hands, for example, mistakenly detecting people carrying baggage as wheelchair users or the elderly, or mistakenly detecting wheelchair users or the elderly as people carrying baggage.

[0006] The problem that the present invention aims to solve is to provide an elevator system that can accurately detect a state in which a user's hands are occupied with luggage or the like, and enables the user to smoothly register a destination floor. [Means for solving the problem]

[0007] In one embodiment, the elevator system detects a user who is in the elevator based on a camera installed in the elevator and a plurality of images continuously captured by the camera. and an object held by the user Detect Examination a feature point detection means for detecting the coordinates of feature points corresponding to each part of the user's body for each of the images; and a feature point detection means for detecting the coordinates of each of the feature points of the user's arms based on the coordinates of each of the feature points of the user. Relative movement and position of the arms and the object The device is characterized by comprising a state determination means for evaluating the state of the user and determining whether both hands of the user are occupied, and a registration control means for registering a destination floor by voice when both hands of the user are occupied. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing a schematic configuration example of an elevator system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing a configuration example of a portion around an entrance in a car in the embodiment. [Diagram 3] FIG. 3 is a diagram for explaining an outline of the operation of the elevator system in the embodiment. [Figure 4] FIG. 4 is a diagram for explaining a method of evaluating the first condition in the embodiment. [Diagram 5] FIG. 5 is a diagram for explaining a method of evaluating the first condition in the embodiment. [Figure 6] FIG. 6 is a diagram for explaining a method of evaluating the second condition in the embodiment. [Figure 7] FIG. 7 is a diagram for explaining a method of evaluating the second condition in the embodiment. [Figure 8]FIG. 8 is a diagram for explaining a method of evaluating the third condition in the embodiment. [Figure 9] FIG. 9 is a diagram for explaining a method of evaluating the third condition in the embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of a processing procedure of the elevator system in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. In addition, the disclosure is merely an example, and the invention is not limited to the contents described in the following embodiments. Modifications that a person skilled in the art can easily conceive are naturally included in the scope of the disclosure. In order to make the explanation clearer, the size, shape, etc. of each part may be changed from the actual embodiment and shown diagrammatically. In multiple drawings, corresponding elements may be given the same reference numerals, and detailed explanations may be omitted.

[0010] 1 is a diagram showing a schematic configuration example of an elevator system according to this embodiment. Note that, although a single car will be described as an example here, the same configuration applies to a system having multiple cars.

[0011] A camera 12 is installed above the entrance of the car 11. Specifically, the camera 12 is installed in a fascia 11a covering the upper part of the entrance of the car 11, with the lens portion facing directly downward, or toward the hall 15 side or the inside of the car 11. The camera 12 has an ultra-wide-angle lens such as a fisheye lens, and captures a wide range of the subject including the inside of the car 11 with a viewing angle of 180 degrees or more. The camera 12 can continuously capture several frames per second (e.g., 30 frames / second). The camera 12 is activated when the car 11 arrives at the hall 15 on each floor, and captures a predetermined range L1.

[0012] The location of the camera 12 does not have to be above the entrance of the car 11, so long as it is near the car door 13. For example, the camera 12 may be installed in any location, such as the ceiling surface near the entrance of the car 11, that can capture an image of the entire car interior including the entire floor area inside the car 11 and the landing 15 near the entrance when the door is open.

[0013] At the landing 15 of each floor, a landing door 14 is installed at the arrival gate of the car 11. The landing door 14 engages with the car door 13 when the car 11 arrives and opens and closes. The power source (door motor) is on the car 11 side, and the landing door 14 simply opens and closes following the car door 13.

[0014] Each image (video) continuously captured by the camera 12 is analyzed in real time by the image processing device 20. For convenience, the image processing device 20 is shown removed from the car 11 in Fig. 1, but in reality, the image processing device 20 is housed in the headboard 11a together with the camera 12.

[0015] The image processing device 20 includes a storage unit 21 and a control unit 22. The storage unit 21 sequentially stores images captured by the camera 12. The storage unit 21 has a buffer area for temporarily storing data necessary for processing by the control unit 22. The storage unit 21 may store images that have been subjected to processing such as distortion correction, enlargement / reduction, and partial cropping as preprocessing of the images captured by the camera 12.

[0016] The control unit 22 executes a process of determining whether or not both hands of the user in the elevator 11 are occupied, using the image captured by the camera 12. In the following, the state in which both hands of the user are occupied may be referred to as a "hands-full state."

[0017] As shown in FIG. 1, the control unit 22 includes a user detection unit 23, an object detection unit 24, a feature point detection unit 25, and a state determination unit .

[0018] The user detection unit 23 detects a user who has boarded the car 11 based on the image captured by the camera 12. Specifically, the user detection unit 23 divides the image captured by the camera 12 into a number of blocks, and performs a motion detection process to detect blocks with movement (movement blocks) from among the blocks by focusing on changes in color information of each block (for example, changes in luminance value). According to this motion detection process, a user shown in the image is detected as a collection of motion detection blocks. The user detection unit 23 detects a user who has boarded the car 11 by the collection of motion blocks moving into the car 11.

[0019] Note that the user detection unit 23 may detect users not by the above-mentioned motion detection process, but by, for example, SSD (Single Shot Multibox Detector), which is one of the image recognition methods using supervised learning. In this case, the user detection unit 23 can efficiently detect users who have boarded the car 11, including their boarding positions, from newly captured images by learning images in which users who have boarded the car 11 are captured as teacher data.

[0020] The object detection unit 24 detects an object in the vicinity of the user detected by the user detection unit 23. In this specification, the term "object" refers to an object that can be moved (carried) by the user by hand, including, for example, luggage, a bag, a dolly, a wheelchair, etc. The object detection unit 24 may detect an object by, for example, the SSD described above. In this case, the object detection unit 24 can efficiently detect objects such as luggage, including their positions, from newly captured images by learning images in which objects such as luggage are captured in advance as teacher data.

[0021] The feature point detection unit 25 detects the coordinates (coordinates on the image) of feature points corresponding to each part of the user's body detected by the user detection unit 23 from the image captured by the camera 12. The details will be described later, but the coordinates of each feature point detected by the feature point detection unit 25 are coordinates in an XY coordinate system (two-dimensional coordinate system) with the lower left vertex of the image captured by the camera 12 as the origin O (see FIG. 6). Note that in this embodiment, it is assumed that the feature point detection unit 25 detects the coordinates of feature points corresponding to the user's right shoulder, left shoulder, right elbow, left elbow, right hand, and left hand as the coordinates of each feature point of the user, but this is not limited thereto, and the feature point detection unit 25 may further detect the coordinates of feature points corresponding to parts other than the above-mentioned parts (for example, both knees and both feet).

[0022] It should be noted that the algorithm used by the feature point detection unit 25 to detect the coordinates of the user's feature points is not particularly limited. However, the feature point detection unit 25 may detect the coordinates of the user's feature points using a known keypoint detection technique such as Open Pose.

[0023] The state determination unit 26 evaluates the state of both arms of the user based on the object near the user detected by the object detection unit 24 and the coordinates of each feature point of the user detected by the feature point detection unit 25, and determines whether both hands of the user are occupied (in other words, determines whether the user is in a hands-full state). Note that the detailed function of the state determination unit 26 will be described later, and therefore a detailed description thereof will be omitted here.

[0024] The elevator control device 30 controls the raising and lowering operation of the car 11 and the operation of various devices (such as destination floor buttons and lighting) installed in the car 11. As shown in FIG. 1, the elevator control device 30 includes a registration control unit 31, a notification unit 32, and a door opening / closing control unit 33.

[0025] The registration control unit 31 registers the destination floor of a user. Specifically, the registration control unit 31 registers the destination floor of the user in response to the user's operation of the destination floor button 44 arranged on the operation panel 45. The destination floor button 44 may be a push button or a non-contact button. The registration control unit 31 also analyzes and processes the user's voice signal input via the voice input device 16 installed in the car 11, and registers the destination floor of the user.

[0026] Hereinafter, the act of analyzing and processing the voice signal to register the destination floor of the user is referred to as "registering a destination floor by voice." Registering a destination floor by voice is realized using a known voice recognition technology. For example, the registration control unit 31 compares the user's voice signal input via the voice input device 16 with a voice signal indicating a floor that has been stored in advance, and if the two match under a predetermined condition, registers the floor corresponding to the voice signal as the destination floor of the user.

[0027] If the function that enables the destination floor to be registered by voice (voice registration function) is always enabled, there is a possibility that the destination floor may be registered erroneously due to conversations of users or noises in the car 11. For this reason, it is desirable that the registration control unit 31 normally disables the voice registration function, and enables the voice registration function when the state determination unit 26 determines that the user has both hands occupied.

[0028] When the user has both hands full, the notification unit 32 notifies the user that the destination floor can be registered by voice via the speaker 46 installed in the car 11. This allows the user to know that the destination floor can be registered by voice even with both hands full.

[0029] The door opening / closing control unit 33 controls the opening and closing of the car door 13 when the car 11 arrives at the landing 15. Specifically, the door opening / closing control unit 33 opens the car door 13 when the car 11 arrives at the landing 15, and closes the door after a predetermined time has elapsed.

[0030] FIG. 2 is a diagram showing an example of the configuration of the area around the entrance / exit inside the car 11. As shown in FIG. A car door 13 is provided at the entrance of the car 11 so as to be freely opened and closed. In the example of Fig. 2, a two-door double-opening car door 13 is shown, and two door panels 13a, 13b constituting the car door 13 open and close in opposite directions along the width direction (horizontal direction). Note that the "width" refers to the entrance of the car 11.

[0031] A display 43, an operation panel 45 including destination floor buttons 44, and a speaker 46 are installed on one or both of the front pillars 41a and 41b. In the example of Fig. 2, the speaker 46 is installed on the front pillar 41a, and the display 43 and the operation panel 45 are installed on the front pillar 41b.

[0032] A camera 12 having a super-wide-angle lens such as a fisheye lens is installed in the center of a panel 11a above the entrance of the car 11 with the lens facing downward.

[0033] A voice input device 16 is installed on the operation panel 45. The voice input device 16 is installed with an input portion facing the inside of the car 11 so that a user can input voice. Note that the voice input device 16 may be installed in a position different from the operation panel 45 as long as it is a position where a user can easily input voice.

[0034] 3 is a diagram for explaining an outline of the operation of the elevator system according to this embodiment. The elevator system according to this embodiment detects a user U who has entered the car 11 based on an image captured by a camera 12 installed above the entrance of the car 11, and judges whether or not both hands of the user U are occupied. If it is judged that both hands of the user U are occupied, the elevator system activates the above-mentioned voice registration function.

[0035] Next, the function of the state determination unit 26 will be described in detail with reference to FIGS. The state determination unit 26 determines whether both of the user's hands are occupied based on the object near the user detected by the object detection unit 24 and the coordinates of each feature point of the user detected by the feature point detection unit 25.

[0036] Specifically, the state determination unit 26 determines that both of the user's hands are occupied when a first condition, a second condition, and a third condition, which will be described later, are all satisfied. First condition: The user's right arm and left arm are parallel. Second condition: The user's right and left arms are not moving relative to each other. Third condition: There is an object extending from one of the user's hands to the other, or there are two objects extending from the user's right hand and left hand, respectively. Below, the methods of evaluating the first condition, the second condition, and the third condition will be explained in order.

[0037] (Evaluation method for the first condition) 4 and 5 are diagrams for explaining a method of evaluating the first condition. In Figures 4 and 5, it is assumed that the feature point detection unit 25 detects (xb1, yb1) as the coordinates of feature point P1 corresponding to the right shoulder of user U, (xb2, yb2) as the coordinates of feature point P2 corresponding to the left shoulder of user U, (xc1, yc1) as the coordinates of feature point P3 corresponding to the right elbow of user U, (xc2, yc2) as the coordinates of feature point P4 corresponding to the left elbow of user U, (xd1, yd1) as the coordinates of feature point P5 corresponding to the right hand of user U, and (xd2, yd2) as the coordinates of feature point P6 corresponding to the left hand of user U.

[0038] The state determination unit 26 determines, based on the coordinates of each feature point of the user U detected by the feature point detection unit 25, whether the right arm and left arm of the user U are parallel to each other.

[0039] Specifically, the state determination unit 26 calculates the tilt a1 of the right upper arm from the right shoulder to the right elbow based on the coordinates (xb1, yb1) of the feature point P1 corresponding to the right shoulder of the user U and the coordinates (xc1, yc1) of the feature point P3 corresponding to the right elbow of the user U. Specifically, the tilt a1 of the right upper arm can be calculated by (yb1-yc1) / (xb1-xc1).

[0040] Furthermore, the state determination unit 26 calculates the tilt a2 of the left upper arm from the left shoulder to the left elbow based on the coordinates (xb2, yb2) of the feature point P2 corresponding to the left shoulder of the user U and the coordinates (xc2, yc2) of the feature point P4 corresponding to the left elbow of the user U. Specifically, the tilt a2 of the left upper arm can be calculated by (yb2-yc2) / (xb2-xc2).

[0041] The state determination unit 26 determines that the right upper arm and the left upper arm are parallel when the calculated inclination a1 of the right upper arm and the inclination a2 of the left upper arm match (or when the calculated inclination a1 of the right upper arm and the inclination a2 of the left upper arm are approximately the same).

[0042] Similarly, the state determination unit 26 calculates the tilt a3 of the right forearm from the right elbow to the right hand based on the coordinates (xc1, yc1) of the feature point P3 corresponding to the right elbow of the user U and the coordinates (xd1, yd1) of the feature point P5 corresponding to the right hand of the user U. Specifically, the tilt a3 of the right forearm can be calculated by (yc1-yd1) / (xc1-xd1).

[0043] Furthermore, the state determination unit 26 calculates the tilt a4 of the left forearm from the left elbow to the left hand based on the coordinates (xc2, yc2) of the feature point P4 corresponding to the left elbow of the user U and the coordinates (xd2, yd2) of the feature point P6 corresponding to the left hand of the user U. Specifically, the tilt a4 of the left forearm can be calculated by (yc2-yd2) / (xc2-xd2).

[0044] The state determination unit 26 determines that the right and left forearms are parallel when the calculated inclination a3 of the right forearm and the inclination a4 of the left forearm match (or when the inclination a3 of the right forearm and the inclination a4 of the left forearm are approximately matched).

[0045] The state determination unit 26 determines that the right arm and left arm of user U are parallel when it is determined that the right upper arm and left upper arm of user U are parallel and the right forearm and left forearm of user U are parallel. Note that both FIG. 4 and FIG. 5 show an example of a state in which the right arm and the left arm of the user U are determined to be parallel to each other.

[0046] (Method of evaluation of the second condition) 6 and 7 are diagrams for explaining a method of evaluating the second condition. In Figures 6 and 7, it is assumed that the feature point detection unit 25 detects (xb1, yb1) as the coordinates of feature point P1 corresponding to the right shoulder of user U, (xb2, yb2) as the coordinates of feature point P2 corresponding to the left shoulder of user U, (xc1, yc1) as the coordinates of feature point P3 corresponding to the right elbow of user U, (xc2, yc2) as the coordinates of feature point P4 corresponding to the left elbow of user U, (xd1, yd1) as the coordinates of feature point P5 corresponding to the right hand of user U, and (xd2, yd2) as the coordinates of feature point P6 corresponding to the left hand of user U from the current image (image at time T2).

[0047] In addition, in Figures 6 and 7, it is assumed that the feature point detection unit 25 detects (xb3, yb3) as the coordinates of feature point P1' corresponding to the right shoulder of user U, (xb4, yb4) as the coordinates of feature point P2' corresponding to the left shoulder of user U, (xc3, yc3) as the coordinates of feature point P3' corresponding to the right elbow of user U, (xc4, yc4) as the coordinates of feature point P4' corresponding to the left elbow of user U, (xd3, yd3) as the coordinates of feature point P5' corresponding to the right hand of user U, and (xd4, yd4) as the coordinates of feature point P6' corresponding to the left hand of user U, from the image immediately before the current image (the image at time T1).

[0048] The state determination unit 26 determines whether the right arm and left arm of the user U are not moving relative to each other based on the coordinates of each feature point of the user U detected from the current image and the coordinates of each feature point of the user U detected from the image immediately before the current image.

[0049] 6(b), based on the coordinates (xb1, yb1) of a feature point P1 corresponding to the right shoulder of the user U detected from the current image (image at time T2) and the coordinates (xb2, yb2) of a feature point P2 corresponding to the left shoulder of the user U, the state determination unit 26 calculates the difference between these two coordinates (first difference). Specifically, the first difference can be calculated by (xb1-xb2, yb1-yb2).

[0050] 6(a), based on the coordinates (xb3, yb3) of a feature point P1' corresponding to the right shoulder of the user U and the coordinates (xb4, yb4) of a feature point P2' corresponding to the left shoulder of the user U, detected from the image immediately preceding the current image (the image at time T1), the state determination unit 26 calculates a difference between these two coordinates (a second difference). Specifically, the second difference can be calculated by (xb3-xb4, yb3-yb4).

[0051] The state determination unit 26 compares the difference (first difference) between the coordinates of the right and left shoulders in the current image (image at time T2) with the difference (second difference) between the coordinates of the right and left shoulders in the previous image (image at time T1), and if there is no discrepancy between these two differences (i.e., if the difference between the first difference and the second difference is within a predetermined allowable range), it determines that the right and left shoulders of the user U have not moved relatively between times T1 and T2. In other words, the state determination unit 26 determines that there has been no change in the posture of the user U (the positional relationship between the right and left shoulders) between times T1 and T2.

[0052] For example, as shown in the lower table of Figure 7, if the coordinates (xb1, yb1) of the right shoulder in the current image (image at time T2) are (6, 7) and the coordinates (xb2, yb2) of the left shoulder are (8, 7), the first difference, which is the difference between these two coordinates, can be calculated as (-2, 0).

[0053] Similarly, as shown in the upper table of Figure 7, if the coordinates (xb3, yb3) of the right shoulder in the previous image (image at time T1) are (2, 3) and the coordinates (xb4, yb4) of the left shoulder are (4, 3), the second difference, which is the difference between these two coordinates, can be calculated as (-2, 0).

[0054] In this case, the state determination unit 26 determines that the first difference and the second difference are both (-2, 0), there is no discrepancy between these two differences, and therefore determines that the right and left shoulders of the user U have not moved relative to each other between times T1 and T2.

[0055] Although a detailed description will be omitted here, the state determination unit 26 also determines whether the right and left elbows and the right and left hands of the user U are not moving relative to each other in a similar manner.

[0056] When the state determination unit 26 determines that the right and left shoulders, right and left elbows, and right and left hands of the user U are not moving relative to each other, it determines that the right and left arms of the user U are not moving relative to each other.

[0057] (Evaluation method for the third condition) (First evaluation method for the third condition) FIG. 8 is a diagram for explaining the first evaluation method for the third condition. The state determination unit 26 determines, based on the object detected by the object detection unit 24, whether or not an object extending from one hand of the user U to the other hand is present.

[0058] Specifically, as shown in Figures 8(a) and (b), when the object detected by the object detection unit 24 has an edge S1 extending from the coordinates of feature point P5 corresponding to the right hand of the user U to the coordinates of feature point P6 corresponding to the left hand of the user U, the state determination unit 26 determines that there is an object extending from one hand of the user U to the other hand.

[0059] Fig. 8(a) is a diagram showing an example of a state in which it is determined that an object extending from one hand to the other hand of a user is present, and shows a state in which the user U is holding a box-shaped package with both arms. Fig. 8(b) is also a diagram showing an example of a state in which it is determined that an object extending from one hand to the other hand of a user is present, and shows a state in which the user U is holding, for example, a dolly by its handle with both hands.

[0060] (Second evaluation method of the third condition) FIG. 9 is a diagram for explaining the second evaluation method for the third condition. The state determination unit 26 determines, based on the objects detected by the object detection unit 24, whether or not two objects are present extending from the right and left hands of the user U.

[0061] Specifically, as shown in Figures 9(a) and (b), when the object detected by the object detection unit 24 has a side S2 extending from the coordinates of feature point P5 corresponding to the right hand of user U, and a side S3 extending from the coordinates of feature point P6 corresponding to the left hand of user U, the state determination unit 26 determines that there are two objects extending from each of the right and left hands of user U.

[0062] Fig. 9(a) is a diagram showing an example of a state in which it is determined that there are two objects extending from the right and left hands of a user, and shows a state in which the user U is, for example, gripping the handles of a wheelchair with the right and left hands. Fig. 9(b) is also a diagram showing an example of a state in which it is determined that there are two objects extending from the right and left hands of a user, and shows a state in which the user U is, for example, gripping the handles of a bag or a plastic bag with the right and left hands.

[0063] When all of the above-mentioned first condition, second condition, and third condition are satisfied, the state determination unit 26 determines that both hands of the user U are occupied (in other words, determines that the user U is in a hands-full state).

[0064] Here, an example of a processing procedure of the elevator system according to this embodiment will be described with reference to the flowchart of FIG. When the elevator car 11 arrives at an arbitrary floor (Yes in step ST11), the door opening / closing control unit 33 opens the elevator car door 13 (step ST12). When the elevator car door 13 opens, the user detection unit 23 detects a user who has entered the elevator car 11 based on the image captured by the camera 12 (step ST13).

[0065] If the user detection unit 23 detects a user who has boarded the elevator car 11 (Yes in step ST13), the object detection unit 24 detects an object present in the vicinity of the user detected by the user detection unit 23 (step ST14).

[0066] If there is no object near the user, i.e., if no object is detected near the user (No in step ST14), it is unlikely that the user has both hands full, so the registration control unit 31 does not enable destination floor registration by voice (i.e., does not enable the voice registration function), and only registers the destination floor in response to the user's button operation (step ST15).

[0067] If an object is present near the user, that is, if an object is detected near the user (Yes in step ST14), the feature point detection unit 25 detects the coordinates of feature points corresponding to each part of the user's body (step ST16).

[0068] Thereafter, the state determination unit 26 determines whether or not both of the user's hands are occupied. Specifically, first, the state determination unit 26 determines whether the right arm and the left arm of the user are parallel based on the coordinates of each feature point of the user detected by the feature point detection unit 25. In other words, the state determination unit 26 determines whether the above-mentioned first condition is satisfied (step ST17). If it is determined that the right arm and the left arm of the user are not parallel, that is, if the above-mentioned first condition is not satisfied (No in step ST17), the state determination unit 26 determines that both hands of the user are not occupied. In this case, the process of step ST15 described above is executed.

[0069] If it is determined that the right arm and the left arm of the user are parallel, that is, if the above-mentioned first condition is satisfied (Yes in step ST17), the state determination unit 26 determines whether the right arm and the left arm of the user are not moving relatively based on the coordinates of each feature point of the user detected from the current image (latest image) and the coordinates of each feature point of the user detected from the image immediately before the current image. In other words, the state determination unit 26 determines whether the above-mentioned second condition is satisfied (step ST18). Note that if it is determined that the right arm and the left arm of the user are moving relatively, that is, if the above-mentioned second condition is not satisfied (No in step ST18), the state determination unit 26 determines that both hands of the user are not occupied. In this case, the process of step ST15 described above is executed.

[0070] If it is determined that the right arm and the left arm of the user are not moving relative to each other, that is, if the above-mentioned second condition is satisfied (Yes in step ST18), the state determination unit 26 determines whether or not there is an object extending from one hand of the user to the other hand. In other words, the state determination unit 26 determines whether or not one of the above-mentioned third conditions is satisfied (step ST19). Note that if it is determined that there is an object extending from one hand of the user to the other hand, that is, if one of the above-mentioned third conditions is satisfied (Yes in step ST19), the process of step ST21 described below is executed.

[0071] On the other hand, if it is determined that there is no object extending from one hand of the user to the other hand, that is, if one of the above-mentioned third conditions is not satisfied (No in step ST19), the state determination unit 26 determines whether or not there are two objects extending from the right hand and the left hand of the user. In other words, the state determination unit 26 determines whether or not the other condition of the above-mentioned third condition is satisfied (step ST20). Note that if it is determined that there are no two objects extending from the right hand and the left hand of the user, that is, if the other condition of the above-mentioned third condition is not satisfied (No in step ST20), the state determination unit 26 determines that both hands of the user are not occupied. In this case, the process of step ST15 described above is executed.

[0072] If it is determined that there are two objects extending from the user's right hand and left hand, respectively, that is, if the other of the third conditions described above is satisfied (Yes in step ST20), the state determination unit 26 determines that both of the user's hands are occupied. In this case, the notification unit 32 notifies the user that the destination floor can be registered by voice (step ST21). The registration control unit 31 also enables the voice registration function, making it possible to use both the registration of the destination floor by voice and the registration of the destination floor using the destination floor button 44 (step ST22).

[0073] Thereafter, when the elevator car 11 closes its doors and departs for the destination floor (step ST23), the series of processes here ends. If the voice registration function is enabled by the process of step ST22 described above, the registration control unit 31 disables the voice registration function when the elevator car 11 doors are all closed in order to prevent erroneous registration of the destination floor due to conversations of users in the elevator car 11.

[0074] As described above, the elevator system according to this embodiment detects the coordinates of feature points corresponding to each part of the body of a user who is in the car 11, evaluates the state of both arms of the user based on the coordinates of each detected feature point, and determines whether both hands of the user are occupied. In other words, the elevator system according to this embodiment can perform processing specialized for detecting users with both hands occupied. This makes it possible to accurately detect users with both hands occupied compared to the case of detecting a wide variety of users with different attributes, such as wheelchair users and elderly people in addition to users with both hands occupied.

[0075] In addition, when the elevator system of this embodiment detects a user with both hands occupied, it enables the user to register a destination floor by voice (voice registration function), allowing the user to register a destination floor hands-free even with both hands occupied.

[0076] When registering a destination floor by voice, the destination floor spoken by the user may not be recognized due to surrounding noise, etc. In this case, the user must speak the destination floor again, so registering a destination floor by voice takes more time than registering a destination floor using the destination floor button. For this reason, when it is determined that the user's hands are occupied and registration of a destination floor by voice is enabled, it is desirable for the door opening / closing control unit 33 to perform control to extend the door open time more than usual. This can prevent the door from closing while the user is registering a destination floor by voice.

[0077] In addition, in this embodiment, a case has been described in which the status determination unit 26 determines whether or not both hands of a user who has boarded the elevator 11 are occupied, but this is not limited to this, and the status determination unit 26 may further determine, using a similar method, whether or not both hands of a user before boarding the elevator 11 (i.e., a user at the platform 15) are occupied.

[0078] According to the embodiment described above, it is possible to provide an elevator system that can accurately detect a state in which a user's hands are occupied with luggage or the like, and enables the user to smoothly register a destination floor.

[0079] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0080] 11...car, 12...camera, 13...car door, 14...landing door, 15...landing, 16...voice input device, 20...image processing device, 21...memory unit, 22...control unit, 23...user detection unit, 24...object detection unit, 25...feature point detection unit, 26...state determination unit, 30...elevator control device, 31...registration control unit, 32...notification unit, 33...door opening / closing control unit, 44...destination floor button, 45...operation panel, 46...speaker.

Claims

1. A camera installed in the car; A detection means for detecting a user who is in the elevator and an object held by the user based on a plurality of images continuously captured by the camera; a feature point detection means for detecting, for each of the images, coordinates of feature points corresponding to each part of the user's body; a state determination means for evaluating a relative movement of both arms of the user and a positional relationship between both arms and the object based on the coordinates of each feature point of the user, and determining whether both hands of the user are occupied; a registration control means for registering a destination floor by voice when the user has both hands full; An elevator system comprising:

2. The state determination means The right and left arms of the user shown in the current image are parallel; The right arm and the left arm of the user shown in the current image and the right arm and the left arm of the user shown in the image immediately before the current image are not moving relative to each other, and When an object extending from one hand to the other hand of the user is present in the current image, it is determined that both hands of the user are occupied.

10. The elevator system of claim 1.

3. The state determination means The right and left arms of the user shown in the current image are parallel; The right arm and the left arm of the user shown in the current image and the right arm and the left arm of the user shown in the image immediately before the current image are not moving relative to each other, and When there are two objects extending from the right hand and the left hand of the user shown in the current image, it is determined that both hands of the user are occupied.

10. The elevator system of claim 1.

4. the feature point detection means detects coordinates of feature points corresponding to the user's right shoulder, left shoulder, right elbow, left elbow, right hand, and left hand as coordinates of each feature point of the user, 10. The elevator system of claim 1.

5. the feature point detection means detects coordinates in a two-dimensional coordinate system having a lower left vertex of each of the images as an origin, 10. The elevator system of claim 1.

6. The present invention is characterized in that the device further includes a notification means for notifying the user by voice that a destination floor can be registered when the state determination means determines that both hands of the user are occupied.

10. The elevator system of claim 1.

7. The vehicle further includes a door opening / closing control means for extending a door opening time of the car when the state determination means determines that both hands of the user are occupied.

10. The elevator system of claim 1.

8. The camera includes at least a part of the hall in its photographing range, The detection means further detects a user at the hall based on each of the images, The feature point detection means further detects, for each of the images, coordinates of feature points corresponding to each body part of the user at the hall; The state determination means evaluates a state of both arms of the user at the platform based on coordinates of each feature point of the user at the platform, and further determines whether both hands of the user at the platform are occupied.

10. The elevator system of claim 1.

Images