Elevator system and elevator control method

Abstract

To provide an elevator system that is able to perform more appropriate operation control of an elevator according to a situation of a predetermined target object such as a wheelchair user.SOLUTION: An elevator system (1) includes: an imaging unit (40) that images inside of a car (21) of an elevator (20) and a landing (50) in a state where a door (24) of the elevator (20) is open; a target recognition unit (112) that detects a predetermined target object (M) from a captured image (G) by image recognition; a position specifying unit (113) that specifies a position of the target object (M) in the captured image (G); and an operation control unit (115) that controls an operation of the elevator (20) according to the position of the target object (M).SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an elevator system and an elevator control method. [Background technology]

[0002] Conventionally, elevator car call buttons and control panels in elevator cars are provided with a wheelchair button for wheelchair users to operate. When the wheelchair button is operated, the door is kept open for a longer period than usual, thereby enhancing the safety of wheelchair users.

[0003] Also, for example, Patent Document 1 discloses a door control device that recognizes the presence of a wheelchair by photographing the inside of the car and the entrance / exit area of ​​the elevator hall, and controls the time the door is open depending on the recognition result. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-156028 A Summary of the Invention [Problem to be solved by the invention]

[0005] The door control device described above only controls the opening and closing of the door depending on whether or not a wheelchair is recognized in the captured image. Therefore, it is not possible to recognize the detailed situation of the wheelchair user, and there is a problem that the control is insufficient.

[0006] An object of one aspect of the present invention is to provide an elevator system capable of performing more appropriate elevator operation control in accordance with the state of a predetermined target object, such as a wheelchair user. [Means for solving the problem]

[0007] In order to solve the above problems, an elevator system according to one embodiment of the present invention includes an imaging unit that images the inside of an elevator car and the landing when the elevator doors are in an open state, a target recognition unit that detects a predetermined target object from an image captured by the imaging unit through image recognition, a position identification unit that identifies the position of the target object detected by the target recognition unit within the captured image, and an operation control unit that controls the operation of the elevator in accordance with the position of the target object identified by the position identification unit.

[0008] According to the above configuration, the position of the target object is identified in the captured image showing the state of the inside of the elevator car and the landing. Therefore, the operation control of the elevator can be changed according to the location of the target object in the area including both the inside of the elevator car and the landing. Therefore, it is possible to perform more appropriate operation control of the elevator according to the state of the target object.

[0009] In an elevator system according to another aspect of the present invention, the target recognition unit generates a bounding box corresponding to the detected target object, and the position identification unit divides the captured image into a plurality of partial image areas and identifies the position of the target object within the captured image based on the positional relationship between the partial image areas and the bounding box generated by the target recognition unit.

[0010] According to the above configuration, by checking the positional relationship between the bounding box and the partial image region, it is possible to specify the position of the target object in the captured image relatively easily.

[0011] In an elevator system according to another aspect of the present invention, the imaging unit is installed at an upper part inside the car, and an angle formed between a straight line connecting the lens of the imaging unit and the center of the floor surface of the car and a normal to the floor surface of the car at the center of the floor surface of the car is within 20 degrees, and the operation control unit may recognize whether the target object is present inside the car depending on the position of the target object identified by the position identification unit.

[0012] According to the above configuration, regardless of where the target object is located in the car, the change range of the imaging angle of the imaging unit imaging the target object from above can be kept low, thereby improving the recognition accuracy of the target object and making it possible to more accurately determine whether the target object is located in the car.

[0013] In an elevator system according to another aspect of the present invention, the operation control unit may recognize, depending on the position of the target object identified by the position identification unit, whether the target object is (1) entirely contained within the car, (2) present across both the car and the landing, or (3) not entirely or partially present within the car.

[0014] According to the above configuration, it is possible to change the operation not only depending on whether the target object is in the car or not, but also when the target object is present both in the car and at the landing, thereby realizing more detailed operation control according to the position of the target object.

[0015] In an elevator system according to another aspect of the present invention, the imaging unit may be installed at an upper portion inside the car, and an angle formed by a straight line from the lens of the imaging unit to the center of the boundary between the inside of the car and the landing and a normal to the floor surface of the car at the center of the boundary may be within 20 degrees.

[0016] According to the above configuration, when a target object is present both inside the car and at the landing, the change range of the imaging angle of the imaging unit imaging the target object from above can be kept low. This improves the recognition accuracy of the target object, making it possible to more accurately determine whether the target object is present inside the car, in the boundary area, or elsewhere.

[0017] In an elevator system according to another aspect of the present invention, when the operation control unit determines that the target object is present both inside the car and at the landing, it may determine whether the target object is moving from the landing to inside the car, or from inside the car to the landing, based on the result of determining the previous location of the target object.

[0018] According to the above configuration, it is possible to perform appropriate elevator operation control depending on whether the target object is moving from the hall to the inside of the car, or from the inside of the car to the hall.

[0019] In an elevator system according to another aspect of the present invention, the operation control unit may control at least one of: (1) opening and closing operations of the doors; (2) display operations or audio output operations for at least one of the interior of the car and the landing; and (3) enabling / disabling switching operations of operation buttons for at least one of the interior of the car and the landing.

[0020] According to the above configuration, depending on the detection result of the target object, the door opening / closing operation can be changed, information can be presented to the user, and the operation buttons can be enabled / disabled, thereby enabling the elevator to operate more appropriately according to the user's condition.

[0021] In an elevator system according to another aspect of the present invention, the operation control unit may control the operation of the elevator when a user gives a call instruction for the elevator before boarding, depending on the relationship between the content of the call instruction and the position of the target object identified by the position identification unit.

[0022] According to the above configuration, it is possible to operate the elevator in a manner appropriate to the situation, depending on the relationship between the content of the call instruction and the position of the target object.

[0023] An elevator system according to another aspect of the present invention may further include a congestion degree calculation unit that calculates a congestion degree inside the car based on the types of target objects detected by the target recognition unit, the number of each type, and the positions identified by the position identification unit.

[0024] According to the above configuration, the degree of congestion inside the car is calculated taking into account the type and number of each type of target object, so that a congestion degree that is more appropriate to the actual situation can be calculated compared to when the congestion degree is simply calculated based only on the number of passengers or their weight.

[0025] An elevator control method according to one embodiment of the present invention is executed by one or more information processing devices, and includes an imaging step of imaging the inside of an elevator car and the landing when the elevator doors are in an open state, a target recognition step of detecting a predetermined target object from an image captured by the imaging step by image recognition, a position identification step of identifying a position within the captured image of the target object detected by the target recognition step, and an operation control step of controlling the operation of the elevator depending on the position of the target object identified by the position identification step.

[0026] The elevator control device according to each aspect of the present invention may be realized by a computer. In this case, the control program of the elevator control device, which causes the computer to operate as each part (software element) of the elevator control device, thereby realizing the elevator control device on a computer, and the computer-readable recording medium on which it is recorded, also fall within the scope of the present invention. Effect of the Invention

[0027] According to one aspect of the present invention, it is possible to provide an elevator system capable of performing more appropriate elevator operation control according to the state of a predetermined target object, such as a wheelchair user. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 is a functional block diagram of an elevator system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram showing a configuration example of the above-mentioned elevator system. [Diagram 3] FIG. 4 is a diagram showing an image captured by an imaging unit of the elevator system. [Figure 4] 3A and 3B are diagrams for explaining in detail the installation position of an imaging unit in the elevator system. [Diagram 5] FIG. 13 is a diagram illustrating an example of a method for detecting a target object. [Figure 6] FIG. 13 is a diagram illustrating an example of identifying the position of a target object. [Figure 7] FIG. 2 is a flow diagram illustrating an example of the operation of the elevator system. [Figure 8] FIG. 13 illustrates a specific variant of the position of the target object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] [Embodiment 1] An embodiment of the present invention will be described in detail below. However, the following description is an example of an elevator system 1 according to the present invention, and the technical scope of the present invention is not limited to the following description and illustrated examples.

[0030] [Elevator System Overview] Fig. 2 is a schematic diagram showing a configuration example of the elevator system 1. Fig. 3 is a diagram showing an image G captured by an imaging unit 40 of the elevator system 1. As shown in Figs. 2 and 3, the elevator system 1 captures an image G including the inside of the car 21 of the elevator 20 and a hall 50 when the door 24 of the elevator 20 is in an open state, by the imaging unit 40 installed near the ceiling inside the car 21. The elevator system 1 controls the operation of the elevator 20 according to the position of a target object M identified in the captured image G.

[0031] This allows the elevator system 1 to change the operation control of the elevator 20 depending on the location of the target object M within an area including both the inside of the car 21 and the hall 50. As a result, it is possible to realize the elevator system 1 that performs more appropriate operation control of the elevator 20 depending on the status of a specific target object M, such as a wheelchair user.

[0032] <Elevator system> The elevator system 1 controls the operation of an elevator 20 in accordance with the position of a target object M. FIG. 1 is a functional block diagram of the elevator system 1 according to a first embodiment of the present invention. As shown in FIGS. 1 and 2, the elevator system 1 includes an elevator control device 10, an elevator 20, a hall operating panel 51, and an alarm unit 52.

[0033] The hall operating panel 51 is an operating panel that is installed at the hall 50 and can register a call for the car 21. The call registration input by the hall operating panel 51 is transmitted to the elevator control device 10 by, for example, a communication device (not shown) installed at the hall 50.

[0034] The notification unit 52 is installed at the landing 50, and notifies various pieces of information under the control of the operation control unit 115 described later. The notification unit 52 may be a display device that displays various pieces of information, or may be an audio output device that outputs audio.

[0035] (Elevator) The elevator 20 includes a car 21, an in-car operation panel 22, a notification unit 23, a door 24, and an imaging unit 40. The car 21 stops at a hall 50. Although there are a plurality of halls 50, each hall 50 has a similar configuration, and therefore, only one hall 50 is shown in Figs. 1 and 2. In addition, in this embodiment, the elevator system 1 is described as including only one elevator 20, but this is not limited to the above. The elevator system 1 may include a plurality of elevators 20, or may include a group of elevators.

[0036] The car internal operation panel 22 is an operation panel that is installed in the car 21 and can register calls for the car 21. The input of call registration by the car internal operation panel 22 is transmitted to the elevator control device 10 by, for example, a communication device (not shown) installed in the car 21.

[0037] The notification unit 23 is installed in the car 21, and notifies various pieces of information under the control of the operation control unit 115 described later. The notification unit 23 may be a display device that displays various pieces of information, or may be an audio output device that outputs audio.

[0038] The doors 24 are installed between the car 21 and the landing 50, on both the car 21 side and the landing 50 side. The doors 24 are opened and closed under the control of an operation control unit 115, which will be described later. At the landing 50 where the car 21 is stopped, the doors 24 installed on the car 21 side and the landing 50 side are controlled to be opened and closed simultaneously.

[0039] (Imaging unit) As shown in Fig. 2, the imaging unit 40 is installed near the ceiling inside the car 21. 2001 in Fig. 2 shows an example in which the imaging unit 40 is installed at a position P1 near the center inside the car 21, and 2002 in Fig. 2 shows an example in which the imaging unit 40 is installed at a position P2 near the door 24 inside the car 21.

[0040] Fig. 3 is a diagram showing an image G captured by the imaging unit 40 of the elevator system 1 installed at position P1. As shown in Fig. 3, the imaging unit 40 captures images of the inside of the car 21 of the elevator 20 and the landing 50 when the door 24 of the elevator 20 is in an open state. The imaging unit 40 captures images G at predetermined intervals when the door 24 is in an open state.

[0041] Fig. 4 is a diagram for explaining in detail the installation position of the imaging unit 40 of the elevator system 1. In 4001 of Fig. 4, the imaging unit 40 is installed at position P1 in the upper part inside the car 21. Position P1 is a position where the angle θ1 between a line L1 and a normal N1 is within 20 degrees. The line L1 is a line connecting the lens of the imaging unit 40 and a center P11 of the floor surface 211 of the car 60. The normal N1 is a normal line of the floor surface 211 at the center P11 of the floor surface 211 of the car 21.

[0042] By positioning the imaging unit 40 at position P1, the range of change in the imaging angle of the target object M captured by the imaging unit 40 from above can be kept low regardless of where the target object M is located within the cage 21.

[0043] In 4002 of Fig. 4, the imaging unit 40 is installed at a position P2 near the door 24 in the upper part of the car 21. The position P2 is a position where the angle θ2 between a line L2 and a normal N2 is within 20 degrees. The line L2 is a line extending from the lens of the imaging unit 40 toward a center P21 of a boundary S1 between the floor surface 211 in the car 21 and the landing 50. The normal N2 is a normal to the floor surface 211 at the center P21 of the boundary S1.

[0044] By positioning the imaging unit 40 at position P2, when the target object M is present across both the car 21 and the hall 50, the range of change in the imaging angle of the imaging unit 40 capturing an image from above with respect to the target object M can be kept low.

[0045] Furthermore, the imaging unit 40 may be disposed in an upper portion of the car 21 such that the angle between the central axis of the imaging angle of view of the imaging unit 40 (the optical axis of the lens of the imaging unit 40) and the straight line L1 connecting the lens of the imaging unit 40 and the center P11 is within 20 degrees. This makes it possible to keep the angle between the imaging direction of the target object M present in the car 21 and the central axis of the imaging angle of view low. Therefore, it is possible to image the target object M present in the car 21 at a position in the captured image G where the influence of wide-angle distortion due to the lens of the imaging unit 40 is small.

[0046] Furthermore, the imaging unit 40 may be disposed in an upper portion within the car 21 such that the angle formed by the central axis of the imaging angle of view of the imaging unit 40 and a straight line L2 extending from the lens of the imaging unit 40 toward the center P21 of the boundary S1 is within 20 degrees. This makes it possible to capture an image of the target object M that exists between the inside of the car 21 and the hall 50 at a position within the captured image G that is less affected by the wide-angle distortion caused by the lens of the imaging unit 40.

[0047] (Elevator Control Device) The elevator control device 10 comprehensively controls the operation of the elevator 20. The elevator control device 10 includes a control unit 11 and a memory unit 12. The memory unit 12 stores various programs executed by the elevator control device 10 and data used by the programs. The control unit 11 comprehensively controls each unit of the elevator 20 and the hall 50. The functions of the control unit 11 may be realized by a CPU (Central Processing Unit) executing the programs stored in the memory unit 12.

[0048] The control unit 11 includes an information acquisition unit 111 , an object recognition unit 112 , a position specification unit 113 , a congestion degree calculation unit 114 , and an operation control unit 115 .

[0049] The information acquisition unit 111 acquires various pieces of information. The information acquisition unit 111 acquires at least the captured image G from the imaging unit 40, and information input to the car internal operating panel 22 and the hall operating panel 51.

[0050] (Target Recognition Unit) The target recognition unit 112 detects a predetermined target object M by image recognition from within the captured image G by the imaging unit 40. As a method for detecting the target object M in the captured image G, for example, there is a method for detecting the target object M using deep learning. In this case, the target recognition unit 112 detects the target object M from the captured image G using an object detection algorithm such as the YOLO algorithm.

[0051] The target recognition unit 112 detects an attribute (type) of the target object M. The attributes of the target object M are, for example, a normal user, a visually impaired person, a dolly, a wheelchair user, a robot, a pet, a stroller, and a shopping cart. The detection of the target object M by the target recognition unit 112 is performed for each attribute of the target object M. The target recognition unit 112 may detect only target objects M having a specific attribute, such as a wheelchair user or a robot.

[0052] The target recognition unit 112 generates a bounding box B according to the detected target object M. For example, the target recognition unit 112 generates a bounding box B so as to surround the detected target object M. Note that, hereinafter, the bounding box may be referred to as a boundary rectangle.

[0053] (Identity determination of target objects) The target recognition unit 112 detects a target object M detected in an image G(T-1) captured at a certain time T-1 in an image G(T) captured at a certain time T after the time T-1. Here, the reference numerals in parentheses indicate the time. The detection method is not particularly limited, and various methods can be used. As an example of a detection method using identity determination, an example in which a boundary rectangle with the largest overlapping area between boundary rectangles captured at different times is determined to be the boundary rectangle of the same target object M will be described below.

[0054] Fig. 5 is a diagram showing an example of a method for detecting a target object M. In Fig. 5, a boundary rectangle B1 and a boundary rectangle B2 are bounding boxes formed for different target objects M1 and M2, respectively. 5001 in Fig. 5 is a diagram explaining the boundary rectangle B1 and the boundary rectangle B2 at time points T-1 and T. 5002 in Fig. 5 is a diagram explaining the overlap of the boundary rectangle B1 and the boundary rectangle B2 at time points T-1 and T, and 5003 in Fig. 5 is a diagram explaining the concept of determining whether the boundary rectangle B(T) is the same at time point T.

[0055] The target recognition unit 112 detects different target objects M1(T-1) and M2(T-1) at time T-1, and forms a bounding rectangle B1(T-1) and a bounding rectangle B2(T-1) for each of them. The target recognition unit 112 also detects different target objects M1(T) and M2(T) at time T, and forms a bounding rectangle B1(T) and a bounding rectangle B2(T) for each of them.

[0056] Next, the target recognition unit 112 performs identity determination on the bounding rectangles B1(T) and B2(T) at time T, starting with the bounding rectangle B1(T) with the largest area. As shown in 5002 of FIG. 5, the target recognition unit 112 identifies an overlapping area J1 between the bounding rectangles B1(T) and B1(T-1), and an overlapping area J2 between the bounding rectangles B1(T) and B2(T-1).

[0057] Next, the target recognition unit 112 compares the area of ​​the overlap region J1 with the area of ​​the overlap region J2. In the case shown in 5003 of Fig. 5, the area of ​​the overlap region J1 is larger than the area of ​​the overlap region J2, and the area of ​​the overlap region J1 is the largest, so the target recognition unit 112 determines that the boundary rectangle B1(T-1) is one and the same as the boundary rectangle B1(T), and determines that the target object M1(T) and the target object M1(T-1) are one and the same. As a result, the target recognition unit 112 detects the target object M1(T) at time T following the target object M1(T-1) at time T-1.

[0058] Since it has been determined that the bounding rectangle B1(T) and the bounding rectangle B1(T-1) are the same body, the identity determination of the next bounding rectangle B2(T) is performed only on the bounding rectangle B2(T-1). Therefore, the area of ​​the region J3 (see 5002 in FIG. 5) where the bounding rectangle B2(T) and the bounding rectangle B2(T-1) overlap is maximized, and the target recognition unit 112 determines that the bounding rectangle B2(T) and the bounding rectangle B2(T-1) are the same body, and determines that the target object M2(T) and the target object M2(T-1) are the same body. As a result, the target recognition unit 112 detects the target object M2(T) at the time T following the target object M2(T-1) at the time T-1.

[0059] If the same object does not exist in the captured image G(T-1), the target recognition unit 112 may perform the same process on the captured image G(T-2). If the same object does not exist in the captured image G(T-1) and the captured image G(T-2), the target recognition unit 112 may treat the detected target object M as a new target object M.

[0060] (Location specifying part) The position identification unit 113 identifies the position of the target object M detected by the target recognition unit 112 within the captured image G. The position identification unit 113 divides the captured image G into a plurality of partial image regions, and identifies the position of the target object M within the captured image G based on the positional relationship between the partial image regions and the bounding box B generated by the target recognition unit 112. This makes it possible to identify the position of the target object M within the captured image G relatively easily by checking the positional relationship between the bounding box B and the partial image regions.

[0061] FIG. 6 is a diagram showing an example of identifying the position of a target object M in a captured image G captured by the imaging section 40 installed at the position P2.

[0062] 6, the position identification unit 113 divides the captured image G into, for example, a first partial image region R1 and a second partial image region R2. The first partial image region R1 is a region corresponding to the inside of the car 21 in the captured image G, and the second partial image region R2 is a region corresponding to the hall 50 in the captured image G. Note that the first partial image region R1 and the second partial image region R2 may be set in advance in the captured image G, or may be automatically divided by determining the car 21 and the hall 50 by image recognition or the like.

[0063] The position identifying unit 113 identifies which of the following positions (1) to (3) the target object M detected in the captured image G is located at: (1) a position (first position) where the target object M is entirely contained in the first partial image region R1, (2) a position (second position) where the target object M straddles both the first partial image region R1 and the second partial image region R2, and (3) a position (third position) where the target object M is neither entirely nor partially present in the first partial image region R1.

[0064] As shown in 6001 in Fig. 6, when the bounding box B is entirely contained in the first partial image region R1, the position identifying unit 113 identifies the target object M as being located at a first position. As shown in 6002 in Fig. 6, when the bounding box B straddles both the first partial image region R1 and the second partial image region R2, the position identifying unit 113 identifies the target object M as being located at a second position. As shown in 6003 in Fig. 6, when the bounding box B is neither entirely nor partially present in the first partial image region R1, the position identifying unit 113 identifies the target object M as being located at a third position.

[0065] Note that the position of the target object M can also be identified in the captured image G captured by the imaging unit 40 installed at position P1 in the same manner as above. When identifying the position of the target object M based on the captured image G captured by the imaging unit 40 installed at position P1, the position identifying section 113 may simply divide the captured image G into a first partial image region R1 and a second partial image region R2 (see FIG. 3) and identify the first position and the third position.

[0066] (Motion control unit) The operation control unit 115 controls the operation of the elevator 20 according to the position of the target object M identified by the position identification unit 113. This makes it possible to change the operation control of the elevator 20 according to the location of the target object M within an area including both the inside of the car 21 and the hall 50. This makes it possible to perform more appropriate operation control of the elevator 20 according to the situation of the target object M.

[0067] The operation control unit 115 recognizes whether or not the target object M is present in the cage 21, depending on the position of the target object M identified by the position identifying unit 113. Specifically, the operation control unit 115 recognizes whether the target object M is in any one of the first position to the third position, depending on the position of the target object M identified by the position identifying unit 113.

[0068] This makes it possible to change the operation not only depending on whether the target object M is present inside the car 21, but also when the target object M is present both inside the car 21 and at the landing 50. Therefore, more detailed operation control according to the position of the target object M can be realized.

[0069] By performing the above recognition using the captured image G by the imaging unit 40 installed at position P2, it becomes possible to more accurately determine whether the target object M is present in the boundary area between the car 21 and the hall 50 or elsewhere. By having the imaging unit 40 at position P2, when the target object M exists straddling both the inside of the car 21 and the hall 50, and when the target object M exists near the boundary S1 between the inside of the car 21 and the hall 50, it is possible to keep low the range of change in the imaging angle of the target object M from the imaging unit 40 capturing an image from above. This is because it is possible to reduce the change in the shape of the target object M in the captured image G due to the difference in the imaging angle, and it is possible to improve the recognition accuracy of the target object M near the second position.

[0070] Moreover, by performing the above recognition using the captured image G by the imaging unit 40 installed at position P1, it becomes possible to more accurately determine whether or not the target object M is present in the car 21. By having the imaging unit 40 at position P1, it is possible to keep the range of change in the imaging angle of the imaging unit 40 with respect to the target object M low regardless of where the target object M is located in the car 21. This is because it is possible to reduce changes in the shape of the target object M in the captured image G due to differences in the imaging angle, and it is possible to improve the recognition accuracy of the target object M.

[0071] In the following description, recognition corresponding to the first position may be referred to as recognition I, recognition corresponding to the second position as recognition II, and recognition corresponding to the third position as recognition III.

[0072] The operation control unit 115 controls at least one of the following (A) to (C): (A) the opening and closing operation of the door 24, (B) the display operation or the audio output operation for at least either one of the inside of the car 21 or the landing 50, and (C) the enable / disable switching operation of the operation button for at least either one of the inside of the car 21 or the landing 50.

[0073] This makes it possible to change the opening and closing operation of the door 24, provide information to the user, and enable / disable the operation buttons depending on the detection results of the target object M, thereby enabling the elevator to operate more appropriately according to the user's condition.

[0074] For example, when the door 24 is open and the recognition I is not recognized for a predetermined time or more, that is, when no one gets on the car 21 for a predetermined time or more, the operation control unit 115 closes the door 24. This reduces unnecessary open states of the door 24, allowing the elevator 20 to operate efficiently.

[0075] Furthermore, the operation control unit 115 may control the operation of the elevator 20 based on the attributes of the target object M in addition to recognition I to recognition III. For example, when recognition III is recognized for the target object M whose attribute is a wheelchair user, the operation control unit 115 may extend the open state of the door 24. When recognition I is recognized for the target object M whose attribute is a wheelchair user, the operation control unit 115 may register the destination floor registered by the in-car operation panel 22 at the floor where the wheelchair user boarded as a wheelchair call.

[0076] Furthermore, when the recognition I is recognized for the target object M, the operation control unit 115 may display a display indicating the attribute of the target object M on the notification unit 52 of each hall 50 of the elevator 20 in which the target object M has boarded. For example, when the recognition I is recognized for the target object M whose attribute is a robot, the operation control unit 115 may display a display indicating "ROBOT ON BOARD" on the notification unit 52 of each hall 50 of the elevator 20 in which the robot has boarded. The same applies when the attribute of the target object M is a wheelchair user or a pet.

[0077] When it is determined that the target object M exists both inside the car 21 and at the landing 50, the operation control unit 115 may determine whether the target object M is moving from the landing 50 to inside the car 21, or from inside the car 21 to the landing 50, based on the determination result of the location of the target object M just before. This makes it possible to perform appropriate elevator operation control depending on whether the target object M is moving from the landing 50 to inside the car 21, or from inside the car 21 to the landing 50.

[0078] For example, when a target object M is recognized as recognition II, if the target object M in the captured image G immediately before the captured image G on which the recognition is based is recognized as recognition III, the operation control unit 115 recognizes that the target object M is "moving from the platform 50 into the car 21."

[0079] When the target object M is recognized as recognition II, if the target object M in the captured image G immediately before the captured image G on which the recognition is based is recognized as recognition I, the operation control unit 115 recognizes that the target object M is "moving from inside the cage 21 to the platform 50."

[0080] When the operation control unit 115 recognizes that the target object M is "moving from the landing 50 into the car 21", the operation control unit 115 may extend the open state of the door 24. When the operation control unit 115 continues to recognize that the target object M is "moving from the landing 50 into the car 21" for a predetermined period of time or more, the operation control unit 115 may cause the notification unit 23 to issue a voice or the like to call attention to "please hurry up and board the car".

[0081] Furthermore, if the in-car operation panel 22 has a non-contact call button, the operation control unit 115 may perform the following operation control of the elevator 20. When the operation control unit 115 recognizes that a target object M whose attribute is a visually impaired person is "moving from the hall 50 to inside the car 21", it may disable the non-contact call button, and when the operation control unit 115 recognizes that the target object M is "moving from inside the car 21 to the hall 50", it may enable the non-contact call button. This makes it possible to prevent erroneous operation of the non-contact call button by visually impaired users.

[0082] When a user issues a call instruction for the elevator 20 before boarding, the operation control unit 115 may control the operation of the elevator 20 according to the relationship between the content of the call instruction and the position of the target object M identified by the position identification unit 113. Here, the "call instruction" includes, for example, a "hall call" in which a call is registered using up and down buttons installed at the hall 50, and a "destination floor registration (provisional call registration)" in which a call is registered in advance using a smartphone or an operation panel installed at the hall 50. This makes it possible to operate the elevator 20 in a manner appropriate to the situation according to the relationship between the content of the call instruction and the position of the target object M.

[0083] For example, when the car 21 stops at the hall 50 in response to a hall call and the door 24 is in an open state and recognizes recognition III for a predetermined period of time or more, the operation control unit 115 may perform the following operation control.

[0084] That is, in the above case, the operation control unit 115 may issue a warning including the destination direction to the landing 50 by the notification unit 52. Specifically, when the car 21 stops at the landing 50 where an upward landing call has been made by the landing operating panel 51, and the door 24 is in an open state and recognizes recognition III for a predetermined period of time or more, the operation control unit 115 may issue a voice notification in the notification unit 52 saying, "We are going up. Please be careful."

[0085] In addition, in the above case, if the direction of the hall call and the direction of movement of the elevator 20 are the same, the operation control unit 115 may issue an alert to encourage the passenger to board the elevator, and if the direction of the hall call and the direction of movement of the elevator 20 are different, an alert may be issued to warn the passenger not to board the elevator by mistake.

[0086] When the car 21 stops at the hall 50 in response to a hall call and the door 24 is in an open state and does not recognize recognition III for a predetermined period of time or more, the operation control unit 115 may close the door 24.

[0087] When the car 21 stops at the boarding platform 50 of the boarding floor for which a destination floor registration call was made and it is recognized that the target object M is "moving from the boarding platform 50 into the car 21", the operation control unit 115 may officially register the destination floor registration call. This makes it possible to perform control such that at the stage when a destination floor registration call is made by the boarding platform operating panel 51, the input destination floor is provisionally registered, and when it is confirmed that a user has actually boarded the car 21 from the boarding platform 50, the input destination floor is officially registered. When the destination floor registration call is officially registered, the operation control unit 115 operates the car 21 to the destination floor registered by the call registered by the destination floor registration call.

[0088] When the car 21 stops at the boarding platform 50 of the boarding floor for a destination floor registration call and the target object M is not recognized as "moving from the boarding platform 50 into the car 21," the operation control unit 115 does not need to register the destination floor registration call as an official registration.

[0089] (Congestion degree calculation section) The congestion degree calculation unit 114 calculates the congestion degree inside the car 21 based on the types and number of each type of target objects M detected by the target recognition unit 112, and the positions identified by the position identification unit 113. As a result, the congestion degree inside the car 21 is calculated taking into account the types and number of each type of target objects M, so that it is possible to calculate a congestion degree that is more suited to the actual situation than when the congestion degree is calculated simply based on the number of passengers or weight. Note that when calculating the congestion degree, it is desirable for the target recognition unit 112 to detect target objects M with multiple types of attributes, rather than detecting only target objects M with a specific attribute.

[0090] The congestion degree calculation unit 114 notifies at least one of the notification unit 23 and the notification unit 52 of a warning such as a full vehicle being present depending on the congestion degree.

[0091] The congestion degree is calculated, for example, by the occupancy area ratio of the target object M to the floor area of ​​the car 21. The congestion degree calculation unit 114 calculates the occupancy area according to the attributes of the target object M, and calculates the occupancy area ratio of the target object M to the floor area of ​​the car 21. When the attributes of the target object M are a cart or a wheelchair, the congestion degree calculation unit 114 calculates the occupancy area to be larger than that of a normal user, and calculates the congestion degree.

[0092] For example, the congestion degree calculation unit 114 calculates the congestion degree based on the determination by the operation control unit 115 for the target object M located at the second position of "moving from the hall 50 to inside the car 21" and "moving from inside the car 21 to the hall 50." Specifically, the congestion degree calculation unit 114 may calculate the congestion degree by (1) adding the occupied area of ​​the target object M located at the second position and determined to be "moving from the hall 50 to inside the car 21" and (2) subtracting the occupied area of ​​the target object M located at the second position and determined to be "moving from inside the car 21 to the hall 50."

[0093] This makes it possible to notify the passenger of a full vehicle condition earlier than calculating the degree of congestion based on the type and number of each type of target object M identified as being located at the first location, making it possible to alert the passenger even if the target object M is in the middle of boarding, for example.

[0094] (Processing operation of elevator control device) 7 is a flow diagram showing an example of the operation of the elevator system 1. First, the imaging unit 40 captures an image of the inside of the car 21 of the elevator 20 and the hall 50 with the door 24 of the elevator 20 in an open state (step S1, imaging step). Next, the target recognition unit 112 detects a predetermined target object M from the captured image G captured by the imaging unit 40 by image recognition (step S2, target recognition step).

[0095] The position specifying unit 113 generates a bounding box B according to the detected target object M (step S3), and divides the captured image G into a first partial image region R1 and a second partial image region R2 (step S4).

[0096] Furthermore, the position identification unit 113 determines whether or not the bounding box B is included in the first partial image region R1 (step S5). If it is determined that the bounding box B is included in the first partial image region R1 (YES in step S5), the position identification unit 113 determines whether or not the bounding box B is entirely included in the first partial image region R1 (step S6).

[0097] If it is determined that the bounding box B is entirely contained within the first partial image region R1 (YES in step S6), the position identifying unit 113 identifies the target object M as entirely contained within the car 21 (step S7, position identifying step). Then, the operation control unit 115 recognizes that the target object M is entirely contained within the car 21 (step S8).

[0098] If it is determined that the bounding box B is not within the first partial image region R1 (NO in step S5), the position identifying unit 113 identifies that the target object M does not exist entirely or partially in the first partial image region R1 (step S11, position identifying step).Then, the operation control unit 115 recognizes that the target object M does not exist entirely or partially in the first partial image region R1 (step S12).

[0099] If it is determined that the bounding box B is included in the first partial image region R1 but is not entirely included in the first partial image region R1 (NO in step S6), the position identification unit 113 identifies the target object M as existing across both the first partial image region R1 and the second partial image region R2 (step S9, position identification step).Then, the operation control unit 115 recognizes that the target object M exists across both the first partial image region R1 and the second partial image region R2 (step S10).

[0100] When the operation control unit 115 recognizes the target object M based on the position determination of the target object M by the position determination unit 113 in steps S8, S10, and S12, the operation control unit 115 controls the operation of the elevator 20 according to the position of the target object M (step S13, operation control step).

[0101] [Modifications] Modifications of the present invention will be described below. For convenience of explanation, the same reference numerals are attached to components having the same functions as those described in the above embodiment, and the description thereof will not be repeated. In the above embodiment, the position specifying unit 113 specifies the position of the target object M in the captured image G by checking the positional relationship between the bounding box B and the partial image area, but the present invention is not limited to the above. As shown in FIG. 8, the position specifying unit 113 may specify the position of the target object M in the captured image G by the following example. FIG. 8 is a diagram showing a modification of specifying the position of the target object M. The captured image G in FIG. 8 is obtained by the imaging unit 40 installed at the position P2.

[0102] (specified by number of vertices) 8001 in Fig. 8 is a diagram illustrating another example of identifying the position of the target object M. As shown in 8001 in Fig. 8, the position identifying unit 113 may identify the position of the target object M based on the number of vertices V of a bounding box B included in the first partial image region R1. In this case, the position identifying unit 113 may calculate the vertices V of the bounding box B, and identify the position of the target object M based on the number of vertices V included in the first partial image region R1.

[0103] For example, when four vertices V are included in the first partial image region R1, the position identifying unit 113 identifies the target object M as being at a first position, and when one to three vertices V are included in the first partial image region R1, the position identifying unit 113 identifies the target object M as being at a second position. When none of the vertices V are included in the first partial image region R1, the position identifying unit 113 identifies the target object M as being at a third position.

[0104] (Specified by area) 8002 in Fig. 8 is a diagram illustrating yet another example of identifying the position of the target object M. As shown in 8002 in Fig. 8, the position identifying unit 113 may identify the position of the target object M based on the area E2 of the bounding box B included in the first partial image region R1. In this case, the position identifying unit 113 may calculate the area E1 of the entire bounding box B and the area E2 of the area where the bounding box B and the first partial image region R1 overlap, and identify the position of the target object M by comparing the areas E1 and E2.

[0105] For example, the position identifying unit 113 may identify the target object M as being at the first position when the ratio of the area E2 to the area E1 is greater than a predetermined value, and may identify the target object M as being at the third position when the ratio of the area E2 to the area E1 is equal to or smaller than a predetermined value. The predetermined value may be, for example, 60%.

[0106] Furthermore, the position identifying unit 113 may set two predetermined values, a first predetermined value and a second predetermined value, and identify the position as follows. For example, the position identifying unit 113 may identify the target object M as being at the first position when the ratio of the area E2 to the area E1 is greater than the first predetermined value. The position identifying unit 113 may identify the target object M as being at the second position when the ratio of the area E2 to the area E1 is greater than the second predetermined value and is equal to or less than the first predetermined value. The position identifying unit 113 may identify the target object M as being at the third position when the ratio of the area E2 to the area E1 is equal to or less than the second predetermined value. Here, the first predetermined value is a value greater than the second predetermined value.

[0107] (Determined by the center point position) 8, the position identifying unit 113 may identify the position of the target object M based on the position of the center point C of a bounding box B in the captured image G. In this case, the position identifying unit 113 calculates the center point C of the bounding box B, and determines whether the center point C is included in the first partial image region R1.

[0108] For example, if the center point C is included in the first partial image area R1, the position identification unit 113 identifies the target object M as being at the first position, and if the center point C is not included in the first partial image area R1, the position identification unit 113 identifies the target object M as being at the third position.

[0109] [Software implementation example] The functions of the elevator control device 10 (hereinafter referred to as the "device") can be realized by a program for causing one or more computers to function as the device, and a program for causing a computer to function as each control block of the device (particularly each part included in the control unit 11).

[0110] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program to realize each function described in each of the above embodiments.

[0111] The program may be non-transitory and may be recorded in one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be provided to the device via any wired or wireless transmission medium.

[0112] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of each of the control blocks can be realized by, for example, a quantum computer.

[0113] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may be operated by the control device or may be operated by another device (for example, an edge computer or a cloud server).

[0114] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in the embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0115] 1. Elevator System 20 Elevator 24 Doors 40 Imaging unit Platform 50 112 Object Recognition Unit 113 Location identification part 114 Congestion degree calculation unit 115 Motion control section 211 Floor B Bounding Box G. Captured image L1, L2 straight line M, M1, M2 Target object N1, N2 normals P11 Center of the cage floor P21 Center of the boundary between the car and the platform R1 1st partial image area (partial image area) R2 Second partial image area (partial image area) S1 boundary θ1, θ2 angle

Claims

1. an imaging unit that images an interior of an elevator car and an elevator hall when the elevator door is in an open state; an object recognition unit that detects a predetermined object from an image captured by the imaging unit through image recognition; a position identification unit that identifies a position of the target object detected by the target recognition unit within the captured image; and an operation control unit that controls the operation of the elevator according to the position of the target object identified by the position identification unit.

2. The target recognition unit generates a bounding box according to the detected target object, 2. The elevator system according to claim 1, wherein the position identification unit divides the captured image into a plurality of partial image regions, and identifies the position of the target object in the captured image based on a positional relationship between the partial image regions and the bounding box generated by the target recognition unit.

3. the imaging unit is installed in an upper portion inside the car, and an angle formed by a straight line connecting a lens of the imaging unit and a center of a floor surface of the car and a normal line of the floor surface of the car at the center of the floor surface of the car is within 20 degrees, The elevator system according to claim 1 , wherein the operation control unit recognizes whether or not the target object is present in the car, depending on the position of the target object identified by the position identifying unit.

4. 2. The elevator system according to claim 1, wherein the operation control unit recognizes, depending on the position of the target object identified by the position identification unit, whether the target object is (1) entirely contained within the car, (2) present across both the car and the landing, or (3) not entirely or partially present within the car.

5. 5. The elevator system according to claim 4, wherein the imaging unit is installed in an upper portion inside the car, and an angle formed by a straight line from a lens of the imaging unit to a center of a boundary between the inside of the car and the landing and a normal to a floor surface of the car at the center of the boundary is within 20 degrees.

6. 5. The elevator system according to claim 4, wherein when it is determined that the target object is present both inside the car and at the landing, the operation control unit determines whether the target object is moving from the landing to inside the car, or from inside the car to the landing, based on a result of determining the previous location of the target object.

7. 2. The elevator system according to claim 1, wherein the operation control unit controls at least one of: (1) opening and closing operations of the doors; (2) display operations or audio output operations for at least one of the interior of the car and the landing; and (3) enabling / disabling switching operations of operation buttons for at least one of the interior of the car and the landing.

8. 2. The elevator system according to claim 1, wherein when a user gives a call instruction for the elevator before boarding the elevator, the operation control unit controls the operation of the elevator in accordance with a relationship between the content of the call instruction and the position of the target object identified by the position identification unit.

9. 2. The elevator system according to claim 1, further comprising a congestion degree calculation unit that calculates a congestion degree inside the car based on the types of the target objects detected by the target recognition unit, the number of each type, and the positions identified by the position identification unit.

10. 1. A method for controlling an elevator executed by one or more computers, comprising: an imaging step of imaging an interior of an elevator car and a landing when the elevator door is in an open state; a target recognition step of detecting a predetermined target object from the captured image by image recognition; a position specifying step of specifying a position of the target object detected by the target recognition step within the captured image; and an operation control step of controlling the operation of the elevator in accordance with the position of the target object identified by the position identification step.

Images