Information processing device and identification method

Abstract

An information processing device (100) comprises: an acquisition unit (120) that acquires a video including people near a door; a detection unit (130) that detects a flow of people on the basis of the video, detects the position of the door on the basis of the position of the flow of people, detects a range where people appeared as the width of the door on the basis of the video, detects the lower side of the door on the basis of the position of the door and one image among a plurality of images constituting the video, and detects the walking stride of people using one image among the plurality of images constituting the video; and a specification unit (140) that specifies a door-closing determination area, which is a region for determining whether or not to close the door, on the basis of the width of the door and the walking stride with the lower side of the door as a reference.

Description

Information processing device and identification method

[0001] The present disclosure relates to an information processing device and a specifying method.

[0002] Doors are present in public transportation facilities, buildings, and the like. A technology for controlling doors has been proposed (see Patent Document 1). In Patent Document 1, a first detection area and a second detection area are set. The control system of Patent Document 1 stops the door or reverses it when it detects a passenger's movement from the first detection area to the second detection area while the door is closing.

[0003] JP 2018-197140 A

[0004] As described above, in the above technology, first and second detection areas are set. Here, areas for determining door control, such as the first and second detection areas, are called door close determination areas. The door close determination area is specified by the user. The user is responsible for specifying the door close determination area. This places a heavy burden on the user.

[0005] The purpose of the present disclosure is to reduce the burden on users.

[0006] According to one aspect of the present disclosure, there is provided an information processing device, the information processing device including: an acquisition unit that acquires a video including a person near a door; a detection unit that detects a flow of people based on the video, detects the position of the door based on a position where the flow of people occurs, detects a range where people will occur based on the video as a width of the door, detects a bottom edge of the door based on one image among a plurality of images constituting the video and the position of the door, and detects a stride of the person using one image among the plurality of images constituting the video; and an identification unit that identifies a door close determination area that is an area for determining whether to close the door based on the width of the door and the stride, using the bottom edge of the door as a reference.

[0007] According to the present disclosure, the burden on the user can be reduced.

[0008] FIG. 1 is a diagram illustrating a control system. FIG. 2 is a diagram illustrating hardware included in an information processing device. FIG. 3 is a block diagram illustrating functions of an information processing device. FIG. 4 is a diagram illustrating a specific example of a process for detecting a door position. FIG. 5 is a diagram illustrating a specific example of a process for detecting a door width. FIG. 6 is a diagram illustrating a specific example of a process for detecting a bottom edge of a door and a stride length. FIG. 7 is a diagram illustrating a specific example of a process for specifying a door closing determination area. FIG. 8 is a flowchart illustrating an example of a process executed by an information processing device. FIG. 9 is a flowchart illustrating an example of a door closing determination process.

[0009] Hereinafter, embodiments will be described with reference to the drawings. The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.

[0010] 1 is a diagram showing a control system. The control system includes an information processing device 100, a camera 200, a door driving device 300, and a door 400. The information processing device 100, the camera 200, the door driving device 300, and the door 400 are connected via a network. The network may be a wired network or a wireless network.

[0011] The information processing device 100 is a device that executes the identification method. The information processing device 100 can acquire an image from a camera 200. The door driving device 300 drives a door 400. For example, the door 400 is a door on a train, a platform door at a station, or a door in a building. In the following description, the door 400 is assumed to be a door on a train.

[0012] Next, a description will be given of the hardware included in the information processing device 100. Fig. 2 is a diagram showing the hardware included in the information processing device 100. The information processing device 100 includes a processor 101, a volatile storage device 102, a non-volatile storage device 103, and a communication interface 104.

[0013] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.

[0014] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a RAM (Random Access Memory). The non-volatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the non-volatile storage device 103 is a HDD (Hard Disk Drive) or an SSD (Solid State Drive). The communication interface 104 communicates with the camera 200 and the door driver 300.

[0015] Next, a description will be given of the functions of the information processing device 100. Fig. 3 is a block diagram showing the functions of the information processing device. The information processing device 100 has a storage unit 110, an acquisition unit 120, a detection unit 130, an identification unit 140, and a determination unit 150.

[0016] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103. Some or all of the acquisition unit 120, detection unit 130, identification unit 140, and determination unit 150 may be realized by a processing circuit. Furthermore, some or all of the acquisition unit 120, detection unit 130, identification unit 140, and determination unit 150 may be realized as program modules executed by the processor 101.

[0017] The storage unit 110 stores various information. The acquisition unit 120 acquires video. For example, the acquisition unit 120 acquires video from the camera 200. The acquisition unit 120 may acquire video via another device. The video includes a person near the door 400.

[0018] The detection unit 130 detects people flow based on the video. The detection unit 130 may detect people flow based on motion vectors such as optical flow. Alternatively, the detection unit 130 may detect people flow using image processing techniques such as background subtraction.

[0019] The detection unit 130 detects the position of the door 400 based on the position where a flow of people occurs. The detection unit 130 may detect the position of the door 400 (hereinafter referred to as the door position) based on the position where a flow of people periodically occurs. The detection process will be described using a specific example.

[0020] FIG. 4 is a diagram showing a specific example of the door position detection process. FIG. 4 shows an image 10. Image 10 is one of multiple images that make up a video. Image 10 shows a train 11, a person 12, and guiding blocks 13 for visually impaired people (i.e., yellow Braille blocks). For ease of explanation, FIG. 4 also shows a flow of people. The detection unit 130 detects a door position 14 based on the position where the flow of people occurs.

[0021] The detection unit 130 detects the range of human appearance as the width of the door 400 based on the video. The detection process will be described in detail. The detection unit 130 detects humans based on the video. More specifically, the detection unit 130 detects humans using image recognition technology. For example, the detection unit 130 detects humans using the video and a trained model. Note that when a trained model is used, the acquisition unit 120 acquires the trained model from the storage unit 110 or an external device. The external device is a device that exists outside the information processing device 100. For example, the external device is a cloud server, an external memory, or the like. The illustration of the external device is omitted. The detection unit 130 can detect the range of human appearance by detecting humans. Then, the detection unit 130 detects the range of human appearance as the width of the door 400 (hereinafter, "door width"). The detection process will be described using a specific example.

[0022] FIG. 5 is a diagram showing a specific example of the door width detection process. FIG. 5 shows an image 20. The image 20 is one of multiple images that make up a video. The detection unit 130 detects a person using the image 20 and a trained model. In this way, the person is detected. For example, the detection of the person is represented by bounding boxes 21 and 22.

[0023] The detection unit 130 can detect the range in which people appear by repeatedly detecting people using video (i.e., multiple images). For example, in Fig. 4, the range in which people appear is indicated by a double-headed arrow 23. The range indicated by the double-headed arrow 23 is the width of the door.

[0024] The detection unit 130 detects the bottom edge of the door 400 based on one of the multiple images constituting the video and the position of the door 400. The detection unit 130 also detects the person's stride using one of the multiple images constituting the video. This image may be the same as the image used to detect the bottom edge of the door 400. In the following description, it is assumed that this image is the same as the image used to detect the bottom edge of the door 400. The process of detecting the bottom edge of the door 400 and the stride will be described using a specific example.

[0025] Fig. 6 is a diagram showing a specific example of the process of detecting the bottom edge of a door and the stride length. Fig. 6 shows an image 30. Image 30 is one of multiple images that make up a video. Image 30 includes people 31 and 32. For ease of explanation, Fig. 6 also shows a door position 33.

[0026] The detection unit 130 detects the bottom edge of the door 400 based on the image 30 and the door position 33. Specifically, the detection unit 130 detects the point where the door position 33 intersects with a line 34 extending from the feet of the person 31 in the direction of the door position 33 as the bottom edge 35 of the door 400.

[0027] The detection unit 130 detects the stride of the person 32 based on the image 30. The stride detection process will be described. For example, the detection unit 130 extracts skeletal information of the person 32 using the image 30 and a trained model. Specifically, the detection unit 130 extracts the skeletal information of the person 32 using OpenPose. The detection unit 130 detects the stride 36 of the person 32 based on the skeletal information of the person 32.

[0028] The detection unit 130 may detect multiple strides based on one or more images constituting the video, and detect the average value of the multiple strides or a representative value of the multiple strides as the person's stride. By using the average value or representative value as the person's stride, the accuracy of the person's stride is improved.

[0029] The identification unit 140 identifies a door close determination area based on the width of the door 400 and the stride length, using the bottom edge of the door 400 as a reference. The door close determination area is an area for determining whether or not to close the door 400. In detail, the identification unit 140 identifies the door close determination area from within the image. The identification process will be described using a specific example.

[0030] FIG. 7 is a diagram showing a specific example of the process for identifying a door-closed determination area. FIG. 7 shows an image 40. Image 40 is one of multiple images that make up a video. FIG. 7 shows a bottom edge 41 of a door 400, a width 42 of the door 400, and a stride 43. The identification unit 140 identifies a door-closed determination area 44 based on the width 42 and stride 43 of the door 400, using the bottom edge 41 of the door 400 as a reference. The door-closed determination area 44 can be expressed by equation (1).

[0031] Door closing judgment area = door width × stride length (1)

[0032] The identification unit 140 may identify the door-closed determination region as follows: When identifying the door-closed determination region, the identification unit 140 adds a constant to at least one of the width of the door 400 and the stride length to identify the door-closed determination region. Specifically, the door-closed determination region can be expressed by any of equations (2) to (4).

[0033] Door closing determination area = (constant × door width) × stride length (2)

[0034] Door closing determination area=door width×(constant×step length) (3)

[0035] Door closing determination area=(constant×door width)×(constant×step length) (4)

[0036] For example, the constant is 2. In this way, by adding a constant, the information processing device 100 can have a margin of error in making a decision when closing the door 400.

[0037] Next, the processing executed by the information processing device 100 will be described using a flowchart. FIG. 8 is a flowchart showing an example of the processing executed by the information processing device. (Step S11) The acquisition unit 120 acquires video. (Step S12) The detection unit 130 detects people flow based on the video. (Step S13) The detection unit 130 detects the position of the door 400 based on the position where people flow occurs. (Step S14) The detection unit 130 detects the range of people generation as the width of the door 400 based on the video.

[0038] (Step S15) The detection unit 130 detects the bottom edge of the door 400 based on one of the multiple images that make up the video and the position of the door 400. (Step S16) The detection unit 130 uses the image to detect the person's stride. (Step S17) The identification unit 140 identifies a door closing determination area based on the width of the door 400 and the stride, using the bottom edge of the door 400 as a reference.

[0039] Next, the processing after the door close determination area is specified will be described. FIG. 9 is a flowchart showing an example of the door close determination processing. (Step S21) The acquisition unit 120 acquires an image from the camera 200. (Step S22) The determination unit 150 sets a door close determination area in the image. (Step S23) The determination unit 150 determines whether or not a person is present in the door close determination area. If a person is present in the door close determination area, the processing proceeds to step S21. If a person is not present in the door close determination area, the processing proceeds to step S24.

[0040] (Step S24) The determination unit 150 determines whether a predetermined time has elapsed since a person was not present in the door close determination area. If the predetermined time has elapsed, the process proceeds to step S25. If the predetermined time has not elapsed, the process proceeds to step S21. (Step S25) The determination unit 150 instructs the door driving device 300 to close the door 400. As a result, the door driving device 300 closes the door 400.

[0041] According to the embodiment, the information processing device 100 automatically identifies the door close determination area. Therefore, the user does not need to perform the work of identifying the door close determination area. Therefore, the information processing device 100 can reduce the burden on the user.

[0042] In the above description, the case where the door close determination area is determined using the information processing device 100 that controls the door driving device 300 has been described. The device that determines the door close determination area does not have to be the device that controls the door driving device 300. For example, a cloud server may determine the door close determination area. In this case, the information processing device 100 is the cloud server. After determining the door close determination area, the cloud server transmits information about the door close determination area to the device that controls the door driving device 300. This allows the device that controls the door driving device 300 to implement the processing shown in FIG. 9.

[0043] 10 Image, 11 Train, 12 Person, 13 Guidance block for visually impaired persons, 14 Door position, 20 Image, 21, 22 Bounding box, 23 Double arrow, 30 Image, 31, 32 Person, 33 Door position, 34 Line, 35 Bottom edge, 36 Stride, 40 Image, 41 Bottom edge, 42 Width, 43 Stride, 44 Door closing determination area, 100 Information processing device, 101 Processor, 102 Volatile storage device, 103 Non-volatile storage device, 104 Communication interface, 110 Storage unit, 120 Acquisition unit, 130 Detection unit, 140 Identification unit, 150 Determination unit, 200 Camera, 300 Door drive device, 400 Door.

Claims

1. An acquisition unit that acquires an image including a person near a door; a detection unit that detects a flow of people based on the image, detects the position of the door based on the position where the flow of people occurs, detects the occurrence range of people as the width of the door based on the image, detects the lower side portion of the door based on one image among a plurality of images constituting the image and the position of the door, and detects the stride of a person using one image among the plurality of images constituting the image; and a specifying unit that specifies a door closing determination region which is a region for determining whether to close the door based on the width of the door and the stride with reference to the lower side portion of the door. An information processing apparatus having the above components.

2. The information processing apparatus according to claim 1, wherein the door is a door of a train, a platform door of a station, or a door existing in a building.

3. The information processing apparatus according to claim 1 or 2, wherein the detection unit detects a plurality of strides based on one or more images constituting the image, and detects an average value or a representative value of the plurality of strides as the stride.

4. The information processing apparatus according to any one of claims 1 to 3, wherein when specifying the door closing determination region, the specifying unit adds a constant to at least one of the width of the door and the stride to specify the door closing determination region.

5. A specifying method, in which an information processing apparatus acquires an image including a person near a door, detects a flow of people based on the image, detects the position of the door based on the position where the flow of people occurs, detects the occurrence range of people as the width of the door based on the image, detects the lower side portion of the door based on one image among a plurality of images constituting the image and the position of the door, detects the stride of a person using one image among the plurality of images constituting the image, and specifies a door closing determination region which is a region for determining whether to close the door based on the width of the door and the stride with reference to the lower side portion of the door.

Images