Control device, learning device, control method, learning method, control program, and learning program

Abstract

To appropriately perform door control.SOLUTION: A control device 100 controls a door drive device 400 for controlling doors of a train or platform doors of a station. The control device 100 includes an acquisition part 120 for acquiring a video obtained by photographing a platform, a movement vector calculation part 130 for calculating a movement vector of a crowd included in the video on the basis of the video, a closed score calculation part 140 for calculating a closed score being a numeric value by using the movement vector, and a determination instruction part 170 for determining whether to close the doors or the platform doors by using the closed score and a threshold, and instructing the door control device 400 so as to close the doors or the platform doors in the case where the closed score is larger than the threshold.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a control device, a learning device, a control method, a learning method, a control program, and a learning program.

Background Art

[0002] A camera is installed on the platform of a station. For example, a technique using an image obtained by photographing with a camera has been proposed (see Patent Document 1). In addition, a door exists on a train. A technique for detecting the timing to close the door has been proposed (see Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, a method of analyzing a person near the door and determining whether to close the door based on the analysis result can be considered. In the analysis, identification and tracking of a person are performed. However, it is difficult to use the determination method during congestion. Therefore, a method for appropriately controlling the door is desired.

[0005] An object of the present disclosure is to appropriately control a door.

Means for Solving the Problems

[0006] A control device according to one aspect of the present disclosure is provided. The control device controls a door drive device that controls a door of a train or a platform door of a station. The control device includes an acquisition unit that acquires an image obtained by photographing a platform, a movement vector calculation unit that calculates a movement vector of a crowd included in the image based on the image, a closed score calculation unit that calculates a closed score, which is a numerical value, using the movement vector, and a determination instruction unit that determines whether to close the door or the platform door using the closed score and a threshold value, and instructs the door drive device to close the door or the platform door when the closed score is greater than the threshold value.

Advantages of the Invention

[0007] According to the present disclosure, the door can be appropriately controlled.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described with reference to the drawings. The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1. <Utilization Phase> FIG. 1 is a block diagram showing the functions of the control device according to Embodiment 1. The control device 100 communicates with the camera 200, the railway operation system 300, and the door drive device 400 via a network.

[0011] The control device 100 is a device that executes a control method. The control device 100 controls the door drive device 400. The camera 200 photographs the platform. The railway operation system 300 outputs operation management information. The operation management information will be described later. The door drive device 400 controls the doors of the train or the platform doors of the station. Note that the train also means a monorail or the like.

[0012] Before explaining the functions of the control device 100, the hardware of the control device 100 will be explained. FIG. 2 is a diagram showing the hardware of the control device according to Embodiment 1. The control device 100 is a computer. The control device 100 includes a processor 101, a volatile memory device 102, a non-volatile memory device 103, an input / output interface 104, and a communication interface 105.

[0013] The processor 101 controls the entire control device 100. For example, the processor 101 is a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), or the like. The processor 101 may be a multi-processor. Further, the control device 100 may include a processing circuit.

[0014] The volatile memory device 102 is the main memory device of the control device 100. For example, the volatile memory device 102 is a RAM (Random Access Memory). The non-volatile memory device 103 is the auxiliary storage device of the control device 100. For example, the non-volatile memory device 103 is an HDD (Hard Disk Drive) or an SSD (Solid State Drive). For example, the input / output interface 104 receives information input by the user. The communication interface 105 communicates with the camera 200, the railway operation system 300, and the door drive device 400.

[0015] Returning to FIG. 1, the functions of the control device 100 will be described. The control device 100 includes a storage unit 110, an acquisition unit 120, a motion vector calculation unit 130, a closed score calculation unit 140, a conversion unit 150, a threshold determination unit 160, and a determination instruction unit 170.

[0016] The storage unit 110 may be realized as a storage area secured in the volatile memory device 102 or the non-volatile memory device 103. Part or all of the acquisition unit 120, the motion vector calculation unit 130, the closed score calculation unit 140, the conversion unit 150, the threshold determination unit 160, and the determination instruction unit 170 may be realized by a processing circuit. Also, part or all of the acquisition unit 120, the motion vector calculation unit 130, the closed score calculation unit 140, the conversion unit 150, the threshold determination unit 160, and the determination instruction unit 170 may be realized as modules of a program executed by the processor 101. For example, the program executed by the processor 101 is also referred to as a control program. For example, the control program is recorded on a recording medium.

[0017] The storage unit 110 stores various information. The storage unit 110 may store a learned model for calculating a closed score, which will be described later.

[0018] The acquisition unit 120 acquires the video obtained by photographing the platform. For example, the acquisition unit 120 acquires the video from the camera 200. The acquisition unit 120 acquires a learned model. For example, the acquisition unit 120 acquires the learned model from the storage unit 110. Also, for example, the acquisition unit 120 acquires the learned model from an external device. Note that the external device is a device existing outside the control device 100. For example, the external device is a cloud server, an external memory, etc. The diagram of the external device is omitted.

[0019] The acquisition unit 120 acquires operation management information from the railway operation system 300. The operation management information is information related to operation. For example, the operation management information is operation status information such as train schedule information, railway signal information, delays, and operation suspensions. The acquisition unit 120 acquires station-related information from the storage unit 110 or an external device. The station-related information is information related to the station. The details of the station-related information will be described later.

[0020] The motion vector calculation unit 130 calculates the motion vectors of the crowd included in the video based on the video. That is, the motion vector calculation unit 130 does not perform tracking of each person included in the video, but calculates the motion vectors of the crowd. For example, when calculating the motion vectors, the motion vector calculation unit 130 uses image processing techniques such as the gradient method and the block matching method, and machine learning models such as FlowNet to calculate the motion vectors.

[0021] The closed score calculation unit 140 calculates a closed score using the motion vectors. For example, the closed score calculation unit 140 calculates the closed score using the learned model and the motion vectors. Specifically, when the closed score calculation unit 140 inputs the motion vectors into the learned model, the learned model outputs the closed score. Here, the closed score is a numerical value used when determining whether to close the train door or the platform door. The larger the closed score, the more it means that the door or the platform door can be closed. The range of the closed score may be in the range of 0 or more and 100 or less, such as [0, 100]. Also, the lower limit value or the upper limit value of the closed score may not be set.

[0022] The conversion unit 150 converts the operation management information into a state used when determining the threshold value. For example, the conversion unit 150 performs the conversion using a table. A specific conversion will be described.

[0023] The case where timetable information is converted will be described. For example, when the difference between the time indicated by the timetable information and the current time is 0, the conversion unit 150 converts the time indicated by the timetable information to "1.0". Also, for example, when the difference between the time indicated by the timetable information and the current time is 120, the conversion unit 150 converts the time indicated by the timetable information to "0.8". In this way, the larger the difference, the smaller the value. A large difference means that it is behind the scheduled departure time. Therefore, it is necessary to close the door earlier. Thus, the larger the difference, the smaller the value. Note that the difference may be represented by an absolute value. Also, the difference may be represented by a positive number (for example, +3 minutes) or a negative number (for example, -3 minutes). Furthermore, the difference may be represented by discrete labels.

[0024] The case where railway signal information is converted will be described. For example, when the railway signal information is "Caution", the conversion unit 150 converts the railway signal information to "1.0". Also, for example, when the railway signal information is "Proceed", the conversion unit 150 converts the railway signal information to "1.3". Also, for example, when the railway signal information is "Stop", the conversion unit 150 converts the railway signal information to the upper limit value. Also, the conversion unit 150 may convert the railway signal information according to the degree of influence exerted by the state indicated by the railway signal information (for example, "Caution").

[0025] The case where operation status information is converted will be described. For example, when the operation status information is "Delay", the conversion unit 150 converts the operation status information to "0.8". As described above, when a delay has occurred, it is necessary to close the door earlier. Therefore, when the operation status information is "Delay", the operation status information is converted to a small value. Also, the conversion unit 150 may convert the operation status information according to the degree of influence exerted by the state indicated by the operation status information (for example, "Delay"). Also, the conversion unit 150 may convert the operation status information according to the assumed time until the state indicated by the operation status information (for example, "Delay") returns to normal operation.

[0026] The conversion unit 150 transmits the converted numerical value of the operation management information to the acquisition unit 120. The acquisition unit 120 acquires the said numerical value. Also, the said numerical value may be expressed as a numerical value corresponding to the operation management information. Here, the conversion process may be performed by an external device. When the conversion process is performed by an external device, the acquisition unit 120 acquires the said numerical value from the external device. Also, the acquisition unit 120 may acquire the numerical value corresponding to the operation management information specified from a table instead of performing conversion.

[0027] The conversion unit 150 converts the station-related information into a state used when determining a threshold value. For example, the conversion unit 150 performs conversion using a table. First, the station-related information will be described. The station-related information is an operation system indicating the station structure, the arrangement of station staff on the platform, and the like. The station structure is either an opposed type or an island type. The opposed type is a system in which only trains in one direction arrive and depart at one platform. The island type is a system in which trains in two directions arrive and depart at one platform. The information regarding the arrangement of station staff is information indicating whether or not station staff are arranged near the doors.

[0028] The specific conversion method will be described. When the station structure is of the opposed type, the conversion unit 150 converts the opposed type indicated by the station structure to "1.0". When the station structure is of the island type, the conversion unit 150 converts the island type indicated by the station structure to "1.2". Note that in the island type, passengers may transfer from the train on the opposite side. That is, the flow of people is complex. Therefore, it is difficult to determine whether to close the doors or not. Thus, a high value is set for the island type. When station staff are arranged near the doors, the conversion unit 150 converts the information indicated by the operation system to "1.0". When station staff are not arranged near the doors, the conversion unit 150 converts the information indicated by the operation system to "1.2". Note that when station staff are not arranged near the doors, human intervention in case of emergency is not possible. Therefore, when station staff are not arranged near the doors, a high value is set. The conversion unit 150 transmits the converted numerical value of the station-related information to the acquisition unit 120.

[0029] Also, when the station-related information acquired by the acquisition unit 120 is already in a state where it is used when determining the threshold value, the conversion unit 150 does not perform the conversion process. Note that the numerical value obtained by converting the station-related information may be expressed as a numerical value corresponding to the station-related information. Further, the conversion process may be performed by an external device. When the conversion process is performed by an external device, the acquisition unit 120 acquires the numerical value obtained by converting the station-related information from the external device. Furthermore, the acquisition unit 120 may acquire, instead of performing the conversion, the numerical value corresponding to the station-related information specified from a table.

[0030] The threshold determination unit 160 determines the threshold value using the numerical value corresponding to the operation management information and the numerical value corresponding to the station-related information. For example, the threshold determination unit 160 determines the threshold value by adding the numerical value corresponding to the operation management information and the numerical value corresponding to the station-related information. The determination process will be described using a specific example. The numerical value corresponding to the operation management information is set to "1.0" (no difference in the train schedule information), "1.3" (the railway signal information is "proceed"), and "1.0" (no delay in the operation status information). The numerical value corresponding to the station-related information is set to "1.2" (the station structure is an island type) and "1.0" (the station staff is arranged near the door). The threshold determination unit 160 determines 5.5 (= 1.0 + 1.3 + 1.0 + 1.2 + 1.0) as the threshold value. Further, the threshold determination unit 160 may multiply the numerical value corresponding to the operation management information and the numerical value corresponding to the station-related information by a reference value. Using the above example, the calculation process will be described. The reference value is set to 70. The threshold determination unit 160 determines 109.2 (= 70 × 1.0 × 1.3 × 1.0 × 1.2 × 1.0) as the threshold value. Note that a lower limit and an upper limit may be set for the threshold value.

[0031] The determination instruction unit 170 determines whether to close the train door or the platform door using the close score and the threshold value. When the close score is greater than the threshold value, the determination instruction unit 170 instructs the door drive device 400 to close the door or the platform door. Thereby, the door drive device 400 closes the door or the platform door.

[0032] Next, the process executed by the control device 100 will be described using a flowchart. FIG. 3 is a flowchart showing an example of the processing executed by the control device according to Embodiment 1. (Step S11) The acquisition unit 120 acquires the video obtained by photographing the platform. (Step S12) The movement vector calculation unit 130 calculates the movement vectors of the crowd included in the video based on the video. (Step S13) The closed score calculation unit 140 calculates a closed score using the learned model and the movement vectors. (Step S14) The acquisition unit 120 acquires operation management information from the railway operation system 300.

[0033] (Step S15) The conversion unit 150 converts the operation management information into a state used when determining the threshold value. (Step S16) The acquisition unit 120 acquires station-related information from the storage unit 110. (Step S17) The conversion unit 150 converts the station-related information into a state used when determining the threshold value. (Step S18) The threshold value determination unit 160 determines a threshold value using the operation management information and the station-related information. (Step S19) The determination instruction unit 170 determines whether to close the train door or the platform door using the closed score and the threshold value. If the closed score is greater than the threshold value, the process proceeds to Step S20. If the closed score is less than or equal to the threshold value, the process proceeds to Step S11. (Step S20) The determination instruction unit 170 instructs the door drive device 400 to close the door or the platform door.

[0034] According to Embodiment 1, the control device 100 calculates the movement vectors of the crowd included in the video. That is, the control device 100 does not track each person included in the video, but calculates the movement vectors of the crowd. By using the movement vectors of the crowd, the control device 100 can appropriately control the door.

[0035] In the above, the case where the threshold value is determined has been described. The threshold value may be a predetermined value. However, the control device 100 can more appropriately control the door by appropriately changing the threshold value according to the situation indicated by the operation management information and the station-related information.

[0036] Next, generation of the learned model will be described. <Learning phase> FIG. 4 is a block diagram showing the functions of the learning device according to the first embodiment. The learning device 500 includes an acquisition unit 510 and a model generation unit 520. Part or all of the acquisition unit 510 and the model generation unit 520 may be realized by a processing circuit included in the learning device 500. Further, part or all of the acquisition unit 510 and the model generation unit 520 may be realized as a module of a program executed by a processor included in the learning device 500.

[0037] The acquisition unit 510 acquires learning data. The model generation unit 520 generates a learned model used by the control device 100 using the learning data.

[0038] Next, the processing executed by the learning device 500 will be described using a flowchart. FIG. 5 is a flowchart showing the processing executed by the learning device according to the first embodiment. (Step S21) The acquisition unit 510 acquires learning data. The learning data is data in which a motion vector is associated with information indicating whether to close the door. The information indicating whether to close the door is a label indicating that the door is to be closed or a label indicating that the door is not to be closed. Note that the motion vector may be a vector calculated from two-frame video or a vector calculated from a plurality of frames (i.e., time-series data) of video.

[0039] Also, in the association, the time period when a person visually closes the door may be recorded, and based on the recorded time period, a movement vector may be associated with information indicating whether to close the train door or the platform door. Further, when data on the control of the door is accumulated, based on the time period when the door is closed and the door open / closed state, a movement vector may be associated with information indicating whether to close the door.

[0040] (Step S22) The model generation unit 520 learns a learning model using the learning data. For example, when the movement vector indicates a direction toward the door, the learning model learns not to close the door. Then, the learning model outputs a confidence score for the label as a close score. The neural network constituting the learning model may be a known neural network such as DNN (Deep Neural Networks), CNN (Convolutional Neural Networks), or RNN (Recurrent Neural Networks). When the learning is completed, the learning model becomes a learned model.

[0041] After the learned model is generated, the learned model is stored in the storage unit 110. Also, the learned model may be stored in an external device.

[0042] In this way, the learning device 500 generates a learned model that outputs a close score when a movement vector is input.

[0043] Embodiment 2. Next, Embodiment 2 will be described. In Embodiment 2, matters different from Embodiment 1 will be mainly described. And in Embodiment 2, the description of matters common to Embodiment 1 will be omitted.

[0044] <Utilization Phase> FIG. 6 is a flowchart showing an example of processing executed by the control device according to Embodiment 2. The processing in FIG. 6 differs from the processing in FIG. 3 in that steps S12a and 13a are executed. Therefore, in FIG. 6, steps S12a and 13a will be described. The description of processing other than steps S12a and 13a will be omitted.

[0045] (Step S12a) The movement vector calculation unit 130 calculates the movement vector of each crowd included in the video based on the video. A specific example will be shown.

[0046] FIG. 7 is a diagram showing an example of the movement vector of each crowd according to Embodiment 2. FIG. 7 shows crowds 10, 11, and 12. Note that crowd 12 is the crowd boarding the train. The movement vector calculation unit 130 calculates movement vectors 10a, 11a, and 12a based on the video.

[0047] The movement vector calculation unit 130 attaches a label to each movement vector. The label is a label indicating the direction. For example, the movement vector calculation unit 130 attaches the label "left" to the movement vector 10a. The movement vector calculation unit 130 attaches the label "right" to the movement vector 11a. The movement vector calculation unit 130 attaches the label "down" to the movement vector 12a.

[0048] (Step S13a) The closed score calculation unit 140 calculates a closed score using the learned model and each movement vector with a label added. Specifically, when the closed score calculation unit 140 inputs each movement vector into the learned model, the learned model outputs a closed score.

[0049] The learned model may calculate a closed score for each label and add up the respective closed scores. For example, the learned model adds the closed score of the movement vector 10a, the closed score of the movement vector 11a, and the closed score of the movement vector 12a. Then, the learned model outputs the value obtained by the addition as the final closed score.

[0050] The closed score calculation unit 140 may calculate a closed score using the movement vectors of the labels related to boarding and alighting and the learned model. The calculation process will be described with reference to FIG. 7. First, the labels related to boarding and alighting are the labels of "up" or "down". In FIG. 7, only the movement vector 12a of the label "down" is detected. Therefore, the closed score calculation unit 140 calculates the closed score using the movement vector 12a and the learned model.

[0051] The closed score calculation unit 140 may calculate the closed score without using the learned model. The closed score calculation unit 140 may calculate the closed score using a mathematical formula. An example is shown below.

[0052] The closed score calculation unit 140 sets a value indicating the presence or absence of a flow of people for each label. When there is a flow of people, "1" is set. When there is no flow of people, "0" is set. When the labels are A, B, C, D and the weights are α, β, γ, δ, the closed score calculation unit 140 calculates the closed score using Equation (1). Also, for example, values of 0 or more are set for the weights related to boarding and alighting. Also, "0" is set for the weights not related to boarding and alighting.

[0053] Closed score = α × A + β × B + γ × C + δ × D ···(1)

[0054] For example, it will be described with reference to FIG. 7. First, label A is "left". Label B is "right". Label C is "down". Label D is "up". In FIG. 7, since label D does not exist (that is, no movement vector is calculated), 0 is set for label D. Since labels A, B, and C exist (that is, movement vectors are calculated), 1 is set for labels A, B, and C. Also, since the "left" and "right" directions are not related to boarding and alighting, 0 is set for α and β. For example, since the "up" and "down" directions are related to boarding and alighting, 1 is set for γ and δ. Therefore, when the closed score calculation unit 140 uses Equation (1), the closed score is calculated as follows.

[0055] Closed score = 1 = 0×1 + 0×1 + 1×1 + 1×0 ···(1)

[0056] In this way, the closed score calculation unit 140 may calculate the closed score using a mathematical formula represented by each label and the weight corresponding to each label.

[0057] Also, the closed score calculation unit 140 may calculate the closed score by the following method. The calculation process will be described using a diagram.

[0058] FIG. 8 is a diagram showing a specific example of calculating the closed score in the second embodiment. FIG. 8 shows the home doors 20a and 20b. First, label A is set to "left". Label B is set to "right". Label C is set to "down". Label D is set to "up". The movement vector of label A is movement vector 21. The movement vector of label B is movement vector 22. The movement vector of label C is movement vector 23. The movement vector of label D is movement vector 24. As will be described later, the movement vector 24 has not been calculated, but for the sake of explanation, the movement vector 24 is drawn.

[0059] The closed score calculation unit 140 calculates the range of the crowd (hereinafter referred to as the crowd range) on which each movement vector in the image is based when calculating. The crowd range on which the movement vector 21 is based when calculating is set to 20% of the image. The crowd range on which the movement vector 22 is based when calculating is set to 15% of the image. The crowd range on which the movement vector 23 is based when calculating is set to 30% of the image. The crowd range on which the movement vector 24 is based when calculating is set to 0% of the image.

[0060] When the weights are α, β, γ, and δ, the closed score calculation unit 140 calculates the closed score using Equation (2). Also, for example, a value of 0 or more is set for the weight related to getting on and off. Also, "0" is set for the weight not related to getting on and off.

[0061] Closed score = α×(100 - population range of group A) + β×(100 - population range of group B) + γ×(100 - population range of group C) + δ×(100 - population range of group D) ··· (2)

[0062] An explanation will be given using a specific example. First, since the "left" and "right" directions have no relation to boarding and alighting, 0 is set for α and β. Since the "up" and "down" directions are related to boarding and alighting, it is assumed that 0.5 is set for γ and δ. Therefore, when using Equation (2), the closed score calculation unit 140 calculates the closed score as follows.

[0063] Closed score = 85 = 0×(100 - 20) + 0×(100 - 15) + 0.5×(100 - 30) + 0.5×(100 - 0) ··· (2)

[0064] In this way, the closed score calculation unit 140 calculates the closed score using a mathematical formula represented by each label, the weight corresponding to each label, and the population range of each movement vector.

[0065] Also, the magnitude of the movement vector may be used. Specifically, the population range in Equation (2) is replaced with the magnitude of the movement vector. Specifically, the closed score calculation unit 140 calculates the closed score using Equation (3).

[0066] Closed score = α×(100 - magnitude of movement vector of A) + β×(100 - magnitude of movement vector of B) + γ×(100 - magnitude of movement vector of C) + δ×(100 - magnitude of movement vector of D) ··· (3)

[0067] In this way, the closed score calculation unit 140 may calculate the magnitude of each movement vector and calculate the closed score using a mathematical formula represented by each label, the weight corresponding to each label, and the magnitude of each movement vector.

[0068] Further, the closed score calculation unit 140 may calculate a closed score using a learned model in which the relationship between the crowd range or the magnitude of the movement vector of each label and the information indicating whether to close the door is learned. Note that the neural network constituting the learned model may be a known neural network such as a DNN, a CNN, or an RNN.

[0069] According to the second embodiment, the control device 100 determines whether to close the door using the label. Therefore, the control device 100 can make a more appropriate determination. Further, as described above, by adding the label, the control device 100 can input the movement vector of the label related to getting on and off into the learned model.

[0070] <Learning phase> The learning device 500 generates a learned model that outputs a closed score when each movement vector with a label added is input.

[0071] Embodiment 3. Next, Embodiment 3 will be described. In Embodiment 3, matters different from Embodiment 1 will be mainly described. And in Embodiment 3, the description of matters common to Embodiment 1 will be omitted.

[0072] <Utilization phase> FIG. 9 is a block diagram showing the functions of the control device according to the third embodiment. The control device 100 further includes a detection unit 180. Part or all of the detection unit 180 may be realized by a processing circuit. Further, part or all of the detection unit 180 may be realized as a module of a program executed by the processor 101.

[0073] The detection unit 180 detects an attribute of a person included in an image based on one or more images constituting the video acquired by the acquisition unit 120. Specifically, the detection unit 180 uses a known image recognition technique to detect the attribute of the person included in the image. For example, the detection unit 180 detects the attribute using the image and a learned model.

[0074] Here, the said attribute will be described. For example, the said attribute is age, group, etc. For example, the group is parent-child, student group, etc. Also, the said attribute may be wheelchair use, white cane use, stroller use, suitcase possession, etc.

[0075] The conversion unit 150 converts the information indicated by the said attribute into a numerical value. For example, the conversion unit 150 performs the conversion using a table. A specific conversion process will be described.

[0076] When the age indicated by the said attribute is elderly, the train door or platform door needs to be opened wide. That is, since the elderly have low mobility, the train door or platform door needs to be opened wide. Therefore, for example, when the age indicated by the said attribute is elderly, the conversion unit 150 converts the information indicated by the said attribute to "1.2".

[0077] When the group indicated by the said attribute is parent-child, there is a possibility that the child may suddenly run out and the parent may chase after. Therefore, the train door or platform door needs to be opened wide. For example, when the group indicated by the said attribute is parent-child, the conversion unit 150 converts the information indicated by the said attribute to "1.1".

[0078] For example, when the age indicated by the said attribute is the adult age, the conversion unit 150 converts the information indicated by the said attribute to "1.0".

[0079] The threshold determination unit 160 determines a threshold value using a numerical value corresponding to the operation management information, a numerical value corresponding to the station-related information, and a numerical value obtained by converting the information indicated by the attribute. For example, the threshold determination unit 160 determines the threshold value by adding a numerical value corresponding to the operation management information, a numerical value corresponding to the station-related information, and a numerical value obtained by converting the information indicated by the attribute. The determination process will be described using a specific example. Assume that the numerical value corresponding to the operation management information is “1.0” (no difference in the train schedule information), “1.3” (the railway signal information is “go”), and “1.0” (no delay in the operation status information). Assume that the numerical value corresponding to the station-related information is “1.2” (the station structure is an island platform) and “1.0” (the station staff is located near the door). Assume that the numerical value obtained by converting the information indicated by the attribute is “1.0” (adult age). The threshold determination unit 160 determines 6.5 (= 1.0 + 1.3 + 1.0 + 1.2 + 1.0 + 1.0) as the threshold value.

[0080] Figure 10 is a flowchart (part 1) showing an example of the process executed by the control device according to Embodiment 3. The process in Figure 10 differs from the process in Figure 3 in that steps S17a and 17b are executed. Therefore, in Figure 10, steps S17a and 17b will be described. The description of the processes other than steps S17a and 17b will be omitted.

[0081] (Step S17a) The detection unit 180 detects an attribute related to a person based on the image. (Step S17b) The conversion unit 150 converts the information indicated by the attribute into a state used when determining the threshold value. Then, the process proceeds to step S18a.

[0082] Figure 11 is a flowchart (part 2) showing an example of the process executed by the control device according to Embodiment 3. The process in Figure 11 differs from the process in Figure 3 in that step S18a is executed. Therefore, in Figure 11, step S18a will be described. The description of the processes other than step S18a will be omitted.

[0083] (Step S18a) The threshold determination unit 160 determines a threshold using a numerical value corresponding to the operation management information, a numerical value corresponding to the station-related information, and a numerical value obtained by converting the information indicated by the attribute.

[0084] According to Embodiment 3, the control device 100 changes the threshold according to the user. Thereby, the control device 100 can appropriately control the door according to the user.

[0085] Embodiment 4. Next, Embodiment 4 will be described. In Embodiment 4, matters different from Embodiment 1 will be mainly described. And in Embodiment 4, the description of matters common to Embodiment 1 will be omitted.

[0086] <Utilization phase> FIG. 12 is a block diagram showing the functions of the control device according to Embodiment 4. The control device 100 further includes a processing unit 190. Part or all of the processing unit 190 may be realized by a processing circuit. Also, part or all of the processing unit 190 may be realized as a module of a program executed by the processor 101.

[0087] The processing unit 190 masks areas other than the areas near the train door or the platform door for each of the plurality of images constituting the video. For example, the masked area is an area where people going to the ticket gate or the entrance / exit exist, or an area where people who do not exist near the yellow braille block exist. A specific example of the masking process will be described. The processing unit 190 identifies the train door or the platform door using image recognition technology. The processing unit 190 masks areas outside a predetermined range centered on the identified train door or platform door. Thereby, only the boarding and alighting people exist in each of the plurality of images.

[0088] The processing unit 190 may mask areas outside the driver's gaze area for each of the plurality of images constituting the video. For example, the driver's gaze area is near the train door or the platform door, near the escalator, near the elevator, near the stairs, near the pillar, etc. Note that the reason why the driver gazes near the escalator, near the elevator, and near the stairs is to check if there is a person making a dash for boarding. Also, the reason why the driver gazes near the pillar is to check if a person hiding in the shadow of the pillar is boarding. An example of the masking process will be described. The processing unit 190 uses image recognition technology to identify the escalator, elevator, stairs, or pillar. The processing unit 190 masks areas outside a predetermined range centered on the identified escalator, elevator, stairs, or pillar. As a result, only the people related to boarding and alighting exist in each of the plurality of images. For example, the people related to boarding and alighting are those who make a dash for boarding or those who board from the shadow of the pillar.

[0089] The movement vector calculation unit 130 calculates the movement vector of the crowd based on the processed video (i.e., the processed plurality of images). Thereby, the movement vector of the crowd getting on and off is calculated. That is, the movement vector calculation unit 130 does not calculate the movement vector of the crowd performing actions not related to boarding and alighting (for example, the crowd heading for the ticket gate). Therefore, the processing load on the control device 100 is reduced.

[0090] Next, the process executed by the control device 100 will be described using a flowchart. FIG. 13 is a flowchart showing an example of the process executed by the control device according to the fourth embodiment. The process in FIG. 13 is different from the process in FIG. 3 in that steps S11a and 12b are executed. Therefore, in FIG. 13, steps S11a and 12b will be described. The description of the processes other than steps S11a and 12b will be omitted.

[0091] (Step S11a) The processing unit 190 masks areas other than the areas near the train door or the platform door for each of the plurality of images constituting the video. (Step S12b) The movement vector calculation unit 130 calculates the movement vectors of the crowd based on the processed video.

[0092] According to the fourth embodiment, the control device 100 does not calculate the movement vectors of the crowd performing actions not related to boarding and alighting. Therefore, the control device 100 can reduce the processing load. Further, only the movement vectors of the crowd boarding and alighting are input to the learned model. In other words, the movement vectors of the crowd performing actions not related to boarding and alighting are not input to the learned model. Therefore, the estimation of the learned model is not affected by the movement vectors of the crowd performing actions not related to boarding and alighting. Thus, the estimation accuracy of the learned model is improved.

[0093] Embodiment 5. Next, Embodiment 5 will be described. In Embodiment 5, the matters different from Embodiment 1 will be mainly described. And in Embodiment 5, the description of the matters common to Embodiment 1 will be omitted.

[0094] <Learning Phase> In the learning data of the first embodiment, the movement vectors are associated with the information indicating whether to close the door. The association is performed by the user. In Embodiment 5, the case where the association is automatically performed will be described.

[0095] FIG. 14 is a block diagram showing the functions of the learning device of the fifth embodiment. The learning device 500 further includes a data generation unit 530. Part or all of the data generation unit 530 may be realized by the processing circuit included in the learning device 500. Also, part or all of the data generation unit 530 may be realized as a module of a program executed by the processor included in the learning device 500. The functions of the data generation unit 530 will be described later.

[0096] Next, the processing executed by the learning device 500 will be described using a flowchart. FIG. 15 is a flowchart showing the processing executed by the learning device according to Embodiment 5. The processing in FIG. 15 differs from the processing in FIG. 5 in that steps S21a and 21b are executed. Therefore, in FIG. 15, steps S21a and 21b will be described. The description of the processing other than steps S21a and 21b will be omitted.

[0097] (Step S21a) The acquisition unit 510 acquires the movement vectors of the crowd calculated based on the video. In addition, the acquisition unit 510 acquires from the railway operation system 300 information indicating whether the train door or the platform door was closed at the time when the video was generated. Note that the information indicating whether the train door or the platform door was closed can be generated based on the operation history of the door.

[0098] (Step S21b) The data generation unit 530 generates learning data by associating the movement vector with the information indicating whether the train door or the platform door was closed. That is, the data generation unit 530 generates the same data as the learning data in Embodiment 1.

[0099] In addition, information indicating that the platform door is to be closed may be associated with the movement vector calculated based on the video when an obstacle is confirmed between the platform door and the train.

[0100] According to Embodiment 5, the learning device 500 automatically associates the movement vector with the information indicating whether to close the door. Therefore, the learning device 500 can reduce the burden on the user.

[0101] The features in the above-described embodiments can be appropriately combined with each other.

Description of Reference Numerals

[0102] Groups 10, 11, 12, movement vectors 10a, 11a, 12a, home doors 20a, 20b, movement vector 21, movement vector 22, movement vector 23, movement vector 24, control device 100, processor 101, volatile memory device 102, non-volatile memory device 103, input / output interface 104, communication interface 105, memory unit 110, acquisition unit 120, movement vector calculation unit 130, closed score calculation unit 140, conversion unit 150, threshold determination unit 160, determination instruction unit 170, detection unit 180, processing unit 190, camera 200, railway operation system 300, door drive device 400, learning device 500, acquisition unit 510, model generation unit 520, data generation unit 530.

Claims

1. A control device for controlling a door drive device that controls a door of a train or a platform door, comprising: an acquisition unit that acquires a video obtained by photographing a platform; a movement vector calculation unit that calculates a movement vector of a crowd included in the video based on the video; a closed score calculation unit that calculates a closed score, which is a numerical value, using the movement vector; a determination instruction unit that determines whether to close the door or the platform door using the closed score and a threshold value, and instructs the door drive device to close the door or the platform door when the closed score is greater than the threshold value; A control device having the above.

2. The acquisition unit acquires a learned model, The closed score calculation unit calculates the closed score using the learned model and the movement vector. The control device according to claim 1.

3. The movement vector calculation unit calculates a movement vector of each crowd included in the video, attaches a label to each movement vector, The closed score calculation unit calculates the closed score using the movement vector of the label related to getting on and off and the learned model. The control device according to claim 2.

4. The control device further includes a processing unit that masks an area other than the area near the door or the platform door for each of a plurality of images constituting the video, The movement vector calculation unit calculates the movement vector based on the processed video. The control device according to claim 2.

5. The control device further includes a processing unit that masks an area other than the area where the driver's attention area is located for each of a plurality of images constituting the video, The movement vector calculation unit calculates the movement vector based on the processed video. The control device according to claim 2.

6. The movement vector calculation unit calculates a movement vector of each crowd included in the video, attaches a label to each movement vector, The closed score calculation unit sets a value indicating the presence or absence of a flow of people for each label, and calculates the closed score using a mathematical formula represented by each label and a weight corresponding to each label, The weight related to getting on and off indicates a value of 0 or more, The weight not related to getting on and off indicates 0. The control device according to claim 1.

7. The closed score calculation unit, calculates the range of the crowd or the magnitude of each movement vector based on which the movement vectors are calculated in the image. The closed score is calculated using a mathematical formula represented by each label, the weight corresponding to each label, the range of the crowd that serves as a basis when calculating each movement vector, or the magnitude of each movement vector. The control device according to claim 6.

8. further comprising a threshold determination unit, The acquisition unit acquires a numerical value corresponding to operation management information, which is information related to operation, and a numerical value corresponding to station-related information, which is information related to a station. The threshold determination unit determines the threshold using the numerical value corresponding to the operation management information and the numerical value corresponding to the station-related information. The control device according to any one of claims 1 to 7.

9. a detection unit that detects an attribute related to a person included in the image based on one or more images constituting the video; a conversion unit that converts the information indicated by the attribute into a numerical value; further comprising The threshold determination unit determines the threshold using the numerical value corresponding to the operation management information, the numerical value corresponding to the station-related information, and the numerical value obtained by converting the information indicated by the attribute. The control device according to claim 8.

10. an acquisition unit that acquires a movement vector of a crowd calculated based on a video and information indicating whether a train door or a platform door of a station was closed at the time when the video was generated; a data generation unit that generates learning data by associating the movement vector with the information indicating whether the door or the platform door was closed; A learning device having

11. A control device that controls a door drive device that controls a train door or a platform door of a station, acquires a video obtained by photographing a platform, calculates a movement vector of a crowd included in the video based on the video, calculates a closed score, which is a numerical value, using the movement vector, determines whether to close the door or the platform door using the closed score and a threshold, and when the closed score is greater than the threshold, instructs the door drive device to close the door or the platform door. A control method.

12. A learning device acquires a movement vector of a crowd calculated based on a video and information indicating whether a train door or a platform door of a station was closed at the time when the video was generated, and generates learning data by associating the movement vector with the information indicating whether the door or the platform door was closed. A learning method.

13. A control device that controls a door drive device that controls the doors of a train or the platform doors of a station, acquires video obtained by photographing the platform, calculates the movement vectors of the crowd included in the video based on the video, calculates a closed score, which is a numerical value, using the movement vectors, determines whether to close the door or the platform door using the closed score and a threshold value, and when the closed score is greater than the threshold value, instructs the door drive device to close the door or the platform door. A control program that causes the processing to be executed.

14. A learning device, acquires the movement vectors of the crowd calculated based on the video and information indicating whether the doors of a train or the platform doors of a station were closed at the time when the video was generated, and generates learning data by associating the movement vectors with the information indicating whether the door or the platform door was closed. A learning program that causes the processing to be executed.

Images