Control device, learning device, control method, learning method, control program, and learning program

Abstract

This control device (100) controls a door drive device (400) that controls a door of a train or a platform door of a station. The control device (100) comprises: an acquisition unit (120) that acquires a video obtained by imaging a platform; a movement vector calculation unit (130) that calculates movement vectors of a crowd included in the video on the basis of the video; a closed score calculation unit (140) that uses the movement vector to calculate a closed score that is a numerical value; and a determination instruction unit (170) that uses the closed score and a threshold to determine whether to close the door or the platform door, and instructs the door drive device (400) to close the door or the platform door when the closed score is greater than the threshold.

Description

Control device, learning device, control method, learning method, control program, and learning program

[0001] The present disclosure relates to a control device, a learning device, a control method, a learning method, a control program, and a learning program.

[0002] Cameras are installed on train platforms. For example, a technology using images captured by the cameras has been proposed (see Patent Document 1). Trains also have doors. A technology for detecting the timing at which the doors close has been proposed (see Patent Document 2).

[0003] JP2019-41207A JP10-203361A

[0004] One method is to analyze people near a door and determine whether to close the door based on the analysis results. The analysis involves identifying and tracking the person. However, this method is difficult to use when the area is crowded. Therefore, a method for appropriately controlling the door is desired.

[0005] The object of the present disclosure is to provide proper door control.

[0006] According to one aspect of the present disclosure, there is provided a control device. The control device controls a door driving device that controls train doors or station platform doors. The control device includes: an acquisition unit that acquires video obtained by photographing a platform; a movement vector calculation unit that calculates a movement vector of a crowd included in the video based on the video; a closed score calculation unit that calculates a numerical closed score using the movement vector; and a determination instruction unit that uses the closed score and a threshold value to determine whether to close the door or the platform door, and that instructs the door driving device to close the door or the platform door if the closed score is greater than the threshold value.

[0007] According to the present disclosure, door control can be performed appropriately.

[0008] 1 is a block diagram showing the functions of a control device of embodiment 1. FIG. 2 is a diagram showing hardware included in the control device of embodiment 1. FIG. 3 is a flowchart showing an example of processing executed by the control device of embodiment 1. FIG. 4 is a block diagram showing the functions of a learning device of embodiment 1. FIG. 5 is a flowchart showing an example of processing executed by the learning device of embodiment 1. FIG. 6 is a flowchart showing an example of processing executed by the control device of embodiment 2. FIG. 7 is a diagram showing an example of each movement vector of each crowd of people of embodiment 2. FIG. 8 is a diagram showing a specific example of calculation of a closure score of embodiment 2. FIG. 9 is a block diagram showing the functions of a control device of embodiment 3. FIG. 10 is a flowchart showing an example of processing executed by the control device of embodiment 3. FIG. 11 is a flowchart showing an example of processing executed by the control device of embodiment 3. FIG. 12 is a flowchart showing an example of processing executed by the control device of embodiment 3. FIG. 13 is a block diagram showing the functions of a control device of embodiment 4. FIG. 14 is a flowchart showing an example of processing executed by the control device of embodiment 4. FIG. 15 is a block diagram showing the functions of a learning device of embodiment 5. FIG. 16 is a flowchart showing the processing executed by the learning device of embodiment 5.

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1. <Utilization Phase> Fig. 1 is a block diagram showing the functions of a control device according to embodiment 1. A control device 100 communicates with a camera 200, a train operation system 300, and a door driving device 400 via a network.

[0011] The control device 100 is a device that executes the control method. The control device 100 controls the door drive device 400. The camera 200 takes pictures of the platform. The railway operation system 300 outputs operation management information. The operation management information will be explained later. The door drive device 400 controls the doors of the train or the platform doors at the station. Note that the train also refers to a monorail or the like.

[0012] Before describing the functions of the control device 100, the hardware of the control device 100 will be described. Fig. 2 is a diagram showing the hardware of the control device of embodiment 1. The control device 100 is a computer. The control device 100 has a processor 101, a volatile storage device 102, a non-volatile storage device 103, an input / output interface 104, and a communication interface 105.

[0013] The processor 101 controls the entire control device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The control device 100 may also include a processing circuit.

[0014] The volatile memory device 102 is a main memory device of the control device 100. For example, the volatile memory device 102 is a RAM (Random Access Memory). The non-volatile memory device 103 is an auxiliary memory device of the control device 100. For example, the non-volatile memory device 103 is a HDD (Hard Disk Drive) or an SSD (Solid State Drive). For example, the input / output interface 104 receives information input by a user. The communication interface 105 communicates with the camera 200, the train operation system 300, and the door driver 400.

[0015] 1 , the functions of the control device 100 will be described. The control device 100 includes a storage unit 110, an acquisition unit 120, a movement vector calculation unit 130, a closure score calculation unit 140, a conversion unit 150, a threshold determination unit 160, and a judgment instruction unit 170.

[0016] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103. Some or all of the acquisition unit 120, the movement vector calculation unit 130, the closed score calculation unit 140, the conversion unit 150, the threshold determination unit 160, and the judgment instruction unit 170 may be realized by a processing circuit. Furthermore, some or all of the acquisition unit 120, the movement vector calculation unit 130, the closed score calculation unit 140, the conversion unit 150, the threshold determination unit 160, and the judgment instruction unit 170 may be realized as program modules executed by the processor 101. For example, the program executed by the processor 101 is also referred to as a control program. For example, the control program is recorded on a recording medium.

[0017] The storage unit 110 stores various information. The storage unit 110 may store a trained model for calculating a closed score, which will be described later.

[0018] The acquisition unit 120 acquires video obtained by photographing the platform. For example, the acquisition unit 120 acquires the video from the camera 200. The acquisition unit 120 acquires a trained model. For example, the acquisition unit 120 acquires the trained model from the storage unit 110. Furthermore, for example, the acquisition unit 120 acquires the trained model from an external device. Note that the external device is a device that exists outside the control device 100. For example, the external device is a cloud server, an external memory, etc. The external device is not shown in the drawing.

[0019] The acquisition unit 120 acquires traffic control information from the railway operation system 300. The traffic control information is information related to operation. For example, the traffic control information is timetable information, railway signal information, operation status information indicating delays, suspension of service, etc. The acquisition unit 120 acquires station-related information from the storage unit 110 or an external device. The station-related information is information related to stations. Details of the station-related information will be explained later.

[0020] The motion vector calculation unit 130 calculates the motion vectors of the crowd included in the video based on the video. That is, the motion vector calculation unit 130 calculates the motion vectors of the crowd without tracking each person included in the video. For example, when calculating the motion vectors, the motion vector calculation unit 130 calculates the motion vectors using an image processing technique such as a gradient method or a block matching method, or a machine learning model such as FlowNet.

[0021] The closed score calculation unit 140 calculates a closed score using the movement vector. For example, the closed score calculation unit 140 calculates a closed score using the learned model and the movement vector. In particular, the closed score calculation unit 140 inputs the movement vector into the learned model, and the learned model outputs a closed score. Here, the closed score is a numerical value used when determining whether or not to close the train doors or platform doors. A larger closed score means that the doors or platform doors may be closed. The range of the closed score may be a range of 0 to 100, such as [0, 100]. Furthermore, a lower limit or upper limit of the closed score may not be set.

[0022] The conversion unit 150 converts the fleet management information into a state that is used when determining the threshold value. For example, the conversion unit 150 performs the conversion using a table. A specific conversion will be described.

[0023] A case where schedule information is converted will be described. For example, if the difference between the time indicated by the schedule information and the current time is 0, the conversion unit 150 converts the time indicated by the schedule information to "1.0." Furthermore, if the difference between the time indicated by the schedule information and the current time is 120, the conversion unit 150 converts the time indicated by the schedule information to "0.8." Thus, the larger the difference, the smaller the value. A large difference means that the train is running behind the scheduled departure time. Therefore, it is necessary to close the doors earlier. Therefore, the larger the difference, the smaller the value. The difference may be expressed as an absolute value. The difference may also be expressed as a positive number (e.g., +3 minutes) or a negative number (e.g., -3 minutes). Furthermore, the difference may be expressed as a discrete label.

[0024] A case where railway signal information is converted will be described. For example, if the railway signal information is "Caution," the conversion unit 150 converts the railway signal information to "1.0." Furthermore, for example, if the railway signal information is "Proceed," the conversion unit 150 converts the railway signal information to "1.3." Furthermore, for example, if the railway signal information is "Stop," the conversion unit 150 converts the railway signal information to an upper limit value. Furthermore, the conversion unit 150 may convert the railway signal information into the degree of impact of the state indicated by the railway signal information (for example, "Caution").

[0025] A case where operation status information is converted will be described. For example, if the operation status information is "delay," the conversion unit 150 converts the operation status information to "0.8." As described above, when a delay occurs, it is necessary to close the doors early. Therefore, when the operation status information is "delay," the operation status information is converted to a smaller value. Furthermore, the conversion unit 150 may convert the operation status information into the degree of impact of the state indicated by the operation status information (e.g., "delay"). Furthermore, the conversion unit 150 may convert the operation status information into an estimated time until the state indicated by the operation status information (e.g., "delay") returns to normal operation.

[0026] The conversion unit 150 transmits the numerical value obtained by converting the traffic management information to the acquisition unit 120. The acquisition unit 120 acquires the numerical value. The numerical value may also be expressed as a numerical value corresponding to the traffic management information. Here, the conversion process may be performed by an external device. When the conversion process is performed by an external device, the acquisition unit 120 acquires the numerical value from the external device. The acquisition unit 120 may also acquire a numerical value corresponding to the traffic management information identified from a table, rather than performing a conversion.

[0027] The conversion unit 150 converts the station-related information into a state that is used when determining the threshold value. For example, the conversion unit 150 performs the conversion using a table. First, the station-related information will be described. The station-related information includes the station structure, the operating system indicating the placement of station staff on the platform, etc. The station structure is either an opposing type or an island type. An opposing type is a type in which trains depart and arrive in only one direction on one platform. An island type is a type in which trains depart and arrive in two directions on one platform. Information regarding the placement of station staff is information indicating whether station staff are placed near the doors.

[0028] The conversion method will be specifically described. If the station structure is an opposing type, the conversion unit 150 converts the opposing type indicated by the station structure to "1.0." If the station structure is an island type, the conversion unit 150 converts the island type indicated by the station structure to "1.2." Note that an island type station has the possibility of passengers transferring from the opposite train. In other words, the flow of people is complex. Therefore, it is difficult to determine whether to close the doors. Therefore, a high value is set for the island type. If a station staff member is stationed near the door, the conversion unit 150 converts the information indicated by the operating system to "1.0." If a station staff member is not stationed near the door, the conversion unit 150 converts the information indicated by the operating system to "1.2." Note that if a station staff member is not stationed near the door, human intervention in an emergency is not possible. Therefore, if a station staff member is not stationed near the door, a high value is set. The conversion unit 150 transmits the numerical value into which the station-related information has been converted to the acquisition unit 120.

[0029] Furthermore, if the station-related information acquired by the acquisition unit 120 is already in a state that can be used when determining the threshold value, the conversion unit 150 does not perform the conversion process. The numerical value into which the station-related information is converted may be expressed as a numerical value corresponding to the station-related information. The conversion process may also be performed by an external device. When the conversion process is performed by an external device, the acquisition unit 120 acquires the numerical value into which the station-related information is converted from the external device. Furthermore, the acquisition unit 120 may acquire a numerical value corresponding to the station-related information identified from a table, rather than performing a conversion.

[0030] The threshold determination unit 160 determines a threshold using a numerical value corresponding to the traffic management information and a numerical value corresponding to the station-related information. For example, the threshold determination unit 160 determines a threshold by adding a numerical value corresponding to the traffic management information and a numerical value corresponding to the station-related information. The determination process will be explained using a specific example. The numerical values ​​corresponding to the traffic management information are assumed to be "1.0" (no difference in timetable information), "1.3" (railway signal information is "ongoing"), and "1.0" (no delay in traffic status information). The numerical values ​​corresponding to the station-related information are assumed to be "1.2" (island station structure) and "1.0" (station staff are stationed near the doors). The threshold determination unit 160 determines 5.5 (= 1.0 + 1.3 + 1.0 + 1.2 + 1.0) as the threshold. The threshold determination unit 160 may also multiply the numerical value corresponding to the traffic management information and the numerical value corresponding to the station-related information by a reference value. The calculation process will be explained using the above example. The reference value is assumed to be 70. The threshold value determining unit 160 determines the threshold value to be 109.2 (=70×1.0×1.3×1.0×1.2×1.0). Note that a lower limit and an upper limit may be set for the threshold value.

[0031] The determination instruction unit 170 uses the closure score and a threshold value to determine whether to close the train doors or platform doors. If the closure score is greater than the threshold value, the determination instruction unit 170 instructs the door drive device 400 to close the doors or platform doors. As a result, the door drive device 400 closes the doors or platform doors.

[0032] Next, the processing executed by the control device 100 will be described using a flowchart. FIG. 3 is a flowchart showing an example of the processing executed by the control device of embodiment 1. (Step S11) The acquisition unit 120 acquires video obtained by photographing the platform. (Step S12) The movement vector calculation unit 130 calculates the movement vectors of the crowd included in the video based on the video. (Step S13) The closed score calculation unit 140 calculates the closed score using the trained model and the movement vectors. (Step S14) The acquisition unit 120 acquires traffic management information from the railway operation system 300.

[0033] (Step S15) The conversion unit 150 converts the traffic control information into a state used when determining the threshold value. (Step S16) The acquisition unit 120 acquires station-related information from the storage unit 110. (Step S17) The conversion unit 150 converts the station-related information into a state used when determining the threshold value. (Step S18) The threshold value determination unit 160 determines a threshold value using the traffic control information and the station-related information. (Step S19) The judgment instruction unit 170 uses the closure score and the threshold value to determine whether to close the train doors or platform doors. If the closure score is greater than the threshold value, the process proceeds to step S20. If the closure score is equal to or less than the threshold value, the process proceeds to step S11. (Step S20) The judgment instruction unit 170 instructs the door drive device 400 to close the doors or platform doors.

[0034] According to the first embodiment, the control device 100 calculates the movement vectors of the crowd included in the video. That is, the control device 100 calculates the movement vectors of the crowd instead of tracking each person included in the video. By using the movement vectors of the crowd, the control device 100 can appropriately control the doors.

[0035] The above describes the case where the threshold value is determined. The threshold value may be a predetermined value. However, the control device 100 can more appropriately control the doors by appropriately changing the threshold value depending on the situation indicated by the traffic management information and station-related information.

[0036] Next, generation of a trained model will be described. <Learning Phase> Fig. 4 is a block diagram showing the functions of the learning device of embodiment 1. The learning device 500 has an acquisition unit 510 and a model generation unit 520. A part or all of the acquisition unit 510 and the model generation unit 520 may be realized by a processing circuit included in the learning device 500. Furthermore, a part or all of the acquisition unit 510 and the model generation unit 520 may be realized as a program module executed by a processor included in the learning device 500.

[0037] The acquisition unit 510 acquires training data. The model generation unit 520 uses the training data to generate a trained model to be used by the control device 100.

[0038] Next, the processing executed by the learning device 500 will be described using a flowchart. FIG. 5 is a flowchart showing the processing executed by the learning device of embodiment 1. (Step S21) The acquisition unit 510 acquires learning data. The learning data is data in which a movement vector is associated with information indicating whether or not to close the door. The information indicating whether or not to close the door is a label indicating to close the door or a label indicating not to close the door. Note that the movement vector may be a vector calculated from two frames of video, or may be a vector calculated from multiple frames of video (i.e., time-series data).

[0039] In addition, the association may involve recording the time periods when people visually close the doors, and associating the movement vector with information indicating whether the train doors or platform doors will be closed based on the recorded time periods. Furthermore, if door control data is accumulated, the movement vector may be associated with information indicating whether the doors will be closed based on the time periods when the doors were closed and the door open / closed status.

[0040] (Step S22) The model generation unit 520 uses the training data to train a training model. For example, if the movement vector indicates a direction toward a door, the training model learns not to close the door. The training model then outputs a confidence score for the label as a closed score. The neural network constituting the training model may be a well-known neural network such as a deep neural network (DNN), a convolutional neural network (CNN), or a recurrent neural network (RNN). When the training is completed, the training model becomes a trained model.

[0041] After the trained model is generated, the trained model is stored in the storage unit 110. The trained model may also be stored in an external device.

[0042] In this way, the learning device 500 generates a learned model that outputs a closed score when a movement vector is input.

[0043] Second Embodiment Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be mainly described. Furthermore, in the second embodiment, descriptions of the commonalities between the first embodiment and the second embodiment will be omitted.

[0044] <Utilization Phase> Fig. 6 is a flowchart showing an example of processing executed by the control device of embodiment 2. The processing in Fig. 6 differs from the processing in Fig. 3 in that steps S12a and S13a are executed. Therefore, steps S12a and S13a will be described in Fig. 6. Description of processing other than steps S12a and S13a will be omitted.

[0045] (Step S12a) The movement vector calculation unit 130 calculates the movement vectors of each of the crowds included in the video based on the video.

[0046] Fig. 7 is a diagram showing examples of movement vectors of each crowd in embodiment 2. Fig. 7 shows crowds 10, 11, and 12. Crowd 12 is a crowd getting on a train. Movement vector calculation unit 130 calculates movement vectors 10a, 11a, and 12a based on the video.

[0047] The motion vector calculation unit 130 assigns a label to each motion vector. The label indicates a direction. For example, the motion vector calculation unit 130 assigns the label "left" to motion vector 10a. The motion vector calculation unit 130 assigns the label "right" to motion vector 11a. The motion vector calculation unit 130 assigns the label "down" to motion vector 12a.

[0048] (Step S13a) The closed score calculation unit 140 calculates a closed score using the trained model and each labeled movement vector. In detail, when the closed score calculation unit 140 inputs each movement vector into the trained model, the trained model outputs a closed score.

[0049] The trained model may calculate a closed score for each label and add the closed scores together. For example, the trained model may add the closed score for movement vector 10a, the closed score for movement vector 11a, and the closed score for movement vector 12a. The trained model then outputs the value obtained by the addition as the final closed score.

[0050] The closed score calculation unit 140 may calculate the closed score using a movement vector of a label related to getting on and off and a learned model. The calculation process will be described with reference to FIG. 7. First, the label related to getting on and off is the label "up" or "down." In FIG. 7, only the movement vector 12a of the label "down" is detected. Therefore, the closed score calculation unit 140 calculates the closed score using the movement vector 12a and a learned model.

[0051] The closed score calculation unit 140 may calculate the closed score without using a trained model. The closed score calculation unit 140 may calculate the closed score using a mathematical formula. An example is shown below.

[0052] The closed score calculation unit 140 sets a value indicating whether or not there is a pedestrian flow to each label. If there is a pedestrian flow, "1" is set. If there is no pedestrian flow, "0" is set. If the labels are A, B, C, and D and the weights are α, β, γ, and δ, the closed score calculation unit 140 calculates the closed score using formula (1). Furthermore, for example, a value greater than or equal to 0 is set for a weight related to boarding and alighting. Furthermore, "0" is set for a weight not related to boarding and alighting.

[0053] Closed score = α × A + β × B + γ × C + δ × D (1)

[0054] For example, a description will be given using FIG. 7 . First, let us say that label A is "left." Label B is "right." Label C is "down." Label D is "up." In FIG. 7 , label D does not exist (i.e., a movement vector has not been calculated), so label D is set to 0. Labels A, B, and C exist (i.e., a movement vector has been calculated), so labels A, B, and C are set to 1. Furthermore, since the "left" and "right" directions are not related to getting on and off, α and β are set to 0. For example, since the "up" and "down" directions are related to getting on and off, γ and δ are set to 1. Therefore, when formula (1) is used, the closed score calculation unit 140 calculates the closed score as follows:

[0055] Closed score = 1 = 0 × 1 + 0 × 1 + 1 × 1 + 1 × 0 ... (1)

[0056] In this way, the closed score calculation unit 140 may calculate the closed score using a mathematical formula expressed by each label and the weight corresponding to each label.

[0057] The closed score calculation unit 140 may also calculate the closed score using the following method: The calculation process will be described with reference to the drawings.

[0058] FIG. 8 is a diagram showing a specific example of calculation of a closed score in embodiment 2. FIG. 8 shows platform doors 20a and 20b. First, label A is set to "left." Label B is set to "right." Label C is set to "down." Label D is set to "up." The movement vector of label A is movement vector 21. The movement vector of label B is movement vector 22. The movement vector of label C is movement vector 23. The movement vector of label D is movement vector 24. As will be described later, movement vector 24 has not been calculated, but is depicted for the sake of explanation.

[0059] The closed score calculation unit 140 calculates the range of the crowd in the image that was used as the basis for calculating each movement vector (hereinafter referred to as the crowd range). The crowd range that was used as the basis for calculating movement vector 21 was set to 20% of the image. The crowd range that was used as the basis for calculating movement vector 22 was set to 15% of the image. The crowd range that was used as the basis for calculating movement vector 23 was set to 30% of the image. The crowd range that was used as the basis for calculating movement vector 24 was set to 0% of the image.

[0060] When the weights are α, β, γ, and δ, the closed score calculation unit 140 calculates the closed score using equation (2). For example, a weight related to getting on and off is set to a value equal to or greater than 0. A weight unrelated to getting on and off is set to "0."

[0061] Closed score = α × (100 - A crowd range) + β × (100 - B crowd range) + γ × (100 - C crowd range) + δ × (100 - D crowd range) (2)

[0062] This will be explained using a specific example. First, since the "left" and "right" directions are not related to getting on and off, α and β are set to 0. Since the "up" and "down" directions are related to getting on and off, γ and δ are set to 0. Therefore, when using formula (2), the closed score calculation unit 140 calculates the closed score as follows:

[0063] Closed score = 85 = 0 × (100 − 20) + 0 × (100 − 15) + 0.5 × (100 − 30) + 0.5 × (100 − 0) ... (2)

[0064] In this way, the closed score calculation unit 140 calculates the closed score using a mathematical formula expressed by each label, the weight corresponding to each label, and the crowd range of each movement vector.

[0065] Alternatively, the magnitude of the movement vector may be used. Specifically, the crowd range in Equation (2) is replaced with the magnitude of the movement vector. Specifically, the closure score calculation unit 140 calculates the closure score using Equation (3).

[0066] Closed score = α × (magnitude of the movement vector of 100-A) + β × (magnitude of the movement vector of 100-B) + γ × (magnitude of the movement vector of 100-C) + δ × (magnitude of the movement vector of 100-D) (3)

[0067] In this way, the closed score calculation unit 140 may calculate the magnitude of each movement vector and calculate the closed score using a formula expressed by each label, the weight corresponding to each label, and the magnitude of each movement vector.

[0068] Furthermore, the closed score calculation unit 140 may calculate the closed score using a trained model that has learned the relationship between the size of the crowd range or movement vector of each label and information indicating whether to close the door. Note that the neural network that constitutes the trained model may be a known neural network such as a DNN, a CNN, or an RNN.

[0069] According to the second embodiment, the control device 100 uses the label to determine whether to close the door. This allows the control device 100 to make a more appropriate determination. Furthermore, as described above, by adding the label, the control device 100 can input the movement vector of the label related to getting in and out of the vehicle into the trained model.

[0070] <Learning Phase> The learning device 500 generates a trained model that outputs a closed score when each labeled movement vector is input.

[0071] Embodiment 3 Next, embodiment 3 will be described. In embodiment 3, differences from embodiment 1 will be mainly described. Furthermore, in embodiment 3, descriptions of matters common to embodiment 1 will be omitted.

[0072] 9 is a block diagram showing the functions of the control device according to the third embodiment. The control device 100 further includes a detection unit 180. Part or all of the detection unit 180 may be implemented by a processing circuit. Alternatively, part or all of the detection unit 180 may be implemented as a program module executed by the processor 101.

[0073] The detection unit 180 detects attributes of people included in an image based on one or more images constituting the video acquired by the acquisition unit 120. Specifically, the detection unit 180 detects attributes of people included in the image using a known image recognition technique. For example, the detection unit 180 detects the attributes using the image and a trained model.

[0074] The attributes will now be described. For example, the attributes may be age, group, etc. For example, a group may be parent and child, a group of students, etc. The attributes may also be wheelchair use, white cane use, stroller use, suitcase possession, etc.

[0075] The conversion unit 150 converts the information indicated by the attribute into a numerical value. For example, the conversion unit 150 performs the conversion using a table. The specific conversion process will be described below.

[0076] If the attribute indicates an elderly age, the train doors or platform doors need to be kept open for a long time. That is, because elderly people have low mobility, the train doors or platform doors need to be kept open for a long time. Therefore, for example, if the attribute indicates an elderly age, the conversion unit 150 converts the information indicated by the attribute to "1.2".

[0077] If the group indicated by the attribute is a parent and child, there is a possibility that the child may suddenly start running and the parent may chase after him. Therefore, the train doors or platform doors need to be kept open for a long time. For example, if the group indicated by the attribute is a parent and child, the conversion unit 150 converts the information indicated by the attribute into "1.1".

[0078] For example, if the age indicated by the attribute is the age of adulthood, the conversion unit 150 converts the information indicated by the attribute into "1.0."

[0079] The threshold determination unit 160 determines a threshold using a numerical value corresponding to the traffic management information, a numerical value corresponding to the station-related information, and a numerical value obtained by converting the information indicated by the attributes. For example, the threshold determination unit 160 determines a threshold by adding a numerical value corresponding to the traffic management information, a numerical value corresponding to the station-related information, and a numerical value obtained by converting the information indicated by the attributes. The determination process will be explained using a specific example. The numerical values ​​corresponding to the traffic management information are assumed to be "1.0" (no difference in timetable information), "1.3" (railway signal information is "ongoing"), and "1.0" (no delay in operation status information). The numerical values ​​corresponding to the station-related information are assumed to be "1.2" (island station structure) and "1.0" (station staff are stationed near the doors). The numerical value obtained by converting the information indicated by the attributes is assumed to be "1.0" (age of adulthood). The threshold determination unit 160 determines 6.5 (= 1.0 + 1.3 + 1.0 + 1.2 + 1.0 + 1.0) as the threshold.

[0080] Fig. 10 is a flowchart (part 1) showing an example of processing executed by the control device of embodiment 3. The processing in Fig. 10 differs from the processing in Fig. 3 in that steps S17a and S17b are executed. Therefore, steps S17a and S17b will be described in Fig. 10. Description of processing other than steps S17a and S17b will be omitted.

[0081] (Step S17a) The detection unit 180 detects attributes related to people based on the image. (Step S17b) The conversion unit 150 converts the information indicated by the attributes into a state used when determining a threshold. Then, the process proceeds to step S18a.

[0082] Fig. 11 is a flowchart (part 2) illustrating an example of processing executed by the control device of embodiment 3. The processing in Fig. 11 differs from the processing in Fig. 3 in that step S18a is executed. Therefore, step S18a will be described in Fig. 11. Description of processing other than step S18a will be omitted.

[0083] (Step S18a) The threshold value determination unit 160 determines a threshold value using a numerical value corresponding to the traffic control information, a numerical value corresponding to the station-related information, and a numerical value obtained by converting the information indicated by the attribute.

[0084] According to the third embodiment, the control device 100 changes the threshold value depending on the type of passenger, thereby enabling the control device 100 to appropriately control the doors depending on the type of passenger.

[0085] Embodiment 4 Next, embodiment 4 will be described. In embodiment 4, differences from embodiment 1 will be mainly described. Furthermore, in embodiment 4, description of matters common to embodiment 1 will be omitted.

[0086] 12 is a block diagram showing the functions of the control device according to the fourth embodiment. The control device 100 further includes a processing unit 190. Part or all of the processing unit 190 may be implemented by a processing circuit. Alternatively, part or all of the processing unit 190 may be implemented as a program module executed by the processor 101.

[0087] The processing unit 190 masks areas other than those near train doors or platform doors for each of the multiple images that make up the video. For example, the masked areas are areas where people are heading toward ticket gates or entrances / exits, or areas where people are not present near the yellow tactile paving blocks. A specific example of masking processing will be described. The processing unit 190 identifies train doors or platform doors using image recognition technology. The processing unit 190 masks areas other than a predetermined range centered on the identified train doors or platform doors. As a result, only people getting on and off are present in each of the multiple images.

[0088] The processing unit 190 may mask areas other than the driver's gaze area for each of the multiple images constituting the video. For example, the driver's gaze area may be near the train door or platform door, near the escalator, near the elevator, near the stairs, near the pillar, etc. The reason the driver gazes near the escalator, near the elevator, and near the stairs is to check whether anyone is running to board the train. The reason the driver gazes near the pillar is to check whether anyone hiding behind the pillar is boarding. An example of masking processing will be described. The processing unit 190 uses image recognition technology to identify the escalator, elevator, staircase, or pillar. The processing unit 190 masks areas other than a predetermined area centered on the identified escalator, elevator, staircase, or pillar. As a result, only people involved in boarding and alighting are present in each of the multiple images. For example, people involved in boarding and alighting may be people running to board the train or people boarding from behind a pillar.

[0089] The movement vector calculation unit 130 calculates the movement vector of the crowd based on the processed video (i.e., the multiple processed images). This calculates the movement vectors of the crowd getting on and off. In other words, the movement vector calculation unit 130 does not calculate the movement vector of the crowd whose behavior is unrelated to getting on and off (for example, a crowd heading toward a ticket gate). This reduces the processing load on the control device 100.

[0090] Next, the processing executed by the control device 100 will be described using a flowchart. Fig. 13 is a flowchart showing an example of the processing executed by the control device of embodiment 4. The processing in Fig. 13 differs from the processing in Fig. 3 in that steps S11a and S12b are executed. Therefore, steps S11a and S12b will be described in Fig. 13. Further, a description of the processing other than steps S11a and S12b will be omitted.

[0091] (Step S11a) The processing unit 190 masks areas other than the areas near the train doors or platform doors for each of the multiple images that make up the video. (Step S12b) The movement vector calculation unit 130 calculates the movement vectors of the crowd based on the processed video.

[0092] According to the fourth embodiment, the control device 100 does not calculate the movement vectors of crowds whose behavior is unrelated to getting on and off. This allows the control device 100 to reduce the processing load. Furthermore, only the movement vectors of crowds who get on and off are input to the trained model. In other words, the movement vectors of crowds whose behavior is unrelated to getting on and off are not input to the trained model. Therefore, the estimation of the trained model is not influenced by the movement vectors of crowds whose behavior is unrelated to getting on and off. This improves the estimation accuracy of the trained model.

[0093] Embodiment 5 Next, embodiment 5 will be described. In embodiment 5, differences from embodiment 1 will be mainly described. Furthermore, in embodiment 5, descriptions of matters common to embodiment 1 will be omitted.

[0094] <Learning Phase> In the learning data of the first embodiment, a movement vector is associated with information indicating whether or not to close the door. The association is performed by the user. In the fifth embodiment, a case where the association is performed automatically will be described.

[0095] 14 is a block diagram showing the functions of the learning device of embodiment 5. The learning device 500 further includes a data generation unit 530. Part or all of the data generation unit 530 may be implemented by a processing circuit included in the learning device 500. Alternatively, part or all of the data generation unit 530 may be implemented as a program module executed by a processor included in the learning device 500. The functions of the data generation unit 530 will be described later.

[0096] Next, the processing executed by the learning device 500 will be described using a flowchart. Fig. 15 is a flowchart showing the processing executed by the learning device of embodiment 5. The processing in Fig. 15 differs from the processing in Fig. 5 in that steps S21a and 21b are executed. Therefore, steps S21a and 21b will be described in Fig. 15. Further, a description of the processing other than steps S21a and 21b will be omitted.

[0097] (Step S21a) The acquisition unit 510 acquires the movement vectors of the crowd calculated based on the video. The acquisition unit 510 also acquires information indicating whether the train doors or platform doors were closed at the time the video was generated from the railway operation system 300. Note that the information indicating whether the train doors or platform doors were closed can be generated based on the door operation history.

[0098] (Step S21b) The data generator 530 generates training data by associating the movement vector with the information indicating whether the train doors or platform doors were closed. That is, the data generator 530 generates the same data as the training data in the first embodiment.

[0099] In addition, the movement vector calculated based on the video when an obstacle is confirmed between the platform doors and the train may be associated with information indicating that the platform doors should be closed.

[0100] According to the fifth embodiment, the learning device 500 automatically associates the movement vector with information indicating whether or not to close the door, thereby reducing the burden on the user.

[0101] The features of the above-described embodiments can be combined with each other as appropriate.

[0102] 10, 11, 12 Crowd, 10a, 11a, 12a Movement vector, 20a, 20b Platform door, 21 Movement vector, 22 Movement vector, 23 Movement vector, 24 Movement vector, 100 Control device, 101 Processor, 102 Volatile storage device, 103 Non-volatile storage device, 104 Input / output interface, 105 Communication interface, 110 Storage unit, 120 Acquisition unit, 130 Movement vector calculation unit, 140 Closed score calculation unit, 150 Conversion unit, 160 Threshold determination unit, 170 Judgment instruction unit, 180 Detection unit, 190 Processing unit, 200 Camera, 300 Railway operation system, 400 Door drive device, 500 Learning device, 510 Acquisition unit, 520 Model generation unit, 530 Data generation section.

Claims

1. A control device for controlling a door driving device that controls train doors or station platform doors, comprising: an acquisition unit that acquires video obtained by photographing a platform; a movement vector calculation unit that calculates a movement vector of a crowd contained in the video based on the video; a closed score calculation unit that uses the movement vector to calculate a numerical closed score; and a determination and instruction unit that uses the closed score and a threshold value to determine whether or not to close the door or the platform door, and instructs the door driving device to close the door or the platform door if the closed score is greater than the threshold.

2. The control device according to claim 1, wherein the acquisition unit acquires a trained model, and the closed score calculation unit calculates the closed score using the trained model and the movement vector.

3. The control device described in claim 2, wherein the movement vector calculation unit calculates each movement vector of each crowd included in the video and labels each movement vector, and the closed score calculation unit calculates the closed score using the movement vector of a label related to boarding and disembarking and the learned model.

4. The control device described in claim 2, further comprising a processing unit that masks areas other than those near the door or the platform door for each of a plurality of images constituting the video, and the movement vector calculation unit calculates the movement vector based on the processed video.

5. The control device described in claim 2, further comprising a processing unit that masks areas other than the driver's gaze area for each of the multiple images that make up the video, and the movement vector calculation unit calculates the movement vector based on the processed video.

6. The control device described in claim 1, wherein the movement vector calculation unit calculates each movement vector of each crowd included in the image and labels each movement vector, the closed score calculation unit sets a value indicating the presence or absence of people flow to each label and calculates the closed score using a formula represented by each label and a weight corresponding to each label, the weight related to boarding and alighting indicates a value of 0 or more, and the weight unrelated to boarding and alighting indicates 0.

7. The control device according to claim 6, wherein the closed score calculation unit calculates the range of the crowd in the image on which each movement vector is calculated or the magnitude of each movement vector, and calculates the closed score using a formula represented by each label, the weight corresponding to each label, and the range of the crowd on which each movement vector is calculated or the magnitude of each movement vector.

8. A control device as described in any one of claims 1 to 7, further comprising a threshold determination unit, wherein the acquisition unit acquires a numerical value corresponding to operation management information, which is information regarding operations, and a numerical value corresponding to station-related information, which is information related to stations, and the threshold determination unit determines the threshold value using the numerical value corresponding to the operation management information and the numerical value corresponding to the station-related information.

9. The control device described in claim 8, further comprising: a detection unit that detects attributes related to people included in one or more images constituting the video, based on the images; and a conversion unit that converts information indicated by the attributes into a numerical value, wherein the threshold determination unit determines the threshold value using a numerical value corresponding to the traffic management information, a numerical value corresponding to the station-related information, and a numerical value obtained by converting the information indicated by the attributes.

10. A learning device having: an acquisition unit that acquires crowd movement vectors calculated based on video and information indicating whether train doors or station platform doors were closed at the time the video was generated; and a data generation unit that generates learning data by associating the movement vectors with the information indicating whether the doors or platform doors were closed.

11. A control method in which a control device that controls a door driving device that controls train doors or station platform doors, acquires video obtained by photographing a platform, calculates a movement vector of a crowd contained in the video based on the video, calculates a numerical closure score using the movement vector, determines whether to close the door or the platform door using the closure score and a threshold value, and instructs the door driving device to close the door or the platform door if the closure score is greater than the threshold value.

12. A learning method, in which a learning device acquires crowd movement vectors calculated based on a video and information indicating whether train doors or station platform doors were closed at the time the video was generated, and generates learning data by associating the movement vectors with the information indicating whether the doors or platform doors were closed.

13. A control program that causes a control device that controls a door driving device that controls train doors or station platform doors to execute the following processes: acquire imagery obtained by photographing a platform, calculate a movement vector of a crowd contained in the image based on the image, use the movement vector to calculate a numerical closure score, use the closure score and a threshold value to determine whether or not to close the door or the platform door, and instruct the door driving device to close the door or the platform door if the closure score is greater than the threshold value.

14. A learning program that causes a learning device to execute a process of acquiring crowd movement vectors calculated based on video and information indicating whether train doors or station platform doors were closed at the time the video was generated, and generating learning data by associating the movement vectors with the information indicating whether the doors or platform doors were closed.

Images