Camera extracting device, video display system, and camera extracting method

Abstract

The objective of the present invention is to provide technology capable of reducing the time required to extract a related camera. This camera extracting device comprises: a position updating unit for updating positions in a moving camera model; a camera model linking unit for linking the moving camera model and a fixed camera model on the basis of the updated positions in the moving camera model and positions in the fixed camera model; and a camera extracting unit for extracting a related camera, which is a camera associated with a designated camera, on the basis of the result of the linking between the moving camera model and the fixed camera model.

Description

Camera extraction device, video display system, and camera extraction method

[0001] The present disclosure relates to a camera extraction device, a video display system, and a camera extraction method.

[0002] In recent years, a technique has been proposed for displaying images captured by a mobile camera mounted on a train and images captured by a fixed camera mounted on the ground on a display device (for example, Patent Documents 1 and 2).

[0003] International Publication No. 2022 / 045166 Japanese Patent Application Laid-Open No. 2022-046277

[0004] However, the conventional technology has a problem in that it is not possible to quickly extract related cameras that are related to a designated camera specified by a user operation from both mobile and fixed cameras, which results in a problem in that it is not possible to quickly display images captured by the related cameras, for example.

[0005] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technology that can reduce the time required to extract related cameras.

[0006] The camera extraction device of the present disclosure includes a memory unit that stores a mobile camera model that associates mobile cameras, whose positions change with those of a train, with each other in terms of their positions, and a fixed camera model that associates fixed cameras, which are installed on the ground and have fixed positions, with each other in terms of their positions; a position update unit that updates the positions in the mobile camera model; a camera model linking unit that links the mobile camera model and the fixed camera model based on the updated positions in the mobile camera model and the updated positions in the fixed camera model; and a camera extraction unit that extracts related cameras that are cameras that are related to a specified camera based on the linking results between the mobile camera model and the fixed camera model.

[0007] According to the present disclosure, the moving camera model and the fixed camera model are linked based on the updated position of the moving camera model and the position of the fixed camera model, and based on the linking result, related cameras that are cameras related to the designated camera are extracted. With this configuration, it is possible to reduce the time required to extract related cameras. Objects, features, aspects, and advantages of the present disclosure will become more apparent from the following detailed description and the accompanying drawings.

[0008] 1 is a block diagram showing the configuration of a video display system according to embodiment 1. FIG. 2 is a diagram schematically showing a real train. FIG. 3 is a diagram showing a moving camera model according to embodiment 1. FIG. 4 is a diagram showing a moving camera model according to embodiment 1. FIG. 5 is a diagram showing a real railroad. FIG. 6 is a diagram showing a fixed camera model according to embodiment 1. FIG. 7 is a diagram showing a fixed camera model according to embodiment 1. FIG. 8 is a diagram showing a moving camera model according to embodiment 1. FIG. 9 is a diagram showing a linkage model according to embodiment 1. FIG. 10 is a flowchart showing the operation of a camera extraction device according to embodiment 1. FIG. 11 is a flowchart showing the operation of a camera extraction device according to variant 1 of embodiment 1. FIG. 12 is a block diagram showing the configuration of a video display system according to variant 2 of embodiment 1. FIG. 13 is a diagram showing an example display of a display device according to embodiment 2. FIG. 14 is a diagram for explaining the operation of a video display system according to variant 2 of embodiment 2. FIG. 15 is a diagram showing an example display of a display device according to variant 2 of embodiment 2. FIG. 16 is a block diagram showing the hardware configuration of a camera extraction device according to another variant. FIG. 17 is a block diagram showing the hardware configuration of a camera extraction device according to another variant.

[0009] <First Embodiment> Fig. 1 is a block diagram showing the configuration of a video display system according to a first embodiment. The video display system of Fig. 1 includes a camera extraction device 1, an input device 2, a video data storage unit 3, and a display device 4.

[0010] The input device 2 is a device that accepts operations from a user, and includes, for example, a keyboard, a mouse, etc. The operations from the user include, for example, an operation to specify the ID of the camera, and operations on various buttons.

[0011] The video data storage unit 3 sequentially stores video data acquired by mobile cameras whose positions change with the train and video data acquired by fixed cameras installed on the ground (for example, in station premises and facilities) whose positions are fixed. The mobile cameras may be installed on the train or in a central control room. The video data includes not only the images captured by each camera, but also additional information about each camera, such as management system information, network address information, and API (Application Programming Interface) query parameters.

[0012] The camera extraction device 1 extracts a designated camera and associated cameras related to the designated camera from among the moving cameras and fixed cameras. In the first embodiment, the designated camera is a first camera designated based on an operation received from a user via the input device 2, but as will be described later, the designated camera is not limited to this. The camera extraction device 1 will be described in detail later.

[0013] The display device 4 uses the additional information of the designated camera and the related cameras to acquire the video data of the designated camera and the related cameras from the video data in the video data storage unit 3, and displays the video included in the video data. This allows the display device 4 to not only display the video taken by the designated camera, but also to display the video taken by the related camera as video taken by a new designated camera.

[0014] <Camera Extraction Device> The camera extraction device 1 according to the first embodiment includes a storage unit 11 and a control unit 12. The control unit 12 includes a position update unit 12a, a camera model linkage unit 12b, a camera extraction unit 12c, and a display control unit 12d.

[0015] The storage unit 11 is, for example, a memory described below. The storage unit 11 stores a mobile camera model in which mobile cameras are associated with each other based on their hierarchical positions, and a fixed camera model in which fixed cameras are associated with each other based on their hierarchical positions.

[0016] First, the moving camera model will be explained. Figure 2 is a diagram that shows a model of an actual train. The train in Figure 2 is made up of three cars 31, each numbered 1 to 3, starting from the front. Moving cameras 21 facing the outside of the car 31 are provided at the front and rear of each car 31, and moving cameras 21 facing the inside of the car 31 are provided near the doors 32 of each car 31.

[0017] Figure 3 is a diagram showing a moving camera model obtained by modeling the actual object in Figure 2. In the moving camera model in Figure 3, vehicle numbers, which are concepts of the position of the moving camera, are associated. In the moving camera model in Figure 3, concepts of the position of the moving camera are associated with each other by a tree structure, and the concepts of the position of the moving camera are applied to nodes indicated by circles, etc. As a general rule, a concept lower in level than the concept of the parent node at the end of the arrow is applied to the child node at the tip of the arrow of the edge that associates the nodes.

[0018] Figure 4 is a diagram showing a moving camera model for vehicle 31 with vehicle number 1 in Figure 3. The terminal node of the moving camera model (the octagonal node in Figure 8) indicates the node for the moving camera. In the moving camera model in Figure 4, the moving cameras are associated with each other in terms of the installation location and camera orientation, which are superordinate concepts of the position of the moving camera and subordinate concepts of the vehicle number.

[0019] Next, the fixed camera model will be described. Fig. 5 is a diagram showing a schematic diagram of an actual railway track on which a train travels. The railway track in Fig. 5 is defined with a plurality of sections, each section being a fixed distance apart. Note that the sections are not limited to sections of a fixed distance, and may be block sections used in general train control, or electrical sections such as track circuits.

[0020] In the example of FIG. 5 , Station A is provided in Section (1). Section (2) is provided with a track connecting Section (1) and Section (3) and Facility X. Section (3) is provided with a first track connecting Section (2) and Section (4) and a second track branching from the first track and connecting Section (2) and Section (6). Section (4) is provided with a track connecting Section (3) and Section (5) and a railroad crossing Y. Station B is provided in Section (5). Section (6) is provided with a track connecting Section (3) and Section (7). Station C is provided in Section (7).

[0021] Fig. 6 is a diagram showing a fixed camera model that models the actual object shown in Fig. 5. In the fixed camera model shown in Fig. 6, sections, which are the concept of the positions of fixed cameras, and installed facilities are associated with each other. The installed facilities are facilities installed or laid on the tracks where fixed cameras are installed, and include, for example, stations, railroad crossings, signals, tunnels, distribution boards, and equipment.

[0022] In the fixed camera model of FIG. 6 , as in the moving camera model, the concepts of fixed camera positions are associated with each other through a tree structure. However, in the first embodiment, the direction of the arrows of the edges connecting the section nodes does not correspond to the hierarchy of the concepts, but corresponds to the direction in which the train moves. For example, in the fixed camera model of FIG. 6 , it is specified that the train moves in the order of section (1), section (2), section (3), section (4), and section (5), and it is also specified that the train moves in the order of section (1), section (2), section (3), section (6), and section (7). Furthermore, the parent node of an installed facility is applied to the section in which the installed facility is located.

[0023] Figure 7 is a diagram showing a fixed camera model for Station A in Figure 6. The terminal nodes of the fixed camera model (octagonal nodes in Figure 7) indicate fixed camera nodes. In the fixed camera model in Figure 7, fixed cameras are associated with each other by their installation locations, which are a higher-level concept of the fixed camera's location and a lower-level concept of the facility (station) where the camera is installed. The example in Figure 7 shows that three fixed cameras are installed on platform 1 of Station A, four fixed cameras are installed on platform 2 of Station A, and one fixed camera is installed at the ticket gate of Station A.

[0024] Note that the association between fixed cameras and between mobile cameras is not limited to the above. For example, as shown in FIG. 8, in a mobile camera model, nodes with different car numbers but the same camera orientation may be associated with each other. In this case, the direction of the edge arrow may correspond to the direction in which the car number increases or the direction in which the car number decreases. Similarly, for example, in a fixed camera model, nodes on platforms at different stations but with trains going to the same destination may be associated with each other. Furthermore, fixed cameras and mobile cameras may be associated with each other not by a tree structure but by a directed graph or a table.

[0025] As will be described in the third embodiment, not only the positions of fixed cameras and mobile cameras but also the attributes of fixed cameras and mobile cameras may be applied to nodes. In such a case, stations and installed facilities may be included not only in the concepts of the positions of fixed cameras and mobile cameras but also in the concepts of the attributes of fixed cameras and mobile cameras.

[0026] The position update unit 12a in Fig. 1 updates the position in the moving camera model. In the first embodiment, the position update unit 12a updates the position in the moving camera model based on the position detected by a position detection device that moves together with the moving camera. The position detection device is, for example, a global positioning system (GPS) provided on the moving camera or a train.

[0027] For example, if a GPS is provided in one moving camera, the position detected by the GPS is saved as the position of one moving camera in the moving camera model. Then, the relative positions of other moving cameras and trains relative to the GPS position are added to the position detected by the GPS and saved as the positions of the other moving cameras and trains in the moving camera model. The relative positions may be saved in table format, or may be saved as the coordinates of the train position with the GPS position as the origin.

[0028] On the other hand, if a GPS is installed on the train, the position detected by the GPS is saved as the position of the train in the moving camera model.Then, the relative position of the moving camera with respect to the GPS position is added to the position detected by the GPS and saved as the position of the moving camera in the moving camera model.The relative position may be saved in table format or may be saved as the coordinates of the moving camera position with the GPS position as the origin.

[0029] In the moving camera model as shown in Figure 9, the node for the moving camera stores the position (coordinates such as latitude and longitude) and ID of the moving camera, and the node for the train stores the position (coordinates such as latitude and longitude) of the train. Updating the position in the moving camera model by the position update unit 12a corresponds to updating the positions (coordinates such as latitude and longitude) of the moving camera and train in the moving camera model. Note that although simplified in the moving camera model of Figure 9, the positions (coordinates such as latitude and longitude) of the vehicle number and installation location may also be stored in nodes such as the vehicle number and installation location.

[0030] Although not shown, in a fixed camera model, the node for a fixed camera stores the position (coordinates such as latitude and longitude) and ID of the fixed camera, and the node for a section stores the position of the section (coordinates such as latitude and longitude). In a fixed camera model, both the position of the fixed camera and the position of the section may be stored, or one of the position of the fixed camera and the position of the section may be stored, and the other may be calculated by adding the relative position of the other. Furthermore, the position of the station or the like (coordinates such as latitude and longitude) may also be stored in a node for a station or the like.

[0031] 1 links the mobile camera model and the fixed camera model based on the updated position in the mobile camera model and the position in the fixed camera model. For example, the camera model linking unit 12b calculates the distance between the train and the section based on the position of the train and the position of the section, and links the mobile camera model and the fixed camera model by linking the train and the section for which the distance is shortest. As a result of the linking, the camera model linking unit 12b generates a linked model in which the mobile camera model and the fixed camera model are linked and associated.

[0032] Fig. 10 is a diagram showing an association model according to the present embodiment 1. In the association model of Fig. 10, trains a, b, and c are associated with sections (1), (3), and (5), respectively.

[0033] In the linkage model of Fig. 10, only the train node of the moving camera model is associated with the fixed camera model. However, in the linkage model, the train and moving camera nodes, or all types of nodes of the moving camera model, may be associated with the fixed camera model. In addition, in the linkage model, trains traveling in the same section may be associated in a list format in order of the train's movement direction.

[0034] 10, only the nodes of the section and the installed facility of the fixed camera model are associated with the mobile camera model. However, in the linked model, the nodes of the section, the installed facility, and the fixed camera, or all the nodes of the fixed camera model, may be associated with the mobile camera model.

[0035] As described above, the linked model may be any model in which at least a part of a moving camera model and at least a part of a fixed camera model are linked.

[0036] 1 extracts related cameras that are cameras related to a designated camera based on the linkage results between the mobile camera model and the fixed camera model. In the first embodiment, the camera extraction unit 12c first extracts, as the designated camera, a camera whose ID matches that of the designated camera from among the mobile cameras and fixed cameras included in the linkage model as shown in FIG.

[0037] Then, for each of the multiple cameras in the collaboration model, including mobile cameras and fixed cameras, the camera extraction unit 12c determines the shortest path, which is the path that passes through the fewest number of nodes from the designated camera through the collaboration model to reach the camera.The camera extraction unit 12c then identifies, as a candidate camera, the camera with the fewest number of nodes on the shortest path among the multiple cameras.Based on the positions of the designated camera and the candidate camera, the camera extraction unit 12c extracts the candidate camera as a related camera if the distance between the designated camera and the candidate camera is equal to or less than a threshold.

[0038] The camera extraction unit 12c then identifies, as a new candidate camera, the camera with the next fewest number of nodes on the shortest path after the candidate camera, from among the multiple cameras consisting of mobile cameras and fixed cameras in the collaboration model. The camera extraction unit 12c repeats the above process until the number of nodes from the designated camera to the candidate camera exceeds a certain number.

[0039] With this configuration, mobile cameras and fixed cameras can be searched for simultaneously. For example, mobile cameras and fixed cameras in a section adjacent to the section in which the designated camera is located can be extracted as related cameras. The method by which the camera extraction unit 12c extracts related cameras based on the linkage results is not limited to the method using the shortest path as described above. Furthermore, the camera extraction unit 12c may update the threshold value used to compare the distance between the designated camera and the candidate camera to the distance value when the related camera was extracted. With this configuration, the camera extraction unit 12c can extract the candidate camera that is closest to the designated camera as the related camera.

[0040] The display control unit 12d controls the display device 4 to appropriately display the images from the designated camera and the related cameras on the display device 4. The display control unit 12d may be provided in the display device 4 instead of the camera extraction device 1.

[0041] <Operation> Fig. 11 is a flowchart showing the operation of the camera extraction device 1 according to the present embodiment 1. In the present embodiment 1, the position update unit 12a sequentially updates the position in the moving camera model in parallel with the operation in Fig. 11, but this may be performed immediately before the linking process in step S2.

[0042] First, in step S1, the camera extraction device 1 determines whether or not the input device 2 has received an operation from the user to specify the ID of a camera to be designated as a specified camera. If it is determined that the operation has been received, the process proceeds to step S2, and if it is determined that the operation has not been received, the process of step S1 is performed again.

[0043] In step S2, the camera model linking unit 12b links the moving camera model and the fixed camera model based on the updated positions in the moving camera model and the fixed camera model.

[0044] In step S3, the camera extraction unit 12c extracts related cameras that are cameras that are related to the designated camera based on the result of linking the moving camera model and the fixed camera model.

[0045] In step S4, the display control unit 12d controls the display device 4 to appropriately display the video from the associated camera on the display device 4.

[0046] Summary of First Embodiment According to the camera extraction device 1 of the first embodiment described above, the camera model linking unit 12b links the mobile camera model and the fixed camera model based on the updated position of the mobile camera model and the updated position of the fixed camera model. This configuration facilitates linking between the mobile camera and the fixed camera. Furthermore, in the first embodiment, the camera extraction unit 12c extracts related cameras, which are cameras related to the designated camera, based on the linking result between the mobile camera model and the fixed camera model. This configuration reduces the time required to extract related cameras from the mobile cameras and the fixed cameras, thereby enabling quick display of footage captured by the related cameras.

[0047] In the first embodiment, in the moving camera model, the moving cameras are related to each other by a tree structure or a directed graph, and in the fixed camera model, the fixed cameras are related to each other by a tree structure or a directed graph. With this configuration, the camera models can be structured using a graph database or the like, which makes it possible to speed up the extraction of related cameras and facilitate the linking of camera models.

[0048] In the first embodiment, the camera model linking unit 12b generates a linking model in which at least a part of the moving camera model and at least a part of the fixed camera model are linked as a linking result. With this configuration, the processing load of the camera extraction device 1 can be reduced.

[0049] Furthermore, in the first embodiment, the camera model linking unit 12b links the moving camera model and the fixed camera model when the user performs an operation. With this configuration, linking is not performed more than necessary, so that the communication load and processing load can be reduced. Note that the camera model linking unit 12b may also be configured to link the moving camera model and the fixed camera model when the train moves. With this configuration, the communication load and processing load can also be reduced.

[0050] In the mobile camera model, the mobile cameras may be associated with each other based on the tracks, and in the fixed camera model, the fixed cameras may be associated with each other based on the tracks and the installed facilities. With this configuration, it is easy to link the mobile cameras and the fixed cameras.

[0051] <First Variation of First Embodiment> The camera model linking unit 12b may link the moving camera model and the fixed camera model based on the updated position in the moving camera model, the position in the fixed camera model, and the ID of the designated camera.

[0052] FIG. 12 is a flowchart showing the operation of the camera extraction device 1 according to the first modification.

[0053] In step S11, the camera extraction device 1 determines whether or not the input device 2 has received an operation from the user to specify the ID of a camera to be designated as the designated camera. If it is determined that the operation has been received, the process proceeds to step S12, and if it is determined that the operation has not been received, the process of step S11 is performed again.

[0054] In step S12, the camera model linking unit 12b determines whether the ID of the designated camera matches that of the moving camera of the moving camera model or the fixed camera of the fixed camera model. If it is determined that the ID of the designated camera matches that of the moving camera, the process proceeds to step S13, and if it is determined that the ID of the designated camera matches that of the fixed camera, the process proceeds to step S15.

[0055] In step S13, the camera model linking unit 12b acquires, from the moving camera model, the position of a train moving with a moving camera whose ID matches that of the designated camera. In step S14, the camera model linking unit 12b acquires, from the fixed camera model, the section closest to the train, and associates the train with that section, thereby linking the moving camera model and the fixed camera model. Then, the process proceeds to step S17.

[0056] In step S15, the camera model linking unit 12b acquires, from the fixed camera model, the position of a section where a fixed camera having an ID matching that of the designated camera is installed. In step S16, the camera model linking unit 12b acquires, from the moving camera model, the train closest to the section, and associates the train with the section, thereby linking the moving camera model and the fixed camera model. Then, the process proceeds to step S17.

[0057] In step S17, the camera extraction unit 12c extracts related cameras that are cameras that are related to the designated camera based on the result of linking the moving camera model and the fixed camera model.

[0058] In step S18, the display control unit 12d controls the display device 4 to appropriately display the video from the associated camera on the display device 4.

[0059] According to the camera extraction device 1 of the present modified example 1, the moving camera model and the fixed camera model are linked not for all trains and sections, but for trains and sections that are highly necessary for extracting the designated camera. With this configuration, it is possible to reduce the processing load associated with linking the moving camera model and the fixed camera model.

[0060] <Modification 2 of Embodiment 1> Fig. 13 is a block diagram showing the configuration of a video display system according to Modification 2. In the configuration of Fig. 13, the camera extraction device 1 is connected to a train control system 71. The train control system 71 includes an operation management device that manages the positions and operation status of trains, and performs automatic train operation based on the positions and operation status of trains.

[0061] In the above-described first embodiment, the position update unit 12a updated the position in the moving camera model based on the position detected by a position detection device such as a GPS that moves along with the moving camera. In contrast, in the present second modification, the position update unit 12a updates the position in the moving camera model based on the train position managed by the train control system 71. Even with this configuration, the same effects as those of the first embodiment can be obtained. Furthermore, the video display system according to the present second modification can be easily applied to systems that are currently in widespread use, which is expected to reduce implementation costs and improve reliability.

[0062] The position update unit 12a may update the position in the moving camera model based on both the position detected by a position detection device that moves together with the moving camera and the position of the train managed by the train control system 71. Furthermore, this modification 2 may be combined with the above-mentioned modification 1.

[0063] <Third Modification of First Embodiment> In the first embodiment, the designated camera is the first camera designated based on an operation received from the user via the input device 2, but it may be a second camera designated based on the analysis result of the video. The second camera may be, for example, a camera that captures video that is the subject of an alarm as a result of the analysis of the video.

[0064] The display control unit 12d, which displays the video from the designated camera on the display device 4, may select either the first camera or the second camera as the designated camera based on the train operation status managed by the train control system 71. With this configuration, the designated camera suitable for the operation status can be used.

[0065] Furthermore, the display control unit 12d may perform machine learning (training) such as deep learning to select a designated camera from the first camera and the second camera, and may select a designated camera using the learning results. The machine learning may be, for example, supervised learning using responses from the user as training data, unsupervised learning, or reinforcement learning. With this configuration, it is possible to use a designated camera that matches the user's usage trends.

[0066] <Second Embodiment> A block diagram showing the configuration of a video display system according to a second embodiment is similar to the block diagram in Fig. 13. Hereinafter, among the components according to the second embodiment, components that are the same as or similar to the components described above will be assigned the same or similar reference numerals, and different components will be mainly described.

[0067] In the first embodiment described above, in the moving camera model, the moving cameras are associated with each other based on their positions, and in the fixed camera model, the fixed cameras are associated with each other based on their positions. In the second embodiment, in the moving camera model, the moving cameras are further associated with each other based on their attributes, and in the fixed camera model, the fixed cameras are further associated with each other based on their attributes.

[0068] The attributes according to the second embodiment include at least one of the following for mobile and fixed cameras: type, purpose, subject, installation facility, installation location, attitude (e.g., direction and angle), resolution, number of effective pixels, refresh rate, angle of view, presence or absence of built-in sensor, presence or absence of microphone, year of manufacture, and manufacturer. Note that in this specification, for example, at least one of A, B, C, ..., and Z means any one of all combinations of one or more types extracted from the group A, B, C, ..., and Z.

[0069] The camera extraction unit 12c according to the second embodiment generates the degrees of association between the designated camera and the mobile and fixed cameras based on the linkage result and the attributes of the mobile and fixed cameras. The camera extraction unit 12c will be described below.

[0070] First, the camera extraction unit 12c extracts, as the designated camera, a camera whose ID matches that of the designated camera from among the mobile cameras and fixed cameras included in the collaboration model. Then, for each of the multiple cameras consisting of mobile cameras and fixed cameras in the collaboration model, the camera extraction unit 12c determines, as the shortest path, the path that passes through the fewest number of nodes from the designated camera through the collaboration model to reach the camera. Then, the camera extraction unit 12c identifies, as the candidate camera, the camera among the multiple cameras that has the fewest number of nodes on the shortest path. The camera extraction unit 12c generates a degree of association between the designated camera and the candidate cameras based on the position and attributes of the designated camera and the position and attributes of the candidate cameras.

[0071] For example, the camera extraction unit 12c calculates the difference (distance) in position between the designated camera and the candidate camera, and acquires a weight from the calculated difference using a function capable of calculating a weight from the difference or a table in which the difference and the weight are associated. Similarly, the camera extraction unit 12c calculates the difference in attributes between the designated camera and the candidate camera, and acquires a weight from the calculated difference. The smaller the difference, the larger the weight. The camera extraction unit 12c generates the relevance degree as the sum of the weight acquired for the difference in position and the weight acquired for the difference in attributes. For example, if the attributes include camera resolution and the weight is set to increase as the camera resolution increases, the camera extraction unit 12c can increase the relevance degree of a related camera with a higher resolution.

[0072] The camera extraction unit 12c extracts the candidate camera as a related camera when the relevance is equal to or greater than a threshold. Then, the camera extraction unit 12c identifies, as a new candidate camera, the camera with the next fewest number of nodes on the shortest path from the candidate camera among the multiple cameras consisting of mobile cameras and fixed cameras in the collaboration model. The camera extraction unit 12c repeats the above process until the number of nodes from the designated camera to the candidate camera exceeds a certain number. Note that, as in the first embodiment, the method by which the camera extraction unit 12c extracts a related camera based on the collaboration result and the attributes of the mobile camera and the fixed camera is not limited to the method using the shortest path as described above. Furthermore, the camera extraction unit 12c may update the threshold used to compare the relevance to the relevance value at the time the related camera was extracted.

[0073] The display control unit 12d simultaneously displays the images captured by the designated camera and the multiple associated cameras on the display device 4. Then, the display control unit 12d displays the images captured by the multiple associated cameras on the display device 4 in descending order of relevance.

[0074] 14 is a diagram showing a display example of display device 4 according to the second embodiment. The screen displayed on display device 4 includes video areas 41a to 41g and display candidate area 42. Video area 41a is larger than video areas 41b and 41c, which are larger than video areas 41d to 41g. The number of video areas is not limited to seven, and may be any number as long as it is plural.

[0075] Video area 41a displays video captured by the designated camera. Video areas 41b to 41g display video captured by six related cameras in descending order of relevance. For example, if the attributes include camera resolution and the weight is set to increase as the camera resolution increases, camera extraction unit 12c can preferentially display video captured by related cameras with higher resolutions in larger video areas 41b and 41c.

[0076] A list of the facilities and locations where the related cameras are installed, or a map with icons of the related cameras, is displayed in the display candidate area 42. When an operation to specify a related camera displayed in the display candidate area 42 is performed, the display device 4 displays the video of the related camera in one of the video areas 41b to 41g.

[0077] Summary of Second Embodiment According to the camera extraction device 1 of the second embodiment described above, the camera extraction unit 12c generates a degree of association between the designated camera and the mobile and fixed cameras, which is used to extract related cameras, based on the linkage result and the attributes of the mobile and fixed cameras. The attributes include at least one of the type, purpose, subject, installation facility, installation location, attitude, resolution, number of effective pixels, refresh rate, angle of view, presence or absence of built-in sensor, presence or absence of microphone, year of manufacture, and manufacturer of the mobile and fixed cameras.

[0078] According to this configuration, for example, the camera extraction unit 12c can adjust the weight to increase the relevance of a camera that is far from the designated camera but has the same subject or posture as the designated camera, thereby preferentially displaying the video from that camera. Furthermore, for example, the camera extraction unit 12c can adjust the weight to increase the relevance of a camera that is far from the designated camera but is located at a station adjacent to the station where the designated camera is installed, thereby preferentially displaying the video from that camera. Furthermore, for example, the camera extraction unit 12c can adjust the weight to decrease the relevance of a camera that is close to the designated camera but is located on a train traveling on the same double track as the train on which the designated camera is installed, thereby preventing the video from being displayed. As described above, the camera extraction unit 12c can adjust the weight to display video from a camera that is far from the designated camera but is highly necessary, and prevent the display of video from a camera that is close to the designated camera but is less necessary.

[0079] The camera extraction unit 12c may perform machine learning (training) such as deep learning to generate the relevance and generate the relevance using the learning results. The machine learning may be, for example, supervised learning using responses from users as training data, unsupervised learning, or reinforcement learning. With this configuration, it is possible to generate relevance that matches the user's usage trends and switch the display.

[0080] In the second embodiment, in the moving camera model, the moving cameras are further associated with each other based on the attributes of the moving camera, and in the fixed camera model, the fixed cameras are further associated with each other based on the attributes of the fixed camera. With this configuration, it is easy to generate the degree of association.

[0081] In the second embodiment, the display control unit 12d, which simultaneously displays the images captured by the designated camera and the multiple associated cameras on the display device 4, displays the images from the multiple associated cameras in descending order of relevance on the display device 4. This configuration allows the user to simultaneously view multiple appropriate images.

[0082] <First Modification of Second Embodiment> In the second embodiment, the display control unit 12d causes the display device 4 to display the images captured by the multiple associated cameras based on the relevance, but this is not limited to this. For example, the display control unit 12d may cause the display device 4 to display the image captured by the single associated camera with the highest relevance.

[0083] These may also be applied to embodiment 1. That is, in embodiment 1, the display control unit 12d may cause the display device 4 to display images captured by multiple associated cameras based on the distance between the designated camera and the candidate camera, or may cause the display device 4 to display the image captured by the single associated camera at the shortest distance. In this way, the only difference between embodiment 1 and embodiment 2 is the algorithm for extracting associated cameras.

[0084] <Variation 2 of Embodiment 2> In this variation 2, the attributes include the attitudes (e.g., direction and angle) of the moving camera and the fixed camera. Fig. 15 is a diagram for explaining the operation of the video display system according to this variation 2, and is a top view of a station platform 51. Fig. 15 shows a train 52 consisting of three cars that can move in the vertical direction of Fig. 15, a pillar 53 attached to the platform 51, cameras 54a, 54b, and 54c attached to the pillar 53, and their shooting ranges 55a, 55b, and 55c. Note that camera 54a is a designated camera, and cameras 54b and 54c are associated cameras.

[0085] Fig. 16 is a diagram showing an example of a display on the display device 4 according to the second modification. The screen displayed on the display device 4 includes an image area 61 and buttons 62a and 62b. The image area 61 displays an image from the camera 54a, which is the designated camera by default. The button 62a is a button for switching the image in the image area 61 to an image from the camera 54b, which is mounted on the same pillar 53 as the camera 54a, and is shown by imaginary lines in Fig. 16. The button 62b is a button for switching the image in the image area 61 to an image from the camera 54c, which has the same attitude (orientation) as the camera 54a.

[0086] If the relevance of camera 54c is higher than that of camera 54b, the display device 4 is controlled by the display control unit 12d to display button 62b without displaying button 62a, as shown in FIG. 16 . In this case, the user can switch the image from camera 54a to the image from camera 54c shifted downward in FIG. 15 by turning on button 62b. On the other hand, if the relevance of camera 54b is higher than that of camera 54c, the display device 4 is controlled by the display control unit 12d to display button 62a without displaying button 62b. In this case, the user can switch the image from camera 54a to the image from camera 54b rotated counterclockwise in FIG. 15 by turning on button 62a.

[0087] The display control unit 12d, which displays the video of the designated camera on the display device 4, may be configured to switch the video of the related camera as the video of the new designated camera based on the relevance when the user operates it. For example, when the user operates one button that is always displayed instead of the buttons 62a and 62b, the display device 4 may display the video of the camera 54b or the camera 54c based on the comparison result between the relevance of the camera 54b and the relevance of the camera 54c.

[0088] Furthermore, the camera extraction unit 12c may generate the relevance based on the collaboration result, the attributes, and the train operation status managed by the train control system 71. For example, when the train 52 is present near the platform 51, the camera extraction unit 12c may prioritize display of the image from the camera 54c by setting the relevance of the camera 54c higher than the relevance of the camera 54b. For example, when the train 52 is not present near the platform 51, the camera extraction unit 12c may prioritize display of the image from the camera 54b by setting the relevance of the camera 54b higher than the relevance of the camera 54c.

[0089] Furthermore, the camera extraction unit 12c may perform machine learning (training) such as deep learning to generate the relevance used to switch the video of the newly specified camera, and may generate the relevance using the learning results. The machine learning may be, for example, supervised learning using responses from the user as training data, unsupervised learning, or reinforcement learning. With this configuration, it is possible to generate the relevance and switch the display in accordance with the user's usage trends.

[0090] <Other Modifications> The position update unit 12a, camera model linking unit 12b, and camera extraction unit 12c shown in FIG. 1 are hereinafter referred to as the "position update unit 12a, etc." The position update unit 12a, etc. are realized by a processing circuit 81 shown in FIG. 17. That is, the processing circuit 81 includes a position update unit 12a that updates the position of the moving camera model, a camera model linking unit 12b that links the moving camera model and the fixed camera model based on the updated position of the moving camera model and the position of the fixed camera model, and a camera extraction unit 12c that extracts related cameras that are cameras related to the designated camera based on the linking result between the moving camera model and the fixed camera model. The processing circuit 81 may be implemented by dedicated hardware or by a processor that executes a program stored in memory. Examples of the processor include a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, and a DSP (Digital Signal Processor).

[0091] When the processing circuit 81 is dedicated hardware, the processing circuit 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each unit, such as the position update unit 12a, may be realized by a circuit in which processing circuits are distributed, or the functions of each unit may be realized together by a single processing circuit.

[0092] When the processing circuit 81 is a processor, the functions of the position update unit 12a and the like are realized in combination with software and the like. Software and the like may include, for example, software, firmware, or software and firmware. The software and the like is written as a program and stored in memory. As shown in FIG. 18 , the processor 82 applied to the processing circuit 81 realizes the functions of each unit by reading and executing a program stored in memory 83. That is, the camera extraction device 1 includes a memory 83 for storing a program that, when executed by the processing circuit 81, results in the following steps: storing a moving camera model and a fixed camera model; updating the position of the moving camera model; linking the moving camera model and the fixed camera model based on the updated position of the moving camera model and the position of the fixed camera model; and extracting a related camera that is a camera related to the designated camera based on the linkage result between the moving camera model and the fixed camera model. In other words, this program can be said to cause a computer to execute the procedures and methods of the position update unit 12a and the like. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read Only Memory), a HDD (Hard Disk Drive), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), a drive device for any of these, or any storage medium to be used in the future.

[0093] The above describes a configuration in which each function of the location update unit 12a, etc. is realized either by hardware or software, etc. However, this is not limited to this, and a configuration in which part of the location update unit 12a, etc. is realized by dedicated hardware and another part is realized by software, etc. For example, the function of the location update unit 12a can be realized by a processing circuit 81 as dedicated hardware, and the other functions can be realized by the processing circuit 81 as a processor 82 reading and executing programs stored in a memory 83.

[0094] As described above, the processing circuitry 81 can realize the above-mentioned functions by hardware, software, or a combination of these.

[0095] The video display system described above may also be a system that appropriately combines a processing device, a communication terminal, the functions of an application installed on at least one of the processing device and the communication terminal, and a server. Communication terminals include, for example, mobile phones, smartphones, and tablets. The functions or components of the camera extraction device described above may be distributed among the devices that make up the system, or may be centralized in one of the devices. For example, the camera extraction device may be configured as a cloud server. This configuration is expected to distribute the processing load.

[0096] In this disclosure, 'a' and 'an' mean one or more. Therefore, 'a', 'an', 'one or more', and 'at least one' can be used interchangeably.

[0097] It should be noted that the embodiments and modifications may be freely combined, and the embodiments (and modifications) may be modified or omitted as appropriate. The above description is illustrative in all respects and is not limiting. It is understood that countless modifications not illustrated can be envisioned.

[0098] Various aspects of the present disclosure are summarized below as appendices.

[0099] (Supplementary Note 1) A camera extraction device comprising: a storage unit that stores a moving camera model in which moving cameras whose positions change with a train are associated with each other in terms of their positions, and a fixed camera model in which fixed cameras that are installed on the ground and whose positions are fixed are associated with each other in terms of their positions; a position update unit that updates the positions in the moving camera model; a camera model linking unit that links the moving camera model and the fixed camera model based on the updated positions in the moving camera model and the updated positions in the fixed camera model; and a camera extraction unit that extracts related cameras that are cameras that are related to a designated camera based on the linking results between the moving camera model and the fixed camera model.

[0100] (Supplementary Note 2) A camera extraction device according to Supplementary Note 1, wherein the position update unit updates the position in the moving camera model based on at least one of a position detected by the moving camera or a position detection device that moves together with the train, and a position of the train managed by an operation management device.

[0101] (Supplementary Note 3) The camera extraction device according to Supplementary Note 1 or Supplementary Note 2, wherein in the moving camera model, the moving cameras are further associated with each other based on attributes of the moving cameras, and in the fixed camera model, the fixed cameras are further associated with each other based on attributes of the fixed cameras.

[0102] (Supplementary Note 4) The camera extraction device according to any one of Supplementary Note 1 to Supplementary Note 3, wherein in the moving camera model, the moving cameras are associated with each other by a tree structure or a directed graph, and in the fixed camera model, the fixed cameras are associated with each other by a tree structure or a directed graph.

[0103] (Supplementary Note 5) The camera extraction device according to any one of Supplementary Notes 1 to 4, wherein the camera model linking unit generates, as the linking result, a linked model in which at least a part of the moving camera model and at least a part of the fixed camera model are linked.

[0104] (Supplementary Note 6) A camera extraction device according to Supplementary Note 1 or Supplementary Note 2, wherein the camera extraction unit generates a degree of association between the designated camera and the mobile camera and the fixed camera, which is used to extract the related camera, based on the collaboration result and attributes of the mobile camera and the fixed camera.

[0105] (Supplementary Note 7) The camera extraction device according to Supplementary Note 6, wherein the camera extraction unit performs machine learning to generate the relevance.

[0106] (Supplementary Note 8) The camera extraction device according to Supplementary Note 6 or Supplementary Note 7, wherein the attributes include at least one of the type, purpose, subject, installation facility, installation location, attitude, resolution, number of effective pixels, refresh rate, angle of view, presence or absence of built-in sensor, presence or absence of microphone, year of manufacture, and manufacturer of the mobile camera and the fixed camera.

[0107] (Supplementary Note 9) A camera extraction device according to any one of Supplementary Note 1 to Supplementary Note 8, wherein in the moving camera model, the moving cameras are associated with each other in terms of tracks, and in the fixed camera model, the fixed cameras are associated with each other in terms of tracks and installed facilities.

[0108] (Supplementary Note 10) A camera extraction device according to any one of Supplementary Note 6 to Supplementary Note 8, wherein the attributes include the attitudes of the mobile camera and the fixed camera, and the camera extraction unit generates the relevance based on the collaboration result, the attributes, and the train operation status managed by an operation management device.

[0109] (Supplementary Note 11) A camera extraction device according to any one of Supplementary Note 6 to Supplementary Note 8, further comprising a display control unit that causes a display device to display the video of the designated camera, and when a user performs an operation, the display control unit switches the video of the related camera as a new video of the designated camera based on the relevance.

[0110] (Supplementary Note 12) The camera extraction device according to Supplementary Note 11, wherein the camera extraction unit performs machine learning to generate the relevance used to switch the video of the designated camera.

[0111] (Supplementary Note 13) The camera extraction device according to any one of Supplementary Note 6 to Supplementary Note 8, further comprising a display control unit that causes a display device to simultaneously display images captured by the designated camera and the plurality of related cameras, and the display control unit causes the display device to display the images captured by the plurality of related cameras in descending order of relevance.

[0112] (Supplementary Note 14) The camera extraction device according to any one of Supplementary Note 1 to Supplementary Note 13, wherein the designated camera includes a first camera designated based on an operation from a user, or a second camera designated based on an analysis result of a video.

[0113] (Supplementary Note 15) The camera extraction device according to Supplementary Note 14, further comprising a display control unit that selects either the first camera or the second camera as the designated camera based on the train operation status managed by an operation management device, and displays the image of the designated camera on a display device.

[0114] (Supplementary Note 16) The camera extraction device according to Supplementary Note 15, wherein the display control unit performs machine learning on the selection of the designated camera from the first camera and the second camera.

[0115] (Appendix 17) A camera extraction device according to any one of appendices 1 to 16, wherein the camera model linking unit links the moving camera model and the fixed camera model when a user performs an operation or when the train moves.

[0116] (Supplementary Note 18) The camera extraction device according to any one of Supplementary Note 1 to Supplementary Note 17, wherein the camera extraction device is configured by a cloud server.

[0117] (Supplementary Note 19) A video display system comprising: a camera extraction device according to any one of Supplementary Note 1 to Supplementary Note 18; an input device that accepts operations from a user; and a display device that can display video from the associated camera.

[0118] (Supplementary Note 20) A camera extraction method comprising: storing a mobile camera model in which mobile cameras whose positions change with a train are associated with each other in terms of their positions; and a fixed camera model in which fixed cameras that are installed on the ground and whose positions are fixed are associated with each other in terms of their positions; updating the positions in the mobile camera model; linking the mobile camera model and the fixed camera model based on the updated positions in the mobile camera model and the updated positions in the fixed camera model; and extracting related cameras that are cameras that are related to a designated camera based on the linking result between the mobile camera model and the fixed camera model.

[0119] 1 camera extraction device, 2 input device, 4 display device, 11 storage unit, 12a position update unit, 12b camera model linkage unit, 12c camera extraction unit, 12d display control unit, 71 train control system.

Claims

1. A camera extraction device used in a train control system for controlling trains operating on a railway line, A mobile camera mounted on the aforementioned train and moving in conjunction with the operation of the aforementioned train, Multiple fixed cameras are fixed in the vicinity of the aforementioned route, A position update unit that updates the position of the aforementioned mobile camera at predetermined intervals, Based on the position updated by the position update unit, a camera extraction unit extracts the relevant camera from among the plurality of fixed cameras, A display device that displays the video from the mobile camera and the video from the related camera extracted by the camera extraction unit from among the plurality of fixed cameras. A camera extraction device equipped with the following features.

2. A camera extraction device according to claim 1, The aforementioned position update unit, A camera extraction device that updates the position in a mobile camera model, which associates the mobile cameras with respect to their respective positions, based on at least one of the positions detected by the mobile camera or the position detection device that moves with the train, and the position of the train managed by the train control system.

3. A camera extraction device according to claim 1, In a mobile camera model in which the mobile cameras are associated with each other in terms of their positions, the mobile cameras are further associated with each other in terms of their attributes, A camera extraction device in which, in a fixed camera model in which fixed cameras are associated with each other in terms of their positions, the fixed cameras are further associated with each other in terms of their attributes.

4. A camera extraction device according to claim 1, In a mobile camera model in which the mobile cameras are associated with each other in terms of their positions, the mobile cameras are associated with each other by a tree structure or a directed graph. In a fixed camera model in which the fixed cameras are associated with each other in terms of their positions, the camera extraction device is such that the fixed cameras are associated with each other by a tree structure or a directed graph.

5. A camera extraction device according to claim 1, A camera extraction device further comprising a camera model linking unit that generates a linking model as a result of linking the mobile camera model and the fixed camera model, which is a linking model formed by linking at least a portion of a mobile camera model that associates the mobile cameras with respect to their positions and at least a portion of a fixed camera model that associates the fixed cameras with respect to their positions.

6. A camera extraction device according to claim 5, The camera extraction unit is A camera extraction device that generates a degree of association between a specified camera and the mobile camera and the fixed camera, used for extracting the related camera, based on the aforementioned linkage results and the attributes of the mobile camera and the fixed camera.

7. A camera extraction device according to claim 6, The camera extraction unit is a camera extraction device that performs machine learning to generate the relevance score.

8. A camera extraction device according to claim 6 or claim 7, A camera extraction device in which the attributes include at least one of the following for the mobile camera and the fixed camera: type, purpose, target, installation facility, installation location, orientation, resolution, effective pixels, refresh rate, field of view, presence or absence of built-in sensor, presence or absence of microphone, year of manufacture, and manufacturer.

9. A camera extraction device according to claim 1, In the mobile camera model in which the mobile cameras are associated with each other in terms of their positions, the mobile cameras are associated with each other in terms of the railway tracks, In a fixed camera model in which the fixed cameras are associated with each other in terms of their positions, the camera extraction device is used to associate the fixed cameras with respect to the railway tracks and installation facilities.

10. A camera extraction device according to claim 6 or claim 7, The aforementioned attributes include the orientation of the mobile camera and the fixed camera, The camera extraction unit is a camera extraction device that generates the correlation score based on the linkage result, the attributes, and the train operation status managed by the operation management device.

11. A camera extraction device according to claim 6 or claim 7, The system further includes a display control unit that displays the image from the designated camera on the display device, The display control unit is a camera extraction device that, when a user performs an operation, switches the image of the related camera as the image of a new designated camera based on the degree of relevance.

12. A camera extraction device according to claim 11, The camera extraction unit is a camera extraction device that performs machine learning to generate the relevance score used for switching the video of the designated camera.

13. A camera extraction device according to claim 6 or claim 7, The system further comprises a display control unit that simultaneously displays images captured by the designated camera and a plurality of related cameras on the display device, The display control unit is a camera extraction device that displays images from the plurality of related cameras on the display device in order of decreasing relevance.

14. A camera extraction device according to claim 6 or claim 7, The camera extraction device includes a first camera designated based on user input, or a second camera designated based on video analysis results.

15. A camera extraction device according to claim 14, A camera extraction device further comprising a display control unit that, based on the operating status of the train managed by the operation management device, selects either the first camera or the second camera as the designated camera and displays the image from the designated camera on the display device.

16. A camera extraction device according to claim 15, The display control unit is a camera extraction device that performs machine learning to select the designated camera from the first camera and the second camera.

17. A camera extraction device according to claim 5, The aforementioned camera model linking unit is A camera extraction device that links the moving camera model and the fixed camera model when a user performs an operation or when the train moves.

18. A camera extraction device according to claim 1 or claim 2, A camera extraction device configured using cloud servers.

19. A camera extraction device according to claim 1 or claim 2, An input device that accepts operations from the user and A video display system equipped with the following features.

20. A camera extraction method used in a train control system that controls trains operating on a railway line, The position update unit updates the position of the mobile camera mounted on the aforementioned train, which moves in conjunction with the operation of the train, at predetermined intervals. Based on the position updated by the position update unit, the camera extraction unit extracts the relevant camera from among a plurality of fixed cameras fixed in the vicinity of the route. A camera extraction method comprising displaying the video from the mobile camera and the video from the related camera extracted by the camera extraction unit from among the plurality of fixed cameras on a display device.