Information processing systems, information processing methods, and programs

Abstract

The system provides an information processing system, etc., that controls an unmanned mobile vehicle according to the conditions at or around the railway crossing. [Solution] The information processing system 10 includes a location information acquisition unit 11, a level crossing information acquisition unit 12, and a plan generation unit 13. The location information acquisition unit 11 acquires location information of a moving object moving along the railway tracks from the moving object. The level crossing information acquisition unit 12 acquires level crossing information that identifies the level crossing that the moving object is about to pass through. The plan generation unit 13 generates an operation plan for at least one of the moving object and the level crossing equipment installed at the level crossing, based on the level crossing information and the location information of the moving object.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program. Background Art

[0002] Use of unmanned aerial vehicles such as drones is becoming widespread for the purpose of inspecting infrastructure such as bridges and overhead lines. As an example of infrastructure inspection using an unmanned mobile object, techniques for realizing inspection, monitoring and the like of railways and surroundings of railway facilities by drones (unmanned aerial vehicles) have been proposed.

[0003] For example, according to Patent Document 1, a management device creates a navigation plan for causing an unmanned aerial vehicle to autonomously navigate during a time period when no train travels within the section indicated by flight section information, based on railway operation information and railway track information. Prior Art Documents Patent Documents

[0004] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2020-006916 Summary of the Invention Problem to be Solved by the Invention

[0005] However, when an unmanned mobile object inspects and monitors railways, the unmanned mobile object needs to pass through a railroad crossing. Therefore, for an unmanned mobile object to pass through a railroad crossing, it is required to take measures in accordance with the condition.

[0006] In view of the above problem, an object of the present disclosure is to provide an information processing system or the like that guides a mobile object in accordance with the situation at or around a railroad crossing. Means for Solving the Problem

[0007] The information processing system described herein includes a location information acquisition unit, a level crossing information acquisition unit, and a plan generation unit. The location information acquisition unit acquires location information of a moving object moving along the railway tracks from the moving object. The level crossing information acquisition unit acquires level crossing information that identifies the level crossing that the moving object is about to pass through. The plan generation unit generates an operation plan for at least one of the moving object and the level crossing equipment installed at the level crossing, based on the level crossing information and the location information of the moving object.

[0008] The information processing method relating to this disclosure involves a computer performing the following steps: The computer obtains position information of a moving object moving along a railway track from the moving object. The computer obtains level crossing information that identifies the level crossing that the moving object is about to pass through. Based on the level crossing information and the position information of the moving object, the computer generates an operation plan for the moving object or the level crossing equipment installed at the level crossing to pass through the level crossing.

[0009] The program relating to this disclosure causes a computer to execute the following information processing method. The information processing method includes a level crossing information acquisition step and a plan generation step. The position information acquisition step acquires position information of a moving object moving along the railway tracks from the moving object. The level crossing information acquisition step acquires level crossing information that identifies the level crossing that the moving object intends to pass through. The plan generation step generates an operation plan for at least one of the moving object and the level crossing equipment installed at the level crossing, based on the level crossing information and the position information of the moving object. [Effects of the Invention]

[0010] According to this disclosure, it is possible to provide an information processing system, information processing method, and program for guiding a moving object according to the conditions of a level crossing or the surrounding area of ​​a level crossing. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram of the operation system according to Embodiment 1. [Figure 2] This is a flowchart of the information processing method according to Embodiment 1. [Figure 3] This is a diagram showing the operation pattern data. [Figure 4] This is a block diagram of the information processing system according to Embodiment 2. [Figure 5] This is a flowchart of the information processing method according to Embodiment 2. [Figure 6] This is a flowchart of the operation plan according to Embodiment 2. [Figure 7] This is a diagram illustrating a scene from an action plan. [Figure 8] This is a block diagram of the information processing system according to Embodiment 3. [Figure 9] This is a diagram illustrating the processes performed by the learning model. [Figure 10] This is a flowchart of the information processing method according to Embodiment 4. [Figure 11] This is a flowchart of the information processing method according to Embodiment 5. [Figure 12] This is a block diagram of the operation system 1 according to Embodiment 6. [Figure 13] This is a flowchart of the information processing method according to Embodiment 6. [Figure 14] This is a block diagram of the operation system 1 according to Embodiment 7. [Figure 15] This is a flowchart of the information processing method according to Embodiment 7. [Figure 16] This is a block diagram illustrating the hardware configuration of a computer. [Modes for carrying out the invention]

[0012] Hereinafter, the present invention will be described through embodiments of the invention, but the invention recited in the claims is not limited to the following embodiments. In addition, not all configurations described in the embodiments are necessarily essential as means for solving the problem. For clarification of the description, the following description and drawings are appropriately omitted and simplified. In each drawing, the same elements are denoted by the same reference numerals, and duplicate descriptions are omitted as necessary.

[0013] <Embodiment 1> Hereinafter, Embodiment 1 will be described. Fig. 1 is a block diagram of an operation system 1 according to Embodiment 1. The operation system 1 is a system for operating a drone that inspects railways. As main components, the operation system 1 includes an information processing system 10, a drone 20, a control terminal 30, a level crossing 40, and an operation management system 50. These components are communicably connected via a network N1. Note that a drone may also be referred to as an unmanned aerial vehicle. An unmanned aerial vehicle is one embodiment of a moving object.

[0014] (Information Processing System 10) The information processing system 10 generates an operation plan for the drone 20 passing through the level crossing 40. The information processing system 10 is, for example, a computer or a server. As main components, the information processing system 10 includes a position information acquisition unit 11, a level crossing information acquisition unit 12, and a plan generation unit 13.

[0015] The position information acquisition unit 11 acquires, from the drone 20, position information of the drone 20 that is a moving object moving along a railway track. The position information is, for example, latitude, longitude, and positioning time acquired by a satellite positioning system. The position information may be acquired by a positioning system using a wireless network.

[0016] The level crossing information acquisition unit 12 acquires level crossing information that identifies the level crossing 40 that the drone 20 is about to pass through. The level crossing information is, for example, identification information of level crossing 40. The level crossing information may also be location information of level crossing 40. The level crossing information acquisition unit 12 may identify the next level crossing 40 that the drone 20 will pass through by referring to the location information of the drone 20 and the location information of level crossing 40 that has been stored in advance. The level crossing information acquisition unit 12 may identify the next level crossing 40 that the drone 20 will pass through by referring to the location information of the drone 20 and the route information that has been stored in advance.

[0017] The planning generation unit 13 generates an action plan for passing through the level crossing 40 based on the position information of the drone 20 and the level crossing information. The action plan includes the actions of the drone 20 and at least one of the level crossing equipment installed at the level crossing 40.

[0018] The motion plan includes information for controlling at least one of the following actions in a time-series manner: the flight action of the drone 20 and the action of the level crossing equipment installed at the level crossing 40. The motion plan for the drone 20 may include, for example, the position, speed, and attitude of the drone 20. The motion plan for the level crossing equipment may include, for example, the illumination of indicator lights and the output of alarm sounds. The plan generation unit 13 presents the generated motion plan to the operator operating the control terminal 30.

[0019] Specifically, for example, the planning generation unit 13 recognizes the direction from which the drone 20 will approach the identified level crossing 40. The planning generation unit 13 also determines an action pattern corresponding to the drone 20's position information and the level crossing information. The planning generation unit 13 may also select one or more action patterns from the action pattern data that stores action patterns corresponding to the level crossing information.

[0020] The information processing system 10 may refer to motion pattern data from a database located outside the information processing system 10. The information processing system 10 may also possess motion pattern data. In this case, the plan generation unit 13 of the information processing system 10 may include motion pattern data. Furthermore, the plan generation unit 13 may generate a motion plan using a learning model that has learned to generate motion patterns corresponding to location information and level crossing information.

[0021] With the above configuration, the planning generation unit 13 can generate an action plan that includes, for example, slowing down the drone 20 when its distance from the level crossing 40 falls below a preset threshold when the drone 20 is about to pass through the level crossing 40. The planning generation unit 13 can also create an action plan that includes, for example, having the drone 20, once it has arrived within range of being able to photograph the situation around the level crossing 40, circle around and determine whether or not there are cars or people at or around the level crossing 40. Furthermore, the planning generation unit 13 can also create an action plan that includes, for example, illuminating the warning indicator of the level crossing 40 when the drone 20 approaches the level crossing 40.

[0022] The level crossing information acquisition unit 12 may also acquire level crossing characteristics corresponding to the level crossing identification information as level crossing information. That is, the level crossing information may include level crossing characteristics in addition to the information described above. In this case, the characteristics of the identified level crossing 40 may include, for example, information regarding the width of the road, the width of the railway track, or the size of the building clearance at the level crossing 40. Structural characteristics may include information regarding the location or size of the surrounding terrain, buildings, and vegetation. The level crossing information may also include two-dimensional or three-dimensional terrain data.

[0023] The characteristics of the level crossing 40 may include qualitative information such as the quality of visibility, the degree of wind speed, and the frequency of objects passing through. The characteristics of the level crossing 40 may also be defined by grouping them according to characteristics that multiple level crossings 40 have in common. In this disclosure, "objects" also include living organisms.

[0024] When the level crossing information acquisition unit 12 acquires the characteristics of the level crossing 40, the plan generation unit 13 may generate an action plan based on the characteristics of the level crossing 40. For example, when passing through a level crossing 40 with poor visibility, the plan generation unit 13 may generate an action plan in which the drone 20 starts moving at a speed below a predetermined threshold when it begins to enter the level crossing 40, and then increases its speed midway through the journey.

[0025] (Drone 20) Next, the configuration of the drone 20 will be described. The drone 20 moves along the railway tracks in the air above them. The drone 20 is connected to the network N1 via wireless communication. The main components of the drone 20 are a sensor group 21, a rotor 22, a drive unit 23, an LED 24, a speaker 25, and a control unit 26.

[0026] The sensor group 21 includes multiple sensors that the drone 20 has in order to perform its functions in the operation system 1. For example, the sensor group 21 includes an antenna for determining its own position using GNSS (Global Navigation Satellite System). The method for determining its own position may also be a wireless network instead of GNSS. The sensor group 21 may also use an image sensor, a distance measuring sensor such as LiDAR (Light Detection and Ranging), a gyroscope, a barometric pressure sensor, etc., for determining its own position. The drone 20 determines its own position and supplies the position information to the information processing system 10.

[0027] The sensor group 21 includes sensors for detecting objects present in the vicinity of the drone 20. In addition to the image sensor and LiDAR mentioned above, the sensor group 21 may include a thermal camera, millimeter-wave sensor, infrared sensor, etc. This allows the drone 20 to move while maintaining a certain distance from surrounding objects. The sensor group 21 may include multiple sensors for object detection. In that case, the drone 20 may recognize objects through multimodal processing. The information processing system 10 may also receive multiple sensor data acquired by the drone 20 and recognize objects around the railroad crossing through multimodal processing.

[0028] The rotor 22 is a rotating blade that lifts and propels the drone 20. The drone 20 preferably has four or eight rotors 22. The drive unit 23 drives the multiple rotors 22 individually under the control of the control unit 26. This enables the drone 20 to fly according to the flight plan.

[0029] LED24 is a light-emitting diode and a display device for presenting visual information to the surroundings. LED24 lights up, flashes, etc., under the control of the control unit 26. Speaker 25 outputs sound based on control signals from the control unit 26. When the drone 20 passes through the railroad crossing 40, for example, it lights up LED24 and outputs sound from speaker 25. This allows the drone 20 to make its presence known to people and others around the railroad crossing 40, and to issue warnings, etc.

[0030] The control unit 26 controls each component of the drone 20. The control unit 26 includes a computing device such as a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory).

[0031] (Pilot terminal 30) The control terminal 30 is a terminal used by the operator to remotely control the drone 20. The control terminal 30 is a computer or a dedicated control terminal. The control terminal 30 mainly consists of an input receiving unit 31 and a control information display unit 32.

[0032] The input receiving unit 31 receives operation instructions from the operator and transmits them to the drone 20. The input receiving unit 31 is, for example, an operation button, switch, or touch panel. The operator of the drone 20 receives information regarding the drone's operation plan from the information processing system 10. In this case, the operator controls the drone 20 via the input receiving unit 31 according to the received operation plan.

[0033] The pilot information display unit 32 is, for example, a display and a speaker. The pilot information display unit 32 presents the pilot with various feedback received from the drone 20. The pilot information display unit 32 also presents information regarding the drone 20's operation plan received from the information processing system 10.

[0034] (Level crossing 40) The level crossing 40 mainly consists of a sensor group 41, a barrier 42, an LED 43, and a speaker 44.

[0035] The sensor group 41 acquires various data that can be used when the drone 20 passes through the level crossing 40. The sensor group 41 includes, for example, object detection sensors for detecting objects present around the level crossing 40. The object detection sensors are, for example, infrared sensors, millimeter-wave sensors, and image sensors. The sensor group 41 may also include an anemometer. Because the level crossing 40 has the sensor group 41, the information processing system 10 can generate an action plan that prevents the drone 20 from entering a dangerous situation before it approaches the level crossing 40. The sensor group 41 may include multiple sensors for object detection. In that case, the information processing system 10 may receive sensor data acquired by multiple sensors from the sensor group 41 and recognize the object through multimodal processing.

[0036] The barrier 42 blocks people and vehicles on the road when a railway vehicle passes through the level crossing 40. The barrier 42 may also be used when a drone 20 passes through the level crossing 40.

[0037] The LED 43 and speaker 44 provide various notifications or warnings to people and vehicles in the vicinity of the level crossing 40. For example, the LED 43 and speaker 44 notify people and vehicles attempting to enter the level crossing 40 that the drone 20 will pass through the level crossing 40.

[0038] (Flight Management System 50) The flight management system 50 manages the operation of the drone 20. The flight management system 50 is a computer or server. The main components of the flight management system 50 are a storage unit 51 and an operation information display unit 53.

[0039] The memory unit 51 stores at least level crossing information 52. The level crossing information 52 includes at least information for identifying a level crossing that has been stored in advance. The level crossing information 52 may also include features of the level crossing that are associated with the identification information of the level crossing. The operation management system 50 may supply at least a portion of the level crossing information 52 to the information processing system 10. In addition to the level crossing information 52, the memory unit 51 may also store various information related to the operation of the drone 20. The information related to the operation of the drone 20 may include information about the drone 20's travel route.

[0040] The flight information display unit 53 displays the status of the drone 20 to the administrator of the flight management system 50. The flight information display unit 53 includes, for example, a display.

[0041] The configurations of the operation system 1 have been described above. With the above configuration, the operation system 1 generates an operation plan that includes at least one of the flight actions of the drone 20 and the operation of the level crossing equipment based on the acquired information. This enables the operation system 1 to operate the drone in accordance with the conditions and characteristics of the level crossing 40.

[0042] (Information processing methods) Next, with reference to Figure 2, the processes executed by the information processing system 10 will be described. Figure 2 is a flowchart of the information processing method according to Embodiment 1. In this embodiment, the information processing method is performed by the information processing system 10 using the following method.

[0043] First, the location information acquisition unit 11 acquires location information from the drone 20 that is moving along the railway tracks (step S11). The location information acquisition unit 11 supplies the acquired location information to at least the plan generation unit 13.

[0044] Next, the level crossing information acquisition unit 12 acquires level crossing information to identify the level crossing 40 that the drone 20 is about to pass through (step S12). Alternatively, the level crossing information acquisition unit 12 may identify the level crossing 40 that the drone 20 is about to pass through from the location information of the drone 20 acquired by the location information acquisition unit 11. The level crossing information acquisition unit 12 supplies the acquired level crossing information to the plan generation unit 13.

[0045] Next, the planning generation unit 13 generates an action plan for the drone 20 or the level crossing equipment installed at the level crossing 40, based on the level crossing information and the position information of the drone 20 (step S13). Once the planning generation unit 13 has generated the action plan, the information processing system 10 terminates its processing.

[0046] The processes performed by the information processing system 10 have been described above. The plan generation unit 13 may present the generated motion plan to the control terminal 30. Alternatively, the control terminal 30 may access the information processing system 10 and obtain the motion plan generated by the plan generation unit 13. With the above configuration, the information processing system 10 associates the current position of the drone 20 with the status of the level crossing 40 and generates a motion plan for the drone 20 or the level crossing equipment. As a result, the information processing system 10 generates a motion plan that appropriately supports passage through the level crossing 40.

[0047] Next, with reference to Figure 3, the data used by the planning generation unit 13 when generating the motion plan will be described. Figure 3 is a diagram showing motion pattern data. The motion pattern data shown in Figure 3 includes the level crossing ID (Identifier), which is identification information assigned to the level crossing 40, the level crossing type, which indicates the type or characteristics of the level crossing 40, and the motion pattern.

[0048] As shown in Figure 3, the operation pattern data includes the items "Level Crossing ID," "Level Crossing Type," and "Operation Pattern." The Level Crossing ID is identification information for identifying Level Crossing 40. In other words, the Level Crossing ID is one embodiment of the Level Crossing information. The Level Crossing Type is information that indicates the characteristics of Level Crossing 40. The Level Crossing Type includes information that indicates the structural characteristics of the Level Crossing. The Level Crossing Type may also include information that indicates the qualitative characteristics of the Level Crossing. The Operation Pattern is an element of operation for generating an operation plan. The operation plan includes at least one operation pattern.

[0049] The first row of Figure 3 associates the level crossing ID "10021" with the level crossing type "AA003" and the action pattern "Decelerate to ** km / h *m before the level crossing." This indicates that level crossing 40, identified by level crossing ID "10021," has the attributes indicated by type "AA003," and shows the specific details of the action pattern adopted when passing through level crossing 40 having those attributes. Note that the level crossing types shown in Figure 3 are codes assigned to each characteristic of a level crossing. "AA003" corresponds, for example, to the structural or qualitative characteristics of the level crossing.

[0050] Similarly, in the second row, the level crossing ID "10022" is associated with level crossing types "AC012" and "AD031". In addition, the level crossing ID "10022" is associated with the operation pattern "Hover *m before the level crossing, at an altitude of *m". As shown here, a single level crossing ID may be associated with multiple level crossing types. In this case, the operation pattern will correspond to the multiple level crossing types.

[0051] In this way, the plan generation unit 13 can generate an action plan from data that associates level crossing information with action patterns. In the example shown in Figure 3, the plan generation unit 13 generates an action plan based on rule-based data, but the plan generation unit 13 may also generate an action plan using a learning model.

[0052] Embodiment 1 has been described above. The storage unit described above may store a computer program (hereinafter simply referred to as "the program") for executing the above-described method. The system described above may also have a processor in a configuration not shown. In this case, the processor loads the computer program from the storage device into a buffer memory such as DRAM (Dynamic Random Access Memory) and executes the program.

[0053] Each component of the system may be implemented with dedicated hardware. Furthermore, some or all of each component may be implemented by general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may be implemented by a single chip or by multiple chips connected via a bus. Some or all of each component may be implemented by a combination of the aforementioned circuits, etc., and programs. Additionally, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array), etc., can be used as the processor. Furthermore, at least some of the functions of this embodiment may be provided in the form of IaaS (Infrastructure as a Service), PaaS (Platform as a Service), or SaaS (Software as a Service).

[0054] Although the above example described a drone as one embodiment of a mobile device, the mobile device may be a vehicle that moves along railway tracks via remote control, or a robot that walks or runs along railway tracks via remote control. Furthermore, the mobile device may or may not have a person on board.

[0055] Embodiment 1 has been described above. According to this embodiment, it is possible to provide an information processing system, information processing method, and program that guide a moving object according to the conditions of the railway crossing or the area surrounding the railway crossing.

[0056] <Embodiment 2> Next, Embodiment 2 will be described. Figure 4 is a block diagram of the information processing system 10 according to Embodiment 2. The information processing system 10 according to this embodiment has a storage unit 19.

[0057] The memory unit 19 stores the operation pattern data 191 and the passage condition data 192. The passage condition data 192 is data that stores conditions to suppress the possibility of contact with other objects when the information processing system 10 passes through the level crossing 40. In other words, the passage condition data 192 is data that stores conditions for passing through the level crossing 40 while maintaining a certain distance from other objects.

[0058] In this disclosure, the distance that must be maintained between the drone 20 and other objects when the drone 20 passes through the level crossing 40 is referred to as the limit distance. In this disclosure, the distance that serves as the threshold for determining whether or not the drone 20 can begin to pass through the level crossing 40 is referred to as the safety distance. While the drone 20 is passing through the level crossing 40, objects may approach the drone 20. Therefore, the safety distance is set to be longer than the limit distance. The passage condition data 192 determines the passage condition based on the safety distance, which is set so that the distance between the drone 20 and the object does not become shorter than the limit distance.

[0059] In this embodiment, the level crossing information acquisition unit 12 acquires structural information regarding the level crossing 40 and the terrain or buildings surrounding the level crossing 40, which has been stored in advance before the drone 20 approaches the level crossing 40. That is, the level crossing information in this embodiment includes the structural information described above. The level crossing information acquisition unit 12 also acquires object information regarding the level crossing 40 and objects around the level crossing 40, which is acquired by sensors included in the sensor group 21 of the drone 20 as it approaches the level crossing 40. That is, the level crossing information in this embodiment includes the object information described above. The level crossing information acquisition unit 12 acquires the structural information and object information as level crossing information.

[0060] In this embodiment, the planning generation unit 13 moves the drone 20 to a determination position to determine the timing of passing through the level crossing 40 based on the level crossing information, as part of the motion plan. The planning generation unit 13 also acquires information about an object entering the level crossing 40 at the determination position. Furthermore, after acquiring information about the object entering the level crossing 40, the planning generation unit 13 determines the timing of passing through according to the pre-set passing conditions. The planning generation unit 13 acquires information about the object entering from the sensor group 41 of the level crossing 40. The information about the object entering may include information about the movement of the object entering.

[0061] The planning generation unit 13 determines the above-mentioned determination position by referring to the motion pattern data 191. The planning generation unit 13 also determines the timing of passage by referring to the passage condition data 192. With the above configuration, the information processing system 10 generates a motion plan for passing through the level crossing 40, after referring to the structural characteristics of the level crossing 40 and information on the dynamic approaching object. The information on the dynamic approaching object is information about the movement of the approaching object. The information on the dynamic approaching object includes at least the direction of movement or the speed of movement of the approaching object.

[0062] Next, with reference to Figure 5, the processing performed by the information processing system 10 according to this embodiment will be described. Figure 5 is a flowchart of the information processing method according to Embodiment 2. The flowchart in Figure 5 has steps S12A and S12B instead of step S12.

[0063] First, the location information acquisition unit 11 acquires location information from the drone 20 that is moving along the railway tracks (step S11).

[0064] Next, the level crossing information acquisition unit 12 acquires structural information regarding the terrain or buildings surrounding the level crossing 40 (step S12A). The level crossing information acquisition unit 12 supplies the acquired structural information to the plan generation unit 13.

[0065] Next, the level crossing information acquisition unit 12 acquires object information regarding the level crossing 40 and objects in the vicinity of the level crossing 40, which is acquired by sensors on the drone 20 as it approaches the level crossing 40 (step S12B). The level crossing information acquisition unit 12 supplies the acquired object information to the plan generation unit 13. The level crossing information acquisition unit 12 may also supply structural information and object information to the plan generation unit 13 together.

[0066] Next, the plan generation unit 13 generates a motion plan based on the drone 20's position information, structural information, and object information (step S13). At this time, as described above, the plan generation unit 13 generates a motion plan that includes determining the timing of passage at the determined position. Once the plan generation unit 13 has generated the motion plan, the information processing system 10 terminates its processing.

[0067] The processes performed by the information processing system 10 according to this embodiment have been described above. By the method described above, the information processing system 10 integrates static structural information and dynamic object information to generate an action plan for passing through a railroad crossing.

[0068] The level crossing information acquisition unit 12 may also acquire object information from the sensor group 41 of the level crossing 40. This allows the information processing system 10 to acquire information about objects that cannot be detected or recognized by the drone 20 as level crossing information.

[0069] Next, with reference to Figure 6, the operation plan generated by the information processing system 10 according to this embodiment will be described. Figure 6 is a flowchart of the operation plan according to Embodiment 2.

[0070] First, the plan generation unit 13 moves the drone 20 to a predetermined determination position (step S101). The predetermined determination position is determined according to the level crossing information acquired by the level crossing information acquisition unit 12. The determination position may vary depending on the object information.

[0071] Next, the planning generation unit 13 causes the drone 20 to acquire information about the approaching object at the determination position (step S102). The planning generation unit 13 causes the drone 20 to acquire object information, for example. The planning generation unit 13 may also acquire object information from the sensor group 41 of the railroad crossing 40.

[0072] Next, the planning generation unit 13 refers to the passage condition data 192 and determines the passage timing (step S103). Furthermore, the planning generation unit 13 has the drone 20 pass through the level crossing 40 according to the determined passage timing (step S104). Once the drone 20 has passed through the level crossing 40, the planning generation unit 13 terminates its processing.

[0073] The above explanation describes the procedure for generating the action plan by the plan generation unit 13. The plan generation unit 13 presents the above action plan to, for example, the control terminal 30. The operator operating the control terminal 30 operates the drone 20 according to the action plan presented by the plan generation unit 13. As a result, the operation system 1 can suppress accidents and other incidents when the drone 20 passes through the railroad crossing 40.

[0074] Next, an example of a motion plan will be explained with reference to Figure 7. Figure 7 is a diagram illustrating a scene in the motion plan. The scene shown in Figure 7 depicts the situation as the drone 20 approaches the level crossing 40 and before it passes over it.

[0075] For the sake of explanation, Figure 7 includes XY coordinates. In Figure 7, the direction from bottom to top is the positive Y-axis direction, and the direction from left to right is the positive X-axis direction. Drone 20 is moving in the positive Y-axis direction.

[0076] Level crossing 40 has a barrier 42. The barrier 42 is in the open position. At level crossing 40, cars E11, E12, and pedestrians E13 are passing on the road that intersects the railway tracks. Also, building E14 is located to the right of level crossing 40, in front of the road, as viewed from the direction of travel of drone 20. At the detection position 102, drone 20 is taking pictures of level crossing 40 and objects around level crossing 40 with its own camera.

[0077] In the situation described above, the drone 20 is located at the determination position 102 just before entering the level crossing 40. The determination position 102 is set to the left side of the tracks (negative X-axis side) so that the overlap between the camera field of view 101 and building E14 is minimized. This position was set in the motion pattern data in accordance with the level crossing information. The planning generation unit 13 decides to move the drone 20 to the determination position 102 as the motion plan. The operator of the drone 20 moves the drone 20 to the determination position 102 according to the instructions from the planning generation unit 13.

[0078] Next, at the determination position 102, the planning generation unit 13 determines the passage conditions for the drone 20 based on the object information and the passage conditions based on the passage condition data 192. For example, the planning generation unit 13 presents information regarding the safe distance between the drone 20 and the object to the control terminal 30 as the passage conditions for the drone 20. The operator moves the drone 20 according to the presented conditions. If the level crossing information acquisition unit 12 can recognize the distance between the drone 20 and the object from the sensor group 21 of the drone 20 or the sensor group 41 of the level crossing 40, it may indicate permission or denial of passage for the drone 20.

[0079] Embodiment 2 has been described above. With the above configuration, the information processing system 10 can generate an action plan that allows safe passage through the level crossing 40 in accordance with the static environment and dynamic conditions of the level crossing 40. Therefore, according to this embodiment, it is possible to provide an information processing system, information processing method, and program that guide a moving object in accordance with structural information and object information of the level crossing and its surroundings.

[0080] <Embodiment 3> Next, Embodiment 3 will be described. Figure 8 is a block diagram of the information processing system according to Embodiment 3. The information processing system 10 according to this embodiment has a learning model 18. The storage unit 19 according to this embodiment stores distance data 193.

[0081] The planning generation unit 13 in this embodiment adjusts the passage conditions based on the learned results regarding the safe distance between the moving object and the approaching object during past crossings. The planning generation unit 13 performs the above functions by utilizing the learning model 18. That is, the planning generation unit 13 inputs the passage condition data 192 and distance data 193 into the learning model 18. The learning model 18 adjusts the passage conditions included in the passage condition data 192 by learning the input passage condition data 192 and distance data 193.

[0082] The learning model 18 accepts distance data 193 and passage condition data 192 as input, determines whether or not to adjust the passage condition data 192, and if it determines that adjustment is necessary, adjusts the passage conditions of the passage condition data 192.

[0083] Distance data 193 is data that records the distance to surrounding objects at the start of passage and the distance to objects during passage when the drone 20 has passed through the level crossing 40 in the past. Distance data 193 may consist of one or more data points. If there are multiple data points, the distance data 193 may be statistically processed, such as by moving average. Furthermore, the distance data 193 may be aggregated for each individual level crossing, for example. The distance data 193 may also be aggregated for each characteristic of the level crossing.

[0084] Next, with reference to Figure 9, the processes performed by the learning model 18 according to this embodiment will be described. Figure 9 is a diagram illustrating the processes performed by the learning model 18.

[0085] The learning model 18 obtains the pre-adjustment passage conditions from the passage condition data 192. The learning model 18 also obtains distance data between the drone 20 and the approaching object during past crossings from the distance data 193. If the distance data between the drone 20 and the approaching object during past crossings is shorter than the limit distance, the learning model 18 adjusts the safety distance included in the passage conditions to be longer. Alternatively, if the distance data between the drone 20 and the approaching object during past crossings is longer than the limit distance by a certain amount, the learning model 18 may adjust the safety distance included in the passage conditions to be shorter.

[0086] Embodiment 3 has been described above. With the above configuration, the information processing system 10 flexibly adjusts the passage conditions and generates an efficient operation plan for the drone 20. Therefore, according to this embodiment, it is possible to provide an information processing system, information processing method, and program that flexibly guides a moving object according to the conditions of the railway crossing or the area surrounding the railway crossing.

[0087] <Embodiment 4> Next, Embodiment 4 will be described. In this embodiment, the information processing system 10 has the following functions: the level crossing information acquisition unit 12 and the plan generation unit 13.

[0088] The level crossing information acquisition unit 12 in this embodiment acquires object information based on sensor data acquired by sensors (sensor group 21 or sensor group 41) installed on the drone 20 or the level crossing 40. The object information is dynamic information about any approaching objects that may be present at the level crossing 40 at the time the drone 20 passes through the level crossing 40. The level crossing information acquisition unit 12 acquires this object information as level crossing information.

[0089] In this embodiment, the planning generation unit 13 determines, based on the level crossing information and position information, whether to prioritize the passage of the drone 20 or the crossing of the level crossing 40 by the approaching object. The planning generation unit 13 generates an action plan based on the result of the above determination.

[0090] The planning generation unit 13 may determine, depending on the type of approaching object, whether to prioritize the passage of the drone 20 or the crossing of the level crossing 40 by the approaching object. For example, if the approaching object is a pedestrian, the planning generation unit 13 will determine to prioritize the pedestrian's crossing. On the other hand, if the approaching object is a car, the planning generation unit 13 will determine the priority based on the car's speed and the estimated time of arrival at the level crossing 40. Furthermore, if the approaching object is stationary, the planning generation unit 13 may determine to prioritize the passage of the drone 20. This allows the information processing system 10 to make decisions according to the attributes of the approaching object.

[0091] The planning generation unit 13 may determine, based on the learning results regarding the type of approaching object and the behavior of the approaching object, whether to prioritize the passage of the drone 20 or the crossing of the railway crossing by the approaching object. In this case, the information processing system 10 stores past passage data. Past performance data is data regarding the behavior corresponding to the type of approaching object. Specifically, for example, the information processing system 10 stores the relationship between an approaching object such as a car and the speed at which the car moves.

[0092] The planning generation unit 13, for example, inputs past passage data into a learning model and estimates the behavior of incoming objects included in the object information. Then, the planning generation unit 13 determines priority targets based on the estimation results and passage condition data 192.

[0093] Furthermore, the learning model 18 may also serve as a learning model to realize the above-mentioned functions performed by the plan generation unit 13. With this configuration, the information processing system 10 can determine priority targets that reflect the actual behavioral tendencies of the approaching object and generate a passage plan adapted to the changing situation. In addition, the plan generation unit 13 may use a neural network that has learned past behavioral data of the approaching object to predict the future movement trajectory of the approaching object and generate a motion plan based on the predicted movement trajectory. The learning model 18 may also be a neural network that has learned past behavioral data of the approaching object. That is, in this case, the learning model 18 learns behavioral data for each type of approaching object and predicts the future behavior of the approaching object from information about the approaching object obtained from the drone 20 or the level crossing 40. Furthermore, the learning model 18 is not limited to a neural network. The learning model 18 may also be a decision tree or a transformer, etc.

[0094] Referring to Figure 10, the processes performed by the information processing system 10 according to this embodiment will be described. Figure 10 is a flowchart of the information processing method according to Embodiment 4.

[0095] First, the level crossing information acquisition unit 12 acquires information about an approaching object (object information) (step S201).

[0096] Next, the planning generation unit 13 determines whether or not to prioritize its own passage (step S202). If the planning generation unit 13 determines to prioritize its own passage (step S202: YES), the planning generation unit 13 generates an action plan that prioritizes its own passage (step S203) and terminates the process. On the other hand, if the planning generation unit 13 does not determine to prioritize its own passage (step S202: NO), the planning generation unit 13 generates an action plan that prioritizes crossing the approaching object (step S204) and terminates the process.

[0097] Embodiment 4 has been described above. With the above configuration, the information processing system 10 determines priority targets according to the type and behavior of objects that may enter the level crossing 40. As a result, the information processing system 10 switches the priority relationships regarding passage through the level crossing 40 according to the situation.

[0098] As described above, according to this embodiment, it is possible to provide an information processing system, information processing method, and program that flexibly guide a moving object according to the conditions of the level crossing or the area surrounding the level crossing, and the type and behavior of the approaching object.

[0099] <Embodiment 5> Next, Embodiment 5 will be described. In this embodiment, the information processing system 10 has the following functions: the level crossing information acquisition unit 12 and the plan generation unit 13.

[0100] The level crossing information acquisition unit 12 according to this embodiment acquires level crossing information including the status of objects entering the level crossing 40. Level crossing information including the status of objects entering is a type of object information. The status of objects entering may be, for example, the number of objects entering. The status of objects entering may also be the speed at which the objects are moving. Alternatively, the level crossing information acquisition unit 12 according to this embodiment may predict the status of objects entering at the time the drone 20 passes through the level crossing 40 using a traffic volume prediction model that has learned past traffic volume data and time information. In this case, the status of objects entering may be information including, for example, the number of objects entering, the frequency of objects passing, or the type of object entering. Alternatively, the level crossing information acquisition unit 12 may recognize the type, position, speed, and attitude (detailed object information) of the objects entering using a deep learning model that integrates and processes camera images, LiDAR point cloud data, and infrared sensor data.

[0101] In this embodiment, the planning generation unit 13 generates an action plan that includes the result of determining whether or not to lower the barrier 42, which is a level crossing facility, based on the status of the approaching object. For example, if the level crossing information acquisition unit 12 acquires that there are many approaching objects at the level crossing 40 and that some object is constantly crossing the level crossing 40, the planning generation unit 13 may determine to lower the barrier 42 in order to temporarily stop the approaching object from crossing. Alternatively, the planning generation unit 13 may determine to activate an alarm using the LED 43 and speaker 44 of the barrier 42 instead of lowering the barrier 42.

[0102] In this embodiment, the planning generation unit 13 may generate an action plan to lower the barrier 42 if there is a possibility that an intruding object is present at the level crossing 40 when the drone 20 passes through the level crossing 40. Similarly, the planning generation unit 13 may generate an action plan to suppress lowering the barrier 42 if there is no possibility that an intruding object is present at the level crossing 40 when the drone 20 passes through the level crossing 40. Alternatively, the planning generation unit 13 may generate an action plan by determining the optimal passing time based on the predicted number and frequency of intruding objects. Or, the planning generation unit 13 may generate an action plan based on detailed object information of the recognized intruding objects.

[0103] Next, with reference to Figure 11, the processes executed by the information processing system 10 according to this embodiment will be described. Figure 11 is a flowchart of the information processing method according to Embodiment 5.

[0104] First, the level crossing information acquisition unit 12 acquires level crossing information (object information) regarding the status of the approaching object (step S301).

[0105] Next, the planning generation unit 13 determines whether or not to lower the barrier 42 (step S302). If the planning generation unit 13 determines that the barrier 42 should be lowered (step S302: YES), the planning generation unit 13 generates an action plan to pass through the level crossing 40 after lowering the barrier 42 (step S303), and terminates the process. On the other hand, if the planning generation unit 13 does not determine that the barrier 42 should be lowered (step S302: NO), the planning generation unit 13 generates an action plan to pass through the level crossing 40 without lowering the barrier 42 (step S304), and terminates the process.

[0106] Embodiment 5 has been described above. With the above configuration, the information processing system 10 generates an action plan to safely pass through the level crossing 40 according to the situation. Furthermore, if it is not necessary to lower the barrier 42, the information processing system 10 generates an action plan that does not operate the barrier 42. Thus, according to this embodiment, it is possible to provide an information processing system, information processing method, and program that safely and rationally guide a moving object according to the situation at or around the level crossing.

[0107] <Embodiment 6> Next, Embodiment 6 will be described. Figure 12 is a block diagram of the flight system 1 according to Embodiment 6. The information processing system 10 according to this embodiment is included in the pilot terminal 30. The information processing system 10 according to this embodiment also includes a plan presentation unit 14.

[0108] The plan presentation unit 14 presents the operation plan to the control terminal 30 used by the operator controlling the mobile vehicle, based on the operation plan. This allows the information processing system 10 to present the operation plan, which was generated prior to passing the level crossing 40, to the operator in a timely manner.

[0109] In this embodiment, the information processing system 10 is included in the control terminal 30. Therefore, the plan presentation unit 14 can display information related to the operation plan on the same screen as the user interface for operation displayed on the control information presentation unit 32 of the control terminal 30. This allows the operator to confirm the operation input screen of the drone 20 in correspondence with the contents of the operation plan.

[0110] Therefore, according to this embodiment, the information processing system 10 can integrally perform the generation and presentation of the operation plan within the control terminal 30. This allows the information processing system 10 to enhance the coordination between the user interface for operation and the operation plan display. The operator can perform operations while intuitively understanding the recommended actions according to the situation at the level crossing 40.

[0111] Next, with reference to Figure 13, the processes executed by the information processing system 10 according to this embodiment will be described. Figure 13 is a flowchart of the information processing method according to Embodiment 6. The flowchart shown in Figure 13 differs from the flowchart shown in Figure 2 in that it has a step S14 after step S13.

[0112] In step S13, after generating the action plan, the plan generation unit 13 presents the generated action plan to the control terminal 30 (step S14). More specifically, the plan generation unit 13 superimposes the action plan onto the control information presentation unit 32. This allows the operator to smoothly make decisions and perform control operations regarding passing through the level crossing 40.

[0113] As described above, this embodiment provides an information processing system, information processing method, and program that intuitively guides a moving object according to the conditions of the railway crossing or the surrounding area.

[0114] <Embodiment 7> Next, Embodiment 7 will be described. Figure 14 is a block diagram of the flight system 1 according to Embodiment 7. In this embodiment, the information processing system 10 includes the drone 20. Also, the flight system 1 according to this embodiment does not include the pilot terminal 30. The drone 20 is capable of autonomous flight. The administrator of the flight system 1 manages the status of the drone 20 using the flight management system 50.

[0115] The information processing system 10 according to this embodiment includes a planning and execution unit 15. The planning and execution unit 15 outputs signals to control the drive unit 23 of the drone 20 based on the motion plan. The planning and execution unit 15 outputs signals to control unit 26 for controlling the drive unit 23. The control unit 26 uses the signals received from the planning and execution unit 15 to control the drive unit 23. As a result, the drone 20 can immediately execute the motion plan generated by the information processing system 10.

[0116] Next, with reference to Figure 15, the processing performed by the information processing system 10 according to this embodiment will be described. Figure 15 is a flowchart of the information processing method according to Embodiment 7. The flowchart shown in Figure 15 differs from the flowchart shown in Figure 2 in that it has step S15 after step S13.

[0117] In step S13, after generating an action plan, the plan generation unit 13 supplies the generated action plan to the plan execution unit 15. The plan execution unit 15 executes the received action plan (step S15). Specifically, the plan execution unit 15 generates a signal to control the drive unit 23 from the received action plan and supplies it to the control unit 26. As a result, the drone 20 can execute an action plan according to the conditions of the railroad crossing 40 during autonomous flight.

[0118] Embodiment 7 has been described above. According to this embodiment, it is possible to provide an information processing system, information processing method, and program that automatically control a moving object according to the conditions of the level crossing or the area surrounding the level crossing.

[0119] <Example hardware configuration> The following describes examples of how each functional configuration of the information processing system described in this disclosure can be realized through a combination of hardware and software.

[0120] Figure 16 is a block diagram illustrating the hardware configuration of a computer. The information processing system 10 in this disclosure can realize the above-described functions using a computer 500 including the hardware configuration shown in the figure.

[0121] Computer 500 may be a portable computer such as a smartphone or tablet, or a stationary computer such as a PC. Computer 500 may also be a server computer operating in a cloud environment. Computer 500 may be a dedicated computer designed to implement each device, or it may be a general-purpose computer. Computer 500 can achieve desired functions by installing a predetermined application.

[0122] Computer 500 includes a bus 501, a processor 510, memory 511, a storage device 512, an input / output interface 513, and a network interface 514. The bus 501 is a data transmission path for the processor 510, memory 511, storage device 512, input / output interface 513, and network interface 514 to send and receive data from each other. However, the method of connecting the processor 510 and other components to each other is not limited to bus connection.

[0123] Processor 510 is a variety of processors such as a CPU, GPU, or FPGA. Memory 511 is main memory implemented using RAM (Random Access Memory), etc.

[0124] The storage device 512 is an auxiliary storage device implemented using a hard disk, SSD, memory card, or ROM (Read Only Memory). The storage device 512 stores a program for realizing a desired function. The processor 510 reads this program into memory 511 and executes it to realize each functional component of each device.

[0125] The input / output interface 513 is an interface for connecting the computer 500 to input / output devices. For example, input devices such as keyboards and output devices such as display devices are connected to the input / output interface 513. The network interface 514 is an interface for connecting the computer 500 to a network.

[0126] Although embodiments have been described above, the configurations of the embodiments described above may be combined with each other, or some configurations may be replaced with other configurations. Furthermore, the configurations of the embodiments described above may be modified in various ways without departing from the spirit of the invention. In addition, the flowcharts used in the above description may be modified in the execution order of the processes performed in each embodiment, to the extent that it does not impede the function realized by the embodiment.

[0127] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments rather than with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps shown in any of the drawings may be changed as appropriate. [Explanation of Symbols]

[0128] 1. Operating System 10 Information Processing Systems 11 Location information acquisition section 12. Level crossing information acquisition unit 13. Planning Generation Unit 14. Planning Presentation Section 15. Planning and Execution Department 18 Learning Models 19 Memory section 20 Drones 21 Sensor Groups 22 rotors 23 Drive unit 24 LED 25 speakers 26 Control Unit 30. Operating terminal 31 Input Reception Section 32. Pilot Information Display Unit 40 railroad crossing 41 Sensor Groups 42. Barrier 43 LED 44 speakers 50 Flight Management Systems 51 Storage section 52 Level Crossing Information 53 Flight Information Display Section 101 Camera field of view 102 Judgment position 191 Operation Pattern Data 192 Passing Condition Data 193 Distance Data 500 Computers 501 Bus 510 Processor 511 memory 512 Storage Devices 513 Input / Output Interfaces 514 Network Interfaces E11 Automobile E12 Automobile E13 Pedestrian E14 Building N1 Network

Claims

1. A position information acquisition unit that acquires position information from a mobile body that moves along the railway tracks and performs railway inspections, A level crossing information acquisition unit acquires level crossing information to identify the level crossing that the moving object is about to pass through, The system includes a planning generation unit that generates an operation plan for at least one of the moving body and the level crossing equipment installed at the level crossing, based on the location information and the level crossing information. The level crossing information acquisition unit acquires the characteristics of the level crossing corresponding to the identification information of the level crossing as the level crossing information, The plan generation unit generates the motion plan based on the characteristics of the level crossing. Information processing system.

2. The level crossing information acquisition unit acquires, as level crossing information, structural information relating to the level crossing and the terrain or buildings surrounding the level crossing, which was stored in advance before the moving body approached the level crossing, and object information relating to the level crossing and objects surrounding the level crossing, which is acquired by sensors on the moving body as it approaches the level crossing. The information processing system according to claim 1.

3. The plan generation unit, as part of the motion plan, moves the moving body to a determination position for determining the timing of passing through the level crossing based on the level crossing information, acquires information about an object entering the level crossing at the determination position, and then determines the passing timing according to the pre-set passing conditions. The information processing system according to claim 2.

4. The plan generation unit adjusts the passage conditions based on the learned results regarding the safe distance between the moving body and the approaching object during past crossings. The information processing system according to claim 3.

5. The level crossing information acquisition unit acquires object information as level crossing information, based on sensor data acquired by the moving body or sensors installed at the level crossing, relating to the movement of any approaching objects that may be present at the level crossing at the time the moving body passes through the level crossing. The information processing system according to claim 1.

6. The plan generation unit determines, based on the level crossing information and the position information, whether to prioritize the passage of the moving body or the crossing of the level crossing by the approaching object, and generates the motion plan based on the result of the determination. The information processing system according to claim 5.

7. The planning generation unit determines, depending on the type of the approaching object, whether to prioritize the passage of the moving body or the crossing of the level crossing by the approaching object. The information processing system according to claim 6.

8. The plan generation unit uses a learning model that has learned the past behavior data of the approaching object to predict the future movement trajectory of the approaching object, and generates a motion plan based on the predicted movement trajectory. The information processing system according to claim 5.

9. The planning generation unit determines, based on the learning results regarding the type of the approaching object and the behavior of the approaching object, whether to prioritize the passage of the moving body or the crossing of the level crossing by the approaching object. The information processing system according to claim 6.

10. The level crossing information acquisition unit acquires the level crossing information, including the status of objects entering the level crossing. The plan generation unit generates the operation plan, which includes the result of determining whether or not to lower the barrier gate, which is part of the level crossing equipment, based on the status of the approaching object. The information processing system according to claim 1.

11. The aforementioned plan generation unit, If there is a possibility that the approaching object is present at the level crossing at the time the moving body passes through the level crossing, it is determined that the barrier should be lowered. The operation plan is generated to suppress lowering the barrier when there is no possibility of the approaching object being present at the level crossing at the time the moving body passes the level crossing. The information processing system according to claim 10.

12. The aforementioned level crossing information acquisition unit is: Using a traffic volume prediction model that has learned from past traffic volume data and time information, the status of the approaching object at the time when a moving object passes through the level crossing is predicted. The aforementioned plan generation unit, Based on the predicted status of the approaching object, the time of passage is determined. The information processing system according to claim 10.

13. The system further includes a plan presentation unit that presents the aforementioned operation plan to a control terminal used by the operator controlling the mobile body, based on the aforementioned operation plan. The information processing system according to any one of claims 1 to 12.

14. The system further includes a planning execution unit that outputs a signal for controlling the drive device of the moving body based on the aforementioned operation plan. The information processing system according to any one of claims 1 to 12.

15. Computers The position information of a mobile body that moves along the railway tracks and performs railway inspections is obtained from the said mobile body. The moving object obtains level crossing information to identify the level crossing it intends to pass through. Based on the location information and the level crossing information, an operation plan is generated for the moving body or the level crossing equipment installed at the level crossing that will pass through the level crossing. Information processing method, The characteristics of the level crossing corresponding to the identification information of the level crossing are acquired as the level crossing information. Based on the characteristics of the aforementioned level crossing, the aforementioned operation plan is generated. Information processing methods.

16. A position information acquisition step involves obtaining position information from a mobile body that moves along the railway tracks and performs railway inspections, A level crossing information acquisition step, which acquires level crossing information to identify the level crossing that the moving object is about to pass through, The system includes a planning generation step that generates an operation plan for the moving body or the level crossing equipment installed at the level crossing, based on the location information and the level crossing information, The aforementioned level crossing information acquisition step acquires the characteristics of the level crossing corresponding to the identification information of the level crossing as the level crossing information, The plan generation step generates the motion plan based on the characteristics of the level crossing. To have a computer execute an information processing method. program.

Images