Vehicle device, control system, control method and control program

Abstract

To reduce a burden on a user during inspection.SOLUTION: A vehicle device 100 according to the present disclosure, which is an unmanned vehicle device that travels on a track, includes: a travel determination unit 133A that determines whether or not the vehicle device 100 can travel, according to an anomaly in a surrounding situation detected based on spatial data acquired by a sensor mounted in the vehicle device 100; and a travel control unit 133C that causes the vehicle device 100 to continue traveling when the travel determination unit 133A determines that the vehicle device can travel, and causes the vehicle device 100 to return when the travel determination unit 133A determines that the vehicle device cannot travel.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a vehicle device, a control system, a control method, and a control program. [Background technology]

[0002] Conventionally, track inspection work has been performed by an inspection vehicle and a user who operates the inspection vehicle. In recent years, technology has been proposed that allows unmanned running inspection vehicles to detect obstacles. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-180702 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional technology has the problem that it does not necessarily reduce the burden on the user during inspection. For example, while some detected obstacles can be avoided or will disappear over time, conventional technology has sometimes caused the inspection vehicle to return to its original starting position when it detects an obstacle, making it impossible to continue the track inspection.

[0005] In order to solve the above-mentioned problems, an object of the present invention is to provide a vehicle device that reduces the burden on the user during inspection. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the vehicle device of the present invention is an unmanned vehicle device that runs on a track, and is characterized by having a running determination unit that determines whether the vehicle device is able to run in accordance with abnormalities in the surrounding conditions detected based on spatial data acquired by a sensor mounted on the vehicle device, and a running control unit that allows the vehicle device to continue running if the running determination unit determines that the vehicle device is able to run, and returns the vehicle device if the running determination unit determines that the vehicle device is not able to run. [Effects of the Invention]

[0007] According to the present invention, the burden on the user during an examination can be reduced. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a control system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a specific example of a vehicle device according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating a specific example of a vehicle device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating a specific example of a vehicle device according to the embodiment. [Figure 5] FIG. 5 is a diagram showing an example of traveling of the vehicle device according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of traveling of the vehicle device according to the embodiment. [Figure 7] FIG. 7 is a diagram showing an example of traveling of the vehicle device according to the embodiment. [Figure 8] FIG. 8 is a diagram showing an outline of implementation by the control system according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the configuration of a vehicle device according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of flight control processing by the flight control unit according to the embodiment. [Figure 11]FIG. 11 is a diagram illustrating an example of an output process by the output unit according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of an output process by the output unit according to the embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of the configuration of an aircraft according to an embodiment. [Figure 14] FIG. 14 is a diagram illustrating the overall processing by the control system according to the embodiment. [Figure 15] FIG. 15 is a flowchart showing an example of the flow of a driving process performed by the vehicle device according to the embodiment. [Figure 16] FIG. 16 is a flowchart showing an example of the flow of imaging processing by the control system according to the embodiment. [Figure 17] FIG. 17 is a flowchart showing an example of the flow of the flying object control process by the vehicle device according to the embodiment. [Figure 18] FIG. 18 is a flowchart showing an example of the flow of the railroad crossing travel processing by the vehicle device according to the embodiment. [Figure 19] FIG. 19 is a flowchart showing an example of the flow of a process for traveling across a railroad crossing without blocking the railroad crossing by the vehicle device according to the embodiment. [Figure 20] FIG. 20 is a flowchart showing an example of the flow of the turnout traveling process performed by the vehicle device according to the embodiment. [Figure 21] FIG. 21 is a diagram showing a modified example of the configuration of the control system according to the embodiment. [Figure 22] FIG. 22 is a diagram illustrating an example of the configuration of a server according to the embodiment. [Figure 23] FIG. 23 illustrates an example of a computer that executes a control program. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, with reference to the drawings, embodiments of a vehicle device, a control system, a control method, and a control program according to the present application will be described in detail. Note that the present invention is not limited to these embodiments. In addition, in the description of the drawings, the same parts are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0010] [Control system configuration] First, the configuration of a control system according to the first embodiment will be described. FIG. 1 is a diagram illustrating an example of the overall configuration of the control system 1 according to the first embodiment. The control system 1 illustrated in FIG. 1 includes a vehicle device 100, an aircraft 200, and a terminal device 300. The vehicle device 100, the aircraft 200, and the terminal device 300 are connected to each other via a predetermined communication network (network N) so as to be able to communicate with each other via wired or wireless communication. For example, the vehicle device 100 and the aircraft 200 are connected via Bluetooth (registered trademark). The vehicle device 100 and the terminal device 300 are also connected via a wireless LAN (Local Area Network). Note that the above connection method is an example, and other known connection methods are also possible.

[0011] The vehicle device 100 is an unmanned vehicle that inspects tracks while traveling on the track by automatic travel or remote control, and is realized by a vehicle equipped with a computer or the like. For example, the vehicle device 100 travels on the track while inspecting for abnormalities in the track, construction gauge, road surface, embankments, cuttings, etc. Note that the construction gauge is the space required for safe train operation. Furthermore, cuttings are areas where a slope such as a mountain has been removed in order to lay tracks on the slope.

[0012] The vehicle device 100 includes various sensors. For example, the vehicle device 100 includes some or all of various sensors, such as an imaging device, a distance measurement sensor such as a LiDAR (Light Detection and Ranging), a millimeter-wave laser, a cross-sectional measurement laser, a rotary rangefinder, a positioning sensor such as a GNSS (Global Navigation Satellite System), an inertial measurement sensor such as an IMU (Inertial Measurement Unit), a barometric pressure sensor, a geomagnetic sensor, an infrared sensor, a microphone sensor, a Bluetooth antenna, an LTE (Long Term Evolution) antenna, a wireless LAN antenna, and a Starlink (registered trademark) antenna. Note that the above are merely examples of sensors included in the vehicle device 100, and the vehicle device 100 may include various other sensors used for inspection purposes.

[0013] The imaging device is a sensor for acquiring image data used to check the surrounding conditions of the vehicle device 100. The distance measurement sensor is a sensor for checking the surrounding conditions by, for example, irradiating light and measuring the distance to an object using the time difference until the reflected light is received. The millimeter-wave laser is a sensor for checking the surrounding conditions by irradiating electromagnetic waves with a wavelength of 1 to 10 mm and measuring the distance to an object using the time difference until the reflected electromagnetic waves are received.

[0014] A cross-sectional measurement laser is a sensor that measures the cross-sectional shape of an object without contact by irradiating the object with a strip-shaped laser and receiving the changes in the reflected light with an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or SPAD (Single Photon Avalanche Diode) to check the surrounding conditions. A rotary rangefinder is a sensor that rolls a wheel installed along the surface of the object to measure the distance traveled by the wheel and checks the surrounding conditions. A positioning sensor is a sensor that identifies the current position by using radio waves to measure the distance between the current position and multiple satellite communications. An inertial measurement sensor is a sensor that measures three-dimensional inertial motion.

[0015] A Bluetooth antenna is an antenna that enables communication using Bluetooth, a short-range wireless standard. An LTE antenna is an antenna that enables communication using LTE, a communication standard exclusively for mobile devices. A wireless LAN antenna is an antenna that enables radio wave communication using a LAN built using wireless communication. A starlink antenna is an antenna that enables radio wave communication via artificial satellites.

[0016] The vehicle device 100 includes various payloads. For example, the vehicle device 100 includes some or all of the various payloads, such as an air vehicle 200, a replacement battery for the air vehicle 200, a wired connection cable for connecting to the air vehicle 200, a light, a robot, a replacement battery for the robot, a wired connection cable for connecting to the robot, a computer for detecting abnormalities, a computer for SLAM (Simultaneous Localization and Mapping), a patrol lamp, a speaker, fuel, an operation remote control, and a railroad crossing control device. The light turns on, changes its illuminance, turns off, etc., depending on a predetermined function. For example, the light functions as a warning light for a railroad crossing and flashes repeatedly at a predetermined interval.

[0017] A robot moves using multiple legs that it possesses. For example, a robot walks or runs using two or four legs that it possesses. A robot can perform processing performed by an air vehicle 200. Furthermore, a robot can be the target of processing for the air vehicle 200, instead of the air vehicle 200. Hereinafter, when processing performed by the air vehicle 200 or processing targeted for the air vehicle 200 is processing performed by a robot or processing targeted for a robot, "air vehicle" will be read as "robot" or "air vehicle or robot." Furthermore, "flight" will be read as "movement." Movement includes methods of movement such as flying, walking, and running.

[0018] Here, specific examples of the vehicle device 100 will be described with reference to FIGS. 2 to 4. FIGS. 2 to 4 are diagrams illustrating specific examples of the vehicle device 100. FIG. 2(1) is a top view of the vehicle device 100. FIG. 2(2) is a front view of the vehicle device 100. FIG. 2(3) is a rear view of the vehicle device 100. FIG. 2(4) is a diagram illustrating a fallen tree on the tracks. As illustrated in FIGS. 2(1) to 2(3), the vehicle device 100 is, for example, a vehicle with a low vehicle height and retractable tires. For example, since the vehicle device 100 has a low vehicle height, it can pass under obstacles such as a fallen tree even in a situation such as that shown in FIG. 2(4), and can continue traveling without hindrance. Note that the vehicle device 100 may be equipped with retractable tires that can travel on track surfaces as well as tires that can travel on non-track surfaces. The vehicle device 100 can store unused tires inside the vehicle.

[0019] As described above, a low vehicle height is important for continuing to travel even when an obstacle is present, but in order to acquire more data as an inspection vehicle, it is desirable to provide sensors such as LiDAR and an image capture device at a higher position. Therefore, as an example of providing sensors at a higher position, the vehicle device 100 can travel on a track with a panel on the top surface of the vehicle device 100 equipped with sensors such as LiDAR, an image capture device, and Bluetooth raised, as shown in FIG. 3. To provide sensors at a higher position, the sensor support portion may be extended and retracted vertically.

[0020] 2(2), the vehicle device 100 can carry an air vehicle 200 such as a drone, and as shown in FIG. 4(1), can control the flight of the air vehicle 200 that can be carried. When the panel on the upper surface of the vehicle device 100 is raised, the panel on the upper surface can be raised in a direction according to the purpose so that various sensors are oriented in the direction in which an abnormality is to be detected or in the direction in which the vehicle device 100 is traveling.

[0021] 4(2), the vehicle device 100 communicates with the flying object 200 in flight using wireless technology such as Bluetooth to acquire absolute position information of the flying object 200 or relative position information of the flying object 200 with the vehicle device 100 as a reference. The vehicle device 100 can also control the flight of the flying object 200 to land it on the vehicle device 100. The vehicle device 100 can also supply power to the flying object 200 that has landed on or been stored in the vehicle device 100.

[0022] Next, the traveling of the vehicle device 100 will be described with reference to Figs. 5 to 7. Figs. 5 to 7 are diagrams for describing an example of traveling of the vehicle device 100. In Fig. 5, the traveling trajectory of the vehicle device 100 is indicated by an arrow, and the railroad tracks are indicated by a thick line. As shown in Fig. 5, the vehicle device 100 can travel by itself from a place where the vehicle device 100 is stored, such as a station building, to a location on the tracks, such as a railroad crossing. Furthermore, the vehicle device 100 can be placed on or off the tracks from a location where it can be left off the tracks. Therefore, the vehicle device 100 can travel from a station building to a destination on the tracks and return to the station building again.

[0023] Here, loading onto a track refers to the act of loading a road-rail vehicle such as the vehicle device 100 onto a target track. For example, loading onto a track includes not only the act of loading a road-rail vehicle onto a track from outside the track, but also the act of moving a road-rail vehicle from a siding to the main track and the act of moving a road-rail vehicle onto the main track using a crossing device. Leaving off the track refers to the act of moving a road-rail vehicle away from a target track. Leaving off the track includes not only the act of moving a road-rail vehicle off the track, but also the act of moving a road-rail vehicle from the main track to a siding and the act of moving a road-rail vehicle onto a siding using a crossing device.

[0024] If the vehicle device 100 is unable to return due to a problem with the vehicle device 100 or the return route, the vehicle device 100 can escape from the tracks via a location where the height of the tracks and the crossing road are approximately the same, such as a railroad crossing, to an empty lot, for example, as shown in Figure 5.

[0025] The vehicle device 100 is a vehicle that meets various requirements for automatic track inspection. For example, the vehicle device 100 is light enough that three or fewer users can manually place it on the track when the vehicle device 100 is stopped. The vehicle device 100 also has a battery capacity or fuel capacity required for long-distance travel. The vehicle device 100 can operate even after being in an inoperative state for a long period of time due to a sleep function or maintenance function. The vehicle device 100 can continue inspections even at night by using a high-sensitivity camera or illuminating a light. The vehicle device 100 is water-resistant or waterproof, allowing it to travel even in bad weather such as rain or wind. The vehicle device 100 can remove small obstacles (e.g., crow-like stones) on the track while traveling by using structures such as mudguards provided near the wheels.

[0026] 6(1) and 6(2) without colliding with people or vehicles. For example, the vehicle device 100 can travel across the railroad crossing by adjusting the short circuit or by issuing a command to the railroad crossing control device.

[0027] Furthermore, the vehicle device 100 can pass through turnouts in addition to the above-mentioned railroad crossings. Here, with reference to FIG. 7, passing through a turnout by the vehicle device 100 will be described. FIG. 7 shows an image of a turnout and the results of image recognition. For example, first, the vehicle device 100 performs image recognition processing on images of turnouts such as those shown on the left side of FIG. 7(1) and the left side of FIG. 7(2). Next, as a result of the image recognition processing, the vehicle device 100 detects (confirms) the opening direction of the turnout as shown on the right side of FIG. 7(1) and the right side of FIG. 7(2). Then, the vehicle device 100 continues traveling when the traveling direction of the vehicle device itself and the opening direction of the turnout match. Note that details of a method for confirming the opening direction of the turnout will be described later in the description of the detection unit 136.

[0028] Returning to the explanation of FIG. 1 , the aircraft 200 is an aircraft that can be connected to the vehicle device 100 via wireless technology such as Bluetooth. The connection between the vehicle device 100 and the aircraft 200 may be a wired connection including a power supply function. For example, the aircraft 200 is a drone that flies under the control of the vehicle device 100. The aircraft 200 can also fly autonomously. The aircraft 200 has various sensors such as an imaging device. The aircraft 200 can acquire its position information using a positioning sensor such as a GNSS. Information from the sensors of the aircraft 200 can be received, processed, and output by the vehicle device 100. As a result, even if the aircraft 200 is unable to communicate via LTE, it can be output to the terminal device 300 via the vehicle device 100.

[0029] The terminal device 300 is a computer such as a tablet or smartphone that can communicate with any vehicle device 100 via a wireless communication network including 4G to 5G (Generations), LTE (Long Term Evolution), and Bluetooth, or via a wired connection. For example, the terminal device 300 displays information output from the vehicle device 100. The terminal device 300 also has a screen such as a liquid crystal display with a touch panel function, and can accept various operations on displayed data such as content, such as tapping, sliding, and scrolling, performed by a user's finger or a stylus.

[0030] Next, an implementation overview using the control system 1 according to the embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining an implementation overview using the control system 1 according to the embodiment. Fig. 8 shows a vehicle device 100 traveling on a track and an aircraft 200 flying under the control of the vehicle device 100.

[0031] Traditionally, railroad track inspection work has been performed by an inspection vehicle that inspects the tracks and a user who operates the inspection vehicle. In recent years, technology has been proposed that enables unmanned running inspection vehicles to detect obstacles. Furthermore, in order to reduce the burden on users, technology has been proposed that detects abnormalities from image data captured by a camera installed on a vehicle traveling on the tracks.

[0032] However, the above-mentioned conventional technology does not necessarily reduce the burden on the user during inspection. For example, it is not possible to automatically determine whether the unmanned vehicle can continue driving when it detects an abnormality such as an obstacle, and to allow the vehicle to continue driving if it is possible to do so. Furthermore, for areas that cannot be photographed by the vehicle's on-board camera, such as an abnormality in the embankment or a collapsed slope, the user conducting the inspection must actually visit the site to check and take photographs of the surrounding conditions using a camera, as in the past.

[0033] It is also anticipated that drones and other flying objects will be used to inspect areas that cannot be inspected by vehicles running on the tracks, but there are technical issues such as the difficulty of accurately obtaining GNSS information on the drone's position and direction due to the influence of high-voltage power lines running over the tracks and other surrounding environmental factors, which can cause the drone to move in an unintended direction or position, resulting in collisions, crashes, landing failures, etc. For this reason, when using drones for track inspections, there have been cases where users have had to operate the drone on-site.

[0034] 8, the vehicle device 100 travels on the track while inspecting the structure gauge and the area around the track, which are spaces necessary for the safe running of a train, using sensors provided therein, and determines whether the vehicle device 100 is able to travel or not in accordance with an abnormality detected based on data acquired from the sensors mounted on the vehicle, and continues traveling if it is able to travel. Furthermore, when an abnormality is detected, the vehicle device 100 controls the flight of the air vehicle 200 while correcting position information of the air vehicle 200 in accordance with the positional relationship between the vehicle device 100 and the air vehicle 200 such as a drone, and acquires image data captured in an area that cannot be captured by the imaging device mounted on the vehicle device 100.

[0035] As a result, the vehicle device 100 collects data while traveling using sensors provided on the vehicle device 100, and even if an abnormality is detected based on the data, if the abnormality allows the vehicle to continue traveling, the vehicle device 100 can continue traveling, thereby automating track inspections and reducing the burden on the user during inspections. Also, by acquiring image data from the sky in cooperation with the air vehicle 200 that can be loaded on the vehicle device 100, it becomes possible to check situations that are difficult to check using the imaging device provided on the vehicle device 100, and the burden on the user during inspections can be further reduced.

[0036] [Vehicle device configuration] Next, the configuration of vehicle device 100 will be described with reference to Fig. 9. As shown in Fig. 9, vehicle device 100 has a communication unit 110, a storage unit 120, a control unit 130, and a drive unit 140. Note that each of these units may be held in a distributed manner by multiple devices. The processing of each of these units will be described below.

[0037] The communication unit 110 is realized by a NIC (Network Interface Card) or the like, and enables communication between an external device and the control unit 130 via an electric communication line such as a LAN or the Internet. For example, the communication unit 110 enables communication between the external device and the control unit 130.

[0038] The storage unit 120 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 has a travel information storage unit 121, a travel route storage unit 122, an image storage unit 123, a model storage unit 124, and a sensor information storage unit 125.

[0039] The travel information storage unit 121 stores information related to travel. For example, the travel information storage unit 121 stores information such as the current location of the vehicle device 100, a start location, a destination location, locations to be monitored intensively, structures, track information, locations where the vehicle can be unloaded, abnormalities in the surrounding conditions, weather, date and time, and instructions according to the opening direction of a turnout.

[0040] The movement route storage unit 122 stores the route taken by the flying object 200. For example, the movement route storage unit 122 stores a flight route including the direction in which the flying object 200 will fly, the position of the flying object 200, and the direction in which an image will be captured by an imaging device mounted on the flying object 200.

[0041] Image storage unit 123 stores image data including still images and moving images. For example, image storage unit 123 stores image data captured by imaging unit 135 (described later) and image data acquired by image acquisition unit 138. Note that the image data stored in image storage unit 123 includes information on the location and date / time at which the image was captured.

[0042] The model storage unit 124 stores a first trained model and a second trained model. Both the first trained model and the second trained model are models that output the presence or absence of an abnormality and the type of abnormality according to input image data. Here, the first trained model is a model that has learned about the relationship between image data captured by the imaging unit 135 and the type of abnormality included in the image data captured by the imaging unit 135. On the other hand, the second trained model is a model that has learned about the relationship between image data captured by the imaging unit 135 and the moving object imaging unit 231 and the type of abnormality included in the image data captured by the imaging unit 135 and the moving object imaging unit 231.

[0043] That is, the first trained model receives as input image data captured by the imaging unit 135 and determines the presence or absence of an abnormality and the type of abnormality. On the other hand, the second trained model receives as input image data captured by the moving object imaging unit 231 in addition to image data captured by the imaging unit 135 and determines the presence or absence of an abnormality and the type of abnormality. The first trained model and the second trained model can recognize objects and object positions used to detect railroad crossing locations, such as nearby railway signs, road signs, cars, and people.

[0044] The model storage unit 124 further stores a third trained model that outputs the opening direction of a turnout in accordance with the input image data. The third trained model is a model that has learned the relationship between the image data captured by the imaging unit 135 and the opening direction of the turnout included in the image data captured by the imaging unit 135.

[0045] The first trained model, the second trained model, and the third trained model can use information obtained by a sensor as input instead of or in addition to image data. For example, the first trained model may be a model that uses point cloud data acquired by LiDAR as input and learns about the relationship between the type of anomaly contained in the point cloud data acquired by LiDAR.

[0046] For example, the second trained model described above may be a model that uses point cloud data acquired by LiDAR as input and learns about the relationship with the types of anomalies contained in the point cloud data.

[0047] For example, the above-mentioned third trained model may be a model that uses point cloud data information acquired by LiDAR as input and learns about the relationship with the opening direction of the switch included in the point cloud data.

[0048] The sensor information storage unit 125 stores information obtained by a sensor. For example, the sensor information storage unit 125 stores various information such as the distance to an object and the shape obtained by LiDAR. Note that the above is merely an example, and the sensor information storage unit 125 stores information obtained by various sensors included in the vehicle device 100, such as a cross-section measurement laser and a positioning sensor.

[0049] The control unit 130 is realized using a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an NP (Network Processor), an FPGA (Field Programmable Gate Array), or the like, and executes a processing program stored in memory. As shown in Fig. 9, the control unit 130 has an acquisition unit 131, a determination unit 132, a travel determination unit 133A, an energy determination unit 133B, a travel control unit 133C, a power supply unit 134, an imaging unit 135, a detection unit 136, a movement control unit 137, an image acquisition unit 138, and an output unit 139. Each unit of the control unit 130 will be described below.

[0050] The acquisition unit 131 acquires information related to traveling. For example, the acquisition unit 131 acquires information such as the start point, current point, destination point, locations to be monitored intensively, locations where the train can be unloaded, structures, track information, and weather from an external device. The acquisition unit 131 also acquires information on abnormalities in the surrounding conditions from the detection unit 136.

[0051] Based on the information related to traveling, the determination unit 132 determines the traveling speed of the vehicle device 100. For example, the determination unit 132 determines the speed at which the vehicle device 100 travels on the track based on information such as the current location, the destination point, locations to be monitored intensively, locations where the vehicle device 100 can leave the track, structures, track information, and weather.

[0052] To explain this using a more specific example, when the current location of the vehicle device 100 overlaps with a location where monitoring is to be prioritized, the decision unit 132 sets the traveling speed of the vehicle device 100 to be less than a predetermined threshold (e.g., 6 km / h). Furthermore, when the weather is bad, such as rain or a strong wind, the decision unit 132 sets the traveling speed of the vehicle device 100 to be less than a predetermined threshold (e.g., 15 km / h). Furthermore, when an abnormality in the surrounding conditions is detected, the decision unit 132 sets the traveling speed of the vehicle device 100 to be less than a predetermined threshold (e.g., 4 km / h). Note that the decision unit 132 may set the speed of the vehicle device 100 depending on the type of abnormality in the surrounding conditions.

[0053] The travel determination unit 133A determines whether or not the vehicle device 100 is allowed to travel, depending on an abnormality in the surrounding conditions detected based on spatial data acquired by a sensor mounted on the vehicle device 100. Here, the spatial data includes, for example, data acquired by a LiDAR, image data captured by an imaging device, etc. For example, the travel determination unit 133A determines whether or not the vehicle device 100 is allowed to travel, depending on the position, size, and type of an obstacle present in the detected construction gauge.

[0054] To explain this using a more specific example, if the detected abnormality in the surrounding conditions is something that can be dealt with by crawling under or removing, such as a fallen tree or a small stone, the travel determination unit 133A determines that the vehicle device 100 can travel. On the other hand, if the detected abnormality in the surrounding conditions is something that cannot be dealt with, such as a landslide, the travel determination unit 133A determines that the vehicle device 100 cannot travel.

[0055] Furthermore, the traveling determination unit 133A determines whether or not the vehicle device 100 is allowed to travel, depending on an abnormality in the surrounding conditions detected based on image data captured by the imaging unit 135. For example, when the type of abnormality in the surrounding conditions detected based on image data captured by the imaging unit 135 is an avalanche, the traveling determination unit 133A determines that the vehicle device 100 is not allowed to travel.

[0056] As another example, when an obstacle can be avoided by a railroad crossing blocking operation, the traveling determination unit 133A determines that traveling of the vehicle device 100 is possible. For example, when the detection unit 136 determines that an obstacle present in the railroad crossing is a human or a vehicle, the traveling determination unit 133A determines that the obstacle can be avoided by a railroad crossing blocking operation and that traveling of the vehicle is possible.

[0057] Furthermore, the running determination unit 133A can determine whether the vehicle device 100 can run based on not only obstacles on the tracks or abnormalities around the tracks, but also the condition of the tracks themselves and the area below the tracks. For example, the running determination unit 133A determines that the vehicle device 100 can run based on data acquired by a sensor such as a LiDAR or a cross-section measurement laser, and detects abnormalities in the track conditions, such as differences in track elevation, distances between tracks, and irregularities in the length direction of the side surfaces, but detects that the tracks are not broken.

[0058] Furthermore, the running determination unit 133A determines that the vehicle device 100 is capable of running based on an abnormality in which there is a slight shortage of stones under the tracks detected from data acquired by sensors such as a millimeter-wave laser, a distance measurement laser, a line sensor, etc. Note that the above-described method for determining whether or not the vehicle device 100 is capable of running is merely an example, and the running determination unit 133A can determine whether or not the vehicle device 100 is capable of running by combining other known methods.

[0059] As another example, the traveling determination unit 133A determines whether the vehicle device 100 can travel through the railroad crossing based on the presence and / or movement of an object around the railroad crossing detected based on spatial data acquired by a sensor mounted on the vehicle device 100.

[0060] The traveling determination unit 133A determines that the vehicle device 100 can travel across the railroad crossing when no object is detected within a predetermined range from the railroad crossing based on data acquired by a sensor such as LiDAR. The predetermined range can be set arbitrarily depending on the purpose, such as preventing accidents or ensuring safety. Furthermore, the traveling determination unit 133A determines that the vehicle device 100 can travel across the railroad crossing when the data acquired by a sensor such as LiDAR detects that the direction of movement of an object around the railroad crossing is a direction away from the railroad crossing.

[0061] The detection direction performed by the sensors that detect the presence and movement direction of objects around the railroad crossing may be lateral to the traveling direction of the vehicle device 100. In other words, the various sensors provided in the vehicle device 100 may be installed or configured so as to detect an object moving toward the tracks from the side of the tracks in an attempt to cross the railroad crossing.

[0062] As another example, the travel determination unit 133A determines whether the vehicle device 100 is allowed to travel, depending on the opening direction of the turnout. For example, the travel determination unit 133A determines that the vehicle device 100 is allowed to travel when the traveling direction identified from the current position and destination of the vehicle device 100 matches the opening direction of the turnout. Conversely, the travel determination unit 133A determines that the vehicle device 100 is not allowed to travel when the traveling direction of the vehicle device 100 does not match the opening direction of the turnout.

[0063] Note that a case where the traveling direction of the vehicle device 100 and the open direction of the switch do not match is also included as a case where the switch is not open. Here, "not open" refers to, for example, a situation where the track branches from direction A to directions B and C, and the section between directions A and B is open, as viewed from direction C toward direction A.

[0064] Next, a description will be given of the processing performed by the traveling determination unit 133A when the opening direction of the turnout does not match the traveling direction of the vehicle device 100. When the opening direction of the turnout does not match the traveling direction of the vehicle device 100, the traveling determination unit 133A determines whether or not it has received an "instruction for when the opening direction of the turnout does not match the traveling direction of the vehicle device 100."

[0065] For example, if the information on instructions according to the turnout opening direction stored in the memory unit 120 includes information such as "instruction when the turnout opening direction does not match the traveling direction of the vehicle device 100: wait," the traveling determination unit 133A determines that the information includes "instruction when the turnout opening direction does not match the traveling direction of the vehicle device 100." Then, the traveling determination unit 133A causes the traveling control unit 133C to perform an operation (such as wait) according to the instruction.

[0066] On the other hand, if the information on instructions according to the turnout opening direction stored in the memory unit 120 does not include "an instruction for when the turnout opening direction does not match the traveling direction of the vehicle device 100," the traveling determination unit 133A determines that it does not include "an instruction for when the turnout opening direction does not match the traveling direction of the vehicle device 100." Then, the traveling determination unit 133A causes the traveling control unit 133C to return the vehicle device 100.

[0067] In the above example, the information on instructions according to the opening direction of the switch is stored in advance in the memory unit 120. However, if the opening direction of the switch does not match the traveling direction of the vehicle device 100, the traveling judgment unit 133A can request instructions according to the opening direction of the switch from an external device such as a server device, and can perform the above processing after receiving the response information.

[0068] The energy determination unit 133B determines whether the remaining energy of a predetermined device is less than a predetermined threshold. For example, the energy determination unit 133B determines whether the remaining energy of the vehicle device 100 or the flying object 200 is less than a threshold. At this time, the energy determination unit 133B can set the predetermined threshold based on information such as the current location, the destination location, and the departure location. For example, the energy determination unit 133B sets the remaining energy amount required for the vehicle device 100 to return to the departure location after arriving at the destination location from the current location as the threshold. Furthermore, when the vehicle device 100 returns, the energy determination unit 133B sets the remaining energy amount required for the vehicle device 100 to return to the departure location from the current location as the threshold.

[0069] As another example, the energy determination unit 133B may set the threshold value to the remaining amount of energy required for the flying object 200 to fly along the flight route and then return to the vehicle device 100. Furthermore, when the flying object 200 returns, the energy determination unit 133B may set the threshold value to the remaining amount of energy required for the flying object 200 to fly back to the vehicle device 100.

[0070] The traveling control unit 133C controls the traveling of the vehicle device 100. For example, when the traveling determination unit 133A determines that the vehicle device 100 is capable of traveling and the energy determination unit 133B determines that the remaining energy amount of the vehicle device 100 is equal to or greater than a threshold, the traveling control unit 133C allows the vehicle device 100 to continue traveling, and when the traveling determination unit 133A determines that the vehicle device 100 is not capable of traveling or the energy determination unit 133B determines that the remaining energy amount of the vehicle device 100 is not equal to or greater than a threshold, the traveling control unit 133C returns the vehicle device 100 to its home position. Note that the traveling control of returning the vehicle device 100 by the traveling control unit 133C also includes off-track situations.

[0071] For example, when the driving determination unit 133A determines that driving is possible and the energy determination unit 133B determines that the energy of the vehicle device 100 is equal to or greater than a predetermined threshold, the driving control unit 133C causes the vehicle device 100 to continue driving toward the destination at the driving speed determined by the determination unit 132.

[0072] On the other hand, when the driving determination unit 133A determines that the vehicle device 100 is not capable of driving, or when the energy determination unit 133B determines that the remaining energy of the vehicle device 100 is not equal to or greater than the predetermined threshold, the driving control unit 133C causes the vehicle device 100 to return toward the departure point at the speed determined by the determination unit 132. At this time, in addition to causing the vehicle device 100 to return, the driving control unit 133C may also cause the vehicle device 100 to wait in a place where it will not obstruct other moving objects.

[0073] When returning the vehicle device 100, if the energy determination unit 133B determines that the remaining energy of the vehicle device 100 is not equal to or greater than a predetermined threshold required for returning, the traveling control unit 133C causes the vehicle device 100 to leave the line from a location where it can be left and retreat to a predetermined position outside the tracks.

[0074] The traveling control unit 133C can also control railroad crossings. For example, the traveling control unit 133C opens and closes the railroad crossing by issuing a command to a railroad crossing control device to open and close the railroad crossing.

[0075] As another example, when the traveling determination unit 133A determines that the vehicle device 100 can travel across the railroad crossing, the traveling control unit 133C causes the vehicle device 100 to issue an alarm while crossing the railroad crossing. Here, the alarm issued when the vehicle device 100 crosses the railroad crossing is an act of informing the surroundings that the vehicle device 100 is about to cross the railroad crossing or is currently crossing the railroad crossing.

[0076] For example, the traveling control unit 133C causes the vehicle device 100 to notify those around it that it is currently crossing a railroad crossing by sounding an alarm through a speaker, playing a voice message such as "You are currently crossing a railroad crossing!", flashing a light, or the like. Then, the traveling control unit 133C causes the vehicle device 100 to cross the railroad crossing while issuing an alarm. At this time, the traveling control unit 133C causes the vehicle device 100 to cross the railroad crossing at a predetermined speed. The predetermined speed can be set arbitrarily depending on the purpose, such as preventing accidents or ensuring safety. For example, the traveling control unit 133C causes the vehicle device 100 to cross the railroad crossing at a slow speed while issuing an alarm.

[0077] The power supply unit 134 supplies power to the flying object 200. For example, the power supply unit 134 charges or replaces the battery of the flying object 200. To explain this in more detail, the power supply unit 134 connects a connector provided in the vehicle device 100 to a charging port of the flying object 200 to charge the battery of the flying object 200. The power supply unit 134 also replaces a replacement battery provided in the vehicle device 100 with the battery provided in the flying object 200. At this time, the power supply unit 134 can charge the replaced battery by connecting it to a connector provided in the vehicle device 100. Note that the charging of the battery provided in the flying object 200 by the power supply unit 134 may be performed using a non-contact method. Furthermore, if the remaining energy of the vehicle device 100 is equal to or less than a predetermined threshold, power supply to the flying object 200 may not be performed.

[0078] The imaging unit 135 captures an image of the surrounding environment using an imaging device mounted on the vehicle device 100. For example, the imaging unit 135 captures an image of the surrounding environment of the vehicle device 100 using a camera mounted on the vehicle device 100. Note that the image data captured by the imaging unit 135 may include the date and time when the image data was captured and location information of the location where the image data was captured.

[0079] The detection unit 136 detects abnormalities in the surrounding conditions based on data acquired by sensors mounted on the vehicle device 100. For example, the detection unit 136 detects the presence of an obstacle at the construction gauge from spatial data acquired by a LiDAR mounted on the vehicle device 100. The detection unit 136 also detects an abnormality when the distance between tracks exceeds a predetermined threshold from data acquired by a cross-section measurement laser mounted on the vehicle device 100. The detection unit 136 also detects an abnormality in the placement of stones under the tracks from data acquired by a millimeter-wave laser mounted on the vehicle device 100.

[0080] As another example, the detection unit 136 performs processing using a model stored in the model storage unit 124. For example, the detection unit 136 inputs image data captured by the imaging unit 135 into a first trained model that outputs the presence or absence of an abnormality and the type of abnormality according to the input image data, and acquires the presence or absence of an abnormality and the type of abnormality. For example, the detection unit 136 inputs image data of a fallen tree captured by the imaging unit 135 into the first trained model that has learned the relationship between the image data captured by the imaging unit 135 and the type of abnormality included in the image data captured by the imaging unit 135, and acquires information that an abnormality is "present" and the type of abnormality is "fallen tree."

[0081] Furthermore, the detection unit 136 can also use as input images captured by an imaging device mounted on the flying object 200. The detection unit 136 acquires the presence or absence of an abnormality and the type of the abnormality by inputting the image data captured by the imaging unit 135 and the image data acquired by the image acquisition unit 138 into a second trained model that outputs the presence or absence of an abnormality and the type of the abnormality according to the input image data.

[0082] For example, the detection unit 136 inputs image data of a rockfall captured by the imaging unit 135 and the mobile body imaging unit 231 into a second trained model that has learned the relationship between the image data captured by the imaging unit 135 and the mobile body imaging unit 231 and the type of abnormality contained in the image data captured by the imaging unit 135 and the mobile body imaging unit 231, and obtains information that an abnormality is "present" and the type of abnormality is "rockfall."

[0083] As another example, the detection unit 136 detects an obstacle. For example, the detection unit 136 determines whether or not an obstacle exists at a railroad crossing using a sensor such as a LiDAR or an imaging device mounted on the vehicle device 100. The detection unit 136 also determines whether or not the obstacle is a predetermined object. For example, the detection unit 136 determines whether or not the obstacle is a human or a vehicle using information acquired from a sensor such as a LiDAR mounted on the vehicle device 100.

[0084] As an example, the detection unit 136 performs image recognition processing on image data of an obstacle captured by an imaging device to determine whether the obstacle is a human or a vehicle. The detection unit 136 may also determine whether the obstacle is a human or a vehicle using one or more of data acquired by LiDAR and image data of the obstacle captured by an imaging device. In addition to the above example, the detection unit 136 can apply known technology to information obtained by various sensors to determine whether the obstacle is a human or a vehicle.

[0085] As another example, the detection unit 136 detects an object. For example, the detection unit 136 detects an object present around a railroad crossing using a sensor such as a LiDAR or an imaging device mounted on the vehicle device 100. For example, the detection unit 136 detects the presence or absence of an object within a predetermined range from the railroad crossing based on data acquired by a sensor such as a LiDAR. Note that the predetermined range can be set arbitrarily depending on the purpose, such as preventing accidents or ensuring safety.

[0086] Furthermore, the detection unit 136 detects the movement direction of an object around the railroad crossing using a sensor such as a LiDAR or an imaging device mounted on the vehicle device 100. For example, the detection unit 136 detects that the movement direction of an object around the railroad crossing is a direction away from the railroad crossing using a sensor such as a LiDAR or an imaging device mounted on the vehicle device 100.

[0087] As another example, the detection unit 136 detects the opening direction of a turnout. For example, the detection unit 136 detects the opening direction of a turnout based on captured image data. For example, the detection unit 136 inputs the image data captured by the imaging unit 135 into a third trained model that outputs the opening direction of a turnout in accordance with the input image data, thereby acquiring the opening direction of the turnout.

[0088] More specifically, the detection unit 136 inputs image data (left side of Figure 7(1)) of a switch with the right side as the opening direction, captured by the imaging unit 135, into a third trained model that has learned the relationship between the image data captured by the imaging unit 135 and the opening direction of the switch contained in the image data captured by the imaging unit 135, and obtains information on the opening direction "right side" ((right side of Figure 7(1))).

[0089] When an abnormality in the surrounding conditions is detected based on the image data captured by the imaging unit 135 and the data acquired by the sensor, and / or when the vehicle device 100 arrives at a preset location, the movement control unit 137 controls the flight of the air vehicle 200 that can be loaded onto the vehicle device 100. Here, the preset location is, for example, a pre-set point or section, such as a location where monitoring is desired to be prioritized.

[0090] For example, the movement control unit 137 sets a flight route including the direction in which the flying object 200 will fly, its position, and the direction in which images will be captured by an imaging device mounted on the flying object 200, and controls the flying object 200 to fly along the set flight route.

[0091] For example, the movement control unit 137 sets a flight route including a direction from the vehicle device 100 as the flying object 200 flies, a coordinate position, and a direction from the flying object 200 in which image capture is performed by an image capture device mounted on the flying object 200, and controls the flying object 200 to fly along the set flight route. For example, when an abnormality in the surrounding conditions is detected and it is difficult for the vehicle device 100 to travel, the movement control unit 137 sets a flight route to capture images of areas in which the vehicle device 100 cannot travel, within a range in which the flying object 200 can communicate with the vehicle device 100, and controls the flying object 200 to fly along the set flight route.

[0092] Furthermore, the movement control unit 137 sets a flight route according to the type of abnormality detected by the detection unit 136. Here, the processing of the movement control unit 137 will be described with reference to FIG. 10. FIG. 10 is a diagram for explaining the flight control processing according to the embodiment. FIG. 10 shows an example of the type of abnormality and the flight route corresponding to the type of abnormality. For example, if the type of abnormality detected by the detection unit 136 is a "rockfall," the movement control unit 137 sets a flight route such that the aircraft flies in a direction with a steeper slope from the location where the abnormality was detected. Furthermore, if the type of abnormality detected by the detection unit 136 cannot be determined, the movement control unit 137 sets a flight route such that the aircraft flies in a direction with a steeper slope from the location where the abnormality was detected. Furthermore, if the movement control unit 137 cannot determine the type of abnormality detected by the detection unit 136, it sets a flight route such that the aircraft flies in a circle within a certain range from the location where the abnormality was detected.

[0093] The image acquisition unit 138 acquires image data captured by an imaging device mounted on the flying object 200. For example, the image acquisition unit 138 acquires image data captured by the moving object imaging unit 231 of the flying object 200.

[0094] The output unit 139 outputs information related to the traveling of the vehicle device 100 acquired by the acquisition unit 131, image data captured by the imaging unit 135, abnormalities in the surrounding conditions detected based on the image data captured by the imaging unit 135, and image data acquired by the image acquisition unit 138. When outputting, the output unit 139 can perform a weighting process on the data to be output and output the weighted data. A known weighting technique is used as the weighting method.

[0095] Here, the processing performed by the output unit 139 will be described with reference to Figs. 11 and 11C. Figs. 11C and 11D are diagrams showing an example of output processing by the output unit 139, in which map information including the tracks between two stations is displayed on the screen of the terminal device 300. For example, the output unit 139 causes the terminal device 300 to display, as a configuration screen for the inspection route, information on the start point and end point of the inspection and locations where monitoring will be focused.

[0096] To explain this using a more specific example, the output unit 139 causes the terminal device 300 to display an inspection route configuration screen, as shown in Figure 11, in which the points on the map where the inspection starts and ends are indicated by circles, the section from the point where the inspection starts to the point where the inspection ends is indicated by a dotted line, and the range of areas where monitoring will be prioritized is indicated by a line.

[0097] In addition, the output unit 139 displays on the terminal device 300, as an operation control screen, information on the start and end points of the inspection, and locations where monitoring is to be prioritized, as well as the current location, the location where an abnormality was detected, and image data.

[0098] For example, as shown in Figure 12, the output unit 139 indicates on the map the points where the inspection starts and ends with a circle, the current point with a triangle, the point where an abnormality was detected with a square, the section from the point where the inspection starts to the point where the inspection ends with a dotted line, and the range of areas where monitoring is prioritized with a line, and at the same time displays image data captured at the point where the abnormality was detected on the terminal device 300.

[0099] Here, for example, the output unit 139 may change the display mode by accepting an operation from the user of the terminal device 300. For example, the output unit 139 accepts a tap operation on image data from the user and causes a square, which indicates a point where an abnormality has been detected in the captured image data, to blink.

[0100] Furthermore, the output unit 139 can further output information about locations where the vehicle device 100 can be placed on or removed from the tracks, which information is acquired by the acquisition unit 131. For example, the output unit 139 causes the terminal device 300 to display on the operation control screen the start point, current location, end point, locations to be monitored intensively, locations where an abnormality has been detected, and image data, as well as information indicating the locations of railroad crossings where the vehicle device 100 can be placed on or removed from the tracks.

[0101] The driving unit 140 drives the wheels provided on the vehicle device 100 to travel the vehicle device 100. For example, the driving unit 140 drives the wheels provided on the vehicle device 100 in accordance with the control of the travel control unit 133C to travel the vehicle device 100 in the direction in which the inspection will be performed.

[0102] [Configuration of the aircraft] Next, the configuration of the flying object 200 will be described using Figure 13. As shown in Figure 13, the flying object 200 has a communication unit 210, a control unit 230, and a storage unit 220. Note that each of these units may be held in a distributed manner by multiple devices. The processing of each of these units will be described below.

[0103] The communication unit 210 is realized by a NIC or the like, and enables communication between the control unit 230 and an external device via a telecommunication line such as a LAN or the Internet, Bluetooth, Wi-Fi (registered trademark), etc. For example, the communication unit 210 enables communication between the vehicle device 100 and the control unit 230 via Bluetooth.

[0104] The storage unit 220 is realized by a semiconductor memory element such as a RAM, a flash memory, or a storage device such as a hard disk, an optical disk, etc. As shown in FIG.

[0105] The image storage unit 221 stores image data including still images and moving images. For example, the image storage unit 221 stores image data captured by a moving object imaging unit 231 (described later). Here, the image data may include the date and time when the image data was captured and location information of the location where the image data was captured.

[0106] The control unit 230 is realized using a GPU, a CPU, an NP, an FPGA, etc., and executes a processing program stored in a memory. As shown in Fig. 13, the control unit 230 has a moving object imaging unit 231, a transmitting unit 232, and an aircraft control unit 233. Each unit of the control unit 230 will be described below.

[0107] The moving body imaging unit 231 images the surrounding conditions using an imaging device mounted on the flying body 200. For example, the moving body imaging unit 231 images the surrounding conditions of the vehicle device 100 of the flying body 200 using a camera mounted on the flying body 200.

[0108] The transmission unit 232 transmits image data captured by the moving body imaging unit 231 to the vehicle device 100. For example, the transmission unit 232 transmits still image data and moving image data captured by the moving body imaging unit 231 to the vehicle device 100 via Bluetooth communication. Note that the transmission destination of the transmission unit 232 may be a server in a remote location.

[0109] The aircraft control unit 233 drives the propellers provided on the aircraft 200 and controls the flight of the aircraft 200. For example, the aircraft control unit 233 drives the propellers provided on the aircraft 200 in accordance with the control of the movement control unit 137, causing the aircraft 200 to fly.

[0110] 〔process〕 Next, the overall processing by the control system 1 will be described with reference to Fig. 14. Fig. 14 is a diagram illustrating the overall processing by the control system 1 according to the embodiment. As illustrated in Fig. 14, first, the acquisition unit 131 acquires information related to traveling. Next, the determination unit 132 determines the traveling speed of the vehicle device 100 based on the information related to traveling. Next, the traveling control unit 133C causes the vehicle device 100 to travel at the traveling speed determined by the determination unit 132.

[0111] At this time, the imaging unit 135 captures an image of the surrounding situation using an imaging device mounted on the vehicle device 100. Next, the detection unit 136 inputs the image data captured by the imaging unit 135 into a first trained model that outputs the presence or absence of an abnormality and the type of abnormality according to the input image data, and acquires information that an abnormality is "present" and the type of abnormality is "fallen tree."

[0112] At this time, the determination unit 132 determines the traveling speed of the vehicle device 100 according to the information on the abnormality in the surrounding conditions that has been detected. For example, the acquisition unit 131 acquires the information on the abnormality in the surrounding conditions that has been detected by the detection unit 136. Then, the determination unit 132 sets the traveling speed of the vehicle device 100 to a speed according to the information on the abnormality in the surrounding conditions that has been acquired by the acquisition unit 131. Note that, if the traveling determination unit 133A determines that the vehicle device 100 is not capable of traveling, the traveling control unit 133C stops the vehicle device 100 from traveling.

[0113] Next, since the detection unit 136 has detected an abnormal "fallen tree," the movement control unit 137 determines a flight route for the flying object 200 that will fly in the longitudinal direction of the fallen tree (toward the tip and base of the fallen tree) and capture an image, and controls the flying object 200 to fly along the flight route. Note that if it is difficult to determine the longitudinal direction of the fallen tree, the movement control unit 137 determines a flight route for the flying object that will capture an image of the entire fallen tree, and controls the flying object 200 to fly along the flight route. Next, the moving object imaging unit 231 uses an imaging device mounted on the flying object 200 to fly in the direction of the tip and base of the fallen tree and capture an image of the fallen tree.

[0114] Next, the transmission unit 232 transmits the image data captured by the moving object imaging unit 231 to the vehicle device 100. Next, the image acquisition unit 138 acquires the image data transmitted by the transmission unit 232. Then, the output unit 139 causes the terminal device 300 to display the information related to traveling acquired by the acquisition unit 131, the image data captured by the imaging unit 135, and the image data captured by the moving object imaging unit 231.

[0115] 〔flowchart〕 Next, the flow of the driving process by the vehicle device 100 will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the flow of the driving process by the vehicle device 100 according to the embodiment. Note that the steps below may be executed in a different order, and some steps may be omitted.

[0116] First, the traveling determination unit 133A sets a line-mounted route and a destination point (step S101). For example, the traveling determination unit 133A sets a location where the line-mounted route can be removed, which is included in the information related to traveling acquired by the acquisition unit 131, as a line-mounted location, and sets the destination point as the destination point of the vehicle device 100.

[0117] Next, the determination unit 132 determines the traveling speed of the vehicle device 100 (step S102). Next, the traveling control unit 133C starts traveling of the vehicle device 100 at the speed determined by the determination unit 132 (step S103). Next, the vehicle device 100 performs an imaging process (step S104). The flow of the imaging process will be described with reference to FIG. 16.

[0118] Next, the detection unit 136 detects an abnormality in the surrounding conditions based on the image data captured by the imaging unit 135 and the data acquired by the sensor (step S105). If the detection unit 136 detects an abnormality in the surrounding conditions (step S105: Yes), the travel determination unit 133A determines whether the vehicle device 100 is allowed to travel (step S106).

[0119] If the travel determining unit 133A determines that travel is possible (step S106: Yes), the energy determining unit 133B determines whether there is a problem with the remaining energy amount of the vehicle apparatus 100 (step S107).

[0120] If there is no problem with the remaining energy of the vehicle device 100 (step S107: Yes), the driving control unit 133C determines whether the vehicle device 100 has reached the destination point (step S108). If the driving control unit 133C determines that the vehicle device 100 has reached the destination point (step S108: Yes), the vehicle device 100 ends the processing. On the other hand, if the driving control unit 133C determines that the vehicle device 100 has not reached the destination point (step S108: No), the processing of step S104 is performed again.

[0121] The branching of step S105 will be described below. If the detection unit 136 does not detect any abnormality in the surrounding situation (step S105: No), the process proceeds to step S107.

[0122] The branching of steps S106 and S107 will be described. If the traveling determination unit 133A determines that the vehicle device 100 cannot travel (step S106: No) and if the energy determination unit 133B determines that there is a problem with the remaining energy of the vehicle device 100 (step S107: No), the traveling determination unit 133A determines whether the vehicle device 100 can return (step S109). For example, the energy determination unit 133B determines whether the remaining energy of the vehicle device 100 is equal to or greater than the amount of energy required to travel from the current location to the starting point.

[0123] Here, if the traveling determination unit 133A determines that the vehicle device 100 can return (step S109: Yes), the traveling control unit 133C causes the vehicle device 100 to return (step S110). On the other hand, if the traveling determination unit 133A determines that the vehicle device 100 cannot return (step S109: No), the traveling control unit 133C causes the vehicle device 100 to retreat from the location where the vehicle device 100 may leave the tracks (step S111). Then, the vehicle device 100 ends the processing.

[0124] Next, the flow of the imaging process by the control system 1 will be described with reference to Fig. 16. Fig. 16 is a flowchart showing an example of the flow of the imaging process by the control system 1 according to the embodiment. Note that the steps below may be executed in a different order, and some processes may be omitted.

[0125] First, the vehicle device 100 captures images of the surrounding environment while traveling (step S201). Next, the detection unit 136 determines whether the detected abnormality or the location to be monitored intensively can be confirmed by the imaging device mounted on the vehicle device 100 (step S202). For example, the detection unit 136 determines whether the entirety of the detected abnormality can be captured by the imaging device mounted on the vehicle device 100. For example, if the detected abnormality is a fallen tree, the detection unit 136 determines whether the image capturing device mounted on the vehicle device 100 can capture an image of the fallen tree from its tip to its base.

[0126] If it is determined that confirmation is not possible (step S202: No), the vehicle device 100 performs an aircraft control process (step S203). The flow of the aircraft control process will be described with reference to Fig. 17. Then, the vehicle device 100 ends the process.

[0127] On the other hand, if it is determined that the confirmation is possible (step S202: Yes), the vehicle device 100 determines whether the communication state is good (step S204). Note that a good communication state means, for example, that a positioning sensor such as GNSS is available and LTE communication is possible using an LTE antenna.

[0128] If it is determined that the communication state is good (step S204: Yes), the vehicle device 100 transmits image data of the check location to an external device such as a server device (step S205).

[0129] On the other hand, if it is determined that the communication state is not good (step S204: No), the vehicle device 100 does not transmit the image data of the check point and stores it in the data transmission list (step S206). Thereafter, the vehicle device 100 reverses (step S207). For example, the vehicle device 100 reverses again to a point where the communication state is good. Then, the process from step S204 onwards is performed again.

[0130] Next, the flow of the flying object control process by the vehicle device 100 will be described with reference to Fig. 17. Fig. 17 is a flowchart showing an example of the flow of the flying object control process by the vehicle device 100 according to the embodiment. Note that the steps below may be executed in a different order, and some processes may be omitted.

[0131] First, the vehicle device 100 determines whether or not there is an abnormality in the airframe of the flying object 200 (step S301). For example, the vehicle device 100 determines whether or not there is an abnormality in the airframe of the flying object 200 by inspecting predetermined items such as the weight, the fixed state of the propeller, the battery state, and the remaining battery level of the airframe of the flying object 200. If it is determined that there is no abnormality in the airframe of the flying object 200 (step S301: Yes), the traveling control unit 133C stops the traveling of the vehicle device 100 (step S302).

[0132] Next, the movement control unit 137 sets a flight route for the flying object 200 (step S303). Next, the movement control unit 137 controls the flying object 200 to fly according to the flight route (step S304).

[0133] Next, the vehicle device 100 communicates position information with the flying object 200 (step S305). Next, the movement control unit 137 controls the flying object 200 to return to the vehicle device 100, and stores the returned flying object 200 (step S306).

[0134] Next, the image acquisition unit 138 receives and collects image data captured by the imaging device mounted on the flying object 200 (step S307). Next, the power supply unit 134 charges the battery of the flying object 200 (step S308). Then, the traveling control unit 133C starts traveling of the vehicle device 100 (step S309).

[0135] Here, the branching of step S301 will be described. If it is determined that there is an abnormality in the airframe of the flying object 200 (step S301: No), the vehicle device 100 determines whether the abnormality in the flying object 200 is a battery abnormality (step S310). For example, the vehicle device 100 checks the state of the battery of the flying object 200 to determine whether there is an abnormality in the battery.

[0136] If it is determined that the abnormality in the flying object 200 is a battery abnormality (step S310: Yes), the power supply unit 134 replaces the battery of the flying object 200 (step S311). On the other hand, if it is determined that the abnormality in the flying object 200 is not a battery abnormality (step S310: No), the vehicle device 100 ends the processing.

[0137] Next, the flow of the railroad crossing travel processing by the vehicle device 100 will be described with reference to Fig. 18. Fig. 18 is a flowchart showing an example of the flow of the railroad crossing travel processing by the vehicle device 100 according to the embodiment. Note that the steps below may be executed in a different order, and some processing may be omitted.

[0138] First, the traveling determination unit 133A detects a railroad crossing (step S401). For example, the traveling determination unit 133A detects the railroad crossing from previously acquired position information of the railroad crossing, signs or barriers obtained by image recognition, etc. Also, for example, the traveling determination unit 133A detects the railroad crossing from an image captured by an imaging device mounted on the vehicle device 100.

[0139] Next, the traveling control unit 133C stops the vehicle device 100 before the railroad crossing (step S402). For example, the traveling control unit 133C stops the traveling of the vehicle device 100 before the railroad crossing using the current position of the vehicle device 100 and the position information of the railroad crossing.

[0140] Next, the traveling control unit 133C closes the railroad crossing (step S403). For example, the traveling control unit 133C closes the railroad crossing by issuing a command to close the railroad crossing to the railroad crossing control device.

[0141] Next, the detection unit 136 determines whether or not an obstacle exists in the railroad crossing (step S404). For example, the detection unit 136 determines whether or not an obstacle exists in the railroad crossing by using a sensor such as a LiDAR mounted on the vehicle device 100.

[0142] Here, if the detection unit 136 determines that there is no obstacle in the railroad crossing (step S404: No), the traveling control unit 133C causes the vehicle device 100 to cross the railroad crossing and stops the traveling of the vehicle device 100 (steps S405, 406).

[0143] Then, the traveling control unit 133C opens the railroad crossing (step S407). For example, the traveling control unit 133C opens the railroad crossing by issuing a command to the railroad crossing control device to open the railroad crossing.

[0144] On the other hand, if the detection unit 136 determines that an obstacle exists in the railroad crossing (step S404: Yes), the detection unit 136 determines whether the obstacle is a human or a vehicle (step S408). For example, the detection unit 136 determines whether the obstacle is a human or a vehicle using information acquired from a sensor such as a LiDAR or an imaging device mounted on the vehicle device 100.

[0145] Here, if the detection unit 136 determines that the obstacle is a human or a vehicle (step S408: Yes), the process from step S405 onwards is carried out after the obstacle no longer exists within the railroad crossing.

[0146] On the other hand, if the detection unit 136 determines that the obstacle is not a human or a vehicle (step S408: No), the detection unit 136 detects an abnormality in the surrounding situation (step S409). At this time, the traveling determination unit 133A determines that the obstacle cannot be avoided by performing a railroad crossing blocking operation. Then, the processing of step S407 is performed.

[0147] Next, the flow of the process of traveling through a railroad crossing without blocking the railroad crossing by the vehicle device 100 will be described with reference to Fig. 19. Fig. 19 is a flowchart showing an example of the flow of the process of traveling through a railroad crossing without blocking the railroad crossing by the vehicle device 100 according to the embodiment. Note that the steps below may be executed in a different order, and some steps may be omitted.

[0148] First, the traveling determination unit 133A detects a railroad crossing (step S501). For example, the traveling determination unit 133A detects the railroad crossing from previously acquired position information of the railroad crossing, signs and barriers obtained by image recognition, etc. Also, for example, the traveling determination unit 133A detects the railroad crossing from an image captured by an imaging device mounted on the vehicle device 100.

[0149] Next, the traveling control unit 133C stops the vehicle device 100 before the railroad crossing (step S502). For example, the traveling control unit 133C stops the traveling of the vehicle device 100 before the railroad crossing using the current position of the vehicle device 100 and the position information of the railroad crossing.

[0150] Next, the detection unit 136 detects the presence and movement of an object around the railroad crossing (step S503). For example, the detection unit 136 detects the presence or absence of an object around the railroad crossing and the direction of movement of the object from data acquired by a sensor such as LiDAR.

[0151] Next, the traveling determination unit 133A determines whether an object exists within a predetermined range from the railroad crossing, or whether the moving direction of an object existing around the railroad crossing is a direction approaching the railroad crossing (step S504). Here, if the traveling determination unit 133A determines that an object exists within a predetermined range from the railroad crossing, or whether the moving direction of an object existing around the railroad crossing is a direction approaching the railroad crossing (step S504: Yes), the traveling determination unit 133A determines that the vehicle device 100 cannot travel across the railroad crossing, and performs the process of S503 again.

[0152] On the other hand, if the traveling determination unit 133A determines that there is no object within a predetermined range from the railroad crossing, or that the direction of movement of an object present around the railroad crossing is not in a direction approaching the railroad crossing (step S504: No), the traveling determination unit 133A determines that the vehicle device 100 is capable of traveling across the railroad crossing, and the traveling control unit 133C causes the vehicle device 100 to cross the railroad crossing while issuing an alarm (step S505).

[0153] For example, the traveling control unit 133C causes the vehicle device 100 to cross a railroad crossing while playing an alarm sound or a voice such as "You are currently crossing a railroad crossing!" from a speaker provided in the vehicle device 100.

[0154] Next, the flow of processing for traveling through a turnout by the vehicle device 100 will be described with reference to Fig. 20. Fig. 20 is a flowchart showing an example of the flow of processing for traveling through a turnout by the vehicle device 100 according to the embodiment. Note that the steps below may be executed in a different order, and some processing may be omitted.

[0155] First, the traveling determination unit 133A detects a turnout (step S601). For example, the traveling determination unit 133A detects the turnout from position information of the turnout acquired in advance, signs or barriers obtained by image recognition, etc. Also, for example, the traveling determination unit 133A detects the turnout from an image captured by an imaging device mounted on the vehicle device 100.

[0156] Next, the traveling control unit 133C stops the vehicle device 100 before the turnout (step S602). For example, the traveling control unit 133C stops the traveling of the vehicle device 100 before the turnout using the current position of the vehicle device 100 and the position information of the turnout.

[0157] Next, the detection unit 136 confirms (detects) the opening direction of the turnout (step S603). For example, the detection unit 136 receives the result of image recognition processing performed using the captured image of the turnout as input (for example, the output result of the third trained model) and confirms the opening direction of the turnout.

[0158] Next, the traveling determination unit 133A determines whether or not the opening direction of the turnout coincides with the traveling direction of the vehicle device 100 (step S604). For example, the traveling determination unit 133A determines whether or not the traveling direction of the vehicle device 100 identified from information on the current position and destination of the vehicle device 100 coincides with the opening direction of the turnout detected by the detection unit 136.

[0159] If the travel determining unit 133A determines that they match (step S604: Yes), the travel determining unit 133A determines that travel is possible, and the vehicle device 100 resumes travel (step S605).

[0160] On the other hand, if the traveling determination unit 133A determines that there is no match (step S604: No), the traveling determination unit 133A determines that traveling is not possible, and then determines whether or not an instruction has been received for when the opening direction of the switch and the opening direction of the vehicle device 100 do not match (step S606).

[0161] Here, if the traveling determination unit 133A determines that an instruction has been received (step S606: Yes), the traveling determination unit 133A causes the traveling control unit 133C to perform an operation according to the instruction (step S607).

[0162] On the other hand, if the traveling determination unit 133A determines that the instruction has not been received (step S606: No), the traveling determination unit 133A causes the traveling control unit 133C to return the vehicle device 100 (step S608).

[0163] 〔effect〕 The vehicle device 100 according to the embodiment is an unmanned vehicle device that travels on a track, and includes a travel determination unit 133A that determines whether the vehicle device 100 is able to travel or not, depending on abnormalities in the surrounding conditions detected based on spatial data acquired by sensors mounted on the vehicle device 100, and a travel control unit 133C that allows the vehicle device 100 to continue traveling if the travel determination unit 133A determines that the vehicle device 100 is able to travel, and returns the vehicle device 100 if the travel determination unit 133A determines that the vehicle device 100 is not able to travel.

[0164] As a result, the vehicle device 100 collects data while traveling using sensors provided in the vehicle device 100, and even if an abnormality is detected based on the data, if the abnormality allows the vehicle to continue traveling, the vehicle device 100 can continue traveling, thereby automating track inspections and reducing the burden on users during inspections.

[0165] In addition, the traveling determination unit 133A of the vehicle device 100 according to the embodiment determines whether or not the vehicle device 100 can travel across the railroad crossing based on the presence and / or movement of an object around the railroad crossing detected based on spatial data acquired by the sensor, and when the traveling determination unit 133A determines that the vehicle device 100 can travel across the railroad crossing, the traveling control unit 133C causes the vehicle device 100 to issue an alarm and cross the railroad crossing.

[0166] As a result, the vehicle device 100 detects the presence and / or movement of objects around the railroad crossing, and if there is no risk of the object coming into contact with the vehicle device 100 crossing the railroad crossing, it issues an alarm while crossing, thereby preventing accidents and allowing the vehicle to cross safely even at railroad crossings where it is not possible to operate the crossing barrier.

[0167] Moreover, the vehicle device 100 according to the embodiment can carry an air vehicle 200 or a robot, and further includes a power supply unit 134 that supplies power to the air vehicle 200 or the robot.

[0168] As a result, the vehicle device 100 can supply power to the flying object 200 or robot that can be loaded onto the vehicle device 100, enabling the flying object 200 or robot to fly, walk, run, or perform other movements that exceed the battery capacity of the flying object 200 or robot.

[0169] Furthermore, the vehicle device 100 according to the embodiment can be placed on or removed from the wire at a location where it can be removed from or removed from the wire.

[0170] This allows the vehicle device 100 to automatically move to a target track, and after completing the inspection, automatically move away from the target track and return.

[0171] Furthermore, in the vehicle device 100 according to the embodiment, the travel determination unit 133A determines whether the vehicle device 100 is allowed to travel or not, depending on the opening direction of the turnout.

[0172] As a result, the vehicle device 100 continues traveling if the opening direction of the switch on the traveling route matches the traveling direction of the vehicle device 100, and if they do not match, it can take action in accordance with instructions from a remote location, such as confirming instructions or suspending traveling.

[0173] Furthermore, the travel determination unit 133A of the vehicle device 100 according to the embodiment determines that the vehicle is allowed to travel when the obstacle can be avoided by performing a railroad crossing blocking operation.

[0174] As a result, even if an abnormality is detected in which an obstacle exists at a railroad crossing, the vehicle device 100 can continue running if it can avoid the obstacle by operating the railroad crossing barrier, thereby automating track inspection and reducing the burden on the user during inspection. Furthermore, by operating the railroad crossing barrier, the vehicle device 100 can prevent collision with obstacles such as people or vehicles that may occur when the vehicle device 100 passes through a railroad crossing.

[0175] In addition, the vehicle device 100 according to the embodiment further includes an energy determination unit 133B that determines whether the remaining energy of the vehicle device 100 is less than a threshold value, and the driving control unit 133C allows the vehicle device 100 to continue driving if the driving determination unit 133A determines that the vehicle device 100 is capable of driving and the energy determination unit 133B determines that the remaining energy of the vehicle device 100 is equal to or greater than a predetermined threshold value, and causes the vehicle device 100 to return home if the driving determination unit 133A determines that the vehicle device 100 is not capable of driving or the energy determination unit 133B determines that the remaining energy of the vehicle device 100 is not equal to or greater than the predetermined threshold value.

[0176] As a result, the vehicle device 100 collects data using sensors provided on the vehicle device 100 while traveling, and even if an abnormality is detected based on the data, if the abnormality allows the vehicle to continue traveling, thereby advancing the automation of track inspections, while if there is a risk that the battery level of the vehicle device 100 will be insufficient, the vehicle device 100 is returned, further reducing the burden on the user during inspections.

[0177] In addition, the vehicle device 100 according to the embodiment has an imaging unit 135 that captures images of the surrounding conditions using an imaging device mounted on the vehicle device 100, and a movement control unit 137 that controls the movement (flying, walking, running) of an air vehicle 200 or robot that can be loaded onto the vehicle device 100 when an abnormality in the surrounding conditions is detected based on image data captured by the imaging unit 135 and data acquired by a sensor, and / or when the vehicle device 100 arrives at a pre-set location.

[0178] As a result, when the vehicle device 100 detects an abnormality from data collected by the equipped sensors or when the vehicle device 100 reaches a location where it is conducting priority monitoring, it controls the flight of the aircraft 200, enabling inspection from positions, angles, and directions that would be difficult to inspect using the vehicle device 100 alone.This eliminates the need for the user to operate the aircraft 200 on site, thereby reducing the burden on the user during inspections.

[0179] Furthermore, the vehicle device 100 according to the embodiment includes an image acquisition unit 138 that acquires image data captured by an imaging device mounted on the flying object 200 or the robot.

[0180] As a result, the vehicle device 100 controls the movement of the flying object 200 or robot and acquires image data captured from positions, angles, and directions that would be difficult to inspect using the vehicle device 100 alone, making it possible for inspection to be performed by the flying object 200 or robot without user operation, thereby reducing the burden on the user during inspection.

[0181] In addition, the movement control unit 137 of the vehicle device 100 according to the embodiment sets a flight route including the direction in which the flying object 200 or robot will fly, its position, and the direction in which imaging will be performed by the imaging device mounted on the flying object 200, and controls the flying object 200 to fly along the set flight route.

[0182] As a result, the vehicle device 100 sets a flight route including the flight direction, position, and imaging direction of the flying object 200, and flies it, making it possible to perform inspections using the flying object 200 without user operation, thereby reducing the burden on the user during inspections.

[0183] In addition, the vehicle device 100 according to the embodiment further includes a detection unit 136 that inputs image data captured by the imaging unit 135 into the first trained model, which outputs the presence or absence of an abnormality and the type of abnormality according to the input image data, and acquires the presence or absence of an abnormality and the type of abnormality, and the movement control unit 137 sets a movement route according to the type of abnormality acquired by the detection unit 136.

[0184] As a result, the vehicle device 100 determines a travel route depending on the type of abnormality detected, and automatically acquires image data captured by the aircraft 200 for the areas that need to be checked for each type of abnormality, thereby reducing the burden on the user during inspection.

[0185] In addition, in the vehicle device 100 according to the embodiment, the detection unit 136 inputs the image data captured by the imaging unit 135 and the image data acquired by the image acquisition unit 138 into a second trained model that outputs the presence or absence of an abnormality and the type of abnormality according to the input image data, and acquires the presence or absence of an abnormality and the type of abnormality.

[0186] As a result, the vehicle device 100 can determine the presence or absence of an abnormality and the type of abnormality by further using as input image data captured from the flying object 200 or robot that has moved along a route corresponding to the type of abnormality detected, thereby identifying the type of abnormality that is difficult to identify from the image acquired by the imaging unit 135, preventing false detections, and reducing the burden on the user during inspection.

[0187] In addition, the vehicle device 100 according to the embodiment further includes an acquisition unit 131 that acquires information regarding the traveling of the vehicle device 100, and a determination unit 132 that determines the traveling speed of the vehicle device 100 based on the information regarding the traveling of the vehicle device 100 acquired by the acquisition unit 131.

[0188] This allows the vehicle device 100 to drive slowly in areas where monitoring is desired intensively, thereby acquiring more data and automatically conducting detailed inspections, thereby reducing the burden on the user during the inspection.

[0189] In addition, the vehicle device 100 according to the embodiment further has an output unit 139 that outputs information regarding the traveling of the vehicle device 100 acquired by the acquisition unit 131, image data captured by an imaging device mounted on the vehicle device 100, information regarding abnormalities in the surrounding conditions detected based on the image data, and image data captured by an imaging device mounted on an aircraft 200 or robot that can be loaded on the vehicle device 100.

[0190] As a result, the vehicle device 100 displays on the terminal device 300 information on the starting point, destination point, current location, areas to be monitored intensively, areas where abnormalities have been detected, and image data, allowing the user to check information on the inspection location without going to the site, thereby reducing the burden on the user during the inspection.

[0191] In addition, in the vehicle device 100 according to the embodiment, the acquisition unit 131 acquires information relating to driving including information on locations where the vehicle device 100 can be unloaded, and the output unit 139 further outputs the information on locations where the vehicle device 100 can be unloaded acquired by the acquisition unit 131.

[0192] As a result, the vehicle device 100 displays on the terminal device 300 the starting point, destination point, current location, locations to be monitored intensively, locations where abnormalities have been detected, image data, and information on locations where the vehicle device 100 can be unloaded, allowing the user to check information on the inspection location without going to the site, thereby reducing the burden on the user during the inspection.

[0193] [Modification] As a modified example, the control system 1 may further include a server 400 that performs processing similar to that of the detection unit 136. The modified example will be described with reference to Figs. 21 and 22. Fig. 21 is a diagram showing an example of the configuration of the control system 1 according to the embodiment. The server 400 is a computer that can communicate with any vehicle device 100 via a telecommunications line such as a LAN or the Internet, or a wireless communication network such as LTE. For example, the server 400 processes image data transmitted from the vehicle device 100.

[0194] Fig. 22 is a diagram showing an example of the configuration of the server 400 according to the embodiment. As shown in Fig. 22, the server 400 has a communication unit 410, a control unit 430, and a storage unit 420. Note that these units may be distributed and held by multiple devices. The processing of these units will be described below.

[0195] The communication unit 410 is realized by a NIC or the like, and enables communication between the control unit 430 and an external device via a telecommunication line such as a LAN or the Internet, or via Bluetooth. For example, the communication unit 410 enables communication between the vehicle device 100 and the control unit 430 via Bluetooth.

[0196] The storage unit 420 is realized by a semiconductor memory element such as a RAM, a flash memory, or a storage device such as a hard disk, an optical disk, etc. As shown in FIG. 22 , the storage unit 420 has an image storage unit 421 and a model storage unit 422.

[0197] The image storage unit 421 stores image data including still images and moving images. For example, the image storage unit 421 stores image data captured by the moving object imaging unit 231. Here, the image data includes the date and time when the image data was captured and the location information of the location where the image data was captured.

[0198] The model storage unit 422 stores a first trained model and a second trained model. Both the first trained model and the second trained model are models that output the presence or absence of an abnormality and the type of abnormality according to input image data. Here, the first trained model is a model that has learned about the relationship between image data captured by the imaging unit 135 and the type of abnormality included in the image data captured by the imaging unit 135. On the other hand, the second trained model is a model that has learned about the relationship between image data captured by the imaging unit 135 and the moving object imaging unit 231 and the type of abnormality included in the image data captured by the imaging unit 135 and the moving object imaging unit 231.

[0199] That is, the first trained model determines the presence or absence of an abnormality and the type of the abnormality using image data captured by the imaging unit 135 as input. On the other hand, the second trained model determines the presence or absence of an abnormality and the type of the abnormality using image data captured by the moving object imaging unit 231 as input in addition to the image data captured by the imaging unit 135.

[0200] The control unit 430 is realized using a CPU, NP, FPGA, etc., and executes a processing program stored in a memory. As shown in Fig. 22, the control unit 430 has an image acquisition unit 431 and a detection unit 432. Each unit of the control unit 430 will be described below.

[0201] The image acquisition unit 431 acquires image data from the vehicle device 100. For example, the image acquisition unit 431 acquires image data captured by the imaging unit 135 and image data captured by the moving object imaging unit 231. The detection unit 432 can perform the same processing as the detection unit 136.

[0202] 〔program〕 The vehicle device 100 according to the above-described embodiment is realized by a computer 1000 having a configuration as shown in Fig. 23, for example. Fig. 23 is a diagram showing an example of a computer that executes a control program. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a central processing unit 1030, a memory 1040, a storage 1050, an output IF (Interface) 1060, an input IF 1070, and a communication interface 1080 are connected via a bus 1090.

[0203] The central processing unit 1030 operates based on programs stored in the memory 1040 or storage 1050, programs read from the input device 1020, and the like, and executes various processes. The memory 1040 is a memory device, such as a RAM, that temporarily stores data used by the central processing unit 1030 for various calculations. The storage 1050 is a storage device in which data used by the central processing unit 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), an HDD (Hard Disk Drive), a flash memory, or the like.

[0204] The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a monitor or a printer, and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (registered trademark) (High Definition Multimedia Interface). The input IF 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, camera, etc., and is realized by a USB, etc. The input device 1020 may be an external storage medium such as a device that reads information from an optical recording medium, a magneto-optical recording medium, a tape medium, a magnetic recording medium, or a semiconductor memory, or a USB memory.

[0205] The communication interface 1080 receives data from other devices via the network N and sends it to the central processing unit 1030, and also transmits data generated by the central processing unit 1030 to other devices via the network N. The central processing unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the central processing unit 1030 loads a program from the input device 1020 or the storage 1050 onto the memory 1040 and executes the loaded program.

[0206] For example, when the computer 1000 functions as the vehicle device 100 , the central processing unit 1030 of the computer 1000 executes a program loaded onto the memory 1040 to realize the functions of the control unit 130 .

[0207] Although some embodiments of the present application have been described above with reference to the drawings, these are merely examples, and the present invention may be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the Disclosure of the Invention section. Furthermore, the above-mentioned "section, module, unit" can be read as "means," "circuit," etc. [Explanation of symbols]

[0208] 1. Control System 100 Vehicle equipment 110 Communications Department 120 Storage section 121 Traveling information storage unit 122 Travel route memory section 123 Image storage unit 124 Model Memory Unit 125 Sensor information storage unit 130 Control Unit 131 Acquisition Department 132 Decision Section 133A Driving judgment unit 133B Energy Determination Unit 133C Driving control unit 134 Power Supply Unit 135 Imaging unit 136 Detection unit 137 Movement control unit 138 Image acquisition unit 139 Output Section 200 flying objects 210 Communications Department 220 Storage section 221 Image storage unit 230 Control Unit 231 Mobile imaging unit 232 Transmitter 300 Terminal Equipment 400 servers 410 Communications Department 420 Storage section 421 Image storage unit 422 Model Memory Unit 430 Control Unit 431 Image Acquisition Unit 432 Detection unit

Claims

1. An unmanned vehicle device that travels on a track, a travel determination unit that determines whether the vehicle device is capable of traveling in accordance with an abnormality in a surrounding situation detected based on spatial data acquired by a sensor mounted on the vehicle device; a travel control unit that allows the vehicle device to continue traveling when the travel determination unit determines that the vehicle device is capable of traveling, and that returns the vehicle device when the travel determination unit determines that the vehicle device is not capable of traveling; A vehicle device comprising:

2. The travel determination unit determining whether the vehicle device is capable of traveling through the railroad crossing in accordance with the presence of an object around the railroad crossing and / or the movement of the object detected based on the spatial data acquired by the sensor; The traveling control unit 2. The vehicle device according to claim 1, wherein, when the travel determination unit determines that travel is possible, the vehicle device is caused to cross the railroad crossing while issuing an alarm.

3. 2. The vehicle device according to claim 1, wherein the vehicle device is capable of carrying an aircraft or a robot, and further comprises a power supply unit that supplies power to the aircraft or the robot.

4. The vehicle device according to claim 1, wherein the vehicle device can be mounted on or demounted from a location where the vehicle device can be demounted from or demounted from the line.

5. The travel determination unit 2. The vehicle device according to claim 1, wherein the vehicle device determines whether it is possible to travel depending on the opening direction of a turnout.

6. 2. The vehicle device according to claim 1, wherein the travel determination unit determines that the vehicle is allowed to travel when an obstacle can be avoided by performing a railroad crossing blocking operation.

7. an energy determination unit that determines whether or not the remaining energy amount of the vehicle device is less than a threshold; The traveling control unit When the travel determination unit determines that travel is possible and the energy determination unit determines that the remaining energy amount of the vehicle device is equal to or greater than a predetermined threshold, the vehicle device is allowed to continue traveling, and when the travel determination unit determines that travel is not possible or the energy determination unit determines that the remaining energy amount of the vehicle device is not equal to or greater than a predetermined threshold, the vehicle device is returned home.

2. The vehicle device according to claim 1.

8. an imaging unit that captures an image of a surrounding situation using an imaging device mounted on the vehicle device; a movement control unit that controls movement of an aircraft or a robot that can be loaded onto the vehicle device when an abnormality in the surrounding situation is detected based on image data captured by the imaging unit and data acquired by a sensor, and / or when the vehicle device arrives at a preset location; 4. The vehicle device according to claim 1, further comprising:

9. an image acquisition unit that acquires image data captured by an imaging device mounted on the flying object or robot; 9. The vehicle device according to claim 8, further comprising:

10. The movement control unit A movement route including a direction in which the flying object or the robot will move, a position, and a direction in which the imaging device mounted on the flying object or the robot will take an image is set, and the flying object or the robot is controlled to move along the set movement route.

10. The vehicle device according to claim 9.

11. A detection unit inputs the image data captured by the imaging unit into a first trained model that outputs the presence or absence of an abnormality and the type of the abnormality according to the input image data, and acquires the presence or absence of the abnormality and the type of the abnormality, The vehicle device according to claim 10 , wherein the movement control unit sets the movement route depending on the type of the abnormality acquired by the detection unit.

12. The detection unit inputs the image data captured by the imaging unit and the image data acquired by the image acquisition unit into a second trained model that outputs the presence or absence of an abnormality and the type of the abnormality in accordance with the input image data, and acquires the presence or absence of an abnormality and the type of the abnormality.

12. The vehicle device according to claim 11.

13. an acquisition unit that acquires information about the traveling of the vehicle device; a determination unit that determines a traveling speed of the vehicle device based on information about the traveling of the vehicle device acquired by the acquisition unit; 2. The vehicle device of claim 1, further comprising:

14. an output unit that outputs information about the traveling of the vehicle device acquired by the acquisition unit, image data captured by an imaging device mounted on the vehicle device, information about abnormalities in the surrounding conditions detected based on the image data, and image data captured by an imaging device mounted on an aircraft or robot that can be loaded on the vehicle device.

14. The vehicle device of claim 13, further comprising:

15. the acquisition unit acquires information about the traveling including information about a location where the vehicle device can be unloaded from a line; The vehicle device according to claim 14 , wherein the output unit further outputs information about a location where the vehicle device can be unloaded from a line, the information being acquired by the acquisition unit.

16. A control system comprising an unmanned vehicle device that travels on a track and an air vehicle that is loaded onto the vehicle device and flies in accordance with the control of the vehicle device, The vehicle device includes: an imaging unit that captures an image of a surrounding situation using an imaging device mounted on the vehicle device; a movement control unit that controls movement of an aircraft or a robot that can be loaded onto the vehicle device when an abnormality in the surrounding situation is detected based on image data captured by the imaging unit; an image acquisition unit that acquires image data captured by an imaging device mounted on the flying object or the robot; and The flying object or the robot is a moving body imaging unit that images a surrounding situation using an imaging device mounted on the flying body or the robot; a transmitting unit that transmits image data captured by the moving body imaging unit to the vehicle device; A control system comprising:

17. A control system comprising an unmanned vehicle device that travels on a track, a flying object or a robot that is loaded on the vehicle device and moves in accordance with the control of the vehicle device, and a server that detects abnormalities, The vehicle device includes: an imaging unit that captures an image of a surrounding situation using an imaging device mounted on the vehicle device; a movement control unit that controls movement of the flying object or the robot that can be loaded onto the vehicle device when an abnormality in the surrounding situation is detected based on image data captured by the imaging unit; an image acquisition unit that acquires image data captured by an imaging device mounted on the flying object or the robot; an image transmission unit that transmits image data captured by the imaging unit and image data acquired by the image acquisition unit to the server; and The flying object or the robot is a moving body imaging unit that images a surrounding situation using an imaging device mounted on the flying body or the robot; a transmitting unit that transmits image data captured by the moving body imaging unit to the vehicle device; Has, The server has a detection unit that detects abnormalities in the surrounding situation from the image data transmitted by the image transmission unit. A control system comprising:

18. A computer-implemented method comprising: a travel determination step of determining whether or not the vehicle device is capable of traveling in accordance with an abnormality in a surrounding situation detected based on spatial data acquired by a sensor mounted on the unmanned vehicle device traveling on a track; a travel control step of continuing the travel of the vehicle device when it is determined in the travel determination step that the vehicle device is capable of traveling, and returning the vehicle device when it is determined in the travel determination step that the vehicle device is not capable of traveling; A control method comprising:

19. a travel determination step of determining whether or not the vehicle device is capable of traveling in accordance with an abnormality in a surrounding situation detected based on spatial data acquired by a sensor mounted on the unmanned vehicle device traveling on a track; a travel control step of continuing the travel of the vehicle device when it is determined in the travel determination step that the vehicle device is capable of traveling, and returning the vehicle device when it is determined in the travel determination step that the vehicle device is not capable of traveling; A control program that causes a computer to execute the above.

Images