Remote monitoring system, automatic driving vehicle, recording medium, and program

Abstract

To enhance safety in automatic driving control by an automatic driving vehicle.SOLUTION: A remote monitoring system includes: a second recognition determination section (242) which performs control to receive data acquired by a first recognition determination section of an automatic driving vehicle (1) from a sensor and a first determination result by the first recognition determination section, and where a first reference in performing at least one of recognition and determination is different from that of the first recognition determination section; a determination section (243) which determines whether a comparison result between the second determination result and the first determination result exceeds a second reference; and a processing section (246) for executing processing based on a determination result by the determination section.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a remote monitoring system, an autonomous driving vehicle, a recording medium, and a program. [Background technology]

[0002] Various technologies for improving the safety of autonomous driving control by autonomous vehicles have been proposed. For example, Patent Document 1 describes an autonomous driving program evaluation system that includes a processor and a memory, an evaluation criterion information storage unit that stores evaluation criterion information, which is information on evaluation criteria for evaluating the driving content of a specified vehicle under specified conditions with a specified numerical value, an execution log receiving unit that receives from a specified information processing system an execution log, which is information including the driving history of the vehicle output as an execution result of a specified program that determines the driving content of the specified vehicle under specified conditions, and an evaluation result information generating unit that generates evaluation result information indicating the evaluation of the driving of the specified vehicle under specified conditions with a specified numerical value based on the received execution log and evaluation criterion information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-123259 A Summary of the Invention [Problem to be solved by the invention]

[0004] The prior art described in Patent Document 1 evaluates virtually executed autonomous driving in order to develop an autonomous driving program. For this reason, the data input to the autonomous driving program execution system 10 is also virtual. The virtual data does not necessarily match the real data. For this reason, when a real autonomous vehicle equipped with an autonomous driving program developed by this prior art runs on a road, it does not necessarily make the same recognition and judgment as that made in the virtual space. In other words, the prior art cannot fully guarantee the safety of the autonomous driving of a real autonomous vehicle. In addition, since the virtual data needs to be artificially generated, it takes a lot of effort to operate the autonomous driving program execution system 10. [Means for solving the problem]

[0005] A remote monitoring system according to one embodiment of the present invention includes a communication unit that communicates with an autonomous vehicle having a first recognition and judgment unit that recognizes at least one of an event inside and outside the autonomous vehicle based on data from a sensor and determines how to control the autonomous vehicle based on the recognition result; a second recognition and judgment unit that controls the communication unit to receive from the autonomous vehicle data acquired by the first recognition and judgment unit from the autonomous vehicle, and further recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result, and has a first criterion for at least one of the recognition and judgment that is different from that of the first recognition and judgment unit; a judgment unit that determines whether a comparison result between the second judgment result by the second recognition and judgment unit and the first judgment result exceeds the second criterion; and a processing unit that executes processing based on the judgment result by the judgment unit.

[0006] an autonomous vehicle according to another aspect of the present invention includes a communication unit that communicates with a remote monitoring device that monitors the operation of the autonomous vehicle; a sensor that detects at least one of an internal and external event of the autonomous vehicle and generates data; a first recognition and judgment unit that recognizes the event based on the data from the sensor and determines how to control the autonomous vehicle based on the recognition result; a driving control unit that controls at least one of acceleration / deceleration or steering based on a first judgment result by the first recognition and judgment unit; and a transmission control unit that controls the communication unit to transmit the first judgment result and the data input to the first recognition and judgment unit to the remote monitoring device, wherein the driving control unit recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result, and when the communication unit receives a predetermined remote control signal transmitted from the remote monitoring device based on the remote monitoring device determining that a comparison result between the first judgment result and a second judgment result by a second recognition and judgment unit that is different from a first criterion for performing at least one of the recognition and judgment exceeds a second criterion, the driving control unit controls at least one of the acceleration / deceleration or steering based on the remote control signal rather than the first judgment result.

[0007] A recording medium according to another aspect of the present invention is a recording medium that records a judgment result of determining whether a comparison result between a first judgment result and a second judgment result exceeds a first criterion, wherein the first judgment result is a judgment result by the autonomous vehicle that recognizes at least one of an event inside and outside the autonomous vehicle based on data from a sensor and determines how to control the autonomous vehicle based on the recognition result, and the second judgment result is a judgment result by an apparatus that recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result, and in which a second criterion for performing at least one of the recognition and judgment is different from that of the autonomous vehicle. In addition, each aspect of the present invention may be realized by a computer. In this case, the present invention also includes a program that causes a computer to operate as each part (software element) of the remote monitoring device and the autonomous vehicle to realize the autonomous vehicle 1 and the remote monitoring device 2, and a computer-readable recording medium on which the program is recorded. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a functional configuration of an autonomous driving vehicle according to an embodiment of the present invention. [Diagram 2] FIG. 11 is a block diagram showing a functional configuration of a remote monitoring device according to an embodiment of another aspect of the present invention. [Diagram 3] 10 is a flowchart showing a flow of a remote monitoring method according to an embodiment of another aspect of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] <1. Self-driving vehicles> First, an embodiment of an autonomously driving vehicle 1 according to one aspect of the present invention will be described in detail.

[0010] [composition] The autonomous vehicle 1 performs autonomous driving of level 4 or higher. As shown in FIG. 1 , the autonomous vehicle 1 includes a sensor 11, a vehicle-side communication unit 12 (communication unit), a vehicle-side storage unit 13, and a vehicle-side control unit 14.

[0011] [Sensor] The sensor 11 detects at least one of events inside and outside the autonomous vehicle 1 to generate data. The internal event is, for example, the state of a person riding in the autonomous vehicle 1. The external event is, for example, at least one of the presence of an object, the moving speed of the object, and brightness (whether it is daytime or nighttime). The object includes, for example, at least one of a roadway, a lane, a sign, a sidewalk, a pedestrian, a preceding vehicle, a following vehicle, an oncoming vehicle, and an obstacle. The obstacle includes at least one of a building, a curb, a roadside tree, a broken-down vehicle, and a fallen object. In addition, a road closure in a part of the roadway is also included in the obstacle because it can be regarded as an obstacle by the vehicle. The sensor 11 may be an image sensor that captures at least one of the inside and the outside of the autonomous vehicle 1 to generate image data. In this case, multiple events can be extracted simultaneously from one image data.

[0012] [Vehicle side communication unit] The vehicle-side communication unit 12 communicates with the remote monitoring device 2. The vehicle-side communication unit 12 according to this embodiment is configured, for example, with a wireless communication module. That is, the vehicle-side communication unit 12 according to this embodiment wirelessly communicates with the remote monitoring device 2. The remote monitoring device 2 is for monitoring the operation of the autonomous driving vehicle 1. The remote monitoring device 2 will be described in detail later.

[0013] [Vehicle side storage unit] The vehicle-side storage unit 13 stores a first criterion when the first recognition judgment unit 142 described later performs at least one of recognition and judgment. The first criterion according to this embodiment is a threshold value used for event recognition (recognition threshold value 131), weighting of the recognized events (coefficient 132 multiplied by the recognition result), and a threshold value used for judgment (judgment threshold value 133). The vehicle-side storage unit 13 according to this embodiment stores a plurality of recognition threshold values ​​131 and a plurality of coefficients 132 according to the type of event. Moreover, the vehicle-side storage unit 13 according to this embodiment is composed of at least one of a semiconductor memory and a hard disk.

[0014] [Vehicle side control unit] The vehicle-side control unit 14 includes an acquisition unit 141, a first recognition determination unit 142 (recognition determination unit), a vehicle-side transmission control unit 143 (transmission control unit), a vehicle-side reception control unit 144, and a driving control unit 145.

[0015] (Acquisition Department) The acquisition unit 141 acquires data generated by the sensor 11. The acquisition unit 141 according to the present embodiment repeatedly acquires data every time a predetermined time elapses. Note that the first recognition determination unit 142 may be configured to directly acquire data. In this case, the vehicle-side control unit 14 may not be provided with the acquisition unit 141.

[0016] (First recognition judgment department) The first recognition judgment unit 142 recognizes at least one of events inside and outside the autonomous driving vehicle 1 based on the data (from the sensor 11) acquired by the acquisition unit 141, and makes a judgment based on the recognition result. The first recognition judgment unit 142 according to this embodiment is constructed by machine learning, for example. Note that the first recognition judgment unit 142 may be constructed using various conventionally known techniques other than machine learning. The first recognition judgment unit 142 according to this embodiment includes an event recognition unit 142a, a weighting unit 142b, and an event judgment unit 142c.

[0017] The event recognition unit 142a uses the data acquired by the acquisition unit 141 and the recognition threshold value 131 stored in the vehicle-side storage unit 13 to recognize at least one of events inside and outside the autonomously driven vehicle 1. The event recognition unit 142a recognizes a plurality of events.

[0018] The weighting unit 142b weights the recognition result by multiplying at least one of the recognition results of the plurality of events by the coefficient 132 stored in the vehicle-side storage unit 13.

[0019] The event determination unit 142c uses the weighted recognition result (multiplied by the coefficient 132) and the determination threshold 133 stored in the vehicle-side storage unit 13 to determine how to control the autonomous vehicle. For example, if the weighted recognition result indicates the presence of a pedestrian ahead of the autonomous vehicle 1, the event determination unit 142c determines that the autonomous vehicle 1 should decelerate or take a detour. At this time, if the pedestrian can be further recognized as, for example, a child or an elderly person, the event determination unit 142c may determine that the pedestrian is one who requires the vehicle's greater attention in accordance with the determination threshold 133, and may temporarily halt the autonomous vehicle 1. Also, if the recognition result indicates only the presence of a roadway, the event determination unit 142c determines that the autonomous vehicle 1 should continue to drive as is. In this case, if the roadway has multiple lanes and the autonomous vehicle 1 is scheduled to turn right or left at the upcoming intersection, the event determination unit 142c may move the autonomous vehicle 1 to the rightmost or leftmost lane according to the determination threshold 133 regardless of the distance to the intersection so that the autonomous vehicle 1 can turn right or left more smoothly, and then continue traveling. Note that the event determination unit 142c may be configured to make a determination directly using (without weighting) the determination result by the event recognition unit 142a. In this case, the first recognition determination unit 142 may not include the weighting unit 142b. Hereinafter, the determination result by the first recognition determination unit 142 will be referred to as a first determination result.

[0020] (Vehicle side transmission control unit) The vehicle-side transmission control unit 143 controls the vehicle-side communication unit 12 to transmit the first determination result by the first recognition determination unit 142 and the same data as the data input to the first recognition determination unit 142 to the remote monitoring device 2. Note that the autonomously driving vehicle 1 may be configured to send the determination result and data by a method other than communication via the vehicle-side communication unit 12. In that case, the autonomously driving vehicle 1 may not be equipped with the vehicle-side transmission control unit 143.

[0021] (Vehicle side reception control unit) The vehicle-side reception control unit 144 controls the vehicle-side communication unit 12 to receive at least one of information for updating and a remote control signal from the remote monitoring device 2. Details of the information for updating and the remote control signal will be described later. Note that the autonomously driving vehicle 1 may be configured to obtain information for updating and a remote control signal by a method other than communication via the vehicle-side communication unit 12. In that case, the autonomously driving vehicle 1 may not be equipped with the vehicle-side reception control unit 144.

[0022] (Operation control unit) The driving control unit 145 controls at least one of acceleration / deceleration or steering based on the first judgment result by the first recognition judgment unit 142. Furthermore, when the vehicle-side communication unit 12 receives a predetermined remote control signal, the driving control unit 145 controls at least one of acceleration / deceleration or steering based on the remote control signal. The predetermined remote control signal is a signal transmitted by the remote monitoring device 2 based on the judgment by the remote monitoring device 2 that the current operation of the autonomous vehicle 1 will lead to an accident. For this reason, when the driving control unit 145 receives the predetermined remote control signal, it performs control based on the predetermined remote control signal in preference to the first judgment result by the first recognition judgment unit 142.

[0023] [Effects] The autonomous vehicle 1 according to this embodiment transmits the first judgment result by the first recognition and judgment unit 142 to the remote monitoring device 2 in addition to the data. As a result, the remote monitoring device 2 can perform a double check using the received first judgment result and the second judgment result by itself in addition to the recognition and judgment using the data. Furthermore, the autonomous vehicle 1 according to this embodiment normally controls acceleration / deceleration / steering based on the first judgment result by the first recognition and judgment unit, and when a predetermined remote control signal is received, controls acceleration / deceleration / steering based on the remote control signal. The predetermined remote control signal is a signal transmitted by the remote monitoring device 2 based on the judgment by the remote monitoring device 2 that the current operation of the autonomous vehicle 1 will lead to an accident. In other words, there is a possibility that the recognition and judgment of the autonomous vehicle when this remote control signal is received is erroneous (causing an accident). Therefore, according to the autonomous vehicle 1 according to this embodiment, when the autonomous vehicle itself makes an erroneous recognition and judgment, it is possible to prevent the acceleration / deceleration / steering based on the erroneous recognition and judgment from being continued.

[0024] <2. Remote monitoring device> Next, an embodiment of the remote monitoring device 2, which is another aspect of the present invention, will be described in detail.

[0025] [composition] As shown in FIG. 2, the remote monitoring device 2 includes a device-side communication unit 21 (communication unit), a device-side storage unit 22, an output unit 23, and a device-side control unit 24.

[0026] [Device side communication section] The device-side communication unit 21 communicates with the autonomously driving vehicle 1 to be monitored. The device-side communication unit 21 according to this embodiment is configured, for example, with a wireless communication module. That is, the device-side communication unit 21 according to this embodiment wirelessly communicates with the autonomously driving vehicle 1. As described above, the autonomously driving vehicle 1 to be monitored has a first recognition judgment unit 142 (see FIG. 1) that recognizes at least one of events inside and outside the autonomously driving vehicle 1 based on data from the sensor 11 and makes a judgment based on the recognition result.

[0027] [Device side storage unit] The device-side storage unit 22 stores a first criterion when the second recognition judgment unit 242 described later performs at least one of recognition and judgment. The first criterion according to this embodiment is a threshold value used for event recognition (device-side recognition threshold value 221), weighting of the recognized events (device-side coefficient 222 multiplied by the recognition result), and a threshold value used for judgment (device-side judgment threshold value 223). The first criterion stored in the device-side storage unit 22 is different from the first criterion stored in the autonomous vehicle 1 in at least one of the device-side recognition threshold value 221, the device-side coefficient 222, and the device-side judgment threshold value 223. Hereinafter, the first criterion stored in the device-side storage unit 22 is referred to as the device-side first criterion. The device-side storage unit 22 according to this embodiment stores a plurality of device-side recognition threshold values ​​221 and a plurality of device-side coefficients 222 according to the type of event. In addition, the device-side storage unit 22 according to this embodiment is composed of at least one of a semiconductor memory and a hard disk.

[0028] The device-side storage unit 22 also stores a second criterion used by the determination unit 243, which will be described later. The second criterion will be described in detail later.

[0029] Further, the device-side storage unit 22 stores a database 224. That is, the remote monitoring device 2 includes the database 224. The database 224 accumulates past verification results (described in detail later) by the verification unit 244. When the verification unit 244 performs new verification, the database 224 according to this embodiment further accumulates the verification result.

[0030] [Output section] The output unit 23 outputs content according to the control of the device-side control unit 24. The output unit 23 is composed of at least one of a communication module that transmits data and signals to other devices (such as a display device), a terminal for connecting to other devices, a liquid crystal panel that displays images, and a speaker that outputs sound.

[0031] [Device-side control section] The device-side controller 24 includes a device-side reception controller 241 (reception controller), a second recognition determination unit 242, a determination unit 243, a verification unit 244, a second determination unit 245, and a processing unit 246.

[0032] (Reception control section) The reception control unit 241 controls the device-side communication unit 21 to receive from the autonomously driving vehicle 1 the same data as the data input to the first recognition judgment unit 142, and the first judgment result by the first recognition judgment unit 142.

[0033] (Second Recognition Judgment Department) The second recognition judgment unit 242 recognizes an event based on the data received by the device-side communication unit 21, and makes a judgment based on the recognition result. The second recognition judgment unit 242 according to this embodiment is constructed by, for example, machine learning, similar to the first recognition judgment unit 142. Note that the second recognition judgment unit 242 may be constructed using various conventionally known techniques other than machine learning. The second recognition judgment unit 242 includes a second event recognition unit 242a, a second weighting unit 242b, and a second event judgment unit 242c.

[0034] The second event recognition unit 242a uses data received by the device-side communication unit 21 and the device-side recognition threshold value 221 stored in the device-side storage unit 22 to recognize at least one of events inside and outside the autonomously driving vehicle 1. The second event recognition unit 242a recognizes a plurality of events.

[0035] The second weighting unit 242b weights the recognition result by multiplying at least one of the recognition results of the plurality of events by the device-side coefficient 222 stored in the device-side storage unit 22.

[0036] The second event judgment unit 242c judges how to control the autonomous vehicle based on the weighted recognition result (multiplied by the device-side coefficient 222). The second event judgment unit 242c may be configured to make a judgment by directly using (without weighting) the judgment result by the second event recognition unit 242a. In that case, the second recognition judgment unit 242 may not include the second weighting unit 242b. Hereinafter, the judgment result by the second recognition judgment unit 242 will be referred to as the second judgment result.

[0037] The second recognition judgment unit 242 is constructed in the same manner as the first recognition judgment unit 142, but as described above, the first criterion (device-side first criterion) when performing at least one of recognition and judgment is different from that of the first recognition judgment unit 142. The device-side first criterion is at least one of the device-side recognition threshold 221 (threshold used for recognizing an event), the device-side coefficient 222 (weighting of multiple recognized events), and the device-side judgment threshold 223 (threshold used for judgment). That is, the second recognition judgment unit 242 may produce a result different from that of the first recognition judgment unit 142 (the first judgment result is significantly different from the first judgment result) in at least one of the recognition of an event, the weighting of the recognition result, and the judgment based on the recognition result. The first recognition judgment unit 142 of the autonomous vehicle 1 developed by each automobile manufacturer has a common basic operation, and only the judgment criterion is different in many cases. For this reason, the second recognition judgment unit 242 according to this embodiment can be configured simply by using one installed in an autonomous vehicle of a manufacturer different from the autonomous vehicle 1.

[0038] (Judgment Department) The determination unit 243 determines whether the comparison result between the second judgment result and the first judgment result by the second recognition judgment unit 242 exceeds a second criterion. The comparison result is, for example, a difference between a first numerical value representing the first judgment result and a second numerical value representing the second judgment result, or a statistical value of the first numerical value and the second numerical value (for example, a ratio of the first numerical value to the second numerical value, etc.). Then, the determination unit 243 determines whether the comparison result exceeds a second criterion. Here, if it is determined that the comparison result between the second judgment result and the first judgment result does not exceed the second criterion, the determination unit 243 considers the second judgment result to be the same as the first judgment result. Then, the determination unit 243 accumulates the first judgment result, the second judgment result, and its own judgment result in the database 224.

[0039] (Verification Department) When the determination unit 243 determines that the comparison result between the second judgment result and the first judgment result exceeds the second standard, the verification unit 244 verifies the difference in the judgment. For example, the verification unit 244 verifies the following contents. Whether the second recognition determination unit 242 (first recognition determination unit 142) recognized the event recognized by the first recognition determination unit 142 (second recognition determination unit 242) Whether the event recognized by the first recognition judgment unit 142 was recognized as the same as that by the second recognition judgment unit 242 Among the recognition results by the second recognition judgment unit 242 (first recognition judgment unit 142), those corresponding to the recognition results multiplied by the large coefficient 132 by the first recognition judgment unit 142 (second recognition judgment unit 242) are also multiplied by the large device side coefficient 222 by the second recognition judgment unit 242 (first recognition judgment unit 142). After completing the verification, the verification unit 244 accumulates the verification results in the database 224 stored in the device-side storage unit 22 .

[0040] (Second Judgment Department) When the determination unit 243 determines that the comparison result between the second determination result and the first determination result exceeds the second standard, the second determination unit 245 determines whether or not the difference in the determination is a problem that leads to an accident, based on the verification result. For example, when the first determination result is a result that there is no abnormality, but the second determination result is a result that the pedestrian is to be avoided, the autonomous vehicle 1 has overlooked the presence of a pedestrian. For this reason, the second determination unit 245 determines that such a difference is a problem that leads to an accident. On the other hand, when the first determination result is a result that the vehicle should decelerate because a car is approaching from the intersecting road, and the second determination result is a result that the vehicle should decelerate because a bicycle is approaching from the intersecting road, the autonomous vehicle 1 or the remote monitoring device 2 has misidentified the target. However, since both are judgments of deceleration, the second determination unit 245 determines that such a difference is not a problem that leads to an accident.

[0041] The verification unit 244 may be configured to verify only when the determination unit 243 determines for the first time that the comparison result between the second judgment result and the first judgment result exceeds the second criterion after at least one of the first criterion of the autonomous driving vehicle 1 and the device-side first criterion of the remote monitoring device 2 is newly stored or updated. In this case, the second determination unit 245 may be configured to verify by matching the newly occurring difference in judgment to which past pattern it applies, with reference to the verification results accumulated in the database 224, when the determination unit 243 determines for the second or subsequent time that the comparison result between the second judgment result and the first judgment result exceeds the second criterion. In this way, the operation of the device-side control unit 24 can be simplified.

[0042] (Processing section) The processing unit 246 executes processing related to remote monitoring based on the determination result by the determination unit 243. The processing unit 246 according to this embodiment executes processing based on the determination result by the determination unit 243 and the second determination result by the second determination unit 245. Furthermore, the processing unit 246 according to this embodiment executes processing based on the verification results accumulated in the database 224. As a result, since not only the latest determination result by the determination unit 243 but also verification results accumulated in the past are used, it is possible to perform remote monitoring with higher accuracy.

[0043] The processing unit 246 according to this embodiment executes at least one of update, remote control, and notification as a process based on the verification result.

[0044] ·update The update is a process executed when the determination unit 243 determines that the comparison result between the second judgment result and the first judgment result exceeds the second standard. When the process executed by the determination unit 243 is an update, the processing unit 246 transmits information for updating at least one of the recognition and judgment functions to at least one of the first recognition judgment unit 142 and the second recognition judgment unit 242 based on the verification result accumulated in the database 224. Specifically, the processing unit 246 generates information for updating and controls the device-side communication unit 21 to transmit the information to the autonomously driven vehicle 1. The information for updating includes, for example, at least one of the following: First criteria for updating (at least one of the recognition threshold 131, the coefficient 132, and the judgment threshold 133) - First device-side criterion for update (at least one of the device-side recognition threshold 221, the device-side coefficient 222, and the device-side judgment threshold 223) Secondary criteria for updates At least one of a program for operating the first recognition determination unit 142 and a program for operating the second recognition determination unit 242 In this way, by using the difference between the recognition / judgment by the first recognition / judgment unit 142 and the recognition / judgment by the second recognition / judgment unit 242 to update at least one of the recognition functions and judgment functions, the accuracy of the recognition or judgment by the first recognition / judgment unit 142 and the second recognition / judgment unit 242 can be further improved.

[0045] Remote Control The remote control is a process executed when the determination unit 243 determines that the comparison result between the second determination result and the first determination result exceeds the second standard, and the second determination unit 245 determines that the difference is a problem that may lead to an accident. When the process executed by the determination unit 243 is remote control, the processing unit 246 remotely controls the autonomously driven vehicle 1. Specifically, the processing unit 246 generates a predetermined remote control signal that instructs an operation that may avoid an accident, and controls the device-side communication unit 21 to transmit this remote control signal to the autonomously driven vehicle 1. The autonomously driven vehicle that receives this remote control signal operates to avoid the accident. As a result, it is possible to prevent the occurrence of an accident based on erroneous recognition and judgment of the autonomously driven vehicle 1.

[0046] ·notification The notification is a process executed when the determination unit 243 determines that the comparison result between the second determination result and the first determination result exceeds the second standard, and the second determination unit determines that the difference is not a problem that leads to an accident. When the process executed by the determination unit 243 is notification, the processing unit 246 notifies the remote monitor of the determination result by the determination unit 243. Specifically, the processing unit 246 controls the output unit 23 so that the output unit 23 outputs the determination result. As a result, the determination result is notified to the remote monitor via the output unit 23. This allows the remote monitor to check only when it receives a notification. Therefore, when multiple autonomous vehicles 1 are remotely monitored, the burden on the remote monitor can be reduced. Note that the processing unit 246 may be configured not to notify (only perform updates or remote control). In that case, the remote monitoring device 2 may not be equipped with the output unit 23.

[0047] [Effects] The second recognition and judgment unit 242 recognizes and judges based on the same data as the actual data obtained from the autonomous vehicle 1 actually traveling. The recognition and judgment made by the second recognition and judgment unit 242 is a second opinion on the recognition and judgment made by the (real) autonomous vehicle 1 to be monitored. If the comparison result between the two (difference between recognition and judgment) is within the second standard, autonomous driving and remote monitoring can be continued without problems. On the other hand, if the comparison result between the two exceeds the second standard, it is considered that a situation that may lead to an accident may have occurred, so remote monitoring will be performed carefully. The remote monitoring device 2 according to this embodiment performs such a double check, and if there is an error in the autonomous driving control of the autonomous vehicle itself, it can be noticed, thereby ensuring the safety of the autonomous driving of the real autonomous vehicle 1. By noticing an error in the autonomous driving control, the error can be improved.

[0048] Furthermore, in autonomous driving of level 4 or higher, remote monitoring by a monitor is required. Therefore, a remote monitor is required to take over the operation of an autonomous vehicle, but from the viewpoint of business viability, it is difficult to perform constant monitoring. However, according to the remote monitoring device 2 according to this embodiment, the remote monitor only needs to respond to autonomous vehicles 1 whose recognitions do not match. As a result, a large number of autonomous vehicles 1 can be easily remotely monitored by a small number of people. In addition, since the data used is detected by the sensor 11 equipped in the actual autonomous vehicle 1, there is no need for a person to generate it. Therefore, the monitor can easily operate the remote monitoring device 2.

[0049] <3. Remote monitoring method> Next, an embodiment of a remote monitoring method, which is another aspect of the present invention, will be described in detail.

[0050] As shown in FIG. 3, the remote monitoring method includes an acquisition step S1, a first recognition and judgment step S2, a second recognition and judgment step S3, a determination step S4, and a processing step S5.

[0051] (Acquisition step) In the first acquisition step S1, data generated by the sensor 11 is acquired. In the acquisition step S1 according to this embodiment, the data is acquired by the acquisition unit 141 of the autonomous driving vehicle 1. Note that in the acquisition step S1, the data may be acquired by the remote monitoring device 2.

[0052] (First recognition and judgment step) After acquiring the data, the process proceeds to a first recognition judgment step S2. In the first recognition judgment step S2, at least one of an internal and external event of the autonomously driven vehicle 1 is recognized based on the data acquired in the acquisition step S1 (from the sensor 11), and a judgment is made based on the recognition result. In the first recognition judgment step S2 according to this embodiment, the judgment is made by the first recognition judgment unit 142 of the autonomously driven vehicle 1. Note that in the first recognition judgment step S2, the recognition of the event and the judgment based on the recognition result may be made by the remote monitoring device 2.

[0053] (Second recognition and judgment step) After recognizing at least one of the events inside and outside the autonomously driven vehicle 1 and making a judgment based on the recognition result, the process proceeds to a second recognition judgment step S3. In the second recognition judgment step S3, the event is recognized based on the same data as that used in the first recognition judgment step S2, and a judgment is made based on the recognition result. In the second recognition judgment step S3 according to this embodiment, the judgment is made by the second recognition judgment unit 242 of the remote monitoring device 2. Note that in the second recognition judgment step S3, the autonomously driven vehicle 1 may recognize the event and make a judgment based on the recognition result.

[0054] (Decision step) After recognizing the event and making a judgment based on the recognition result, the process proceeds to judgment step S4. In judgment step S4, it is determined whether or not a comparison result between the second judgment result in the second recognition judgment step S3 and the first judgment result in the first recognition judgment step S2 exceeds a second criterion. In judgment step S4 according to this embodiment, the judgment unit 243 of the remote monitoring device 2 makes a judgment. Note that in judgment step S4, the autonomously driven vehicle 1 may determine whether or not a comparison result between the second judgment result and the first judgment result exceeds the second criterion.

[0055] (Processing Steps) After determining whether the comparison result between the second judgment result and the first judgment result exceeds the second standard, the process proceeds to processing step S5. In processing step S5, processing related to remote monitoring is executed based on the judgment result in the above-mentioned judgment step S4. Processing step S5 according to this embodiment is executed by the processing unit 246 of the above-mentioned remote monitoring device 2. Note that in processing step S5, the processing related to remote monitoring may be executed by the autonomously driving vehicle 1. Also, in processing step S5, if it is determined that the comparison result between the second judgment result and the first judgment result exceeds the second standard, it is not necessary to execute the processing related to remote monitoring.

[0056] <4.Other> The functions of the autonomous vehicle 1 and the remote monitoring device 2 (hereinafter, devices 1 and 2) can be realized by a program for making a computer function as the devices 1 and 2, and for making a computer function as each control block of the devices 1 and 2 (particularly each unit included in the vehicle-side control unit 14 and the device-side control unit 24). In this case, the devices 1 and 2 include a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The functions described in the above embodiment are realized by executing the program using the control device and the storage device. The program may be recorded on one or more computer-readable recording media, not on a temporary basis. The recording media may or may not be included in the devices. In the latter case, the program may be supplied to the devices via any wired or wireless transmission medium.

[0057] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of each of the control blocks can be realized by, for example, a quantum computer.

[0058] Furthermore, the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0059] According to each aspect of the present invention described above, by providing effects that are suitable for safe and sustainable public transportation such as buses, it can contribute to the achievement of Goal 11 of the Sustainable Development Goals (SDGs), "Sustainable cities and communities". [Explanation of symbols]

[0060] 1. Self-driving vehicles 11 Sensors 12 Vehicle side communication unit 13 Vehicle side memory unit 131 Recognition Threshold 132 coefficient 133 Judgment Threshold 14 Vehicle side control unit 141 Acquisition Department 142 First Recognition Judgment Department 142a Event Recognition Section 142b Weighting section 142c Event Judgment Department 143 Vehicle side transmission control unit 144 Vehicle side receiving control unit 145 Operation control unit 2. Remote monitoring equipment 21 Device side communication section 22 Device side storage unit 221 Device side recognition threshold 222 Equipment side coefficient 223 Device-side judgment threshold 224 Database 23 Output section 24 Device side control section 241 Device side reception control unit (reception control unit) 242 Second Recognition Judgment Department 242a Second Event Recognition Unit 242b Second weighting section 242c Second Event Judgment Department 243 Judgment section 244 Verification Department 245 Second Judgment Department 246 Processing section

Claims

1. a communication unit that communicates with the autonomous vehicle and has a first recognition and judgment unit that recognizes at least one of events inside and outside the autonomous vehicle based on data from a sensor and determines how to control the autonomous vehicle based on the recognition result; a reception control unit that controls the communication unit to receive, from the autonomous driving vehicle, the data acquired by the first recognition determination unit from the sensor and a first determination result by the first recognition determination unit; a second recognition and determination unit that recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result; a determination unit that determines whether a comparison result between the first determination result and a second determination result by the second recognition / determination unit exceeds a comparison standard; a processing unit that executes a process related to remote monitoring based on a result of the determination by the determination unit; Equipped with The second recognition / judgment unit uses a judgment criterion for performing at least one of recognition and judgment, which is different from that of the first recognition / judgment unit. Remote monitoring system.

2. A remote monitoring system for remotely monitoring an autonomous vehicle that determines how to control the vehicle based on data from a sensor, comprising: A communication unit that receives, from the autonomous driving vehicle, the data and a first determination result that determines how to control the autonomous driving vehicle; a determination unit that determines how to control the autonomous vehicle based on the data as a second determination result, and determines whether a comparison result between the first determination result of the autonomous vehicle and the second determination result exceeds a comparison standard; A processing unit that executes a process related to remote monitoring of the autonomous driving vehicle based on a determination result by the determination unit; and Equipped with Remote monitoring system.

3. A remote monitoring system for monitoring a vehicle that includes a sensor that detects events inside or outside the vehicle and generates data, a first recognition and judgment unit that receives as input the data generated by the sensor and outputs a first judgment result on how to control the vehicle, and a driving control unit that controls at least one of acceleration / deceleration and steering of the vehicle based on the judgment of the first recognition and judgment unit, a receiving unit that receives data input to the first recognition and judgment unit from a vehicle to be monitored and the first judgment result corresponding to the data; a second recognition / judgment unit that outputs a second judgment result regarding how to control the monitored vehicle based on the data input to the first recognition / judgment unit; a comparison unit that compares the first judgment result and the second judgment result output based on the same data; a processing unit that executes a process related to remote monitoring of the vehicle to be monitored based on a comparison result by the comparison unit, the second recognition and judgment unit performs a calculation for generating a judgment result based on input data different from that of the first recognition and judgment unit; Remote monitoring system.

4. A remote monitoring system capable of determining how to control an autonomous vehicle equipped with a sensor, comprising: A communication unit that receives, from the autonomous driving vehicle, data acquired by the sensor and a first determination result that determines how to control the autonomous driving vehicle based on the data; a determination unit that determines, as a second determination result, how to control the autonomous vehicle based on the data received by the communication unit, and determines whether a comparison result between the first determination result and the second determination result exceeds a comparison standard; and A processing unit that executes a process related to remote monitoring of the autonomous driving vehicle based on a determination result by the determination unit; and Equipped with Remote monitoring system.

5. A communication unit that receives from an autonomous vehicle data detected by a sensor and a first determination result that determines how to control the autonomous vehicle; A determination unit that determines how to control the autonomous vehicle based on the data as a second determination result. a determination unit that determines whether a comparison result between a first determination result of the autonomous driving vehicle and the second determination result exceeds a comparison standard; and A processing unit that executes a process related to remote monitoring of the autonomous driving vehicle based on a determination result by the determination unit; and Equipped with Remote monitoring system.

6. The first judgment result and the second judgment result differ due to differences in any of the following: recognition of an event based on data acquired by the sensor used in the judgment; weighting of the recognized event; and a threshold for making a judgment based on the recognition. The remote monitoring system according to claim 4.

7. The judgment unit judges whether or not a difference between a first numerical value representing the first judgment result and a second numerical value representing the second judgment result, or a ratio of the first numerical value to the second numerical value, exceeds the comparison criterion. The remote monitoring system according to claim 4.

8. The first judgment result and the second judgment result differ due to differences in at least one of the thresholds used in the judgment to recognize at least one of the events inside and outside the autonomous driving vehicle, the weighting of the recognized events, and the thresholds used in the judgment. The remote monitoring system according to claim 5.

9. a verification unit that verifies a difference in the judgment when the determination unit determines that a comparison result between the second judgment result and the first judgment result exceeds the comparison standard; a database that accumulates the verification results by the verification unit; a second determination unit that, when the determination unit determines that a comparison result between the second determination result and the first determination result exceeds the comparison standard, determines whether or not the difference in judgment is a problem that may lead to an accident, based on the verification result; Further comprising: When the second determination unit determines that the difference is a problem that may lead to an accident, the processing unit remotely controls the autonomous vehicle. The remote monitoring system of claim 1 .

10. An autonomous vehicle, A communication unit that communicates with a remote monitoring system that monitors the operation of the autonomous vehicle; A sensor that detects at least one of an internal and external event of the autonomous vehicle and generates data; a first recognition and determination unit that recognizes the event based on data from the sensor and determines how to control the autonomous vehicle based on the recognition result; A driving control unit that controls at least one of acceleration / deceleration and steering based on a first determination result by the first recognition determination unit; a transmission control unit that controls the communication unit to transmit the first determination result and data input to the first recognition determination unit to the remote monitoring system; the driving control unit, when the communication unit receives a predetermined remote control signal transmitted by the remote monitoring system based on the remote monitoring system determining that a comparison result between a second determination result by a second recognition determination unit that recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result exceeds a comparison standard, controls at least one of acceleration / deceleration or steering based on the remote control signal rather than the first determination result; A judgment criterion for performing at least one of the recognition and judgment by the second recognition / judgment unit is different from that of the first recognition / judgment unit. Self-driving vehicles.

11. When the driving control unit receives information for updating at least one of the functions of the recognition and the judgment in the first recognition judgment unit from the remote monitoring system, the driving control unit updates the function.

11. The autonomous vehicle of claim 10.

12. A recording medium storing a judgment result of whether or not a comparison result between a first judgment result and a second judgment result exceeds a comparison standard, the first judgment result is a judgment result by the autonomous vehicle that recognizes at least one of an internal and external event of the autonomous vehicle based on data from a sensor and determines how to control the autonomous vehicle based on the recognition result; the second determination result is a determination result by a system that recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result; The judgment criteria for at least one of the recognition and judgment of the system are different from those of the autonomous vehicle; Recording medium.

13. The first judgment result and the second judgment result are further recorded. The recording medium according to claim 12.

Images