Remote monitoring system, autonomous vehicle, recording medium, and program

Abstract

To enhance safety in automatic driving control by an automatic driving vehicle.SOLUTION: A remote monitoring system includes: a second recognition determination section (242) which performs control to receive data acquired by a first recognition determination section of an automatic driving vehicle (1) from a sensor and a first determination result by the first recognition determination section, and where a first reference in performing at least one of recognition and determination is different from that of the first recognition determination section; a determination section (243) which determines whether a comparison result between the second determination result and the first determination result exceeds a second reference; and a processing section (246) for executing processing based on a determination result by the determination section.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a remote monitoring system, an autonomous vehicle, a recording medium, and a program.

Background Art

[0002] Various techniques for enhancing the safety of autonomous driving control by autonomous vehicles have been conventionally proposed. For example, Patent Document 1 describes an autonomous driving program evaluation system including an evaluation criterion information storage unit that stores evaluation criterion information, which is information on evaluation criteria for evaluating the content of driving of a predetermined vehicle under predetermined conditions by a numerical value, using a processor and a memory; an execution log reception unit that receives, from a predetermined information processing system, an execution log, which is information including the driving history of the vehicle, output as a result of execution of a predetermined program for determining the content of driving of a predetermined vehicle under predetermined conditions; and an evaluation result information generation unit that generates evaluation result information indicating, by a numerical value, an evaluation of the driving of a predetermined vehicle under predetermined conditions based on the received execution log and the evaluation criterion information.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The prior art described in Patent Document 1 evaluates virtually executed autonomous driving in order to develop an autonomous driving program. Therefore, the data input to the autonomous driving program execution system 10 is also virtual. Virtual data does not necessarily match real data. For this reason, when a real autonomous vehicle equipped with an autonomous driving program developed using this prior art drives on the road, it may not make the same recognition and judgments as it did in the virtual space. In other words, the prior art cannot adequately guarantee the safety of autonomous driving in a real autonomous vehicle. Furthermore, since virtual data needs to be artificially generated, it is time-consuming to operate the autonomous driving program execution system 10. [Means for solving the problem]

[0005] A remote monitoring system according to one aspect of the present invention includes: a communication unit that communicates with an autonomous vehicle having a first recognition and determination unit that recognizes at least one event inside or outside the autonomous vehicle based on data from sensors and determines how to control the autonomous vehicle based on the recognition results; a second recognition and determination unit that controls the communication unit to receive from the autonomous vehicle data acquired by the first recognition and determination unit from the sensors and a first determination result by the first recognition and determination unit, and further recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition results, and whose first criterion for performing at least one of recognition and determination is different from that of the first recognition and determination unit; a determination unit that determines whether the comparison result between the second determination result by the second recognition and determination unit and the first determination result exceeds the second criterion; and a processing unit that executes processing based on the determination result by the determination unit.

[0006] An autonomous vehicle according to another aspect of the present invention includes a communication unit that communicates with a remote monitoring device that monitors the operation of the autonomous vehicle; a sensor that detects at least one event inside or outside the autonomous vehicle and generates data; a first recognition and determination unit that recognizes the event based on the data from the sensor and determines how to control the autonomous vehicle based on the recognition result; an operation control unit that controls at least one of acceleration / deceleration or steering based on a first determination result by the first recognition and determination unit; and a transmission control unit that controls the communication unit to transmit the first determination result and the data input to the first recognition and determination unit to the remote monitoring device. The operation control unit recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result. Furthermore, if the communication unit receives a predetermined remote control signal transmitted by the remote monitoring device based on the remote monitoring device's determination that the comparison result between the first determination result and a second determination result by a second recognition and determination unit, which has a different first criterion for recognition and determination than that of the first recognition and determination unit, exceeds the second criterion, the operation control unit controls at least one of acceleration / deceleration or steering based on the remote control signal rather than the first determination result.

[0007] A recording medium according to another aspect of the present invention is a recording medium that records a determination result in which a comparison result between a first determination result and a second determination result is determined to be whether or not it exceeds a first criterion, wherein the first determination result is a determination result by the autonomous vehicle that recognizes at least one event inside or outside the autonomous vehicle based on data from sensors and determines how to control the autonomous vehicle based on the recognition result, and the second determination result is a determination result by a device that recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result, and whose second criterion for performing at least one of the recognition and determination is different from that of the autonomous vehicle. Furthermore, each aspect of the present invention may be implemented by a computer. In this case, a program that enables the computer to implement the autonomous vehicle 1 and the remote monitoring device 2 by operating the computer as the respective parts (software elements) of the remote monitoring device and the autonomous vehicle, and a computer-readable recording medium on which this program is recorded, also fall within the scope of the present invention. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the functional configuration of an autonomous vehicle according to one embodiment of the present invention. [Figure 2] This is a block diagram showing the functional configuration of a remote monitoring device according to another embodiment of the present invention. [Figure 3] This flowchart shows the flow of a remote monitoring method according to another embodiment of the present invention. [Modes for carrying out the invention]

[0009] <1. Autonomous vehicles> First, an embodiment of an autonomous vehicle 1, which is one aspect of the present invention, will be described in detail.

[0010] [composition] Autonomous vehicle 1 performs autonomous driving at Level 4 or higher. As shown in Figure 1, autonomous vehicle 1 includes a sensor 11, a vehicle-side communication unit 12 (communication unit), a vehicle-side storage unit 13, and a vehicle-side control unit 14.

[0011] [Sensor] Sensor 11 detects events inside and outside the autonomous vehicle 1 and generates data. Internal events include, for example, the state of people inside the autonomous vehicle 1. External events include, for example, the presence of an object, the speed of movement of an object, and brightness (whether it is daytime or nighttime). Objects include, for example, at least one of roadways, lanes, signs, sidewalks, pedestrians, preceding vehicles, following vehicles, oncoming vehicles, and obstacles. Obstacles include at least one of buildings, curbs, street trees, broken-down vehicles, and fallen objects. Road closures in certain sections of roadways are also considered obstacles for vehicles and are therefore included as obstacles. Sensor 11 may also be an image sensor that captures images of at least one of the inside and outside of the autonomous vehicle 1 and generates image data. In this case, multiple events can be extracted simultaneously from a single image data.

[0012] [Vehicle-side communication unit] The vehicle-side communication unit 12 communicates with the remote monitoring device 2. In this embodiment, the vehicle-side communication unit 12 is composed of, for example, a wireless communication module. That is, the vehicle-side communication unit 12 in this embodiment communicates wirelessly with the remote monitoring device 2. The remote monitoring device 2 is for monitoring the operation of the autonomous vehicle 1. Details of this remote monitoring device 2 will be described later.

[0013] [Vehicle-side memory unit] The vehicle-side storage unit 13 stores a first criterion for the first recognition and judgment unit 142, which will be described later, when it performs recognition and judgment. The first criterion in this embodiment is a threshold used for recognizing an event (recognition threshold 131), a weighting of the multiple recognized events (a coefficient 132 multiplied by the recognition result), and a threshold used for judgment (judgment threshold 133). The vehicle-side storage unit 13 in this embodiment stores a plurality of recognition thresholds 131 and a plurality of coefficients 132 according to the type of event. Furthermore, the vehicle-side storage unit 13 in this embodiment is composed of at least one of semiconductor memory and a hard disk.

[0014] [Vehicle-side control unit] The vehicle - side control unit 14 includes an acquisition unit 141, a first recognition and determination unit 142 (recognition and determination unit), a vehicle - side transmission control unit 143 (transmission control unit), a vehicle - side reception control unit 144, and a driving control unit 145.

[0015] (Acquisition unit) The acquisition unit 141 acquires the data generated by the sensor 11. The acquisition unit 141 according to the present embodiment repeatedly acquires data every time a predetermined time elapses. Note that the acquisition of data may be configured to be directly performed by the first recognition and determination unit 142. In that case, the vehicle - side control unit 14 may not include the acquisition unit 141.

[0016] (First recognition and determination unit) The first recognition and determination unit 142 recognizes at least one of the events inside and outside the autonomous vehicle 1 based on the data (from the sensor 11) acquired by the acquisition unit 141, and makes a determination based on the recognition result. The first recognition and determination unit 142 according to the present embodiment is constructed by, for example, machine learning. Note that the first recognition and determination unit 142 may be constructed using various conventionally known techniques other than machine learning. The first recognition and determination unit 142 according to the present embodiment includes an event recognition unit 142a, a weighting unit 142b, and an event determination unit 142c.

[0017] The event recognition unit 142a uses the data acquired by the acquisition unit 141 and the recognition threshold 131 stored in the vehicle - side storage unit 13 to recognize at least one of the events inside and outside the autonomous vehicle 1. The event recognition unit 142a recognizes a plurality of events.

[0018] The weighting unit 142b weights the recognition results by multiplying at least any one of the recognition results of the plurality of events by the coefficient 132 stored in the vehicle - side storage unit 13.

[0019] The event determination unit 142c uses the weighted (multiplied by the coefficient 132) recognition result and the determination threshold value 133 stored in the vehicle-side memory unit 13 to determine how to control the autonomous vehicle. For example, when the weighted recognition result indicates the presence of a pedestrian in front of the autonomous vehicle 1, the event determination unit 142c determines that the autonomous vehicle 1 should decelerate or perform a detour. At this time, if it can be further recognized that the pedestrian is, for example, a child or an elderly person, the event determination unit 142c determines, according to the determination threshold value 133, that the pedestrian requires more attention for the vehicle, and the autonomous vehicle 1 may be stopped temporarily. Also, when the recognition result only indicates the presence of the lane, the event determination unit 142c determines that the autonomous vehicle 1 should continue to travel as it is. At this time, if the lane consists of multiple lanes and a right turn or a left turn of the autonomous vehicle 1 is scheduled at the next intersection, the event determination unit 142c, according to the determination threshold value 133, moves the autonomous vehicle 1 to the rightmost or leftmost lane regardless of the distance to the intersection so that the right turn or left turn can be performed more smoothly, and then continues the travel. Note that the event determination unit 142c may be configured to make a determination directly using the determination result of the event recognition unit 142a (without performing weighting). In that case, the first recognition determination unit 142 may not include the weighting unit 142b. Hereinafter, the determination result by the first recognition determination unit 142 is referred to as the first determination result.

[0020] (Vehicle-side transmission control unit) The vehicle-side transmission control unit 143 controls the vehicle-side communication unit 12 to transmit the first determination result by the first recognition determination unit 142 and the same data as the data input to the first recognition determination unit 142 to the remote monitoring device 2. Note that the autonomous vehicle 1 may be configured to send the determination result and data by a method other than communication via the vehicle-side communication unit 12. In that case, the autonomous vehicle 1 may not include the vehicle-side transmission control unit 143.

[0021] (Vehicle-side reception control unit) The vehicle-side receiving control unit 144 controls the vehicle-side communication unit 12 to receive at least one of the information for updating and / or remote control signals from the remote monitoring device 2. Details of the information for updating and / or remote control signals will be described later. Note that the autonomous vehicle 1 may be configured to acquire the information for updating and / or remote control signals by means other than communication via the vehicle-side communication unit 12. In that case, the autonomous vehicle 1 does not need to have the vehicle-side receiving control unit 144.

[0022] (Operation Control Unit) The driving control unit 145 controls at least one of acceleration / deceleration or steering based on the first judgment result from the first recognition / determination unit 142. Furthermore, if the vehicle-side communication unit 12 receives a predetermined remote control signal, the driving control unit 145 controls at least one of acceleration / deceleration or steering based on the remote control signal. The predetermined remote control signal is a signal transmitted by the remote monitoring device 2 based on its determination that the current operation of the automated driving vehicle 1 could lead to an accident. Therefore, when the driving control unit 145 receives the predetermined remote control signal, it takes precedence over the first judgment result from the first recognition / determination unit 142 and performs control based on the predetermined remote control signal.

[0023] [Effects and Effects] In this embodiment, the autonomous vehicle 1 transmits data, as well as a first judgment result from the first recognition and judgment unit 142, to the remote monitoring device 2. This allows the remote monitoring device 2 to perform a double check using the received first judgment result and its own second judgment result, in addition to recognition and judgment using the data. Furthermore, the autonomous vehicle 1 in this embodiment normally controls acceleration, deceleration, and steering based on the first judgment result from the first recognition and judgment unit, and when it receives a predetermined remote control signal, it controls acceleration, deceleration, and steering based on the remote control signal. The predetermined remote control signal is a signal transmitted by the remote monitoring device 2 based on the remote monitoring device 2's determination that the current operation of the autonomous vehicle 1 would lead to an accident. In other words, there is a possibility that the autonomous vehicle's recognition and judgment is incorrect (causing an accident) when this remote control signal is received. For this reason, the autonomous vehicle 1 in this embodiment can prevent the continuation of acceleration, deceleration, and steering based on incorrect recognition and judgment if it makes an error in its own recognition and judgment.

[0024] <2. Remote monitoring device> Next, an embodiment of the remote monitoring device 2, which is another aspect of the present invention, will be described in detail.

[0025] [composition] As shown in Figure 2, the remote monitoring device 2 comprises a device-side communication unit 21 (communication unit), a device-side storage unit 22, an output unit 23, and a device-side control unit 24.

[0026] [Device-side communication unit] The device-side communication unit 21 communicates with the autonomous vehicle 1 to be monitored. The device-side communication unit 21 according to this embodiment is composed of, for example, a wireless communication module. That is, the device-side communication unit 21 according to this embodiment communicates wirelessly with the autonomous vehicle 1. As described above, the autonomous vehicle 1 to be monitored has a first recognition and determination unit 142 (see Figure 1) that recognizes at least one event inside or outside the autonomous vehicle 1 based on data from the sensor 11 and makes a decision based on the recognition result.

[0027] [Device side storage unit] The device-side storage unit 22 stores a first criterion for the second recognition and judgment unit 242, described later, when it performs recognition and judgment. The first criterion in this embodiment is a threshold used for recognizing an event (device-side recognition threshold 221), a weighting of the multiple recognized events (device-side coefficient 222 multiplied by the recognition result), and a threshold used for judgment (device-side judgment threshold 223). The first criterion stored in the device-side storage unit 22 differs from the first criterion stored by the autonomous vehicle 1 in at least one of the device-side recognition threshold 221, device-side coefficient 222, and device-side judgment threshold 223. Hereinafter, the first criterion stored in the device-side storage unit 22 will be referred to as the device-side first criterion. The device-side storage unit 22 in this embodiment stores a plurality of device-side recognition thresholds 221 and a plurality of device-side coefficients 222 according to the type of event. Furthermore, the device-side storage unit 22 in this embodiment is composed of at least one of a semiconductor memory and a hard disk.

[0028] Furthermore, the device-side storage unit 22 stores the second criterion used by the determination unit 243, which will be described later. Details of this second criterion will be discussed later.

[0029] Furthermore, the device-side storage unit 22 stores the database 224. In other words, the remote monitoring device 2 is equipped with a database 224. The database 224 stores past verification results (details described later) from the verification unit 244. In this embodiment, when the verification unit 244 performs a new verification, the database 224 further stores the verification results.

[0030] [Output section] The output unit 23 outputs content according to the control of the device-side control unit 24. The output unit 23 consists of at least one of the following: a communication module for transmitting data and signals to other devices (such as display devices), terminals for connecting to other devices, a liquid crystal panel for displaying images, and a speaker for outputting sound.

[0031] [Device-side control unit] The device-side control unit 24 comprises a device-side receiving control unit 241 (receiving control unit), a second recognition determination unit 242, a determination unit 243, a verification unit 244, a second determination unit 245, and a processing unit 246.

[0032] (Receiver control unit) The receiving control unit 241 controls the device-side communication unit 21 to receive the same data as the data input to the first recognition and judgment unit 142, as well as the first judgment result from the first recognition and judgment unit 142, from the autonomous driving vehicle 1.

[0033] (Second Recognition Judgment Department) The second recognition and determination unit 242 recognizes an event based on the data received by the device-side communication unit 21 and makes a decision based on the recognition result. The second recognition and determination unit 242 according to this embodiment is constructed, for example, by machine learning, similar to the first recognition and determination unit 142 described above. However, the second recognition and determination unit 242 may be constructed using various conventionally known technologies other than machine learning. The second recognition and determination unit 242 comprises a second event recognition unit 242a, a second weighting unit 242b, and a second event determination unit 242c.

[0034] The second event recognition unit 242a recognizes at least one event inside or outside the autonomous vehicle 1 using the data received by the device-side communication unit 21 and the device-side recognition threshold 221 stored in the device-side storage unit 22. The second event recognition unit 242a recognizes multiple events.

[0035] The second weighting unit 242b weights the recognition results by multiplying at least one of the recognition results of multiple events by a device-side coefficient 222 stored in the device-side storage unit 22.

[0036] The second event determination unit 242c determines how to control the autonomous vehicle based on the weighted (multiplied by the device-side coefficient 222) recognition result. The second event determination unit 242c may also be configured to make a decision directly using the determination result from the second event recognition unit 242a (without weighting). In that case, the second recognition determination unit 242 does not need to include a second weighting unit 242b. Hereinafter, the determination result from the second recognition determination unit 242 will be referred to as the second determination result.

[0037] The second recognition and judgment unit 242 is constructed similarly to the first recognition and judgment unit 142, but as described above, the first criterion (device-side first criterion) used when performing at least one of recognition and judgment is different from that of the first recognition and judgment unit 142. The device-side first criterion is at least one of the device-side recognition threshold 221 (threshold used for event recognition), device-side coefficient 222 (weighting of multiple recognized events), and device-side judgment threshold 223 (threshold used for judgment). In other words, the second recognition and judgment unit 242 may produce different results from the first recognition and judgment unit 142 in at least one of the event recognition, weighting of recognition results, and judgment based on recognition results (the first judgment result may differ significantly from the first judgment result). The first recognition and judgment unit 142 of the autonomous driving vehicle 1 being developed by each automobile manufacturer has common basic operation, and in many cases only the judgment criteria differ. For this reason, the second recognition and judgment unit 242 according to this embodiment can be constructed simply by using one installed in an autonomous driving vehicle from a different manufacturer than the autonomous driving vehicle 1.

[0038] (Judgment Department) The determination unit 243 determines whether the comparison result between the second determination result and the first determination result from the second recognition determination unit 242 exceeds the second criterion. The comparison result is, for example, the difference between the first numerical value representing the first determination result and the second numerical value representing the second determination result, or statistical values ​​of the first numerical value and the second numerical value (for example, the ratio of the first numerical value to the second numerical value). The determination unit 243 then determines whether the comparison result exceeds the second criterion. If the determination unit 243 determines that the comparison result between the second determination result and the first determination result does not exceed the second criterion, the determination unit 243 considers the second determination result to be the same as the first determination result. The determination unit 243 then stores the first determination result, the second determination result, and its own determination result in the database 224.

[0039] (Verification Department) The verification unit 244 verifies the difference in judgment when the determination unit 243 determines that the comparison result between the second judgment result and the first judgment result exceeds the second criterion. For example, the verification unit 244 verifies the following: Whether or not the second recognition determination unit 242 (first recognition determination unit 142) recognized the event that the first recognition determination unit 142 (second recognition determination unit 242) recognized. Whether the second recognition and judgment unit 242 recognized the event recognized by the first recognition and judgment unit 142 as the same event or not. • For recognition results from the second recognition determination unit 242 (first recognition determination unit 142), if the first recognition determination unit 142 (second recognition determination unit 242) multiplies the result by a large coefficient 132, does the second recognition determination unit 242 (first recognition determination unit 142) also multiply it by a large device-side coefficient 222? After completing the verification, the verification unit 244 stores the verification results in the database 224 stored in the device-side storage unit 22.

[0040] (Second Judgment Department) The second judgment unit 245 determines, based on the verification results, whether the difference in judgment between the second judgment result and the first judgment result is a problem that could lead to an accident, when the judgment unit 243 determines that the comparison result between the second judgment result and the first judgment result exceeds the second criterion. For example, if the first judgment result indicates that there is no abnormality, but the second judgment result indicates that the vehicle should avoid the pedestrian, then the autonomous vehicle 1 has overlooked the presence of the pedestrian. For this reason, the second judgment unit 245 determines that such a difference is a problem that could lead to an accident. On the other hand, if the first judgment result indicates that the vehicle should slow down because a car is approaching from an intersecting road, and the second judgment result indicates that the vehicle should slow down because a bicycle is approaching from an intersecting road, then the autonomous vehicle 1 or the remote monitoring device 2 has misidentified the object. However, since both are judgments to slow down, the second judgment unit 245 determines that such a difference is not a problem that could lead to an accident.

[0041] Furthermore, the verification unit 244 may be configured to perform verification only when the determination unit 243 first determines that the comparison result between the second judgment result and the first judgment result exceeds the second criterion, after at least one of the first criterion of the autonomous vehicle 1 and the device-side first criterion of the remote monitoring device 2 has been newly stored or updated. In that case, the second determination unit 245 may be configured to perform verification when the determination unit 243 determines for the second time or later that the comparison result between the second judgment result and the first judgment result exceeds the second criterion, by referring to the verification results stored in the database 224 and matching which past pattern the newly arising difference in judgment corresponds to. This simplifies the operation of the device-side control unit 24.

[0042] (processing) The processing unit 246 executes processing related to remote monitoring based on the determination result from the determination unit 243. In this embodiment, the processing unit 246 executes processing based on the determination result from the determination unit 243 and the second determination result from the second determination unit 245. Furthermore, the processing unit 246 in this embodiment executes processing based on the verification results stored in the database 224. As a result, since it uses not only the latest determination result from the determination unit 243 but also verification results stored in the past, more accurate remote monitoring can be performed.

[0043] The processing unit 246 according to this embodiment performs at least one of the following processes based on the verification results: update, remote control, and notification.

[0044] ·update An update is a process executed when the determination unit 243 determines that the comparison result between the second determination result and the first determination result exceeds the second criterion. If the process executed by the determination unit 243 is an update, the processing unit 246 transmits information to at least one of the first recognition determination unit 142 and the second recognition determination unit 242 to update at least one of the recognition and determination functions, based on the verification results stored in the database 224. Specifically, it generates information for updating and controls the device-side communication unit 21 to transmit this information to the autonomous driving vehicle 1. The information for updating includes, for example, at least one of the following: • Primary criteria for updates (at least one of the recognition threshold 131, coefficient 132, and judgment threshold 133) • Device-side primary criteria for updates (at least one of the device-side recognition threshold 221, device-side coefficient 222, and device-side judgment threshold 223) • Second standard for updates - At least one of the programs for operating the first recognition and determination unit 142 and the program for operating the second recognition and determination unit 242. In this way, by using the difference between the recognition and judgment performed by the first recognition and judgment unit 142 and the recognition and judgment performed by the second recognition and judgment unit 242 to update at least one of their respective recognition and judgment functions, the accuracy of the recognition or judgment performed by the first recognition and judgment unit 142 and the second recognition and judgment unit 242 can be further improved.

[0045] • Remote control Remote control is a process executed when the determination unit 243 determines that the comparison result between the second determination result and the first determination result exceeds the second criterion, and the second determination unit 245 determines that the discrepancy is a problem that could lead to an accident. If the process executed by the determination unit 243 is remote control, the processing unit 246 remotely controls the autonomous vehicle 1. Specifically, it generates a predetermined remote control signal that instructs an action that can avoid an accident, and controls the device-side communication unit 21 to transmit this remote control signal to the autonomous vehicle 1. The autonomous vehicle, upon receiving this remote control signal, takes action to avoid an accident. As a result, it is possible to prevent accidents based on incorrect perceptions and judgments by the autonomous vehicle 1.

[0046] ·notification Notification is a process executed when the judgment unit 243 determines that the comparison result between the second judgment result and the first judgment result exceeds the second criterion, and the second judgment unit determines that the difference is not a problem that would lead to an accident. If the process executed by the judgment unit 243 is notification, the processing unit 246 notifies the remote monitor of the judgment result made by the judgment unit 243. Specifically, the output unit 23 is controlled to output the judgment result. As a result, the judgment result is notified to the remote monitor via the output unit 23. This means that the remote monitor only needs to check when they receive a notification. For this reason, the burden on the remote monitor can be reduced when multiple autonomous vehicles 1 are being remotely monitored. The processing unit 246 may be configured not to send notifications (only perform updates or remote control). In that case, the remote monitoring device 2 does not need to have an output unit 23.

[0047] [Effects and Effects] The second recognition and judgment unit 242 performs recognition and judgment based on the same data as the actual data obtained from the autonomous vehicle 1 that is actually in motion. The recognition and judgment performed by the second recognition and judgment unit 242 serves as a second opinion on the recognition and judgment performed by the monitored (actual) autonomous vehicle 1. If the comparison result (difference in recognition and judgment) between the two is within the second standard, autonomous driving and remote monitoring can continue without problems. On the other hand, if the comparison result between the two exceeds the second standard, it is possible that a situation that could lead to an accident has occurred, and remote monitoring will be performed carefully. The remote monitoring device 2 according to this embodiment performs such a double check and can detect errors in the autonomous driving control of the autonomous vehicle itself, thereby ensuring the safety of the autonomous driving of the actual autonomous vehicle 1. By detecting errors in the autonomous driving control, those errors can be corrected.

[0048] Furthermore, for autonomous driving at Level 4 or higher, remote monitoring by a human supervisor is required. Therefore, a remote supervisor needs to take over the role of human supervisor during the operation of autonomous vehicles, but from a business perspective, continuous monitoring is difficult. However, with the remote monitoring device 2 according to this embodiment, the remote supervisor only needs to respond to autonomous vehicles 1 whose perceptions do not match those of the human supervisor. As a result, a large number of autonomous vehicles 1 can be easily remotely monitored by a small number of people. In addition, the data used is detected by the sensors 11 installed in the actual autonomous vehicles 1, so there is no need for human generation. For this reason, the supervisor can easily operate the remote monitoring device 2.

[0049] <3. Remote Monitoring Method> Next, an embodiment of a remote monitoring method, which is another aspect of the present invention, will be described in detail.

[0050] The remote monitoring method, as shown in Figure 3, comprises an acquisition step S1, a first recognition judgment step S2, a second recognition judgment step S3, a determination step S4, and a processing step S5.

[0051] (Acquisition Step) In the initial acquisition step S1, data generated by sensor 11 is acquired. In this embodiment, acquisition step S1 is performed by the acquisition unit 141 of the autonomous driving vehicle 1. Alternatively, in acquisition step S1, the data may be acquired by the remote monitoring device 2.

[0052] (First recognition and judgment step) After acquiring the data, the process moves to the first recognition and judgment step S2. In the first recognition and judgment step S2, at least one event inside or outside the autonomous vehicle 1 is recognized based on the data (from the sensor 11) acquired in the acquisition step S1, and a judgment is made based on the recognition result. In this embodiment, the first recognition and judgment step S2 is performed by the first recognition and judgment unit 142 of the autonomous vehicle 1. Alternatively, in the first recognition and judgment step S2, the recognition of the event and the judgment based on the recognition result may be performed by the remote monitoring device 2.

[0053] (Second recognition and judgment step) After recognizing at least one event inside or outside the autonomous vehicle 1 and making a decision based on the recognition result, the process moves to the second recognition decision step S3. In the second recognition decision step S3, the event is recognized based on the same data used in the first recognition decision step S2, and a decision is made based on the recognition result. In this embodiment, the second recognition decision step S3 is made by the second recognition decision unit 242 of the remote monitoring device 2. In addition, the autonomous vehicle 1 may make the recognition of the event and the decision based on the recognition result in the second recognition decision step S3.

[0054] (Judgment step) After recognizing an event and making a judgment based on the recognition result, the process moves to determination step S4. In determination step S4, it is determined whether the comparison result between the second judgment result in the second recognition judgment step S3 and the first judgment result in the first recognition judgment step S2 exceeds the second criterion. In this embodiment, determination step S4 is made by the determination unit 243 of the remote monitoring device 2. Alternatively, in determination step S4, the autonomous driving vehicle 1 may determine whether the comparison result between the second judgment result and the first judgment result exceeds the second criterion.

[0055] (Processing step) After determining whether the comparison result between the second judgment result and the first judgment result exceeds the second criterion, the process proceeds to processing step S5. In processing step S5, processing related to remote monitoring is performed based on the judgment result in the judgment step S4. In this embodiment, processing step S5 is performed by the processing unit 246 of the remote monitoring device 2. In addition, the autonomous driving vehicle 1 may perform the processing related to remote monitoring in processing step S5. Furthermore, if it is determined in processing step S5 that the comparison result between the second judgment result and the first judgment result exceeds the second criterion, the processing related to remote monitoring does not need to be performed.

[0056] <4. Others> The functions of the above-described autonomous vehicle 1 and remote monitoring device 2 (hereinafter referred to as "devices 1 and 2") can be realized by a program that causes a computer to function as devices 1 and 2, and by a program that causes a computer to function as each control block of devices 1 and 2 (particularly the parts included in the vehicle-side control unit 14 and the device-side control unit 24). In this case, devices 1 and 2 include a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the above program. By executing the above program with this control device and storage device, each of the functions described in the above embodiment is realized. The above program may be recorded on one or more computer-readable recording media, not temporary. This recording media may or may not be provided by the above-described device. In the latter case, the above program may be supplied to the above-described device via any wired or wireless transmission medium.

[0057] Furthermore, some or all of the functions of each of the above control blocks can also be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of the present invention. In addition, it is also possible to realize the functions of each of the above control blocks by, for example, a quantum computer.

[0058] Furthermore, the present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included within the technical scope of the present invention.

[0059] According to each aspect of the present invention described above, the invention can contribute to achieving Sustainable Development Goal (SDG) 11, "Make cities and human settlements inclusive, safe, and sustainable," by providing safe and sustainable public transportation such as buses. [Explanation of Symbols]

[0060] 1. Autonomous vehicles 11 sensors 12. Vehicle-side communication unit 13 Vehicle-side storage unit 131 Recognition threshold 132 coefficients 133 Judgment threshold 14 Vehicle-side control unit 141 Acquisition Department 142 First Recognition Judgment Department 142a Event Recognition Unit 142b Weighting section 142c Event Judgment Unit 143 Vehicle-side transmission control unit 144 Vehicle-side receiving control unit 145 Operation Control Unit 2 Remote monitoring device 21. Device-side communication unit 22 Device side storage unit 221 Threshold for device-side recognition 222 Device-side coefficient 223 Threshold for device-side decision 224 Databases 23 Output section 24 Device-side control unit 241 Device-side receiving control unit (receiving control unit) 242 Second Recognition Judgment Department 242a Second Event Recognition Unit 242b Second weighting section 242c Second Event Judgment Unit 243 Judgment section 244 Verification Department 245 Second Judgment Department 246 Processing Unit

Claims

1. A communication unit that communicates with an autonomous vehicle, which has a first recognition determination unit that recognizes at least one event inside or outside the autonomous vehicle based on data from sensors and determines how to control the autonomous vehicle based on the recognition result, A receiving control unit controls the communication unit to receive the data acquired by the first recognition and determination unit from the sensor and the first determination result from the first recognition and determination unit from the autonomous vehicle. A second recognition and determination unit recognizes the event based on the aforementioned data and determines how to control the autonomous vehicle based on the recognition result, A determination unit that determines whether the comparison result between the first determination result and the second determination result by the second recognition determination unit exceeds a comparison standard, A processing unit that performs remote monitoring processing based on the determination result by the determination unit, If the determination unit determines that the comparison result between the second determination result and the first determination result exceeds the comparison criterion, the verification unit verifies the difference in the determination, If the determination unit determines that the comparison result between the second determination result and the first determination result exceeds the comparison criterion, the second determination unit determines, based on the results of the verification, whether or not the difference in the determination is a problem that could lead to an accident. Equipped with, The criteria used by the second recognition and judgment unit when performing recognition and judgment are different from those of the first recognition and judgment unit. If the second determination unit determines that the aforementioned difference is a problem that could lead to an accident, the processing unit remotely controls the autonomous vehicle. Remote monitoring system.

2. The first and second judgment results differ due to differences in any of the following: the recognition of events based on the data acquired by the sensor, the weighting of the recognized events, or the threshold used to make a judgment based on the recognition. The remote monitoring system according to claim 1.

3. The determination unit determines whether the difference between the first numerical value representing the first judgment result and the second numerical value representing the second judgment result, or the ratio of the first numerical value to the second numerical value, exceeds the comparison standard. The remote monitoring system according to claim 1.

4. Further comprising a database for storing verification results from the verification unit, The remote monitoring system according to claim 1.

5. It is an autonomous vehicle, A communication unit that communicates with a remote monitoring system that monitors the operation of the autonomous vehicle, A sensor that detects at least one event inside or outside the autonomous vehicle and generates data, A first recognition and determination unit recognizes the event based on data from the aforementioned sensor and determines how to control the autonomous vehicle based on the recognition result, An operation control unit that controls at least one of acceleration / deceleration or steering based on the first determination result from the first recognition determination unit, The system comprises a transmission control unit that controls the communication unit to transmit the first judgment result and the data input to the first recognition judgment unit to the remote monitoring system, The driving control unit recognizes the event based on the data, and when the communication unit receives a predetermined remote control signal transmitted by the remote monitoring system based on the remote monitoring system's determination that the comparison result between the second determination result of the second recognition determination unit, which recognizes the event based on the data and determines how to control the autonomous vehicle based on the recognition result, exceeds a comparison criterion, the driving control unit controls at least one of acceleration / deceleration or steering based on the remote control signal, rather than the first determination result. The criteria for making at least one of the recognition and judgments of the second recognition and judgment unit are different from those of the first recognition and judgment unit. When the operation control unit receives information from the remote monitoring system to update at least one of the recognition and judgment functions in the first recognition and judgment unit, it updates the function. Self-driving vehicles.

Images