Information processing method and information processing device
By performing initial preprocessing on the autonomous driving vehicle and outputting data to an external advanced processing device, the problem of insufficient performance of the autonomous mobile body is solved, and more advanced driving control decisions and safety improvements are achieved.
Patent Information
- Application Number
- CN202411391334.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2020-02-14
- Filing Date
- 2020-12-23
- Publication Date
- 2024-11-15
AI Technical Summary
The computers equipped with autonomous driving vehicles are limited in terms of cost, power consumption and space, resulting in insufficient performance of autonomous mobile systems and it is difficult to effectively improve the safety and performance of autonomous mobile.
After performing initial preprocessing on the autonomous mobile body, the sensing data is output to the external device for advanced preprocessing, and the driving control decision is made using the higher processing power of the external device, and the difference between the processing results and the initial results is corrected or Limit to improve driving control accuracy and safety.
The driving performance and safety of autonomous mobile bodies are improved, and the driving control capabilities of autonomous driving vehicles are enhanced through the introduction of advanced external processing results, and the driving area is expanded.
Smart Images

Figure CN118968797A_ABST
Abstract
Description
[0001] This application was submitted on December 23, 2020, the Chinese patent application number is 202080031968.4 (international application number PCT / JP2020 / 048205), and the invention name is "Information processing method, information processing system and information processing device". The application is a divisional application of the patent application. Technical Field
[0002] The present disclosure relates to an information processing method, an information processing system, and an information processing device related to autonomous movement of an autonomous moving object. Background Art
[0003] In order to improve the safety of the autonomous driving system of an autonomous driving vehicle, it is desirable to overlap the hardware or software. For example, Patent Documents 1 and 2 propose the following method, that is, in an autonomous driving vehicle, the safety is improved by overlapping the power supply, detection function, and control function systems.
[0004] (Prior art literature)
[0005] (Patent Document)
[0006] Patent Document 1: International Publication No. 2018 / 154860
[0007] Patent Document 2: Japanese Patent No. 3881197 Summary of the invention
[0008] Problems to be solved by the invention
[0009] However, from the perspectives of cost, power consumption, and space, there are limits on the computers that can be mounted on autonomous vehicles such as self-driving cars, and sometimes the performance of autonomous mobile systems mounted on autonomous mobile bodies is low. Therefore, in the methods disclosed in the above-mentioned patent documents 1 and 2, the safety of autonomous mobility can be improved by overlapping the hardware or software mounted on the autonomous mobile body, but sometimes there are deficiencies in the performance of autonomous mobility.
[0010] Therefore, the present disclosure provides an information processing method and the like capable of improving the performance of autonomous mobility.
[0011] Means used to solve the problem
[0012] The information processing method disclosed herein causes an information processing device mounted on an autonomous mobile body to perform the following processing: performing a first preprocessing based on sensor data obtained from the autonomous mobile body and obtaining a first processing result, wherein the first preprocessing refers to a preprocessing of a driving control process in an automatic driving of the autonomous mobile body; outputting the sensor data to an external device; obtaining a second processing result obtained by performing a second preprocessing based on the sensor data, which is a preprocessing higher than the first preprocessing, from the external device; wherein the second preprocessing is performed by the external device connected to the autonomous mobile body via a network; The processing capability of the external device is higher than that of the information processing device. When there is a communication delay between the autonomous mobile body and the external device, or when there is a processing delay in the second preprocessing, the driving of the autonomous mobile body is restricted. When there is no communication delay between the autonomous mobile body and the external device, or when there is no processing delay in the second preprocessing, the driving of the autonomous mobile body is controlled according to the first processing result or the third processing result, and the third processing result is the second processing result, or a processing result obtained by correcting the first processing result.
[0013] The information processing device disclosed herein is mounted on an autonomous mobile body and performs the following processing: performing a first preprocessing based on sensor data obtained from the autonomous mobile body and obtaining a first processing result, wherein the first preprocessing refers to preprocessing of driving control processing in automatic driving of the autonomous mobile body; outputting the sensor data to an external device; obtaining a second processing result obtained by performing a second preprocessing that is a higher level preprocessing than the first preprocessing based on the sensor data from the external device, wherein the second preprocessing is performed by the external device connected to the autonomous mobile body via a network, and the processing capability of the external device is higher than that of the information processing device; restricting the driving of the autonomous mobile body when there is a communication delay between the autonomous mobile body and the external device or when there is a processing delay in the second preprocessing; and controlling the driving of the autonomous mobile body based on the first processing result or a third processing result when there is no communication delay between the autonomous mobile body and the external device or when there is no processing delay in the second preprocessing, wherein the third processing result is the second processing result or a processing result obtained by correcting the first processing result.
[0014] The information processing method disclosed herein causes an external device connected to an autonomous mobile body via a network to perform the following processing: obtaining a first processing result as a result of a first preprocessing from the autonomous mobile body and sensor data obtained by the autonomous mobile body, wherein the first preprocessing refers to a preprocessing of a driving control process in an autonomous movement process of the autonomous mobile body, wherein the first preprocessing is performed by an information processing device of the autonomous mobile body based on the sensor data, and a second preprocessing as a preprocessing higher than the first preprocessing is performed based on the sensor data to obtain a second processing result, wherein the processing capacity of the external device is higher than that of the information processing device. The processing device is high, and when there is a communication delay between the autonomous mobile body and the external device, or when there is a processing delay in the second preprocessing, a driving restriction instruction for restricting the driving of the autonomous mobile body is output to the autonomous mobile body, and when there is no communication delay between the autonomous mobile body and the external device, or when there is no processing delay in the second preprocessing, a change instruction for changing the first processing result to the third processing result is output to the autonomous mobile body, and the third processing result is the second processing result, or a processing result obtained by correcting the first processing result.
[0015] The information processing method involved in the present disclosure enables a computer to perform the following processing: obtaining a first processing result as a result of a first preprocessing from an autonomous moving body and sensor data obtained by the autonomous moving body, the first preprocessing refers to the preprocessing of a driving control processing in the autonomous moving processing of the autonomous moving body, performing a second preprocessing as a preprocessing higher than the first preprocessing based on the sensor data to obtain a second processing result, judging the difference between the first processing result and the second processing result, and outputting a change instruction to change the first processing result to a third processing result according to the judged difference to the autonomous moving body, the third processing result being a result obtained based on at least one of the first processing result and the second processing result.
[0016] In addition, these general or specific solutions may be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, methods, integrated circuits, computer programs, and recording media.
[0017] Effects of the Invention
[0018] The information processing method and the like according to one embodiment of the present disclosure can improve the performance of autonomous mobility. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1This is a block diagram showing an example of an autonomous driving vehicle and a remote autonomous driving server according to Implementation Example 1.
[0020] Figure 2 This is a flowchart showing an example of the information processing method according to the first embodiment.
[0021] Figure 3 This is a flowchart showing another example of the information processing method according to the first embodiment.
[0022] Figure 4 This is a block diagram showing an example of an autonomous driving vehicle, a remote autonomous driving server, and a remote processing management server according to a variation of Implementation Example 1.
[0023] Figure 5 This is a flowchart showing an example of an information processing method according to a variation of the first embodiment.
[0024] Figure 6 This is a block diagram showing an example of an autonomous driving vehicle and a remote autonomous driving server according to Implementation Example 2.
[0025] Figure 7 This is a flowchart showing an example of the operation of the automatic driving vehicle according to the second embodiment.
[0026] Figure 8 This is a block diagram showing an example of an autonomous driving vehicle, a remote autonomous driving server, and a remote processing management server according to a variation of Implementation Example 2.
[0027] Fig. 9 This is a flowchart showing an example of the operation of an automatic driving vehicle according to a variation of Implementation Example 2.
[0028] Fig.10 This is a flowchart showing an example of an information processing method according to a modified example common to each embodiment. DETAILED DESCRIPTION
[0029] An information processing method involved in one scheme of the present disclosure causes a computer to execute the following processing: obtaining a first processing result as a result of a first preprocessing from an autonomous moving body and sensor data obtained by the autonomous moving body, the first preprocessing being a preprocessing of a driving control process in an autonomous moving process of the autonomous moving body, executing a second preprocessing as a preprocessing higher than the first preprocessing based on the sensor data to obtain a second processing result, judging a difference between the first processing result and the second processing result, and outputting a change instruction for changing the first processing result to a third processing result according to the judged difference to the autonomous moving body, the third processing result being a result obtained based on at least one of the first processing result and the second processing result.
[0030] Unlike the first preprocessing performed by an autonomous mobile body such as an automatic driving car, which has limited computers that can be mounted from the perspective of cost, power consumption, and space, a second preprocessing that is more advanced than the first preprocessing is performed on a server or the like with fewer restrictions on cost, power consumption, and space, and a difference between a first processing result that is a result of the first preprocessing and a second processing result that is a result of the second preprocessing is determined. In addition, according to the difference, a change instruction is output to the autonomous mobile body to change the first processing result, which is the result of the first preprocessing, that is, the preprocessing of the driving control processing of the autonomous mobile body, to a third processing result based on the result of the second preprocessing that is more advanced, and the change instruction is output to the autonomous mobile body. Alternatively, according to the difference, a change instruction is output to the autonomous mobile body to change the first processing result to a result obtained by correcting or limiting the first processing result, that is, the third processing result. Thus, in the driving control processing of the autonomous mobile body, the third processing result that is more advanced than the first processing result is used, so that the performance of the autonomous mobile body can be improved.
[0031] Furthermore, the first preprocessing may be performed using a first resource, and the second preprocessing may be performed using a second resource, and the first resource and the second resource may be different resources.
[0032] Thus, the second preprocessing which is higher than the first preprocessing can be performed using the second resource different from the first resource used when performing the first preprocessing. Therefore, the second preprocessing can be performed with higher accuracy, higher speed or lower latency than the first preprocessing.
[0033] Furthermore, the first preprocessing may be performed using a first algorithm, and the second preprocessing may be performed using a second algorithm, and the first algorithm and the second algorithm may be different algorithms.
[0034] Thus, the second preprocessing which is more advanced than the first preprocessing can be performed using a second algorithm different from the first algorithm used when performing the first preprocessing. Therefore, the second preprocessing can be a processing with higher accuracy, higher speed, lower latency or more functions than the first preprocessing.
[0035] Furthermore, the first preprocessing and the second preprocessing may include a recognition process for recognizing an environment in which the autonomous mobile body is located.
[0036] In this way, each preprocessing may include recognition processing, and the result of the recognition processing may be used in the driving control processing of the autonomous mobile body. Therefore, the safety or comfort of the autonomous mobile body can be improved.
[0037] Furthermore, the first pre-processing and the second pre-processing may include a driving determination process of the autonomous moving body.
[0038] In this way, each preprocessing may include the driving judgment processing of the autonomous mobile body, and the result of the driving judgment processing of the autonomous mobile body may be used in the driving control processing of the autonomous mobile body. Therefore, the safety or comfort of the autonomous mobile body can be improved.
[0039] Furthermore, the third processing result may be the second processing result.
[0040] As described above, in the travel control process of the autonomous moving body, the second processing result of the second preprocessing which is more advanced than the first preprocessing is used, so the performance of the autonomous moving body can be improved.
[0041] Furthermore, the third processing result may be a result obtained by correcting the first processing result based on the difference.
[0042] Through the above, the third processing result is used in the driving control of the autonomous moving body, so the performance of the autonomous movement can be improved. The third processing result is the result of correcting the first processing result based on the difference between the second processing result of the second preprocessing that is more advanced than the first preprocessing and the first processing result.
[0043] Furthermore, when a request for executing the second pre-processing is received from the autonomous moving body, the second pre-processing is executed to obtain a second processing result, and the change instruction may be output to the autonomous moving body as a response to the request.
[0044] As described above, for example, the second pre-processing can be executed at the timing when the autonomous moving body wishes to execute the advanced second pre-processing.
[0045] Furthermore, the request may include information designating a specific process in the second pre-processing, and the specific process may be executed to obtain the second process result.
[0046] As described above, it is possible to selectively execute a specific process, for example, desired by the autonomous moving body, which is specified in the second pre-processing.
[0047] Furthermore, when the request is accepted, it is determined whether to execute the second preprocessing, and when it is determined that the second preprocessing is not to be executed, the request is rejected or ignored. For example, the determination of whether to execute the second preprocessing can be made based on at least one of the resources of the autonomous mobile body, the moving state of the autonomous mobile body, the external environment of the autonomous mobile body, the time, and the response time to the request.
[0048] When a request is received, the performance of the autonomous movement may not be improved even if the second pre-processing is performed, depending on the situation. For example, depending on the resources of the autonomous movement body, the movement state of the autonomous movement body, the external environment of the autonomous movement body, the time, and the response time to the request, the performance of the autonomous movement may not be improved even if the second pre-processing is performed and the driving control process of the autonomous movement body is performed using the third processing result. In such a case, the request can be rejected or ignored.
[0049] An information processing system involved in one scheme of the present disclosure is an information processing system that can communicate with an autonomous moving body, in which a first processing result as a result of a first preprocessing and sensor data obtained by the autonomous moving body are obtained from the autonomous moving body, the first preprocessing refers to the preprocessing of a driving control process in the autonomous movement of the autonomous moving body, a second preprocessing as a preprocessing higher than the first preprocessing is performed based on the sensor data to obtain a second processing result, a difference between the first processing result and the second processing result is judged, and according to the judged difference, a change instruction for changing the first processing result to a third processing result is output to the autonomous moving body, the third processing result being a result obtained based on at least one of the first processing result and the second processing result.
[0050] Through the above, it is possible to provide an information processing system that improves the performance of autonomous mobility.
[0051] An information processing device involved in one scheme of the present disclosure is an information processing device mounted on an autonomous mobile body, in which a first preprocessing is performed to obtain a first processing result, wherein the first preprocessing refers to the preprocessing of the driving control processing in the autonomous movement of the autonomous mobile body, the sensor data obtained by the autonomous mobile body is output to an external device, and a second processing result is obtained from the external device, wherein the second processing result is a result obtained by performing a second preprocessing which is a preprocessing higher than the first preprocessing based on the sensor data, and a difference between the first processing result and the second processing result is judged, and according to the judged difference, the first processing result is changed to a third processing result, and the third processing result is a result obtained based on at least one of the first processing result and the second processing result.
[0052] For example, unlike the first preprocessing performed by an autonomous mobile body such as an autonomous driving vehicle, which has limited computers that can be mounted from the perspectives of cost, power consumption, and space, a second preprocessing that is more advanced than the first preprocessing is performed by an external device such as a server that has fewer restrictions on cost, power consumption, and space, and a second processing result that is the result of the second preprocessing is sent to the autonomous driving vehicle, and the autonomous driving vehicle determines the difference between the result of the first preprocessing, i.e., the first processing result, and the result of the second preprocessing, i.e., the second processing result. And according to the difference, the first processing result, i.e., the result of the preprocessing in the driving control process of the autonomous mobile body, i.e., the result of the first preprocessing, is changed to the following third processing result, which is a result based on the result of the second preprocessing that is more advanced, i.e., the second processing result. Or, according to the difference, the first processing result is changed to the following third processing result, which is a result obtained by correcting or limiting the first processing result. Thus, in the driving control process of the autonomous mobile body, the third processing result that is more advanced than the first processing result is used, so that the performance of the autonomous mobile body can be improved.
[0053] Furthermore, a request for executing the second pre-processing may be output to the external device, and the second processing result may be obtained as a response to the request.
[0054] As described above, for example, when the autonomous moving body wishes to execute the advanced second pre-processing, the external device can be caused to execute the second pre-processing.
[0055] In addition, the request may be output to the external device based on at least one of the resources possessed by the autonomous mobile body, the moving state of the autonomous mobile body, the external environment of the autonomous mobile body, the time, and the response time to the inquiry to the external device.
[0056] Depending on the situation, the performance of autonomous mobility may not be improved even if the second preprocessing is performed. For example, even if the second preprocessing is performed to perform driving control processing of the autonomous mobility using the third processing result, the performance of autonomous mobility may not be improved depending on the resources of the autonomous mobility, the movement state of the autonomous mobility, the external environment of the autonomous mobility, the time, and the response time to the request. In such a case, the request may not be output. In other words, depending on the situation, the performance of autonomous mobility may be improved by performing the second preprocessing. In such a situation where the performance of autonomous mobility can be improved, the request may be output.
[0057] Furthermore, the request may include information designating a specific process in the second pre-process, and the second process result may be a result obtained by executing the specific process.
[0058] As described above, by designating, for example, a specific process desired by the autonomous moving body in the second pre-processing, it is possible to cause an external device to selectively execute the specific process.
[0059] Furthermore, when the second processing result is obtained more than a prescribed time period from the output of the sensor data or the output of the request, (A) the judgment of the difference is not performed, or (B) a judgment is made on the difference between a portion of the processing result in the first processing result and a processing result in the second processing result corresponding to the portion of the processing result, and according to the judged difference, the portion of the processing result in the first processing result is changed to the third processing result.
[0060] As described above, when the second processing result is obtained after a predetermined time has passed since the output of the sensing data or the output of the request, there is a possibility that a communication delay occurs with the external device. In this case, the difference judgment is not performed, or the difference judgment is performed only on a part of the processing results that are not affected by the delay, thereby reducing the processing load of the information processing device.
[0061] Hereinafter, embodiments and the like will be described with reference to the drawings.
[0062] In addition, the embodiments described below are all general or specific examples. The values, shapes, materials, components, configuration positions of components, connection forms, steps, and the order of steps shown in the following embodiments are all examples, and their purpose is not to limit the present disclosure.
[0063] (Implementation Method 1)
[0064] use Figures 1 to 3 Implementation method 1 will be described.
[0065] Figure 1 This is a block diagram showing an example of the autonomous driving vehicle (specifically, the information processing device 20 mounted on the autonomous driving vehicle) and the remote autonomous driving server 10 according to the first embodiment.
[0066] For example, an autonomous vehicle is a vehicle that can travel automatically without human driving. An autonomous vehicle is equipped with sensors such as cameras, thermal imagers, radars, LiDAR (Light Detection and Ranging), sonar, GPS (Global Positioning System) or IMU (Inertial Measurement Unit), which can recognize the surrounding environment and travel autonomously. In addition, an autonomous vehicle is an example of an autonomous mobile object. An autonomous mobile object can be a mobile robot, a flying object such as a drone, or a ship.
[0067] The remote automatic driving server 10 can wirelessly communicate with the automatic driving vehicle and remotely control the automatic driving of the automatic driving vehicle. In addition, automatic driving is an example of autonomous movement.
[0068] The automatic driving vehicle is equipped with an information processing device 20. The information processing device 20 is a computer including a processor, a memory, and a communication interface. The memory is a ROM (Read Only Memory) and a RAM (Random Access Memory), etc., which can store programs executed by the processor. The information processing device 20 includes: a sensor data acquisition unit 21, a sensor data transmission unit 22, a vehicle automatic driving system 23, a first processing result transmission unit 24, a processing result change unit 25, a communication confirmation unit 26, and a driving restriction unit 27. The sensor data acquisition unit 21, the sensor data transmission unit 22, the vehicle automatic driving system 23, the first processing result transmission unit 24, the processing result change unit 25, the communication confirmation unit 26, and the driving restriction unit 27 are implemented by a processor that executes a program stored in the memory.
[0069] The sensor data acquisition unit 21 acquires sensor data from sensors such as a camera, thermal imager, radar, LiDAR, sonar, GPS, or IMU that the autonomous driving vehicle has. In addition, the sensor data acquisition unit 21 can acquire sensor data from sensors installed on other vehicles or traffic lights.
[0070] The sensor data transmission unit 22 transmits the sensor data obtained by the sensor data acquisition unit 21 to the remote automatic driving server 10. The sensor data transmission unit 22 transmits the sensor data to the remote automatic driving server 10, for example, via a communication interface provided by the information processing device 20. The sensor data transmission unit 22 can transmit the sensor data with low latency by utilizing data compression technology and high-speed transmission technology such as 5G.
[0071] The vehicle automatic driving system 23 performs the preprocessing of the driving control process in the automatic driving process of the automatic driving vehicle, i.e., the first preprocessing, based on the sensor data obtained by the sensor data obtaining unit 21, and obtains the first processing result. The first processing result is the result of the first preprocessing. For example, the preprocessing includes recognition processing or driving judgment processing of the automatic driving vehicle. The recognition processing includes the processing of recognizing the environment in which the automatic driving vehicle is located. The environment is the self-position, surrounding objects, road surface conditions, weather, or road conditions. Specifically, the preprocessing includes the estimation processing of the self-position of the automatic driving vehicle, the detection processing of the surrounding objects of the automatic driving vehicle, the movement prediction processing of the surrounding objects of the automatic driving vehicle, the driving judgment processing of the automatic driving vehicle, or the path planning processing of the automatic driving vehicle. For example, the vehicle automatic driving system 23 can use a learning model to perform these processes. For example, the processing result includes the recognition result or the driving judgment result of the automatic driving vehicle. Specifically, the processing result includes the estimation result of the self-position of the automatic driving vehicle, the detection result of the surrounding objects of the automatic driving vehicle, the movement prediction result of the surrounding objects of the automatic driving vehicle, the driving judgment result of the automatic driving vehicle, or the path planning result of the automatic driving vehicle.
[0072] The first processing result sending unit 24 sends the first processing result, which is the result of the first preprocessing performed by the vehicle automatic driving system 23, to the remote automatic driving server 10. The first processing result sending unit 24 sends the first processing result to the remote automatic driving server 10, for example, via a communication interface provided in the information processing device 20.
[0073] In addition, the sensor data sending unit 22 may send only the sensor data used when the vehicle automatic driving system 23 performs the first preprocessing, or may send not only the sensor data used when the vehicle automatic driving system 23 performs the first preprocessing, but also the sensor data not used when performing the first preprocessing (for example, high-resolution sensor data that cannot be processed by the vehicle automatic driving system 23).
[0074] The processing result changing unit 25 changes the first processing result to the third processing result according to the change instruction received from the remote automatic driving server 10. In other words, the processing result changing unit 25 changes the first processing result to the third processing result, thereby using the third processing result instead of the first processing result originally used for driving control of the automatic driving vehicle.
[0075] The communication confirmation unit 26 confirms the communication status of the wireless communication between the autonomous driving vehicle and the remote autonomous driving server 10. For example, the communication confirmation unit 26 sends an inquiry to the remote autonomous driving server 10 via the communication interface of the information processing device 20, and confirms the communication status according to the response to the inquiry. Specifically, the communication confirmation unit 26 can determine that there is no communication connection when there is no response to the inquiry, and can determine that a communication delay has occurred when there is a delay in the response to the inquiry. When it is determined that there is no communication connection or when it is determined that a communication delay has occurred, the communication confirmation unit 26 notifies the driving restriction unit 27 of a driving restriction instruction to restrict driving.
[0076] The driving restriction unit 27 restricts the driving of the autonomous driving vehicle according to the driving restriction instruction. Specifically, the driving restriction unit 27 decelerates the autonomous driving vehicle, stops the autonomous driving vehicle, or increases the avoidance margin. Thus, even when the communication between the autonomous driving vehicle and the remote autonomous driving server 10 is cut off, the safety of the autonomous driving vehicle can be ensured.
[0077] The remote autonomous driving server 10 is a computer including a processor, a memory, a communication interface, etc. The remote autonomous driving server 10 is an example of an information processing system that can communicate wirelessly with an autonomous driving vehicle. The memory is a ROM and a RAM, etc., which can store programs executed by the processor. The remote autonomous driving server 10 includes a sensor data acquisition unit 11, a simulated autonomous driving system 12, a first processing result acquisition unit 13, a difference judgment unit 14, and a change instruction output unit 15. The sensor data acquisition unit 11, the simulated autonomous driving system 12, the first processing result acquisition unit 13, the difference judgment unit 14, and the change instruction output unit 15 are implemented by a processor that executes a program stored in the memory. In addition, the components that constitute the remote autonomous driving server 10 can be distributed in multiple servers.
[0078] The sensor data acquisition unit 11 acquires sensor data acquired by the autonomous driving vehicle from the autonomous driving vehicle. For example, the sensor data acquisition unit 11 acquires sensor data sent from the autonomous driving vehicle and received via a communication interface or the like provided by the remote autonomous driving server 10.
[0079] The simulated automatic driving system 12 performs a second preprocessing, which is a preprocessing of the driving control process in the automatic driving of the automatic driving vehicle, based on the sensor data obtained by the sensor data obtaining unit 11, and obtains a second processing result. The second preprocessing is a preprocessing that is more advanced than the first preprocessing. For example, the simulated automatic driving system 12 may perform the second preprocessing using a learning model. The second processing result is the result of the second preprocessing.
[0080] For example, from the perspective of cost, power consumption, and space, the number of computers that can be installed in an autonomous vehicle is limited. Therefore, the resources used by the vehicle autonomous driving system 23 are reduced, or the amount of calculation of the algorithm is reduced. Here, resources refer to the processing capacity or processing speed of the processor, storage capacity, or power. On the other hand, the remote autonomous driving server 10 has fewer restrictions on cost, power consumption, and space. Therefore, the resources used in the simulated autonomous driving system 12 can be increased, or the amount of calculation of the algorithm can be increased. Therefore, the second preprocessing performed by the remote autonomous driving server 10 can be a more advanced process than the first preprocessing performed by the information processing device 20 installed in the autonomous driving vehicle. For example, the first preprocessing is performed using the first resource (e.g., the small resource of the autonomous driving vehicle), and the second preprocessing is performed using the second resource different from the first resource (e.g., the large resource of the remote autonomous driving server 10), so that the second preprocessing can be made more advanced than the first preprocessing. In other words, the second resource is richer than the first resource. In addition, for example, the first preprocessing is performed using the first algorithm (e.g., an algorithm with a small amount of calculation that the autonomous driving car can handle), and the second preprocessing is performed using a second algorithm different from the first algorithm (e.g., an algorithm with a large amount of calculation that the remote autonomous driving server 10 can handle), so that the second preprocessing can be made more advanced than the first preprocessing. That is, the second algorithm is more advanced than the first algorithm. In addition, the first preprocessing can be performed using both the first resource and the first algorithm, and the second preprocessing can be performed using both the second resource and the second algorithm.
[0081] The first processing result obtaining unit 13 obtains the first processing result from the autonomous driving vehicle. For example, the first processing result obtaining unit 13 obtains the first processing result, which is a processing result sent from the autonomous driving vehicle and received via a communication interface or the like provided in the remote autonomous driving server 10.
[0082] The difference determination unit 14 determines the difference between the first processing result obtained by the first processing result obtaining unit 13 and the second processing result obtained by the simulated automatic driving system 12. The operation of the difference determination unit 14 will be described in detail later.
[0083] The change instruction output unit 15 outputs a change instruction to the autonomous vehicle to change the first processing result to the third processing result according to the difference determined by the difference determination unit 14. In other words, the change instruction output unit 15 issues an instruction to the autonomous vehicle to change the first processing result to the third processing result, so that the third processing result is used in the driving control of the autonomous vehicle instead of the first processing result originally used for the driving control of the autonomous vehicle. A specific example of the third processing result will be described later.
[0084] Next, the operation of the remote automatic driving server 10 is controlled by using Figure 2 To explain.
[0085] Figure 2 1 is a flowchart showing an example of the information processing method of Embodiment 1. For example, the information processing method of Embodiment 1 is a method executed by a computer (specifically, a processor) provided in the remote automatic driving server 10. Figure 2 1 is a flowchart showing the operation of the remote automatic driving server 10 .
[0086] First, the remote autonomous driving server 10 obtains the first processing result as a result of the first preprocessing from the autonomous driving vehicle and the sensor data obtained by the autonomous driving vehicle, wherein the first preprocessing refers to the preprocessing of the driving control processing in the autonomous driving processing of the autonomous driving vehicle (step S11). For example, the remote autonomous driving server 10 obtains the result of the detection processing obtained by the autonomous driving vehicle detecting obstacles around the autonomous driving vehicle based on the sensor data (e.g., detecting the number of obstacles or detecting the position of obstacles, etc.). In addition, for example, the remote autonomous driving server 10 obtains the result of the estimation processing obtained by the autonomous driving vehicle estimating the position of the autonomous driving vehicle based on the sensor data. In addition, for example, the remote autonomous driving server 10 obtains the result of the driving judgment processing obtained by the autonomous driving vehicle performing the driving judgment processing of the autonomous driving vehicle (e.g., judging whether to continue driving or stop, etc.) based on the sensor data.
[0087] Next, the remote autonomous driving server 10 performs a second preprocessing, which is a preprocessing higher than the first preprocessing, based on the sensor data, and obtains a second processing result (step S12). For example, the remote autonomous driving server 10 performs a detection process on obstacles around the autonomous driving vehicle based on the sensor data (e.g., detection of the number of obstacles or detection of the positions of obstacles, etc.), and obtains the result of the detection process. In addition, for example, the remote autonomous driving server 10 performs an estimation process on the position of the autonomous driving vehicle based on the sensor data, and obtains the result of the estimation process. In addition, for example, the remote autonomous driving server 10 performs a driving judgment process on the autonomous driving vehicle based on the sensor data (e.g., judgment on whether to continue driving or stop, etc.), and obtains the result of the driving judgment process.
[0088] Next, the remote autonomous driving server 10 determines whether there is a processing delay in the second pre-processing (step S13). For example, in the remote autonomous driving server 10, the resource occupancy rate for the second pre-processing is high, so there may be a processing delay in the second pre-processing.
[0089] The remote autonomous driving server 10, when there is a processing delay in the second preprocessing ("Yes" in step S13), outputs an instruction to restrict the driving of the autonomous driving vehicle (step S14). For example, it is preferable that the driving control of the autonomous driving vehicle is performed with as little delay as possible from the timing of obtaining the sensor data. This is because if it takes time from the acquisition of the sensor data to the use of the result of the preprocessing to perform driving control, the autonomous driving vehicle will move a lot from the location where the sensor data is obtained, causing the result of the preprocessing performed based on the sensor data to lose its validity at the current position of the autonomous driving vehicle that has deviated from the location where the sensor data is obtained. Therefore, when there is a processing delay in the second preprocessing, the autonomous driving vehicle cannot effectively use the result of the advanced second preprocessing performed by the remote autonomous driving server 10, and the autonomous driving vehicle may be in a dangerous state, so the remote autonomous driving server 10 outputs an instruction to restrict the driving of the autonomous driving vehicle to the autonomous driving vehicle. The instruction to restrict the driving of the autonomous driving vehicle is, for example, an instruction to slow down the autonomous driving vehicle, stop the autonomous driving vehicle, and expand the avoidance margin. In addition, when there is a processing delay in the second preprocessing, the remote autonomous driving server 10 can notify the autonomous driving vehicle of an alarm.
[0090] The remote autonomous driving server 10, when there is no processing delay in the second preprocessing ("No" in step S13), determines the difference between the first processing result and the second processing result (step S15). For example, the remote autonomous driving server 10 determines the difference between the number of obstacles shown in the first processing result and the number of obstacles shown in the second processing result. In addition, for example, the remote autonomous driving server 10 determines the difference between the position of the obstacle shown in the first processing result and the position of the obstacle shown in the second processing result (for example, the RMS (Root Mean Square) error value of the position of the obstacle shown in the first processing result when the position of the obstacle shown in the second processing result is correct). In addition, for example, the remote autonomous driving server 10 determines the difference between the position of the autonomous driving vehicle shown in the first processing result and the position of the autonomous driving vehicle shown in the second processing result (for example, the RMS error value of the position of the autonomous driving vehicle shown in the first processing result when the position of the autonomous driving vehicle shown in the second processing result is correct). In addition, for example, the remote autonomous driving server 10 determines the difference between the driving judgment processing result of the autonomous driving vehicle represented by the first processing result and the driving judgment processing result of the autonomous driving vehicle represented by the second processing result (for example, the number of times the driving judgment processing result represented by the first processing result and the driving judgment processing result represented by the second processing result become different results within a certain period). In addition, these are shown as examples of the judgment of the difference, and are not limited to this.
[0091] Next, the remote autonomous driving server 10 determines whether the determined difference satisfies a prescribed condition (step S16). The prescribed condition is, for example, a condition related to the size of the determined difference. For example, the remote autonomous driving server 10 determines whether the number of obstacles represented by the first processing result is different from the number of obstacles represented by the second processing result. In addition, for example, the remote autonomous driving server 10 determines whether the RMS error value is above a prescribed threshold. In addition, for example, the remote autonomous driving server 10 determines whether the number of times the driving judgment processing result represented by the first processing result and the driving judgment processing result represented by the second processing result become different results within a certain period is above a prescribed threshold. In addition, these are shown only as an example of prescribed conditions and are not limited to this.
[0092] When the determined difference satisfies the prescribed condition ("Yes" in step S16), the remote automatic driving server 10 outputs a change instruction to change the first processing result to the third processing result to the automatic driving vehicle (step S17). The cases where the determined difference satisfies the prescribed condition include, for example, the case where the number of obstacles represented by the first processing result is different from the number of obstacles represented by the second processing result, the case where the RMS error value is greater than a prescribed threshold value, and the case where the number of times the driving judgment processing result represented by the first processing result and the driving judgment processing result represented by the second processing result become different results within a certain period is greater than a prescribed threshold value.
[0093] The third processing result is a result obtained based on at least one of the first processing result or the second processing result. The second processing result is the result of the advanced second preprocessing, so the result of the processing based on the second processing result, that is, the third processing result, is the advanced processing result. For example, the third processing result may be the second processing result. In this way, in the driving control of the autonomous moving body, the first processing result is not used, and the second processing result as the result of the advanced second preprocessing is used, so the performance of the automatic driving can be improved. In addition, for example, the third processing result can be obtained by correcting the first processing result according to the determined difference. For example, according to the difference between the second processing result and the first processing result, only the necessary range or the processable range in the first processing result is corrected. Therefore, in the driving control of the automatic driving vehicle, the first processing result is not used, but the result after the first processing result is corrected according to the difference between the second processing result as the result of the advanced second preprocessing and the first processing result (in other words, the corrected first processing result) is used, which can improve the performance of the automatic driving.
[0094] In addition, the remote autonomous driving server 10 may output an instruction to restrict the driving of the autonomous driving vehicle when the determined difference satisfies the prescribed condition ("Yes" in step S16). In this case, the remote autonomous driving server 10 may notify the remote monitor or passenger of the autonomous driving vehicle of the abnormality.
[0095] The remote automatic driving server 10 does not output the change instruction to the automatic driving vehicle (step S18) when the determined difference does not satisfy the prescribed condition ("No" in step S16). The cases where the determined difference does not satisfy the prescribed condition include, for example, the case where the number of obstacles represented by the first processing result is the same as the number of obstacles represented by the second processing result, the case where the RMS error value is less than a prescribed threshold value, and the case where the number of times the driving judgment processing result represented by the first processing result and the driving judgment processing result represented by the second processing result become different results within a certain period is less than a prescribed threshold value. In this case, it can be said that the first processing result is not inferior to the second processing result, and the automatic driving vehicle does not receive the change instruction, and uses the first processing result that is not inferior to the second processing result to perform driving control of the automatic driving vehicle. In other words, when it is confirmed that the first processing result is not inferior to the second processing result, or the degree to which the first processing result is inferior to the second processing result is within an allowable range, the first processing result can be used in the driving control of the automatic driving vehicle.
[0096] In addition, when the automatic driving vehicle is delayed in receiving the change instruction due to communication delay, the received change instruction can be ignored and the driving of the automatic driving vehicle can be restricted. This is similar to the case where there is a processing delay in the second pre-processing described above, because even if the first processing result is changed to the third processing result, the automatic driving vehicle cannot effectively use the third processing result, and the automatic driving vehicle may be in a dangerous state.
[0097] In addition, the sensor data obtained by the autonomous driving vehicle is easily affected by the weather. The vehicle autonomous driving system 23 uses the sensor data affected by the weather to perform the first preprocessing, and sometimes cannot obtain the correct first processing result. For example, when the weather is rainy or cloudy, and the autonomous driving vehicle has poor vision around it, the autonomous driving vehicle may not be able to correctly identify the obstacles around it. Therefore, it is also possible to judge whether the weather is suitable for the autonomous driving performed by the vehicle autonomous driving system 23. In this regard, using Figure 3 To explain.
[0098] Figure 3 This is a flowchart showing another example of the information processing method according to the first embodiment.
[0099] Figure 3 The flowchart represented by Figure 2Compared with the flowchart shown in FIG. 1 , the difference is that step S19 is added. The rest (in other words, from step S11 to step S18) is the same as Figure 2 The steps are the same, so the description is omitted.
[0100] When there is no processing delay in the second pre-processing ("No" in step S13), the remote automatic driving server 10 determines whether the weather is suitable for the automatic driving by the vehicle automatic driving system 23 (step S19). For example, when the weather is sunny, it is determined that the weather is suitable for the automatic driving by the vehicle automatic driving system 23, and when the weather is rainy or cloudy, it is determined that the weather is not suitable for the automatic driving by the vehicle automatic driving system 23.
[0101] When the weather is not suitable for the automatic driving by the vehicle automatic driving system 23 ("No" in step S19), the remote automatic driving server 10 outputs a change instruction (step S17) without performing the judgment process of the difference between the first processing result and the second processing result in step S15. When the weather is not suitable for the automatic driving by the vehicle automatic driving system 23, it is estimated that the first processing result is inferior to the second processing result without specifically judging the difference.
[0102] The remote automatic driving server 10, when the weather is suitable for the automatic driving performed by the vehicle automatic driving system 23 ("Yes" in step S19), Figure 2 The same as the description of step S15 and subsequent processing are performed. Because the first processing result may be a result that is not inferior to the second processing result when the weather is suitable for the automatic driving of the vehicle automatic driving system 23, it is preferable to judge the difference and determine whether to output a change instruction.
[0103] In this way, it can be determined whether the weather is suitable for autonomous driving performed by the vehicle's autonomous driving system 23.
[0104] As described above, unlike the first preprocessing performed by the autonomous driving vehicle (specifically, the information processing device 20 mounted on the autonomous driving vehicle) which has limited computers that can be mounted from the viewpoints of cost, power consumption, and space, the remote autonomous driving server 10 which has fewer restrictions on cost, power consumption, and space, performs the second preprocessing which is higher than the first preprocessing, and judges the difference between the first processing result of the first preprocessing and the second processing result of the second preprocessing. And, according to the difference, a change instruction is outputted to the autonomous driving vehicle, which is a change instruction for changing the first processing result, which is a preprocessing of the driving control processing of the autonomous driving vehicle, to a third processing result based on the second processing result, and the second processing result is the result of the second preprocessing which is higher than the first processing result. Or, according to the difference, a change instruction is outputted to the autonomous driving vehicle, which is a change instruction for changing the first processing result to a third processing result obtained by correcting or limiting the first processing result. Therefore, in the driving control processing of the autonomous driving vehicle, the third processing result which is higher than the first processing result is used, so that the performance of the autonomous driving can be improved. For example, by improving the performance of autonomous driving, it is possible to expand the areas where autonomous vehicles can travel.
[0105] (Variation of Implementation Example 1)
[0106] For example, when a request is received from the autonomous driving vehicle, the second pre-processing may be started on the remote autonomous driving server. Figure 4 as well as Figure 5 To explain.
[0107] Figure 4 This is a block diagram showing an example of an autonomous driving vehicle (specifically, an information processing device 20a mounted on the autonomous driving vehicle), a remote autonomous driving server 10a, and a remote processing management server 30 according to a variation of Implementation Example 1.
[0108] The information processing device 20a is different from the information processing device 20 of the first embodiment in that it further includes a remote processing request unit 28, and includes a sensor data acquisition unit 21a, a sensor data transmission unit 22a, and a first processing result transmission unit 24a instead of the sensor data acquisition unit 21, the sensor data transmission unit 22, and the first processing result transmission unit 24. The rest is the same as the information processing device 20, so the description is omitted. In addition, the remote processing request unit 28 is implemented by a processor or the like that executes a program stored in a memory, like the other components.
[0109] The remote processing request unit 28 outputs a request for executing the second preprocessing to the remote automatic driving server 10a via the remote processing management server 30. For example, when there are multiple remote automatic driving servers 10a having the function of executing the second preprocessing, the remote processing management unit 31 of the remote processing management server 30, upon receiving the request, inquires the multiple remote automatic driving servers 10a whether the second preprocessing can be executed, and selects the remote automatic driving server 10a that can execute the second preprocessing from the multiple remote automatic driving servers 10a according to the result of the inquiry. The remote processing management unit 31 notifies the automatic driving vehicle of the selected remote automatic driving server 10a. The remote processing request unit 28 instructs the sensor data acquisition unit 21a to acquire sensor data, instructs the sensor data transmission unit 22a to transmit sensor data to the selected remote automatic driving server 10a, and instructs the first processing result transmission unit 24a to transmit the first processing result to the selected remote automatic driving server 10a.
[0110] The sensor data acquisition unit 21a acquires sensor data acquired by sensors included in the autonomous driving vehicle by receiving instructions from the remote processing request unit 28. Other parts related to the sensor data acquisition unit 21a are the same as those of the sensor data acquisition unit 21, and thus description thereof is omitted.
[0111] The sensor data transmission unit 22a transmits the sensor data obtained by the sensor data acquisition unit 21a to the selected remote automatic driving server 10a. The other parts related to the sensor data transmission unit 22a are the same as those of the sensor data transmission unit 22, and thus the description thereof is omitted.
[0112] The first processing result sending unit 24a sends the first processing result, which is the result of the first preprocessing performed by the vehicle automatic driving system 23, to the selected remote automatic driving server 10a. The other parts related to the first processing result sending unit 24a are the same as those of the first processing result sending unit 24, so the description thereof is omitted.
[0113] The remote automatic driving server 10a is different from the remote automatic driving server 10 of the first embodiment in that it further includes a request acquisition unit 16 and a second pre-processing execution judgment unit 17, and includes a sensor data acquisition unit 11a and a simulated automatic driving system 12a instead of the sensor data acquisition unit 11 and the simulated automatic driving system 12. The rest is the same as the remote automatic driving server 10, so the description is omitted. In addition, the request acquisition unit 16 and the second pre-processing execution judgment unit 17 are implemented by a processor or the like that executes a program stored in a memory, like other components.
[0114] The request acquisition unit 16 acquires a request for executing the second pre-processing from the autonomous driving vehicle. For example, the request acquisition unit 16 acquires a request for executing the second pre-processing from the autonomous driving vehicle via the remote processing management server 30 .
[0115] The second preprocessing execution judgment unit 17 judges whether to execute the second preprocessing when the acceptance is requested. For example, the second preprocessing execution judgment unit 17 can also judge whether to execute the second preprocessing according to the task status or resource status of the remote autonomous driving server 10a. For example, when the remote autonomous driving server 10a executes many tasks or the resources become scarce, it is judged that the second preprocessing is not executed. In addition, for example, the second preprocessing execution judgment unit 17 can judge whether to execute the second preprocessing according to at least one of the resources of the autonomous driving vehicle, the moving state of the autonomous driving vehicle, the external environment of the autonomous driving vehicle, the time, and the response time to the request. The moving state is the moving speed, acceleration, deceleration, moving direction (steering angle), etc. of the autonomous driving vehicle. For example, when the resources of the autonomous driving vehicle are insufficient, when the moving state of the autonomous driving vehicle is, for example, a state of high vehicle speed, a state of high acceleration, or a state of large steering angle, when the external environment of the autonomous driving vehicle is an environment where obstacles are close, the number of obstacles is large, or the type is a moving object, when the location of the autonomous driving vehicle is an environment where the traffic volume is large (for example, an intersection), the surrounding brightness is dark, the weather is rainy or cloudy, when the time is at night, when the response time to the request is short, etc., it is determined that the second pre-processing is executed. For example, when the resources of the autonomous driving vehicle are sufficient, when the moving state of the autonomous driving vehicle is, for example, a state of low vehicle speed, a state of low acceleration, or a state of small steering angle, when the external environment of the autonomous driving vehicle is an environment where obstacles are far, the number of obstacles is small, or the type is a non-moving object, when the location of the autonomous driving vehicle is an environment where the traffic volume is small, the surrounding brightness is bright, the weather is sunny, etc., when the time is during the day, when the response time to the request is long, etc., it is determined that the second pre-processing is not executed.
[0116] Thus, when a request is received, the performance of the autonomous driving may not be improved even if the second pre-processing is performed, depending on the situation. For example, depending on the resources of the autonomous driving vehicle, the movement state of the autonomous driving vehicle, the external environment of the autonomous driving vehicle, the time or the response time to the request, the performance of the autonomous driving may not be improved even if the second pre-processing is performed and the driving control of the autonomous driving vehicle is performed using the third processing result. In such a case, the remote autonomous driving server 10a can reject or ignore the request.
[0117] The sensor data acquisition unit 11a acquires sensor data when it is determined that the second pre-processing is to be executed upon receiving a request from the automatic driving vehicle. The other parts related to the sensor data acquisition unit 11a are the same as those of the sensor data acquisition unit 11, and thus description thereof is omitted.
[0118] When receiving a request from the automatic driving vehicle, the simulated automatic driving system 12a determines that the second pre-processing is to be performed, and performs the second pre-processing based on the sensor data obtained by the sensor data obtaining unit 11a to obtain the second processing result. The other parts related to the simulated automatic driving system 12a are the same as those of the simulated automatic driving system 12, so the description thereof is omitted.
[0119] Next, regarding the operation of the remote automatic driving server 10a, Figure 5 To explain.
[0120] Figure 5 This is a flowchart showing an example of an information processing method of a modified example of Implementation 1. For example, the information processing method of the modified example of Implementation 1 is a method executed by a computer (specifically, a processor) possessed by the remote automatic driving server 10a. Therefore, Figure 5 This is also a flowchart showing the operation of the remote automatic driving server 10a.
[0121] Figure 5 The flowchart represented by Figure 2 The difference from the flowchart shown in FIG. 1 is that step S21, step S22 and step S23 are added. The rest (in other words, from step S11 to step S18) is the same as Figure 2 The expressions are the same, so the description is omitted.
[0122] The remote automatic driving server 10a determines whether a request to execute the second pre-processing has been received from the automatic driving vehicle (step S21).
[0123] When the remote automatic driving server 10a has not received a request to execute the second pre-processing from the automatic driving vehicle (No in step S21), the remote automatic driving server 10a repeats the process of step S21 until the request is received.
[0124] When receiving a request to execute the second pre-processing from the autonomous driving vehicle (Yes in step S21 ), the remote autonomous driving server 10 a determines whether to execute the second pre-processing (step S22 ).
[0125] When the remote autonomous driving server 10a determines that the second pre-processing is not to be performed ("No" in step S22), it rejects or ignores the request (step S23). Thus, for example, the remote processing management server 30 does not select the remote autonomous driving server 10a that rejects or ignores the request as the server to perform the second pre-processing, but selects a server to perform the second pre-processing from other remote autonomous driving servers 10a. In addition, when all remote autonomous driving servers 10a reject or ignore the request, the remote processing management server 30 can instruct the autonomous driving vehicle to restrict driving.
[0126] When the remote automatic driving server 10a determines that the second pre-processing is to be performed ("Yes" in step S22), Figure 2 The same as described above, the processing after step S11 is performed.
[0127] In addition, the request from the autonomous driving vehicle may include information specifying a specific process in the second preprocessing. In this case, the remote autonomous driving server 10a performs a specific process in step S12 to obtain a second process result. This is because, depending on the movement state of the autonomous driving vehicle or the external environment, it is sometimes possible to judge the difference only for a specific process (for example, only for obstacle detection processing, only for driving judgment processing, etc.). Through the above, the remote autonomous driving server 10a can selectively execute a specific process specified in the second preprocessing, such as that desired by the autonomous driving vehicle. In addition, in the case of only obstacle detection, the area for obstacle detection can also be limited. For example, in the case where the autonomous driving vehicle changes lanes, obstacle detection can be performed on the lane after the lane change.
[0128] In addition, when the remote processing management server 30 receives requests from multiple autonomous driving vehicles, it can select an autonomous driving vehicle that receives the request with priority based on at least one of the resources possessed by each autonomous driving vehicle, the movement status of each autonomous driving vehicle, the external environment of each autonomous driving vehicle, the time, and the response time to the request.
[0129] As described above, the execution of the second pre-processing performed by the remote autonomous driving server 10a can be started when a request is received from the autonomous driving vehicle. In this way, the remote autonomous driving server 10a can execute the second pre-processing at the timing when the autonomous driving vehicle wishes to execute the advanced second pre-processing.
[0130] (Implementation Method 2)
[0131] In the first embodiment, an example is described in which the remote autonomous driving server 10 performs the judgment on the difference between the first processing result and the second processing result. However, the judgment may be performed by an information processing device mounted on the autonomous driving vehicle. Figure 6 as well as Figure 7 To explain.
[0132] Figure 6 This is a block diagram showing an example of an autonomous driving vehicle (specifically, an information processing device 200 mounted on the autonomous driving vehicle) and a remote autonomous driving server 100 according to Embodiment 2.
[0133] The remote autonomous driving server 100 communicates wirelessly with the autonomous driving vehicle.
[0134] An information processing device 200 is mounted on the autonomous driving vehicle. The information processing device 200 is a computer including a processor, a memory, and a communication interface. The memory is a ROM and a RAM, etc., which can store programs executed by the processor. The information processing device 200 includes: a sensor data acquisition unit 201, a sensor data transmission unit 202, a vehicle autonomous driving system 203, a second processing result acquisition unit 204, a difference judgment unit 205, a processing result change unit 206, a communication confirmation unit 207, and a driving restriction unit 208. The sensor data acquisition unit 201, the sensor data transmission unit 202, the vehicle autonomous driving system 203, the second processing result acquisition unit 204, the difference judgment unit 205, the processing result change unit 206, the communication confirmation unit 207, and the driving restriction unit 208 are implemented by a processor that executes a program stored in the memory.
[0135] The functions of the sensor data acquisition unit 201, the sensor data sending unit 202, the vehicle automatic driving system 203, the communication confirmation unit 207 and the driving restriction unit 208 are basically the same as the functions of the sensor data acquisition unit 21, the sensor data sending unit 22, the vehicle automatic driving system 23, the communication confirmation unit 26 and the driving restriction unit 27 in implementation mode 1, so the description is omitted.
[0136] The second processing result obtaining unit 204 obtains the second processing result from the remote autonomous driving server 100. The second processing result is a result of performing a second preprocessing, which is a preprocessing higher than the first preprocessing, on the sensor data. For example, the second processing result obtaining unit 204 obtains the second processing result, which is sent from the remote autonomous driving server 100 and received via a communication interface or the like provided in the autonomous driving vehicle.
[0137] The difference determination unit 205 determines the difference between the first processing result obtained by the vehicle automatic driving system 203 and the second processing result obtained by the second processing result obtaining unit 204. The difference between the difference determination unit 205 of the second embodiment and the difference determination unit 14 of the first embodiment is whether the information processing device 200 has the difference determination unit or the remote automatic driving server 10 has the difference determination unit, and their functions are basically the same. Therefore, a detailed description of the difference determination unit 205 is omitted.
[0138] The processing result changing unit 206 changes the first processing result to the third processing result according to the determined difference. In the first embodiment, the processing result changing unit 25 changes the first processing result to the third processing result according to the change instruction from the remote automatic driving server 10. In the second embodiment, the processing result changing unit 206 changes the first processing result to the third processing result according to the difference determined by the automatic driving vehicle itself.
[0139] The remote autonomous driving server 100 is a computer including a processor, a memory, a communication interface, etc. The remote autonomous driving server 100 is an example of a device external to the information processing device 200. The memory is a ROM and a RAM, etc., which can store programs executed by the processor. The remote autonomous driving server 100 includes a sensor data acquisition unit 101, a simulated autonomous driving system 102, and a second processing result sending unit 103. The sensor data acquisition unit 101, the simulated autonomous driving system 102, and the second processing result sending unit 103 are implemented by a processor that executes a program stored in the memory. In addition, the components constituting the remote autonomous driving server 100 can be distributed on multiple servers.
[0140] The functions of the sensor data acquisition unit 101 and the simulated automatic driving system 102 are basically the same as the functions of the sensor data acquisition unit 11 and the simulated automatic driving system 12 in the first embodiment, and thus the description thereof will be omitted.
[0141] The second processing result sending unit 103 sends the second processing result, which is the result of the second pre-processing performed by the simulated automatic driving system 102, to the automatic driving vehicle. The second processing result sending unit 103 sends the second processing result to the automatic driving vehicle via a communication interface or the like provided in the remote automatic driving server 100.
[0142] In implementation mode 2, the second processing result is sent from the remote autonomous driving server 100 to the autonomous driving vehicle, and the judgment on the difference between the first processing result and the second processing result is not performed by the remote autonomous driving server 100 but by the autonomous driving vehicle.
[0143] Next, regarding the actions of the self-driving car, Figure 7To explain.
[0144] Figure 7 This is a flowchart showing an example of the operation of the automatic driving vehicle (specifically, the information processing device 200) according to the second embodiment.
[0145] First, the autonomous driving vehicle determines whether the autonomous driving vehicle is connected to the remote autonomous driving server 100 for communication (step S41).
[0146] When the autonomous driving vehicle is not connected to the remote autonomous driving server 100 for communication ("No" in step S41), the driving of the autonomous driving vehicle is restricted (step S42). When the autonomous driving vehicle is not connected to the remote autonomous driving server 100 for communication, the autonomous driving vehicle cannot obtain the second processing result, that is, cannot determine the difference between the first processing result and the second processing result, and cannot change the first processing result to the third processing result. Therefore, when the autonomous driving vehicle is not connected to the remote autonomous driving server 100 for communication, the autonomous driving vehicle may be in a dangerous state, so the driving of the autonomous driving vehicle is restricted. Therefore, even when the communication between the autonomous driving vehicle and the remote autonomous driving server 10 is cut off, the safety of the autonomous driving vehicle can be ensured.
[0147] When the autonomous driving vehicle determines that the autonomous driving vehicle is in communication connection with the remote autonomous driving server 100 ("Yes" in step S41), the autonomous driving vehicle performs a preprocessing of the driving control processing in the autonomous driving of the autonomous driving vehicle, i.e., the first preprocessing, based on the sensor data, and obtains the first processing result (step S43). For example, the autonomous driving vehicle performs a detection process of obstacles around the autonomous driving vehicle (e.g., detection of the number of obstacles or detection of the positions of obstacles, etc.) based on the sensor data, and obtains the result of the detection process. In addition, for example, the autonomous driving vehicle performs an estimation process of the position of the autonomous driving vehicle based on the sensor data, and obtains the result of the estimation process. In addition, for example, the autonomous driving vehicle performs a driving judgment process of the autonomous driving vehicle (e.g., judgment on whether to continue driving or stop, etc.) based on the sensor data, and obtains the result of the driving judgment process.
[0148] Next, the autonomous driving vehicle outputs the sensor data obtained by the autonomous driving vehicle to the remote autonomous driving server 100 (step S44). The remote autonomous driving server 100 that has received the sensor data performs a second preprocessing that is a higher level preprocessing than the first preprocessing based on the sensor data to obtain a second processing result. Furthermore, the remote autonomous driving server 100 sends the obtained second processing result to the autonomous driving vehicle.
[0149] Next, the autonomous driving vehicle determines whether the second processing result transmitted from the remote autonomous driving server 100 is obtained without delay (step S45). For example, if the autonomous driving vehicle obtains the second processing result after a predetermined time or more has passed since the output of the sensor data or the output of the request, it determines that the second processing result is not obtained without delay. If a communication delay occurs between the autonomous driving vehicle and the remote autonomous driving server 100, the second processing result may not be obtained without delay.
[0150] If the autonomous driving vehicle does not obtain the second processing result without delay ("No" in step S45), the difference determination is not performed, and the driving of the autonomous driving vehicle is restricted (step S42). As in the case where there is a processing delay in the second pre-processing described in the first embodiment, the autonomous driving vehicle cannot effectively use the third processing result even if the first processing result is changed to the third processing result, and the autonomous driving vehicle may be in a dangerous state.
[0151] When the autonomous driving vehicle obtains the second processing result without delay ("Yes" in step S45), it determines the difference between the first processing result and the second processing result (step S46), and determines whether the determined difference satisfies the prescribed condition (step S47). The processing of steps S46 and S47 is the same as that of the automatic driving vehicle except that the automatic driving vehicle performs the processing. Figure 2 The processing of step S15 and step S16 described above is the same, so the description is omitted.
[0152] When the determined difference satisfies the prescribed condition ("Yes" in step S47), the autonomous driving vehicle executes a process of changing the first processing result to the third processing result (step S48). Furthermore, the driving control process is executed using the third processing result. In this way, the first processing result is not used for the driving control process of the autonomous driving vehicle, but the third processing result based on the result of the advanced second preprocessing, that is, the second processing result, is used for the driving control of the autonomous driving vehicle, which can improve the performance of the autonomous driving.
[0153] In addition, when the determined difference satisfies the prescribed condition ("Yes" in step S47), the autonomous driving vehicle may restrict the driving of the autonomous driving vehicle. In this case, the autonomous driving vehicle may notify the remote monitor or passenger of the autonomous driving vehicle of the abnormality.
[0154] When the determined difference does not satisfy the prescribed condition ("No" in step S47), the autonomous driving vehicle does not perform the changed processing (step S49). In this case, for example, if the first processing result is not inferior to the second processing result, the autonomous driving vehicle may not change the first processing result to the third processing result, and may use the first processing result that is not inferior to the second processing result to perform driving control of the autonomous driving vehicle.
[0155] Furthermore, the autonomous driving vehicle determines whether it has reached the destination (step S50). If it has not reached the destination ("No" in step S50), the processing of steps S41 to S49 is repeated until it reaches the destination. If it has reached the destination ("Yes" in step S50), the autonomous driving vehicle stops and the processing ends.
[0156] As described above, unlike the first preprocessing performed by the autonomous driving vehicle, which has limited computers that can be mounted from the perspectives of cost, power consumption, and space, the second preprocessing that is more advanced than the first preprocessing is performed by an external device (e.g., the remote autonomous driving server 100) that has fewer restrictions on cost, power consumption, and space, and the result of the second preprocessing, i.e., the second processing result, is sent to the autonomous driving vehicle, and the autonomous driving vehicle determines the difference between the result of the first preprocessing, i.e., the first processing result, and the result of the second preprocessing, i.e., the second processing result. Furthermore, according to the difference, the first processing result is changed to a third processing result based on the second processing result, the first processing result being the result of the first preprocessing of the driving control processing of the autonomous driving vehicle, and the second processing result being the result of the second preprocessing that is the advanced level. Alternatively, according to the difference, the first processing result is changed to a third processing result obtained by correcting or limiting the first processing result. Thus, in the driving control processing of the autonomous driving vehicle, the third processing result that is more advanced than the first processing result is used, so that the performance of the autonomous driving can be improved. For example, by improving the performance of the autonomous driving, the area in which the autonomous driving vehicle can travel can be expanded.
[0157] (Variation of Implementation Example 2)
[0158] For example, the execution of the second pre-processing in the remote autonomous driving server may be started when there is a request from the autonomous driving vehicle. Figure 8 as well as Fig. 9 To explain.
[0159] Figure 8 This is a block diagram showing an example of an autonomous driving vehicle (specifically, an information processing device 200a mounted on the autonomous driving vehicle), a remote autonomous driving server 100a, and a remote processing management server 30 according to a variation of Implementation Example 2.
[0160] The information processing device 200a is different from the information processing device 200 of the second embodiment in that it further includes a remote processing request unit 210, and includes a sensor data acquisition unit 201a and a sensor data transmission unit 202a instead of the sensor data acquisition unit 201 and the sensor data transmission unit 202. The other parts are the same as the information processing device 200, so the description is omitted. In addition, the remote processing request unit 210 is implemented by a processor or the like that executes a program stored in a memory, like other components.
[0161] The remote processing request unit 210 outputs a request to execute the second preprocessing to the remote automatic driving server 100a via the remote processing management server 30. The remote processing management server 30 has the same functions as those of the modified example of the first embodiment, so the description is omitted. For example, the remote processing request unit 210 outputs the request to the remote automatic driving server 100a based on at least one of the resources of the automatic driving vehicle, the moving state of the automatic driving vehicle, the external environment of the automatic driving vehicle, the time, and the response time to the inquiry to the remote automatic driving server 100a. For example, when the resources of the automatic driving vehicle are insufficient, the moving state of the automatic driving vehicle is, for example, a state of high vehicle speed, a state of high acceleration, or a state of large steering angle, the external environment of the automatic driving vehicle is an environment where obstacles are close, the number of obstacles is large, or the type of obstacles is a moving body, the location of the automatic driving vehicle is an environment where there is a lot of traffic (such as an intersection), the surrounding brightness is dark, the weather is rainy or cloudy, the time is at night, and the response time to the request is short, etc., the request is output. For example, when the autonomous driving vehicle has sufficient resources, the moving state of the autonomous driving vehicle is, for example, a slow vehicle speed, a small acceleration, or a small steering angle, the external environment of the autonomous driving vehicle is an environment where obstacles are far away, few in number, or of a type of non-moving objects, the location of the autonomous driving vehicle is an environment with little traffic, the surrounding brightness is bright, the weather is sunny, the time is daytime, the response time to the request is long, etc., the request is not output.
[0162] In this way, depending on the situation, even if the second preprocessing is performed, the performance of the autonomous driving may not be improved. For example, depending on the resources of the autonomous driving vehicle, the movement state of the autonomous driving vehicle, the external environment of the autonomous driving vehicle, the time or the response time to the request, even if the second preprocessing is performed and the driving control of the autonomous driving vehicle is performed using the third processing result, the performance of the autonomous driving may not be improved. In such a case, the autonomous driving vehicle can be set not to output the request. In other words, depending on the situation, the performance of the autonomous driving may be improved by performing the second preprocessing. The autonomous driving vehicle can output the request under such a situation that the performance of the autonomous driving can be improved.
[0163] The remote processing request unit 210 instructs the sensor data acquisition unit 201 a to acquire sensor data, and instructs the sensor data transmission unit 202 a to transmit the sensor data to the remote automatic driving server 100 a selected by the remote processing management unit 31 .
[0164] The sensor data acquisition unit 201a acquires sensor data obtained by sensors provided in the autonomous driving vehicle according to instructions from the remote processing request unit 210. Other parts related to the sensor data acquisition unit 201a are the same as those of the sensor data acquisition unit 201, and thus description thereof is omitted.
[0165] The sensor data transmission unit 202a transmits the sensor data obtained by the sensor data acquisition unit 201a to the selected remote automatic driving server 100a. The other parts related to the sensor data transmission unit 202a are the same as those of the sensor data transmission unit 202, and thus the description thereof is omitted.
[0166] The remote automatic driving server 100a is different from the remote automatic driving server 100 of the second embodiment in that it further includes a request obtaining unit 104 and a second pre-processing execution judgment unit 105, and includes a sensor data obtaining unit 101a and a simulated automatic driving system 102a instead of the sensor data obtaining unit 101 and the simulated automatic driving system 102. The other parts are the same as the remote automatic driving server 100, so the description is omitted. In addition, the request obtaining unit 104 and the second pre-processing execution judgment unit 105 are implemented by a processor or the like that executes a program stored in a memory, like other components.
[0167] The request obtaining unit 104 is basically the same as the request obtaining unit 16 of the modification of the first embodiment, and therefore the description thereof will be omitted.
[0168] The second preprocessing execution determination unit 105 is basically the same as the second preprocessing execution determination unit 17 of the modification of the first embodiment, and thus the description thereof will be omitted.
[0169] The sensor data acquisition unit 101a acquires sensor data when it is determined that the second pre-processing is to be executed upon receiving a request from the automatic driving vehicle. The other parts related to the sensor data acquisition unit 101a are the same as those of the sensor data acquisition unit 101, and thus description thereof is omitted.
[0170] When receiving a request from the automatic driving vehicle, the simulated automatic driving system 102a performs the second pre-processing based on the sensor data obtained by the sensor data obtaining unit 101a when it is determined that the second pre-processing is to be performed, and obtains the second processing result. The other parts related to the simulated automatic driving system 102a are the same as those of the simulated automatic driving system 102, so the description thereof is omitted.
[0171] Next, for the actions of the self-driving car, use Fig. 9 To explain.
[0172] exist Fig. 9 In the flowchart represented by Figure 7 The difference from the flowchart shown in FIG. 1 is that step S51 is added instead of step S41. As for the other parts (in other words, steps S42 to S50), the same Figure 7 The same is shown, so the description is omitted.
[0173] The autonomous driving vehicle outputs a request for executing the second preprocessing to the remote autonomous driving server 100a (step S51). For example, after outputting the request, the autonomous driving vehicle receives a notification from the remote processing management server 30, which indicates to which remote autonomous driving server 100a, as a server capable of executing the second preprocessing, the sensor data should be output. And as a response to the request, the second processing result is obtained in step S45. In addition, after the autonomous driving vehicle outputs the request, if there is no response to the request, it can be determined that there is no communication connection with the remote processing management server 30, etc., and the driving of the autonomous driving vehicle is restricted.
[0174] In addition, the request for execution of the second preprocessing may include information specifying a specific process in the second preprocessing. Depending on the movement state of the autonomous driving vehicle or the external environment, it is sometimes sufficient to make a difference judgment only for a specific process (e.g., only for an obstacle detection process, only for a driving judgment process, etc.). Thus, the autonomous driving vehicle can specify a specific process desired by the autonomous driving vehicle from the second preprocessing, for example, and selectively cause the remote autonomous driving server 100a to execute the specific process. In this case, the autonomous driving vehicle obtains the result obtained by executing the specific process, i.e., the second process result, in steps S45 and S46, and makes a difference judgment only for the specific process.
[0175] In addition, even if the autonomous driving vehicle obtains the second processing result after a predetermined time or more has passed since the output of the sensor data or the output of the request (in other words, even if it is "No" in step S45), it can proceed to step S46 to determine the difference between a part of the processing result in the first processing result and the processing result corresponding to the part of the processing result in the second processing result. For example, the part of the processing result is a part that is not easily affected by the delay. For example, the part of the processing result is the result of the surrounding recognition processing, and the result of the surrounding recognition processing is not easily affected by the delay. For example, the result of the self-position estimation processing is easily affected by the delay, so the difference determination may not be performed for this processing result. Moreover, the autonomous driving vehicle may change the part of the processing result in the first processing result to the third processing result according to the determined difference. In this way, when the second processing result is obtained after a predetermined time or more has passed since the output of the sensor data or the output of the request, there is a possibility that a communication delay occurs with the remote autonomous driving server 100a. In this case, the difference determination is not performed, or the difference determination is performed only for the part of the processing result that is not affected by the delay, thereby reducing the processing load of the autonomous driving vehicle (specifically, the information processing device 200a).
[0176] As described above, the autonomous driving vehicle can output a request to start the second pre-processing etc. at the remote autonomous driving server. Therefore, the second pre-processing can be executed at the timing when the autonomous driving vehicle wishes to execute the advanced second pre-processing.
[0177] (Modifications common to various embodiments)
[0178] In the above-described embodiments, the change instruction or change control of the preprocessing result is performed according to whether there is a delay related to the second preprocessing. However, if there is a delay, it may be performed according to whether correction processing can be performed for the delay.
[0179] In addition, the change instruction or change control of the preprocessing result and the driving restriction instruction or driving restriction may be performed according to the ODD (Operational Design Domain) corresponding to the delay and the difference, in addition to the difference between the first processing result and the second processing result. The ODD is set, for example, based on the time period, region, driving state (speed, acceleration, steering angle, etc.), and environment (weather, illumination, etc.) as elements.
[0180] For the above processing, refer to Fig.10 To explain. Fig.10 1 is a flowchart showing an example of an information processing method of a modified example common to each embodiment. In addition, description of substantially the same processing as that of each of the above-mentioned embodiments is omitted.
[0181] The server (e.g., remote autonomous driving server), when there is a delay in the second pre-processing ("Yes" in step S13), determines whether the delay can be corrected (step S60). Specifically, when there is a communication delay between the server and the autonomous driving vehicle or when there is the above-mentioned processing delay, the server determines whether the time deviation in the processing result due to the delay can be corrected (suppressed, reduced). For example, it is determined whether the delay amount is below a threshold.
[0182] If the correction process can be performed on the delay (Yes in step S60 ), the server executes the correction process (step S61 ). Specifically, if the delay amount is equal to or less than the threshold, the server executes the correction process on the second pre-processing result.
[0183] When there is no delay in the second pre-processing (No in step S13 ), or after the correction process is executed, the process proceeds to steps S15 and S16 .
[0184] If the delay cannot be corrected ("No" in step S60), the server determines whether the first ODD is satisfied (step S62). Specifically, the first ODD is an ODD set for the autonomous driving of the autonomous driving vehicle. For example, the first ODD is that the area is outside the intersection, the speed is less than 20 km / h, and the weather is sunny.
[0185] When the first ODD is satisfied, the server does not output a change instruction (step S18). In this case, the driving control process in the automatic driving is performed using the result of the first pre-processing executed in the automatic driving vehicle.
[0186] When the difference between the first preprocessing and the second preprocessing satisfies the prescribed conditions ("Yes" in step S16), the server determines whether the second ODD is satisfied (step S63). Specifically, the second ODD is an ODD different from the first ODD set for the autonomous driving of the autonomous driving vehicle. For example, the second ODD is the entire area, the speed is below 15 km / h, and the weather is sunny or rainy. In this way, the second ODD is at least partially relaxed compared to the first ODD. On the other hand, communication is used when the second processing result is used, so the items that affect the delay in the second ODD are the same as or stricter than the first ODD.
[0187] If the difference satisfies the prescribed conditions and satisfies the second ODD ("Yes" in step S63), the server outputs a change instruction (step S17). In this case, the driving control process in the automatic driving is performed using the result of the second preprocessing executed on the server or the corrected result of the first preprocessing.
[0188] When the difference between the first pre-processing and the second pre-processing does not satisfy the predetermined condition (No in step S16), the server determines whether the second ODD is satisfied (step S64). This process is substantially the same as the process in step S63.
[0189] When the difference does not satisfy the prescribed condition but satisfies the second ODD ("Yes" in step S64), the server does not output a change instruction (step S18). The reason why the second ODD is used when the difference does not satisfy the prescribed condition is that the difference between the preprocessing result in the server and the preprocessing result in the autonomous driving vehicle is not enough to satisfy the prescribed condition. In other words, the preprocessing result in the autonomous driving vehicle can be used equally with the preprocessing result in the server.
[0190] When the first ODD is not satisfied (No in step S62 ) or when the second ODD is not satisfied (No in step S63 or S64 ), the server outputs a driving restriction instruction (step S14 ).
[0191] In addition, Fig.10 The example in which the server determines whether or not a correction process for the delay can be performed, and determines an example of a change instruction for the preprocessing result and a driving restriction instruction according to the ODD corresponding to the delay and the difference is shown below. However, the autonomous driving vehicle may also determine whether or not a correction process for the delay can be performed, and execute a change control of the preprocessing result and a driving restriction according to the ODD corresponding to the delay and the difference.
[0192] In addition, the change indication or change control of the preprocessing result can also be performed according to whether there is a communication failure related to the second preprocessing. Specifically, the communication failure is the loss of communication data. For example, the change indication or change control of the preprocessing result can be performed according to whether the packet loss rate is above a threshold. In addition, the communication failure can also include the above-mentioned communication delay.
[0193] In addition, in the case of a communication failure, a change instruction or change control of the preprocessing result can be performed according to whether the communication failure can be corrected. Specifically, a change instruction or change control of the preprocessing result can be performed according to whether the data missing due to the loss of communication data can be supplemented. For example, a change instruction or change control of the preprocessing result can be performed according to the degree to which the lost data packets can be supplemented.
[0194] (Other Embodiments)
[0195] The above describes the information processing method, information processing system (such as a remote autonomous driving server) and information processing device involved in one or more forms of the present disclosure based on the embodiments, but the present disclosure is not limited to these embodiments. Without departing from the scope of the present disclosure, forms obtained by implementing various modifications that can be thought of by those skilled in the art in various embodiments and forms formed by combining constituent elements in different embodiments are also included in the scope of one or more forms of the present disclosure.
[0196] For example, the present disclosure implements the steps included in the information processing method as a program for causing a processor to execute. Furthermore, the present disclosure can be implemented as a non-transitory computer-readable recording medium such as a CD-ROM recording the program.
[0197] For example, when the present disclosure is implemented by a program (software), the program is executed by using the hardware resources such as the CPU, memory, and input / output circuit of the computer, thereby executing each step. In other words, the CPU obtains data from the memory or the input / output circuit and performs calculations, or outputs the calculation results to the memory or the input / output circuit, thereby executing each step.
[0198] In addition, in each of the above-mentioned embodiments, each component included in the information processing system and the information processing device may be formed by dedicated hardware, or implemented by executing a software program suitable for each component. Each component may be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.
[0199] Part or all of the functions of the information processing system and information processing device involved in the embodiments are typically implemented by an integrated circuit, i.e., an LSI. These can be monolithic, or monolithic in a manner that includes part or all of them. In addition, integrated circuitization is not limited to LSI, and can be implemented by a dedicated circuit or a general-purpose processor. It is also possible to use a programmable FPGA (Field Programmable Gate Array) after LSI manufacturing, or a reconfigurable processor that can reconfigure the connection and settings of the circuit units inside the LSI.
[0200] Furthermore, various modifications made to the embodiments of the present disclosure within the scope that can be conceived by those skilled in the art without departing from the gist of the present disclosure are also included in the present disclosure.
[0201] Industrial Applicability
[0202] The present disclosure can be applied to a remote control system of an autonomous driving vehicle.
[0203] Explanation of symbols
[0204] 10, 10a, 100, 100a Remote Autopilot Server
[0205] 11, 11a, 21, 21a, 101, 101a, 201, 201a sensor data acquisition unit
[0206] 12, 12a, 102, 102a Simulation of autonomous driving system
[0207] 13. First processing result obtaining unit
[0208] 14,205 Difference Judgment Department
[0209] 15Change instruction output unit
[0210] 16,104 Request for Ministry
[0211] 17, 105 Second pre-processing execution judgment unit
[0212] 20, 20a, 200, 200a information processing device
[0213] 22, 22a, 202, 202a sensor data transmission unit
[0214] 23,203 Vehicle autonomous driving system
[0215] 24, 24a First processing result sending unit
[0216] 25,206 Processing Result Change Department
[0217] 26,207 Communication Confirmation Department
[0218] 27,208 Driving Restriction Department
[0219] 28,210 Remote Processing Request Department
[0220] 30 Remote Processing Management Server
[0221] 31 Remote Processing Management Department
[0222] 103 Second processing result sending unit
[0223] 204 Second processing result acquisition unit
Claims
1. An information processing method, wherein an information processing device mounted on an autonomous mobile body performs the following processing: performing a first preprocessing on the sensor data obtained by the autonomous mobile body and obtaining a first processing result, wherein the first preprocessing is a preprocessing of a driving control process in the automatic driving of the autonomous mobile body, outputting the sensing data to an external device, obtaining, from the external device, a second processing result obtained by performing a second preprocessing, which is a preprocessing higher than the first preprocessing, based on the sensing data, wherein the second preprocessing is performed by the external device connected to the autonomous moving body via a network, and the processing capability of the external device is higher than that of the information processing device, If there is a communication delay between the autonomous mobile body and the external device, or if there is a processing delay in the second pre-processing, restricting the travel of the autonomous mobile body, When there is no communication delay between the autonomous mobile body and the external device, or when there is no processing delay in the second pre-processing, driving control of the autonomous mobile body is performed based on the first processing result or the third processing result. The third processing result is the second processing result, or a processing result obtained by correcting the first processing result.
2. The information processing method according to claim 1, When a change instruction to change the first processing result to the third processing result is received, if reception of the change instruction is delayed due to the communication delay, the received change instruction is ignored and travel of the autonomous mobile body is restricted.
3. The information processing method according to claim 1, When the second processing result is not obtained without delay due to the communication delay, the travel of the autonomous moving body is restricted.
4. The information processing method according to claim 1, In the case where the second processing result is not obtained without delay due to the communication delay, or when there is a processing delay in the second pre-processing, when the delay can be corrected, the driving control of the autonomous mobile body is performed based on the first processing result or the third processing result.
5. The information processing method according to claim 1, In the case where the second processing result is not obtained without delay due to the communication delay, or when there is a processing delay in the second preprocessing, when the delay cannot be corrected, when the specified ODD, i.e., the design operating domain, is met, the driving of the autonomous mobile body is controlled according to the first processing result, and when the specified ODD is not met, the driving of the autonomous mobile body is restricted.
6. The information processing method according to any one of claims 1 to 5, and then, In a case where there is no communication delay between the autonomous moving body and the external device, or in a case where there is no processing delay in the second pre-processing, a difference between the first processing result and the second processing result is determined, The first processing result is changed to the third processing result according to the determined difference, and the driving control of the autonomous mobile body is performed based on the third processing result.
7. The information processing method according to claim 6, When the difference satisfies a predetermined condition, the first processing result is changed to the third processing result, and the driving control of the autonomous mobile body is performed based on the third processing result.
8. The information processing method according to claim 7, When the difference satisfies the specified conditions, when the specified ODD is met, the first processing result is changed to the third processing result, and the driving of the autonomous moving body is controlled based on the third processing result. When the specified ODD is not met, the driving of the autonomous moving body is restricted.
9. The information processing method according to claim 6, When the difference satisfies a predetermined condition, the travel of the autonomous mobile object is restricted.
10. The information processing method according to claim 6, When the difference satisfies a predetermined condition, an abnormality is notified.
11. The information processing method according to claim 6, When the weather is not suitable for the autonomous moving body's automatic driving, the difference is not determined, the first processing result is changed to the third processing result, and the driving control of the autonomous moving body is performed based on the third processing result.
12. The information processing method according to claim 6, When the second processing result is obtained after a predetermined time or longer has passed since the output of the sensing data, a difference between a part of the first processing result and a part of the second processing result is determined.
13. The information processing method according to any one of claims 1 to 5, Furthermore, based on at least one of the resources possessed by the autonomous mobile body, the moving state of the autonomous mobile body, the external environment of the autonomous mobile body, the time, and the response time to the inquiry to the external device, the request for execution of the second preprocessing is output to the external device.
14. An information processing device, mounted on an autonomous mobile body, performs the following processing: performing a first preprocessing based on the sensor data obtained from the autonomous mobile object and obtaining a first processing result, wherein the first preprocessing refers to a preprocessing of a driving control process in the automatic driving of the autonomous mobile object, outputting the sensing data to an external device, obtaining, from the external device, a second processing result obtained by performing a second preprocessing, which is a preprocessing higher than the first preprocessing, based on the sensing data, wherein the second preprocessing is performed by the external device connected to the autonomous moving body via a network, and the processing capability of the external device is higher than that of the information processing device, If there is a communication delay between the autonomous mobile body and the external device, or if there is a processing delay in the second pre-processing, restricting the travel of the autonomous mobile body, When there is no communication delay between the autonomous mobile body and the external device, or when there is no processing delay in the second pre-processing, driving control of the autonomous mobile body is performed based on the first processing result or the third processing result. The third processing result is the second processing result, or a processing result obtained by correcting the first processing result.
15. An information processing method, causing an external device connected to an autonomous mobile body via a network to perform the following processing: obtaining a first processing result as a result of a first preprocessing from the autonomous moving body, and sensor data obtained by the autonomous moving body, wherein the first preprocessing is a preprocessing of a driving control process in an autonomous moving process of the autonomous moving body, and the first preprocessing is performed by an information processing device of the autonomous moving body based on the sensor data, performing a second preprocessing which is a preprocessing higher than the first preprocessing based on the sensing data to obtain a second processing result, wherein the processing capability of the external device is higher than that of the information processing device, outputting, to the autonomous mobile body, a driving restriction instruction for restricting driving of the autonomous mobile body in a case where there is a communication delay between the autonomous mobile body and the external device, or in a case where there is a processing delay in the second pre-processing, When there is no communication delay between the autonomous moving body and the external device, or when there is no processing delay in the second pre-processing, a change instruction for changing the first processing result to the third processing result is output to the autonomous moving body, The third processing result is the second processing result, or a processing result obtained by correcting the first processing result.
16. The information processing method according to claim 15, In the case where there is a communication delay between the autonomous mobile body and the external device, or in the case where there is a processing delay in the second preprocessing, when the delay cannot be corrected, when the specified ODD, i.e., the design operating domain, is satisfied, the change indication is not output to the autonomous mobile body, and when the specified ODD is not satisfied, the driving restriction indication is output to the autonomous mobile body.
17. The information processing method according to claim 15, and then, In a case where there is no communication delay between the autonomous moving body and the external device, or in a case where there is no processing delay in the second pre-processing, a difference between the first processing result and the second processing result is determined, The change instruction is output to the autonomous mobile object according to the determined difference.
18. The information processing method according to claim 17, When the difference satisfies a predetermined condition, the change instruction is output to the autonomous moving object.
19. The information processing method according to claim 18, When the difference satisfies a predetermined condition, the change instruction is output to the autonomous moving object when the predetermined ODD is satisfied, and the driving restriction instruction is output to the autonomous moving object when the predetermined ODD is not satisfied.
20. The information processing method according to any one of claims 15 to 19, In the case where requests for executing the second preprocessing are received from a plurality of the autonomous mobile objects, an autonomous mobile object that preferentially receives the request is selected from among the plurality of autonomous mobile objects based on at least one of the resources possessed by the plurality of autonomous mobile objects, the movement states of the plurality of autonomous mobile objects, the external environments of the plurality of autonomous mobile objects, the time, and the response time to the request.
