Information processing device and program

Abstract

An information processing device is provided with: an acquisition unit that acquires information relating to a vehicle from a detection unit that includes a sensor that detects a state of the surroundings of the vehicle including an obstacle, with a second period that is shorter than a first period in which the surroundings of the vehicle are imaged as a period in which the state of the surroundings of the vehicle is detected; a calculation unit including a setting unit that sets a travel strategy for the vehicle to avoid the obstacle on the basis of the plurality of acquired information, and calculates a control variable for controlling the behavior of the vehicle on the basis of the plurality of acquired information and the set travel strategy; and a control unit that controls the behavior of the vehicle on the basis of the calculated control variable.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a program. Background Art

[0002] JP-A-2022-035198 describes a vehicle having an autonomous driving function. Summary of the Invention

[0003] Problems to be Solved by the Invention

[0004] According to an embodiment of the present invention, there is provided an information processing apparatus. The information processing apparatus includes: an acquisition unit that acquires, as a period for detecting the state of the surroundings of the vehicle, a second period shorter than a first period for photographing the surroundings of the vehicle, a plurality of information related to the vehicle from a detection unit including sensors that detect the state of the surroundings of the vehicle including obstacles; a calculation unit that includes a setting unit that sets a driving strategy for the vehicle to avoid the obstacles based on the acquired plurality of information, and calculates a control variable for controlling the behavior of the vehicle based on the acquired plurality of information and the set driving strategy; and a control unit that controls the behavior of the vehicle based on the calculated control variable.

[0005] According to an embodiment of the present invention, in the information processing apparatus, the calculation unit calculates the control variable based on a multivariate analysis using an integration method based on deep learning according to the index value.

[0006] According to an embodiment of the present invention, in the information processing apparatus, the acquisition unit acquires the plurality of information in units of 1 nanosecond; the calculation unit calculates the control variable using the information acquired in units of 1 nanosecond; and the control unit controls the behavior of the vehicle in units of 1 nanosecond for the control variable.

[0007] According to an embodiment of the present invention, in the information processing apparatus, the setting unit applies a driving mode related to a driving path to a destination for avoiding the obstacle as the driving strategy, and sets a driving mode in which the size of the space between the vehicle and the obstacle exceeds a predetermined value among a plurality of different driving modes.

[0008] According to an embodiment of the present invention, in the information processing apparatus, the calculation unit calculates the control variable so that the vehicle travels according to the set driving mode.

[0009] According to an embodiment of the present invention, in the above information processing device, the above control variables are the speed of the above vehicle and the timing at which the speed of the above vehicle changes.

[0010] According to an embodiment of the present invention, the above information processing device further includes a transmitting unit that, when it is determined based on the above control variables calculated by the above calculating unit that the above vehicle performs an emergency action, transmits the above control variables to other vehicles existing around the above vehicle.

[0011] According to an embodiment of the present invention, the above information processing device further includes a transmitting unit that, when it is determined based on the above control variables calculated by the above calculating unit that the above vehicle performs an emergency action, transmits a signal indicating that the above vehicle performs an emergency action to other vehicles existing around the above vehicle.

[0012] According to an embodiment of the present disclosure, in the above information processing device, the above calculating unit calculates the above control variables based on the above index values through multivariate analysis using an integration method based on deep learning.

[0013] According to an embodiment of the present disclosure, there is provided an information processing device. The above information processing device includes: an acquisition unit that, as a period for detecting the state of the surroundings of the above vehicle, acquires, from a detection unit including sensors, a plurality of pieces of information related to the above vehicle at a second period shorter than a first period for photographing the surroundings of the vehicle, the sensors detecting the state of the surroundings of the above vehicle including obstacles; a calculation unit that includes a setting unit that sets a driving strategy for the above vehicle to avoid the above obstacles based on the plurality of pieces of information acquired above, and calculates control variables for controlling the behavior of the above vehicle based on the plurality of pieces of information acquired above and the set driving strategy; a control unit that controls the behavior of the above vehicle based on the above calculated control variables; a receiving unit that, when the above control unit determines that there is a possibility of collision with the above obstacle and the above obstacle is another vehicle, receives a plurality of pieces of information related to the above other vehicle including the state of the surroundings of the above other vehicle acquired by the sensors of the above other vehicle; and a transmitting unit that calculates, through the above calculating unit, control variables for controlling the behavior of the above other vehicle based on at least the plurality of pieces of information related to the above other vehicle and the plurality of pieces of information related to the above vehicle, and transmits the calculated control variables for controlling the behavior of the above other vehicle to the above other vehicle.

[0014] According to an embodiment of the present disclosure, in the above information processing device, a control variable for controlling the behavior of the other vehicle is calculated based on at least a plurality of information related to the other vehicle, a plurality of information related to the vehicle, and a control variable for controlling the behavior of the vehicle.

[0015] According to an embodiment of the present disclosure, in the above information processing device, the calculation unit calculates the control variable according to the index value through multivariate analysis based on an integration method using deep learning.

[0016] According to an embodiment of the present invention, there is provided an information processing device. The information processing device includes: a registration unit that registers different multiple driving modes, which are driving modes that the vehicle has performed in the past to avoid obstacles; an acquisition unit that acquires a plurality of information related to the vehicle from a detection unit including sensors, and the sensors detect the surrounding conditions of the vehicle including the obstacles; a calculation unit that includes a setting unit, the setting unit uses the acquired plurality of information to selectively set a driving mode for the vehicle to avoid the obstacles from the registered multiple driving modes, and calculates a control variable for controlling the behavior of the vehicle based on the acquired plurality of information and the set driving mode, so that the vehicle travels according to the driving mode; and a control unit that controls the behavior of the vehicle based on the calculated control variable.

[0017] According to an embodiment of the present invention, in the above information processing device, the registration unit registers the driving modes for each vehicle as the control object or for each vehicle model of the vehicle.

[0018] According to an embodiment of the present invention, in the above information processing device, the obstacle is another vehicle other than the vehicle.

[0019] According to an embodiment of the present invention, an information processing apparatus is provided. The information processing apparatus includes: an acquisition unit that acquires a plurality of pieces of information related to a vehicle from a detection unit including sensors that detect the state of the surroundings of the vehicle including obstacles; a calculation unit that includes a setting unit that sets a driving strategy for the vehicle to avoid the obstacles based on the plurality of pieces of information acquired, and calculates a control variable for controlling the behavior of the vehicle based on the plurality of pieces of information acquired and the set driving strategy; and a control unit that controls the behavior of the vehicle based on the calculated control variable; wherein the setting unit applies a driving mode related to a driving path to reach a destination for avoiding the obstacles as the driving strategy, and sets, among a plurality of different driving modes, the driving mode in which the size of the space that changes over time according to the movement of at least one of the vehicle and the obstacle between the vehicle and the obstacle is the largest.

[0020] According to an embodiment of the present invention, in the information processing apparatus, the obstacle is another vehicle other than the vehicle.

[0021] According to an embodiment of the present invention, an information processing apparatus is provided. The information processing apparatus includes: an acquisition unit that acquires a plurality of pieces of information related to a vehicle from a detection unit including sensors that detect the state of the surroundings of the vehicle including obstacles; a calculation unit that includes a setting unit that sets a driving strategy for the vehicle to avoid the obstacles based on the plurality of pieces of information acquired, and calculates a control variable for controlling the behavior of the vehicle based on the plurality of pieces of information acquired and the set driving strategy; and a control unit that controls the behavior of the vehicle based on the calculated control variable; wherein the setting unit applies a driving mode related to a driving path to reach a destination for avoiding the obstacles as the driving strategy, and for a plurality of different driving modes, uses the plurality of pieces of information acquired to set a weighting value corresponding to a collision risk including contact with the obstacle, and selectively sets a driving mode from among the plurality of different driving modes according to the set weighting value.

[0022] According to an embodiment of the present invention, in the information processing apparatus, the weighting value is a value indicating a low degree of the risk or a value indicating a high degree of the risk.

[0023] According to an embodiment of the present invention, in the information processing apparatus, the obstacle is another vehicle other than the vehicle.

[0024] According to an embodiment of the present invention, in the above information processing apparatus, when the vehicle exceeds the driving lane, the setting unit sets a higher risk value as the above weight value compared to the case where it does not exceed the driving lane.

[0025] According to an embodiment of the present invention, in the above information processing apparatus, the larger the size of the space between the vehicle and the obstacle, the lower the risk value set by the setting unit as the above weight value.

[0026] According to an embodiment of the present invention, there is provided a program for causing a computer to function as the above information processing apparatus.

[0027] In addition, the above summary of the invention does not enumerate all the essential features of the present invention. Moreover, sub - combinations of these feature groups can also be the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a diagram schematically showing the danger prediction ability of the AI for ultra - high - performance autonomous driving according to the present embodiment.

[0029] Figure 2 is a diagram schematically showing an example of the network configuration in the vehicle according to the present embodiment.

[0030] Figure 3 is a flowchart executed by the Central Brain according to the present embodiment.

[0031] Figure 4 is a first explanatory diagram for explaining a control example of autonomous driving based on the Central Brain according to the present embodiment.

[0032] Figure 5 is a second explanatory diagram for explaining a control example of autonomous driving based on the Central Brain according to the present embodiment.

[0033] Figure 6 is a third explanatory diagram for explaining a control example of autonomous driving based on the Central Brain according to the present embodiment.

[0034] Figure 7 is a fourth explanatory diagram for explaining a control example of autonomous driving based on the Central Brain according to the present embodiment.

[0035] Figure 8 is a fifth explanatory diagram for explaining a control example of autonomous driving based on the Central Brain according to the present embodiment.

[0036] Figure 9It is a schematic diagram showing the state of other vehicles driving around the vehicle when controlling autonomous driving based on the central brain according to this embodiment.

[0037] Figure 10 It is the first explanatory diagram illustrating an example of setting a driving strategy based on the central brain according to this embodiment.

[0038] Figure 11 It is the second explanatory diagram illustrating an example of setting a driving strategy based on the central brain according to this embodiment.

[0039] Figure 12 It is a block diagram showing an example of the configuration of an information processing device including a central brain according to this embodiment.

[0040] Figure 13 It is a schematic diagram showing the first relationship between the host vehicle and other vehicles related to the driving strategy set by the central brain according to this embodiment.

[0041] Figure 14 It is a schematic diagram showing the first relationship between the host vehicle and other vehicles related to the driving strategy set by the central brain according to this embodiment.

[0042] Figure 15 It is a flowchart executed by the central brain according to this embodiment.

[0043] Figure 16 It is a diagram schematically showing an example of the hardware configuration of a computer that functions as a central brain.

[0044] Figure 17A It is a block diagram showing an example of the functional configuration of the central brain according to the second embodiment.

[0045] Figure 17B It is an example of a flowchart executed by the central brain according to the second embodiment.

[0046] Figure 17C It is an example of a flowchart executed by the central brain according to the second embodiment.

[0047] Figure 18A It is a block diagram showing an example of the functional configuration of the central brain according to the third embodiment.

[0048] Figure 18B It is an example of a flowchart executed by the central brain according to the third embodiment.

[0049] Figure 18C It is an example of a flowchart executed by the central brain according to the third embodiment.

[0050] Figure 19A It is an explanatory diagram showing an example of setting a driving strategy based on the central brain according to the fourth embodiment.

[0051] Figure 19B It is a flowchart executed by the central brain according to the fourth embodiment.

[0052] Figure 19C It is a flowchart executed by the central brain according to the fourth embodiment.

[0053] Figure 20A It is a schematic diagram showing the relationship between the host vehicle and other vehicles related to the driving strategy set by the central brain according to the fifth embodiment.

[0054] Figure 20B It is a flowchart executed by the central brain according to the fifth embodiment.

[0055] Figure 21A It is a schematic diagram showing the relationship between the host vehicle and other vehicles related to the driving strategy set by the central brain according to the sixth embodiment.

[0056] Figure 21B It is a flowchart executed by the central brain according to the sixth embodiment.

[0057] Figure 22A It is a conceptual diagram showing a driving pattern based on a representative proportionality constant.

[0058] Figure 22B It is a conceptual diagram showing a driving pattern based on a combination of multiple proportionality constants. Detailed Embodiments

[0059] Hereinafter, the present invention will be described by way of embodiments of the invention. However, the following embodiments do not limit the invention described in the claims. In addition, the combinations of features described in the embodiments are not all necessary for the solution means of the invention.

[0060] The information processing apparatus of the present disclosure can accurately obtain an index value required for driving control based on a plurality of pieces of information related to vehicle control. Therefore, the information processing apparatus of the present disclosure can be at least partially mounted on a vehicle to achieve control of the vehicle.

[0061] In addition, the information processing device of the present disclosure can achieve autonomous driving in real time based on data obtained from various sensor inputs in Level 6 AI / multivariate analysis / goal seek / strategy formulation / optimal probability solution / optimal speed solution / optimal course management / edge, and can provide a driving system adjusted based on the delta optimal solution.

[0062] "Level 6" represents the level of autonomous driving, which is equivalent to a level higher than Level 5 that represents fully autonomous driving. Although Level 5 represents fully autonomous driving and is at the same level as human driving, there is still a probability of accidents occurring. Level 6 represents a level higher than Level 5, which is equivalent to a level with a lower probability of accidents than Level 5.

[0063] The computing power in Level 6 is about 1000 times that of Level 5. Therefore, it is possible to achieve high-performance driving control that cannot be achieved in Level 5.

[0064] Figure 1 Schematically shows the danger prediction ability of the AI for ultra-high-performance autonomous driving according to the first embodiment. In this embodiment, various sensor information is digitalized by AI and stored in the cloud. The AI predicts and judges the optimal mixture of situations every nanosecond (one billionth of a second) to optimize the operation of the vehicle 12.

[0065] Figure 2 Is a schematic diagram showing an example of the vehicle 12 equipped with the central brain 120. The central brain 120 can be an example of the information processing device 1 according to this embodiment. As Figure 2 Shown, the central brain 120 is communicably connected to a plurality of gateways. The central brain 120 according to this embodiment can achieve Level 6 autonomous driving based on a plurality of information obtained via the gateway. The central brain 120 is an example of an information processing device.

[0066] As Figure 2 Shown, a plurality of gateways are communicably connected to the central brain 120. The central brain 120 is connected to the external cloud via the gateway 130. The central brain 120 is configured to be able to access the external cloud via the gateway 130. On the other hand, due to the existence of the gateway 130, it is configured that the central brain 120 cannot be directly accessed from the outside.

[0067] The central brain 120 outputs a request signal to the server every predetermined time. Specifically, the central brain 120 outputs a request signal indicating an inquiry to the server every one billionth of a second.

[0068] As an example of the sensors included in the vehicle 12 used in this embodiment, there may be mentioned radar, lidar (LiDAR), high-pixel / long-focus / ultra-wide-angle / 360-degree / high-performance cameras, visual recognition, faint sounds, ultrasonic waves, vibrations, infrared rays, ultraviolet rays, electromagnetic waves, temperature, humidity, spot AI weather forecasting, high-precision multi-channel Global Positioning System (GPS), low-altitude satellite information, long-tail event AI data, etc. Long-tail event AI data refers to the trip data of a Level 5 vehicle.

[0069] The above sensors include sensors that detect the conditions around the vehicle. The sensors that detect the conditions around the vehicle detect the conditions around the vehicle at a second period shorter than the first period for photographing the surroundings of the vehicle using a camera or the like as the period for detecting the conditions around the vehicle.

[0070] As the sensor information obtained from various sensors, there may be mentioned the movement of the center of gravity of the weight, the detection of the road material, the detection of the external air temperature, the detection of the external air humidity, the detection of the inclination angles in the up / down direction, lateral direction, and diagonal direction of the slope, the freezing mode of the road, the detection of the moisture content, the material, wear condition, and air pressure of each tire, the road width, the presence or absence of overtaking prohibition, oncoming vehicles, the vehicle type information of the vehicles in front and behind, the cruising states of these vehicles, the surrounding conditions (birds, animals, footballs, accident vehicles, earthquakes, fires, winds, typhoons, heavy rains, light rains, snowstorms, fogs, etc.). In this embodiment, these detections are performed every one billionth of a second.

[0071] The central brain 120, which functions as an example of the information processing device according to this embodiment, at least includes the respective functions of an acquisition unit capable of acquiring a plurality of pieces of information related to the vehicle, a calculation unit that calculates a control variable based on the plurality of pieces of information acquired by the acquisition unit, and a control unit that performs driving control of the vehicle based on the control variable.

[0072] For example, the central brain 120 functions by calculating a control variable using one or more pieces of sensor information detected by the above sensors. The control variables include control variables for controlling the wheel speed and inclination of each of the four wheels of the vehicle, and control variables for controlling the wheel speed, inclination, and suspension of each suspension supporting the wheels. It should be noted that the inclination of the wheel includes the inclination of the wheel with respect to the axis horizontal to the road and the inclination of the wheel with respect to the axis perpendicular to the road.

[0073] Here, one or more sensor information may apply sensor information from sensors that detect the conditions around the vehicle. In addition, when using multiple sensor information as one or more sensor information, a predetermined number of sensor information may be applied. The predetermined number is, for example, three. Based on the three sensor information, index values for controlling wheel speed, tilt, and suspension are calculated. The number of index values calculated according to the combination of the three sensor information is, for example, three. Among the index values for controlling wheel speed, tilt, and suspension, there are included: for example, an index value calculated based on information related to air resistance in the sensor information, an index value calculated based on information related to road resistance in the sensor information, and an index value calculated based on information related to slip coefficient in the sensor information, etc.

[0074] In addition, the index values calculated for each of the combinations of multiple sensor information that differ according to one sensor information or a combination of sensor information are aggregated, and a control variable for controlling wheel speed, tilt, and suspension is calculated. For example, index values are calculated based on the sensor information of sensors that detect the conditions around the vehicle, and a control variable is calculated. In addition, when using multiple sensor information, for example, multiple index values are calculated using the combination of sensors 1, 2, and 3, multiple index values are calculated using the combination of sensors 4, 5, and 6, and multiple index values are calculated using the combination of sensors 1, 3, and 7, and these index values are aggregated to calculate the control variable. In this way, while changing the combination of sensor information, a predetermined number, for example, 300 index values, are calculated, and the control variable is calculated. Specifically, the calculation unit may use machine learning, and more specifically, deep learning, to calculate the control variable based on the sensor information. In other words, the calculation unit that calculates the index value and the control variable may be constituted by artificial intelligence (AI, Artificial Intelligence).

[0075] The calculation unit uses the computing power of level 6 to perform multivariate analysis (see, for example, Equation (2)) based on the integration method shown in the following Equation (1) on the data collected per nanosecond by multiple sensor groups, etc., thereby enabling accurate control variables to be obtained. More specifically, by using the computing power of level 6 to obtain the integral value of various ultra-high-resolution increment values, and at the same time obtaining the indexed values of each variable at the edge level and in real time, the highest probability value of the result that appears in the next nanosecond can be obtained.

[0076] [Expression 1]

[0077]

[0078] [Expression 2]

[0079] V n= DL(f(A, B, C, ..., N)(dA n / dt)) (2)

[0080] In addition, DL in the formula represents deep learning, and A, B, C, D, ..., N are index values calculated based on sensor information. A, B, C, D, ..., N represent, for example, index values calculated based on air resistance, index values calculated based on road resistance, index values calculated based on road elements, and index values calculated based on slip coefficients, etc. When the number of index values calculated while changing the combination of a predetermined number of sensor information is 300, the index values of A to N in the formula are also 300, and the 300 index values are summarized. V n is a control variable.

[0081] Furthermore, in the above formula (2), the wheel speed (V) is calculated, but the control variables for controlling the tilt and suspension are also calculated in the same way.

[0082] Specifically, the central brain 120 calculates a total of 16 control variables, and the total 16 control variables are used to control the wheel speeds of the four wheels respectively, the tilts of the four wheels respectively with respect to the axis horizontal to the road, the tilts of the four wheels respectively with respect to the axis perpendicular to the road, and the suspensions respectively supporting the four wheels. In this embodiment, the calculation of the above 16 control variables is performed every one billionth of a second. It should be noted that the wheel speeds of the four wheels respectively can also be referred to as "the rotation speeds (rotational speeds) of the in-wheel motors mounted on the four wheels respectively", and the tilts of the four wheels respectively with respect to the axis horizontal to the road can also be referred to as "the horizontal angles of the four wheels respectively". And, for example, when the vehicle is driving on a mountain road, the above control variables are the values for performing the optimal steering matching the mountain road, and when the vehicle is parked in a parking lot, the above control variables are the values for driving at the optimal angle matching the parking lot.

[0083] In addition, in this embodiment, the central brain 120 calculates a total of 16 control variables, and the total 16 control variables are used to control the wheel speeds of the four wheels respectively, the tilts of the four wheels respectively with respect to the axis horizontal to the road, the tilts of the four wheels respectively with respect to the axis perpendicular to the road, and the suspensions respectively supporting the four wheels, but this calculation does not have to be performed by the central brain 120, and a dedicated AnchorChip for calculating the above control variables can also be set separately. In this case, DL in formula (2) also represents deep learning, and A, B, C, D, ..., N represent index values also calculated based on sensor information. When the total number of summarized index values is 300 as described above, the number of index values in this formula is also 300.

[0084] In addition, in the present embodiment, the central brain 120 functions as a control unit that controls autonomous driving in units of one billionth of a second based on the calculated control variables described above. Specifically, the central brain 120 controls the in-wheel motors respectively mounted on the four wheels based on the above 16 control variables, thereby controlling the wheel speed, tilt of each of the four wheels of the vehicle 12, and the suspension that supports the four wheels respectively, so as to perform autonomous driving.

[0085] In addition, the index value calculated based on the sensor information can be calculated according to one sensor, for example, when the sensor for detecting the surrounding conditions of the vehicle detects the surrounding conditions of the vehicle, the sensor information of the sensor is used to calculate the index value, and the control variable is calculated.

[0086] The above calculation unit includes a setting unit that sets a driving strategy related to the driving of the vehicle according to the acquired multiple pieces of information. In an example of the driving strategy related to the driving of the vehicle, when there is an obstacle in the driving path of the vehicle, a driving strategy such as a driving mode for the vehicle to avoid the obstacle can be applied. In this driving strategy, information including a driving mode related to the driving path until reaching the destination of avoiding the obstacle can be applied.

[0087] The setting unit can set the position that should be reached after a predetermined time elapses from the current position during vehicle driving as the destination, and set the information related to the driving from this current position to reaching the destination as the driving strategy. In addition, the position predetermined from the current position can also be set as the destination, and the information related to the driving from this current position to reaching the destination can be set as the driving strategy.

[0088] In addition, the setting unit can also set the information related to the driving from the current location to the destination as the driving strategy based on the information of the destination input by the vehicle occupants, etc., and the traffic information between the current location and the destination. At this time, the information at the moment when the calculation strategy is set, that is, the data currently acquired by the information acquisition unit, can also be considered. This is because not only a simple route calculation to the destination is considered, but also the surrounding conditions at that moment are taken into account to calculate a more realistic theoretical value. The driving strategy can be configured to include at least one theoretical value of the optimal route (strategy route) to the destination, driving speed, tilt, and braking. The driving strategy is preferably composed of all the theoretical values of the above optimal route, driving speed, tilt, and braking.

[0089] The multiple theoretical values that make up the driving strategy set by the setting unit can be used for the autonomous driving control of the control unit. In addition to this, the control unit can also include an update unit that updates the driving strategy based on the difference between the multiple index values calculated by the calculation unit and each theoretical value set by the setting unit.

[0090] The index value calculated by the calculation unit is information obtained during vehicle driving. Specifically, it is detected during actual driving. For example, it is deduced based on the friction coefficient. Therefore, by updating the driving strategy by the update unit, it is possible to cope with the changes that occur at any time when passing through the strategic route. Specifically, in the update unit, by calculating the difference (increment value) based on the theoretical value and the index value included in the driving strategy, the optimal solution can be derived again, and the strategic route can be re-formulated. Thereby, it is possible to achieve autonomous driving control with minimal slip. In addition, during such update processing, since the computing power of the above-mentioned level 6 can be used, correction and fine-tuning can be performed in units of one billionth of a second, enabling more precise driving control.

[0091] In addition, when the acquisition unit has the above-mentioned vehicle lower sensor, since this vehicle lower sensor also detects the temperature and material of the ground, etc., it is possible to cope with the changes that occur at any time when passing through the strategic route. When calculating the driving route included in the driving strategy, independent smart tilt can also be implemented. Furthermore, even when other information (flying tires, debris, animals, etc.) is detected, by coping with the changes that occur at any time when passing through the strategic route, it is possible to instantly recalculate the optimal driving route and implement optimal route management.

[0092] The central brain 120 repeatedly executes Figure 3 the flowchart shown.

[0093] In step S10, the central brain 120 acquires sensor information including the road information detected by the sensor. Then, the central brain 120 proceeds to step S11. The processing of step S10 is an example of the function of the acquisition unit.

[0094] In step S11, the central brain 120 calculates the above-mentioned 16 control variables based on the sensor information acquired in step S10. Then, the central brain 120 proceeds to step S12. The processing of step S11 is an example of the functions of the setting unit and the calculation unit.

[0095] In step S12, the central brain 120 controls autonomous driving based on the control variables calculated in step S11. Then, the central brain 120 ends the processing of this flowchart. The processing of step S12 is an example of the function of the control unit.

[0096] Figures 4 to 8 It is an explanatory diagram for explaining an example of the control of autonomous driving by the central brain 120. It should be noted that Figures 4 to 6 it is an explanatory diagram from the perspective of observing the vehicle 12 from the front, Figure 7 and Figure 8It is an explanatory diagram of the perspective of observing the vehicle 12 from below.

[0097] Figure 4 It shows the vehicle 12 traveling on a flat road R1. The central brain 120 controls the in-wheel motors 31 respectively mounted on the four wheels 30 based on the above 16 control variables calculated according to the road R1. Thereby, it controls the rotation speed and inclination of each of the four wheels 30, and the suspensions 32 supporting each of the four wheels 30, so as to perform autonomous driving.

[0098] Figure 5 It shows the vehicle 12 traveling on a mountain road R2. The central brain 120 controls the in-wheel motors 31 respectively mounted on the four wheels 30 based on the above 16 control variables calculated according to the mountain road R2. Thereby, it controls the rotation speed and inclination of each of the four wheels 30, and the suspensions 32 supporting each of the four wheels 30, so as to perform autonomous driving.

[0099] Figure 6 It shows the vehicle 12 traveling in a puddle R3. The central brain 120 controls the in-wheel motors 31 respectively mounted on the four wheels 30 based on the above 16 control variables calculated according to the puddle R3. Thereby, it controls the rotation speed and inclination of each of the four wheels 30, and the suspensions 32 supporting each of the four wheels 30, so as to perform autonomous driving.

[0100] Figure 7 It shows the vehicle 12 turning in the direction indicated by the arrow A1. The central brain 120 controls the in-wheel motors 31 respectively mounted on the four wheels 30 based on the above 16 control variables calculated according to the incoming curve. Thereby, it controls the rotation speed, inclination of each of the four wheels 30, and the suspensions 32 (not shown) respectively supporting the four wheels 30, so as to perform autonomous driving.

[0101] Figure 8 It shows the vehicle 12 moving parallel in the direction indicated by the arrow A2. The central brain 120 controls the in-wheel motors 31 respectively mounted on the four wheels 30 based on the above 16 control variables calculated according to the parallel movement in the direction indicated by the arrow A2. Thereby, it controls the rotation speed, inclination of each of the four wheels 30, and the suspensions 32 (not shown) respectively supporting the four wheels 30, so as to perform autonomous driving.

[0102] It should be noted that Figures 4 to 8 The states (inclinations) of the wheels 30 and suspensions 32 shown are only examples. Of course, there will be states of the wheels 30 and suspensions 32 different from those shown in each figure.

[0103] Among them, although the existing in-wheel motors mounted on vehicles can independently control their respective drive wheels, in this vehicle, it is impossible to control the in-wheel motors by analyzing road conditions or the like. Therefore, in this vehicle, for example, when driving on mountain roads or in waterlogged sections, etc., appropriate autonomous driving based on road conditions or the like cannot be performed.

[0104] However, according to the vehicle 12 according to the present embodiment, based on the structure described above, autonomous driving that can control speed, steering, etc. according to the environment such as road conditions can be performed.

[0105] As described above, based on the control variables, the behavior of the vehicle 12 can be controlled to perform autonomous driving.

[0106] However, when the vehicle is driving, there are cases where obstacles approach the vehicle. In such cases, it is preferable for the vehicle to change its behavior and drive to avoid contact or collision with the obstacles. As an example of an obstacle, other vehicles, wall surfaces, guardrails, curbs, and other installations other than the vehicle itself during driving can be cited. In the following description, as an example of an obstacle, the case where another vehicle approaching the vehicle 12 is applied will be described. In addition, an obstacle is an example of an object.

[0107] Figure 9 FIG. schematically shows a state in which the vehicle 12 traveling on a two-lane two-way road is taken as the vehicle 12A, and other vehicles 12B, 12C, and 12D are traveling around the vehicle 12A. In the example of the figure, it is a state in which the other vehicle 12D is traveling behind the vehicle 12A, the other vehicle 12D is traveling ahead in the oncoming lane, and the other vehicle 12C is traveling behind the other vehicle 12D.

[0108] The central brain 120 of the vehicle 12A controls the in-wheel motors 31 respectively mounted on the four wheels 30 based on the above-described control variables calculated according to the state of traveling on the driving road that changes moment by moment. Thereby, the wheel speeds and tilts of the four wheels 30 respectively, and the suspensions 32 supporting the four wheels 30 respectively are controlled, so as to perform autonomous driving. In addition, the central brain 120 detects the behaviors of other vehicles 12B, 12C, and 12D around the vehicle 12A through sensors and obtains sensor information.

[0109] As Figure 9 shown, when the other vehicle 12D enters the driving path of the vehicle 12A, the central brain 120 drives in order to at least avoid the obstacle, and the above setting unit sets a driving strategy from the current position to the destination according to the situation around the vehicle 12A obtained from the sensor information. In Figure 9In the example shown, a driving mode 12Ax1 in which the vehicle 12A is traveling in the current driving state will collide with another vehicle 12D that enters. Therefore, the central brain 120 sets, for example, a driving mode 12Ax2 that avoids a collision with the other vehicle 12D.

[0110] In the driving mode, a driving mode that can avoid a collision with the other vehicle 12D is selected and set from among a plurality of different driving modes determined in advance. Specifically, it is sufficient to select a driving mode in which there is no other vehicle 12D in the driving mode. The plurality of driving modes may be stored in the memory in advance. The driving mode may apply a path that records the vehicle trajectory.

[0111] Figure 10 It is a schematic diagram of an example of a plurality of different driving modes. Figure 10 The example shown shows a driving mode PL with one turn such as an L-shaped curve, a driving mode PM with three turns such as an M-shaped curve, and a driving mode PS with four turns such as an S-shaped curve. Ten variations correspond to each mode, and it is sufficient to select one driving mode that can avoid a collision with the other vehicle 12D from a total of 30.

[0112] In addition, in order to drive while at least avoiding an obstacle, the central brain 120 only needs to have a function of predicting the position of a collision including contact between the obstacle and the vehicle based on the acquired sensor information. Therefore, the central brain 120 sets a driving mode that can avoid other vehicles at the predicted position. Then, a control variable corresponding to the driving mode is calculated. The control variable can calculate a control variable that can be controlled to a driving state according to a driving mode that can avoid other vehicles by changing at least one of acceleration / deceleration and steering angle in time series corresponding to the current speed of the vehicle 12A.

[0113] For example, as Figure 11 shown, even for the same driving mode, the shape of the driving mode can be changed by changing the gain (amplification degree) of a part of the driving mode. Therefore, the control variable includes calculating a magnification factor for the control variable that matches the calculated driving mode. In addition, the control variable may apply at least one of the steering angle of the vehicle and the speed of the vehicle.

[0114] The central brain 120 that can implement the above-mentioned autonomous driving is further described. The central brain 120 is configured as Figure 12 the information processing device 10 shown. In addition, the above-mentioned central brain 120 is a general processing device that functions as an information processing device including a gateway, and the central brain 125 described later is a narrow processing device when classifying functions by each processor.

[0115] Figure 12FIG. 0 is a block diagram showing an example of the configuration of an information processing apparatus 10 including a central brain according to an embodiment. The information processing apparatus 10 includes an image processing unit (IPU) 121, a motion processing unit (MoPU) 122, a central brain 125, and a memory 126. The central brain 125 is configured to include a graphics neural network processing unit (GNPU) 123 and a central processing unit (CPU) 124.

[0116] The IPU 121 may be built into a super high-resolution camera (not shown) provided on a vehicle. The IPU 121 performs predetermined image processing such as Bayer conversion, demosaicing, denoising, and sharpening on an image of an object existing around the vehicle, and outputs the processed image of the object at a frame rate of, for example, 10 frames per second and a resolution of 12 million pixels. The image output from the IPU 121 is provided to the central brain 125 and the memory 126.

[0117] The MoPU 122 may be built into a low-resolution camera different from the super high-resolution camera provided on the vehicle. The MoPU 122 outputs motion information indicating the movement of the photographed object at a frame rate of, for example, 1920 frames per second. That is, the frame rate of the output of the MoPU 122 is 100 times the frame rate of the output of the IPU 121. The MoPU 122 outputs vector information indicating the movement of a point representing the existence position of the object along a predetermined coordinate axis as the motion information. That is, the motion information output from the MoPU 122 does not include information required to identify what the photographed object is (for example, a person or an obstacle), but only includes information indicating the movement (movement direction and movement speed) of the center point (or center of gravity point) of the object on the coordinate axes (x-axis, y-axis, z-axis). The image output from the MoPU 122 is provided to the central brain 125 and the memory 126. By not including image information in the motion information, the amount of information transmitted to the central brain 125 and the memory 126 can be suppressed.

[0118] The present disclosure includes a first processor that outputs an image of a captured object at a first frame rate, and a second processor that outputs movement information indicating the movement of the captured object at a second frame rate higher than the first frame rate. That is, as a cycle for detecting the situation around the vehicle of the present disclosure, a detection unit that captures the situation around the vehicle at a first cycle is an example of the above "first processor", and the IPU 121 is an example of the "first processor". In addition, a detection unit including a sensor that detects the situation around the vehicle at a second cycle shorter than the first cycle of the present disclosure is an example of the above "second processor", and the MoPu 122 is an example of the "second processor".

[0119] The central brain 125 performs driving control of the vehicle based on the image output from the IPU 121 and the movement information output from the MoPU 122. For example, the central brain 125 identifies objects (people, animals, roads, traffic lights, signs, crosswalks, obstacles, buildings, etc.) existing around the vehicle based on the image output from the IPU 121. In addition, the central brain 125 identifies the movement of the objects where something has been identified around the vehicle based on the movement information output from the MoPU 122. The central brain 125 performs, for example, control (speed control) of the motor for driving the wheels, braking control, and steering wheel control based on the identified information. In the central brain 125, the GNPU 123 may be responsible for processing related to image recognition, and the CPU 124 may be responsible for processing related to vehicle control.

[0120] Generally, an ultra-high-resolution camera is used for image recognition in autonomous driving. It is possible to identify what objects are included in the image captured by the high-resolution camera. However, in the autonomous driving in the level 6 era, it is not enough to only reach this level. In the level 6 era, it is also necessary to identify the movement of the object. By identifying the movement of the object, for example, it is possible to perform an avoidance operation for a vehicle traveling through autonomous driving to avoid obstacles with higher accuracy. However, in a high-resolution camera, only about 10 frames of images can be acquired per second, and it is difficult to analyze the movement of the object. On the other hand, in a camera equipped with the MoPu 122, although the resolution is low, it can output at a high frame rate of, for example, 1920 frames per second.

[0121] Therefore, in the technology of the present disclosure, two independent processors, namely the IPU 121 and the MoPU 122, are used. The high-resolution camera (IPU 121) has the function of acquiring the image information required to identify what the captured object is, and the MoPU 122 has the function of detecting the movement of the object. The MoPU 122 takes the object as a point and analyzes in which direction and at what speed the coordinates of this point move on the x-axis, y-axis, and z-axis. Since the overall contour of the object and the detection of what the object is can be carried out through the image from the high-resolution camera, through the MoPU 122, as long as it is known how the center point of the object moves, it is known what behavior the entire object will have.

[0122] According to the method of only analyzing the movement and speed of the center point of the object, compared with judging how the entire image of the object moves, the amount of information transmitted to the central brain 125 can be greatly suppressed, thereby greatly reducing the amount of calculation in the central brain 125. For example, when an image of 1000 pixels × 1000 pixels is sent to the central brain 15 at a frame rate of 1920 frames per second, if color information is included, 4 billion bits per second of data will be sent to the central brain 125. The MoPU 122 can compress the amount of data transmitted to the central brain 125 to 20,000 bits per second by only sending the movement information indicating the movement of the center point of the object. That is, the amount of data transmitted to the central brain 125 is compressed to one two-hundred-thousandth.

[0123] In this way, by combining the use of the low frame rate and high-resolution image output from the IPU 121 and the high frame rate and lightweight movement information output from the MoPU 122, object recognition including object movement can be achieved with a smaller amount of data.

[0124] In addition, in the case of using one MoPU 122, vector information indicating the movement of the point representing the existence position of the object along each of the two coordinate axes (x-axis and y-axis) in the three-dimensional rectangular coordinate system can be obtained. The principle of a stereo camera can also be used, and two MoPU 122s can be used to output vector information indicating the movement of the point representing the existence position of the object along each of the three coordinate axes (x-axis, y-axis, and z-axis) in the three-dimensional rectangular coordinate system. The z-axis is the axis along the depth direction (vehicle driving).

[0125] In the present embodiment, as sensor information, the surrounding conditions of the vehicle 12 are detected at a second period shorter than a first period of photographing the surroundings of the vehicle using a camera or the like. That is, in the above-mentioned Level 5, for example, the surrounding conditions can be detected twice by a camera or the like in 0.3 seconds, but in the present embodiment, in Level 6, for example, the surrounding conditions can be detected 576 times. Then, an index value and a control variable can be calculated for each of the 576 detections of the surrounding conditions, and driving that is faster and safer than the autonomous driving performed by Level 5 can be executed, thereby enabling the behavior of the vehicle to be controlled.

[0126] In addition, in order for the present vehicle 12A to drive while at least avoiding other vehicles 12D, there are sometimes multiple driving modes. Therefore, in the present embodiment, the central brain 120 can determine the optimal driving mode from multiple driving modes and set it as the driving strategy.

[0127] Figure 13 It schematically shows Figure 9 a diagram of the driving modes applicable to the present vehicle 12A in the driving state showing the relationship between the present vehicle 12A and other vehicles 12B, 12C, 12D shown.

[0128] In Figure 13 the example shown, when adopting the driving mode 12Ax1 as the path of the present vehicle 12A, the present vehicle 12A will collide with other vehicles 12D. On the other hand, when adopting the driving modes 12Ax2 and 12Ax3, the present vehicle 12A can avoid collisions including contact with other vehicles 12D. Therefore, the central brain 120 calculates the driving modes 12Ax2 and 12Ax3 respectively and sets any one of them as the driving mode. The setting of this driving mode sets the driving mode with the minimum risk relationship for the mutual relationship between the present vehicle 12A and other vehicles 12D. Specifically, as Figure 14 shown, the driving mode 12Ax2 with the largest distance between the present vehicle 12A and other vehicles 12D is set. That is, the driving mode 12Ax2 in which the size of the space between the vehicle and other vehicles as obstacles exceeds a predetermined value (for example, the maximum value) determined in advance among multiple different driving modes is set.

[0129] Figure 15 It is a flowchart showing an example of the processing flow in the central brain 120 in which the present vehicle 12A can drive while avoiding other vehicles 12D. In Figure 15 it, the processing of step S11 shown in Figure 3 is replaced with steps S11A, 11B, and 11C and executed. The central brain 120 can repeatedly execute the processing shown in Figure 15 in place of the processing shown in Figure 3 shown.

[0130] In step S11A, the central brain 120 derives a driving strategy for the vehicle 12 based on sensor information. Then, in step S11B, the optimal driving strategy (driving mode) is set from the derived driving strategies. Then, in step S11C, a control variable corresponding to the driving strategy (driving mode) set to be able to drive while avoiding obstacles (e.g., other vehicle 12D) is calculated. The processes of steps S11A to S11C are an example of the function of the calculation unit, and the processes of steps S11A and S11B are an example of the function of the setting unit.

[0131] The above driving strategy (driving mode) can also change the driving mode according to the situation around the vehicle that changes all the time. For example, the driving strategy (driving mode) can include timings that affect the control variable and change vehicle behaviors such as the vehicle speed. In addition, the control variable can also include the above timings.

[0132] In addition, for the control variable related to the speed of the vehicle 12, 10 modes each of large, medium, and small (L, M, S) can be set for the acceleration side and a total of 30 modes can be applied, and 10 modes each of large, medium, and small (L, M, S) can be set for the deceleration side and a total of 30 modes can be applied for selection. In this case, the relationship between the own vehicle and other vehicles changes all the time. In the state where the own vehicle approaches other vehicles, the distance changes in the approaching direction. Therefore, the options for selecting a mode become smaller all the time, the processing time for selection can be reduced, and further, the adjustment of the incremental difference can be reduced.

[0133] In addition, the above describes the case where the other vehicle 12D is an other vehicle relative to the own vehicle 12A, but it is also possible to calculate a control variable that reduces the collision risk with respect to at least one of the other vehicles 12B, 12C, 12D around the own vehicle 12A, and further all other vehicles.

[0134] Therefore, the central brain 120 controls, for example, the in-wheel motors 31 respectively mounted on the four wheels 30 based on the control variable calculated according to the prediction of a collision including contact. Thus, by controlling the wheel speed and inclination of each of the four wheels 30 and the suspension 32 that supports each of the four wheels 30, it is possible to avoid a collision or reduce the damage to the vehicle during a collision and perform autonomous driving.

[0135] [Second Embodiment]

[0136] [Notification (1)]

[0137] Figure 17AIt is a block diagram showing the functional configuration of the central brain 2120. For example, the central brain 2120A is the central brain of the vehicle 12A, and the English letter at the end of the reference numeral indicates the corresponding vehicle. Hereinafter, when there is no need to distinguish, the English letter at the end is omitted. In addition, the English letter at the end is used to distinguish the vehicle and the central brain in the description, but it is not specific in implementation. For example, the vehicle 12A can be any one of the vehicles 12B to 12N.

[0138] As used above Figure 1 and Figure 2 described, the central brains 2120A to 2120N can communicate with each other via the cloud to send and receive data. As an example, as Figure 17A shown, the central brain 2120 of the vehicle 12 functions as an acquisition unit 2200, a calculation unit 2210, a control unit 2220, a transmission unit 2230, and a reception unit 2240. The acquisition unit 2200 of the vehicle 12 acquires sensor information including road information and the cruising state of surrounding vehicles from sensors mounted on the vehicle 12.

[0139] The calculation unit 2210 uses the sensor information of the vehicle 12 to calculate a total of 16 control variables for the wheel speeds, tilts, and suspensions supporting the wheels of the four wheels of the vehicle 12. The control unit 2220 controls the autonomous driving of the vehicle 12 based on the control variables calculated by the calculation unit 2210.

[0140] When the control unit 2220 determines that the vehicle 12 performs an emergency behavior to avoid an obstacle (performs an emergency avoidance), the control unit 2220 sends the control variables of the vehicle 12 to other vehicles 12 via the transmission unit 2230. The emergency behavior may refer to, for example, a behavior in which the vehicle deviates from the current traveling direction by a predetermined distance or more within a predetermined time. In addition, the emergency behavior may also refer to a behavior in which the vehicle performs acceleration or deceleration more than a predetermined amount within a predetermined time.

[0141] Other vehicles 12 receive the sent control variables via the reception unit 2240. In addition to the sensor information acquired by the acquisition unit 2200, the calculation unit 2210 of other vehicles 12 also uses the control variables of the vehicle 12 to calculate the control variables of other vehicles 12. The central brain 2120A of the vehicle 12A repeatedly executes Figure 17B the flowchart shown. Except for steps S2120 and S2150, Figure 17B the processing of the flowchart can also be substantially the same as Figure 3 the processing of the flowchart.

[0142] In step S2010, the central brain 2120A acquires the sensor information detected by the sensor. In step S2011, the central brain 2120A calculates a total of 16 control variables based on the sensor information acquired in step S2010.

[0143] In step S2120, the central brain 2120A determines whether the vehicle 12A performs an emergency behavior (performs an emergency avoidance) to avoid an obstacle based on the sensor information and / or the calculated control variable. When the determination in step S2120 is negative, the central brain 2120A proceeds to step S2012.

[0144] When the determination in step S2120 is positive, in step S2150, the central brain 2120A sends the calculated control variable to the central brain 2120D of another vehicle 12D. In step S2012, the central brain 2120 controls the autonomous driving based on the control variable calculated in step S2011 and ends the processing of this flowchart. In addition to the sensor information acquired by the vehicle 12D, the central brain 2120D of the vehicle 12D also uses the control variable of the vehicle 12A to calculate the control variable of the vehicle 12D.

[0145] As described above, when the vehicle 12 performs an emergency behavior (performs an emergency avoidance) to avoid an obstacle, by sending the control variable of the vehicle 12 to other vehicles 12, it notifies other vehicles 12 in advance of the emergency behavior of the vehicle 12 that is difficult for other vehicles 12 to predict. Thereby, other vehicles 12 can sense the occurrence of the emergency behavior of the vehicle 12 before it is detected by sensors or the like after the behavior occurs, and thus can respond more quickly to the occurrence of the emergency behavior of the vehicle 12. In addition, by using the control variable of the vehicle 12 for the calculation of the control variable of other vehicles 12, an appropriate control variable of other vehicles 12 can be calculated by preventing accidents.

[0146] In addition, the above description is an example. Instead of sending the control variable of the vehicle 12 to other vehicles 12, the vehicle 12 may send an emergency avoidance signal indicating that the vehicle 12 performs an emergency behavior (performs an emergency avoidance) to avoid an obstacle to other vehicles 12. The other vehicle 12 that receives the emergency avoidance signal, for example, uses the sensor information acquired by the other vehicle 12 with the emergency avoidance signal as a trigger to calculate the control variable of the other vehicle 12. The other vehicle 12 may also estimate the emergency avoidance signal transmission time based on information such as the inter-vehicle distance, and focus on the sensor information of the vehicle 12 at the estimated emergency avoidance signal transmission time to calculate the control variable of the other vehicle 12. In addition, for the sake of convenience of explanation, an example in which the vehicle 12A sends a control variable to the vehicle 12D is given, but the present embodiment is not limited to the above description. For example, the vehicle 12A may send a control variable to a plurality of vehicles 12 within a predetermined distance from the vehicle 12A, or to vehicles 12 having a predetermined positional relationship with the vehicle 12A (existing in the front, rear, left, or right).

[0147] 〔Hardware (1)〕

[0148] Further describe the central brain 2120 that can achieve the above-mentioned autonomous driving. The central brain 2120 is configured as Figure 12 the information processing device 10 shown in the figure. In addition, the above-mentioned central brain 2120 is a general processing device that functions as an information processing device including a gateway, and the central brain 125 described later is a narrow processing device when classifying functions by each processor.

[0149] Figure 12 FIG. is a block diagram showing an example of the configuration of the information processing device 10 including the central brain according to the embodiment. The information processing device 10 includes an image processing unit (IPU, Image Processing Unit) 121, a motion processing unit (MoPU, Motion Processing Unit) 122, a central brain 125, and a memory 126. The central brain 125 is configured to include a graphics neural network processing unit (GNPU, Graphics Neuralnetwork Processing Unit) 123 and a central processing unit (CPU, Central Processing Unit) 124.

[0150] The IPU 121 can be built into a super high-resolution camera (not shown) provided on the vehicle. The IPU 121 performs predetermined image processing such as Bayer transformation, demosaicing, denoising, and sharpening on the images of the objects existing around the vehicle, and outputs the processed images of the objects at a frame rate of, for example, 10 frames per second and a resolution of 12 million pixels. The images output from the IPU 121 are provided to the central brain 125 and the memory 126.

[0151] The MoPU 122 can be built into a low-resolution camera different from the super high-resolution camera provided on the vehicle. The MoPU 122 outputs motion information indicating the movement of the photographed object at a frame rate of, for example, 1920 frames per second. That is, the frame rate of the output of the MoPU 122 is 100 times the frame rate of the output of the IPU 121. The MoPU 122 outputs vector information indicating the movement of the point representing the existence position of the object along a predetermined coordinate axis as the motion information. That is, in the motion information output from the MoPU 122, the information required to identify what the photographed object is (for example, a person or an obstacle) is not included, but only the information indicating the movement (movement direction and movement speed) of the center point (or center of gravity point) of the object on the coordinate axes (x-axis, y-axis, z-axis) is included. The images output from the MoPU 122 are provided to the central brain 125 and the memory 126. By not including image information in the motion information, the amount of information transmitted to the central brain 125 and the memory 126 can be suppressed.

[0152] The present disclosure includes a first processor that outputs an image of a captured object at a first frame rate, and a second processor that outputs movement information indicating the movement of the captured object at a second frame rate higher than the first frame rate. That is, as a cycle for detecting the conditions around the vehicle of the present disclosure, a detection unit that captures the surroundings of the vehicle at a first cycle is an example of the above "first processor", and the IPU 121 is an example of the "first processor". In addition, a detection unit including a sensor that detects the conditions around the vehicle at a second cycle shorter than the first cycle of the present disclosure is an example of the above "second processor", and the MoPu 122 is an example of the "second processor".

[0153] The central brain 125 performs driving control of the vehicle based on the image output from the IPU 121 and the movement information output from the MoPU 122. For example, the central brain 125 identifies objects (people, animals, roads, traffic lights, signs, crosswalks, obstacles, buildings, etc.) existing around the vehicle based on the image output from the IPU 121. In addition, the central brain 125 identifies the movement of the objects for which something has been identified existing around the vehicle based on the movement information output from the MoPU 122. The central brain 125 performs, for example, control (speed control) of the motor that drives the wheels, braking control, and steering wheel control based on the identified information. In the central brain 125, the GNPU 123 may be responsible for processing related to image recognition, and the CPU 124 may be responsible for processing related to vehicle control.

[0154] Generally, an ultra-high-resolution camera is used for image recognition in autonomous driving. It is possible to identify what objects are included in the image captured by the high-resolution camera. However, in level 6 autonomous driving, this is not enough. In level 6, it is also necessary to identify the movement of objects. By identifying the movement of objects, for example, it is possible to perform an avoidance operation for a vehicle traveling through autonomous driving to avoid obstacles with higher accuracy. However, in a high-resolution camera, only about 10 frames of images can be acquired per second, and it is difficult to analyze the movement of objects. On the other hand, in a camera equipped with the MoPU 122, although the resolution is low, for example, it can output at a high frame rate of 1920 frames per second.

[0155] Therefore, in the technology of the present disclosure, two independent processors, i.e., the IPU 121 and the MoPU 122, are used. The high-resolution camera (IPU 121) functions to acquire the image information required to identify what the captured object is, and the MoPU 122 functions to detect the movement of the object. The MoPU 122 takes the object as a point and analyzes in which directions on the x-axis, y-axis, and z-axis and at what speed the coordinates of this point move. Since the overall contour of the object and the detection of what the object is can be performed through the image from the high-resolution camera, through the MoPU 122, as long as it is known how the center point of the object moves, it is known what behavior the entire object will have.

[0156] According to the method of only analyzing the movement and speed of the center point of the object, compared with judging how the entire image of the object moves, the amount of information transmitted to the central brain 125 can be significantly suppressed, thereby significantly reducing the amount of calculation in the central brain 125. For example, when an image of 1000 pixels × 1000 pixels is sent to the central brain 125 at a frame rate of 1920 frames per second, if color information is included, 4 billion bits per second of data will be sent to the central brain 125. The MoPU 122 can compress the amount of data transmitted to the central brain 125 to 20,000 bits per second by only sending the movement information indicating the movement of the center point of the object. That is, the amount of data transmitted to the central brain 125 is compressed to one two-hundred-thousandth.

[0157] In this way, by combining the use of the low frame rate and high-resolution image output from the IPU 121 and the high frame rate and lightweight (low-resolution) movement information output from the MoPU 122, object recognition including object movement can be achieved with a smaller amount of data.

[0158] In addition, in the case of using one MoPU 122, vector information representing the movement of the point indicating the existence position of the object along each of the two coordinate axes (x-axis and y-axis) in the three-dimensional rectangular coordinate system can be obtained. The principle of a stereo camera can also be utilized, and two MoPU 122s can be used to output vector information representing the movement of the point indicating the existence position of the object along each of the three coordinate axes (x-axis, y-axis, and z-axis) in the three-dimensional rectangular coordinate system. The z-axis is the axis along the depth direction (the driving direction of the vehicle).

[0159] In this embodiment, as sensor information, the condition around the vehicle 12 is detected at a second period shorter than the first period of photographing the surroundings of the vehicle using a camera or the like. That is, in the above-mentioned Level 5, for example, the surroundings can be detected twice by a camera or the like in 0.3 seconds, but in this embodiment, in Level 6, for example, the surroundings can be detected 576 times. Then, an index value and a control variable can be calculated for each of the 576 detections of the surrounding conditions, and driving that is faster and safer than the autonomous driving performed by Level 5 can be executed, thereby enabling the behavior of the vehicle to be controlled.

[0160] [Avoidance (2)]

[0161] In addition, in order for the present vehicle 12A to drive while at least avoiding other vehicles 12D, there are sometimes multiple driving modes. Therefore, in this embodiment, the central brain 2120 can determine the optimal driving mode from the multiple driving modes and set it as the driving strategy.

[0162] Figure 13 is a diagram schematically showing Figure 9 the driving modes applicable to the present vehicle 12A in the driving state showing the relationship between the present vehicle 12A and other vehicles 12B, 12C, 12D.

[0163] In Figure 13 the example shown, when adopting the driving mode 12Ax1 as the path of the present vehicle 12A, the present vehicle 12A will collide with other vehicles 12D. On the other hand, when adopting the driving modes 12Ax2 and 12Ax3, the present vehicle 12A can avoid collisions including contact with other vehicles 12D. Therefore, the central brain 2120 calculates the driving modes 12Ax2 and 12Ax3 respectively and sets any one of them as the driving mode. The setting of this driving mode sets the driving mode with the minimum risk relationship for the mutual relationship between the present vehicle 12A and other vehicles 12D. Specifically, as Figure 14 shown, the driving mode 12Ax2 with the largest distance between the present vehicle 12A and other vehicles 12D is set. That is, the driving mode 12Ax2 in which the size of the space between the vehicle and other vehicles as obstacles exceeds a predetermined value (for example, becomes the maximum value) among the multiple different driving modes is set.

[0164] Figure 15 is a flowchart showing an example of the processing flow in the central brain 2120 in which the present vehicle 12A can drive while avoiding other vehicles 12D. In Figure 15 it, the processing of step S11 shown in Figure 3 is replaced with steps S11A, 11B, and 11C and executed. The central brain 2120 can repeatedly execute Figure 15The processing shown is used instead of Figure 3 the processing shown.

[0165] In step S2011A, the central brain 2120 derives the driving strategy of the vehicle 12 based on the sensor information. Then, in step S2011B, the optimal driving strategy (driving mode) is set from the derived driving strategy. Then, in step S2011C, the control variables corresponding to the driving strategy (driving mode) set to be able to drive while avoiding obstacles (such as other vehicles 12D) are calculated. The processing of steps S2011A to S2011C is an example of the function of the calculation unit, and the processing of steps S2011A and S2011B is an example of the function of the setting unit.

[0166] The above driving strategy (driving mode) can also change the driving mode according to the conditions around the vehicle that change moment by moment. For example, the driving strategy (driving mode) can include the timing that affects the control variables and changes vehicle behaviors such as the vehicle speed. In addition, the control variables can also include the above timing.

[0167] In addition, for the control variables related to the speed of the vehicle 12, 10 modes each of large, medium, and small (L, M, S) can be set for the acceleration side and a total of 30 modes can be applied, and 10 modes each of large, medium, and small (L, M, S) can be set for the deceleration side and a total of 30 modes can be applied for selection. In this case, the relationship between the own vehicle and other vehicles changes moment by moment. In the state where the own vehicle is approaching other vehicles, the distance changes in the approaching direction, so the options for selecting modes become smaller moment by moment, which can reduce the processing time for selection and further reduce the adjustment of the incremental difference.

[0168] In addition, the above has described the case where the other vehicle 12D is an other vehicle relative to the own vehicle 12A, but it is also possible to calculate the control variables for reducing the collision risk with respect to at least one, and further all, of the other vehicles 12B, 12C, 12D around the own vehicle 12A.

[0169] Therefore, the central brain 2120 controls, based on the control variables calculated according to the prediction of collisions including contact, for example, the in-wheel motors 31 respectively mounted on the four wheels 30. Thus, by controlling the wheel speed and inclination of each of the four wheels 30 and the suspension 32 supporting each of the four wheels 30, it is possible to avoid collisions or reduce the damage to the vehicle during collisions and perform autonomous driving.

[0170] 〔Notification (2)〕

[0171] The central brain 2120A of the vehicle 12A repeatedly executes Figure 17CThe flowchart shown. Except for steps S2120 and S2150, Figure 17C the processing of the flowchart of Figure 17B can also be substantially the same as the processing of the flowchart of Figure 17A as shown in the example. The functional configuration of the central brain 2120 is as

[0172] In step S2010, the central brain 2120A acquires sensor information detected by the sensors. In step S2011A, the central brain 2120A derives a driving strategy for the vehicle 12A based on the sensor information acquired in step S2010. In step S2011B, the central brain 2120A sets the optimal driving strategy (driving mode) from the derived driving strategies.

[0173] In step S2011C, the central brain 2120A calculates a total of 16 control variables corresponding to the driving strategy (driving mode) set to be able to drive while avoiding obstacles. In step S2120, the central brain 2120A determines whether the vehicle 12A performs an emergency behavior of avoiding obstacles (performs an emergency avoidance) based on the sensor information and / or the calculated control variables. If the determination in step S2120 is negative, the central brain 2120A proceeds to step S2012.

[0174] If the determination in step S2120 is positive, in step S2150, the central brain 2120A sends the calculated control variables to the central brain 2120D of the vehicle 12D.

[0175] In step S2012, the central brain 2120A controls the autonomous driving based on the control variables calculated in step S2011C and ends the processing of this flowchart. In addition to the sensor information acquired by the vehicle 12D, the central brain 2120D of the vehicle 12D also uses the control variables of the vehicle 12A to calculate the control variables of the vehicle 12D.

[0176] As described above, when the vehicle 12 performs an emergency behavior of avoiding obstacles (performs an emergency avoidance), by sending the control variables of the vehicle 12 to other vehicles 12, other vehicles 12 are notified in advance of the emergency behavior of the vehicle 12 that is difficult for other vehicles 12 to predict. Thus, other vehicles 12 can perceive the occurrence of the emergency behavior of the vehicle 12 before it is detected by sensors or the like after the behavior occurs, and can thus respond more quickly to the occurrence of the emergency behavior of the vehicle 12. In addition, by using the control variables of the vehicle 12 for the calculation of the control variables of other vehicles 12, appropriate control variables for other vehicles 12 can be calculated by preventing accidents.

[0177] In addition, the above description is an example. Instead of sending the control variables of vehicle 12 to other vehicles 12, vehicle 12 may also send an emergency avoidance signal indicating that vehicle 12 performs an emergency behavior of avoiding an obstacle (performing an emergency avoidance) to other vehicles 12. Other vehicles 12 that receive the emergency avoidance signal, for example, trigger based on the emergency avoidance signal and calculate the control variables of other vehicles 12 using the sensor information obtained by other vehicles 12. Other vehicles 12 may also estimate the emergency avoidance signal transmission time based on information such as the inter-vehicle distance, etc., and focus on the sensor information of vehicle 12 at the estimated emergency avoidance signal transmission time to calculate the control variables of other vehicles 12. In addition, for the sake of convenience of explanation, an example in which vehicle 12A sends control variables to vehicle 12D is given, but the present embodiment is not limited to the above description. For example, vehicle 12A may send control variables to a plurality of vehicles 12 within a predetermined distance from vehicle 12A, or to vehicles 12 having a predetermined positional relationship (existing in the front, rear, left, or right) with vehicle 12A.

[0178] [Third Embodiment]

[0179] [Notification (1)]

[0180] Figure 18A is a block diagram illustrating the functional configuration of the central brain 3120. For example, the central brain 3120A is the central brain of vehicle 12A, and the English letter at the end of the reference numeral indicates the corresponding vehicle. Hereinafter, when there is no need to distinguish, the English letter at the end is omitted. In addition, the English letter at the end distinguishes the vehicle and the central brain in the description, but there is no particularity in implementation. For example, vehicle 12A may be any one of vehicles 12B to 12N.

[0181] As used above Figure 1 and Figure 2 described, the central brains 2120A to 2120N can mutually transmit and receive data via the cloud. As an example, as Figure 18A shown, the central brain 3120 of vehicle 12 functions as an acquisition unit 3200, a calculation unit 3210, a control unit 3220, a transmission unit 3230, and a reception unit 3240. The acquisition unit 3200 of vehicle 12 acquires sensor information including road information and the cruising state of surrounding vehicles from sensors mounted on vehicle 12.

[0182] The calculation unit 3210 calculates a total of 16 control variables for controlling the wheel speeds, tilts, and suspensions of the four wheels of vehicle 12 using the sensor information of vehicle 12. The control unit 3220 controls the autonomous driving of vehicle 12 based on the control variables calculated by the calculation unit 3210.

[0183] When the control unit 3220 determines that there is a possibility of collision between the vehicle 12 and other vehicles 12, it receives the sensor information of the other vehicles 12 via the receiving unit 3240 of the vehicle 12. For example, the vehicle 12 can send a request signal to send the sensor information to the other vehicles 12, and the other vehicles 12 that receive the send request signal send the sensor information to the vehicle 12 via the sending unit 3230. Alternatively, each vehicle 12 can send the sensor information to the server at a predetermined time interval, and the vehicle 12 selectively receives the sensor information of the other vehicles 12 from the server.

[0184] For example, it can be determined whether there is a possibility of collision between the vehicle 12 and other vehicles 12 based on the sensor information acquired by the acquisition unit 3200. For example, it can be determined whether the collision object is a vehicle based on the cruising state of the surrounding vehicles included in the sensor information (such as the image information of the surrounding of the vehicle captured by the in-vehicle camera, etc.). In addition, the determination of whether the collision object is a vehicle can be performed, for example, by a server that sends and stores the sensor information of each vehicle (such as including the GPS information of each vehicle, etc.), and the vehicle 12 receives the determination result. The possibility of collision can mean, for example, that the distance between the vehicle 12 and other vehicles 12 is equal to or less than a predetermined distance, and it is predicted that the distance between the vehicle 12 and other vehicles 12 will shorten in the future.

[0185] The calculation unit 3210 of the vehicle 12 generates a control variable for the other vehicle 12 based on the received sensor information of the other vehicle 12 and the sensor information of the vehicle 12, and sends the control variable of the other vehicle 12 to the other vehicle 12 via the sending unit 3230. The calculation unit 3210 of the vehicle 12 can generate the control variable of the other vehicle 12 not only using the sensor information, but also using the control variables of the vehicle 12 and / or other vehicles 12. When also using the control variable of the other vehicle 12, in addition to the sensor information of the other vehicle 12, the vehicle 12 receives the control variable of the other vehicle 12 via the receiving unit 3240.

[0186] The other vehicle 12 receives the control variable sent from the vehicle 12 via the receiving unit 3240. The control unit 3220 of the other vehicle 12 uses the received control variable, that is, the control variable of the other vehicle 12 calculated by the vehicle 12, to control the autonomous driving of the other vehicle 12. The central brain 3120A of the vehicle 12A repeatedly executes Figure 18B the flowchart shown. Except for steps S3160, S3170, and S3180, Figure 18B the processing of the flowchart can also be substantially the same as Figure 3 the processing of the flowchart.

[0187] In step S3010, the central brain 3120A acquires sensor information detected by sensors. In step S3011, the central brain 3120A calculates a total of 16 control variables based on the sensor information acquired in step S3010.

[0188] In step S3160, the central brain 3120A determines whether vehicle 12A has a possibility of colliding with another vehicle, such as vehicle 12D, based on the sensor information and / or the calculated control variables. When the determination in step S3160 is negative, the central brain 3120A proceeds to step S3012.

[0189] When the determination in step S3160 is affirmative, in step S3170, the central brain 3120A receives the sensor information of the other vehicle 12D. In step S3180, the central brain 3120A calculates the control variables of the other vehicle 12D based on the sensor information of the other vehicle 12D and the sensor information of vehicle 12A, and sends the calculated control variables to the other vehicle 12D. In step S3012, the central brain 3120A controls the autonomous driving according to the control variables calculated in step S3011, and ends the processing of this flowchart. The central brain 3120D of vehicle 12D uses the control variables received from vehicle 12A to control the autonomous driving of vehicle 12D.

[0190] As described above, when vehicle 12 has a possibility of colliding with another vehicle 12, it generates the control variables of the other vehicle 12 and sends the control variables of the other vehicle 12 to the other vehicle 12. Therefore, it is possible to control the autonomous driving that takes into account the movements of both vehicle 12 and the other vehicle 12 in both vehicle 12 and the other vehicle 12. Thus, it is possible to attempt to avoid a collision between vehicle 12 and the other vehicle 12, and even if a collision occurs, it is possible to reduce the mutual damage.

[0191] In addition, the above description is an example. For the sake of easy explanation, an example in which vehicle 12A sends the control variables of vehicle 12D to vehicle 12D is given, but the present embodiment is not limited to the above description. For example, vehicle 12A may also calculate the control variables of each of a plurality of other vehicles 12 that may collide with vehicle 12A and send them to each of the other vehicles 12.

[0192] 〔Hardware (1)〕

[0193] The central brain 3120 that can implement the above autonomous driving is further described. The central brain 3120 is configured as Figure 12 the information processing device 10 shown. In addition, the above central brain 3120 is a general processing device that functions as an information processing device including a gateway, and the later-described central brain 125 is a narrow processing device when classifying functions by each processor.

[0194] Figure 12 This is a block diagram showing an example of the configuration of an information processing apparatus 10 including a central brain according to an embodiment. The information processing apparatus 10 includes an image processing unit (IPU, Image Processing Unit) 121, a motion processing unit (MoPU, Motion Processing Unit) 122, a central brain 125, and a memory 126. The central brain 125 is configured to include a graphics neural network processing unit (GNPU, Graphics Neuralnetwork Processing Unit) 123 and a central processing unit (CPU, Central Processing Unit) 124.

[0195] The IPU 121 can be built into a super high-resolution camera (not shown) provided on a vehicle. The IPU 121 performs predetermined image processing such as Bayer conversion, demosaicing, denoising, and sharpening on an image of an object existing around the vehicle, and outputs the processed image of the object at a frame rate of, for example, 10 frames per second and a resolution of 12 million pixels. The image output from the IPU 121 is provided to the central brain 125 and the memory 126.

[0196] The MoPU 122 can be built into a low-resolution camera different from the super high-resolution camera provided on the vehicle. The MoPU 122 outputs motion information indicating the movement of the photographed object at a frame rate of, for example, 1920 frames per second. That is, the frame rate of the output of the MoPU 122 is 100 times the frame rate of the output of the IPU 121. The MoPU 122 outputs vector information indicating the movement of a point representing the existence position of the object along a predetermined coordinate axis as the motion information. That is, in the motion information output from the MoPU 122, information required to identify what the photographed object is (for example, a person or an obstacle) is not included, and only information indicating the movement (movement direction and movement speed) of the center point (or center of gravity point) of the object on the coordinate axes (x-axis, y-axis, z-axis) is included. The image output from the MoPU 122 is provided to the central brain 125 and the memory 126. By not including image information in the motion information, the amount of information transmitted to the central brain 125 and the memory 126 can be suppressed.

[0197] The present disclosure includes a first processor that outputs an image of a captured object at a first frame rate, and a second processor that outputs movement information indicating the movement of the captured object at a second frame rate higher than the first frame rate. That is, as a period for detecting the situation around the vehicle of the present disclosure, a detection unit that captures the situation around the vehicle at a first period is an example of the above "first processor", and the IPU 121 is an example of the "first processor". In addition, a detection unit including a sensor that detects the situation around the vehicle at a second period shorter than the first period of the present disclosure is an example of the above "second processor", and the MoPu 122 is an example of the "second processor".

[0198] The central brain 125 performs driving control of the vehicle based on the image output from the IPU 121 and the movement information output from the MoPU 122. For example, the central brain 125 identifies objects (people, animals, roads, traffic lights, signs, crosswalks, obstacles, buildings, etc.) existing around the vehicle based on the image output from the IPU 121. In addition, the central brain 125 identifies the movement of the objects where something has been identified existing around the vehicle based on the movement information output from the MoPU 122. The central brain 125 performs, for example, control (speed control) of the motor for driving the wheels, braking control, and steering wheel control based on the identified information. In the central brain 125, the GNPU 123 may be responsible for processing related to image recognition, and the CPU 124 may be responsible for processing related to vehicle control.

[0199] Generally, an ultra-high-resolution camera is used for image recognition in autonomous driving. It is possible to identify what objects are included in the image captured by the high-resolution camera. However, in autonomous driving in the level 6 era, this is not enough. In the level 6 era, it is also necessary to identify the movement of objects. By identifying the movement of objects, for example, it is possible to perform an avoidance operation for a vehicle traveling through autonomous driving to avoid obstacles with higher accuracy. However, in a high-resolution camera, only about 10 frames of images can be acquired per second, and it is difficult to analyze the movement of objects. On the other hand, in a camera equipped with the MoPU 122, although the resolution is low, it can output at a high frame rate of, for example, 1920 frames per second.

[0200] Therefore, in the technology of the present disclosure, two independent processors, i.e., the IPU 121 and the MoPU 122, are used. The high-resolution camera (IPU 121) functions to obtain the image information required to identify what the captured object is, and the MoPU 122 functions to detect the movement of the object. The MoPU 122 takes the object as a point and analyzes in which directions on the x-axis, y-axis, and z-axis and at what speed the coordinates of this point move. Since the overall contour of the object and the detection of what the object is can be performed through the image from the high-resolution camera, through the MoPU 122, as long as it is known how the center point of the object moves, it is known what behavior the entire object will have.

[0201] According to the method of only analyzing the movement and speed of the center point of the object, compared with judging how the entire image of the object moves, the amount of information transmitted to the central brain 125 can be significantly suppressed, thereby significantly reducing the amount of calculation in the central brain 125. For example, when sending an image of 1000 pixels × 1000 pixels to the central brain 125 at a frame rate of 1920 frames per second, if color information is included, 4 billion bits per second of data will be sent to the central brain 125. The MoPU 122 can compress the amount of data transmitted to the central brain 125 to 20,000 bits per second by only sending the movement information indicating the movement of the center point of the object. That is, the amount of data transmitted to the central brain 125 is compressed to one two-hundred-thousandth.

[0202] In this way, by combining the use of the low frame rate and high-resolution image output from the IPU 121 and the high frame rate and lightweight (low-resolution) movement information output from the MoPU 122, object recognition including object movement can be achieved with a smaller amount of data.

[0203] In addition, in the case of using one MoPU 122, vector information indicating the movement of the point representing the existence position of the object along each of the two coordinate axes (x-axis and y-axis) in the three-dimensional rectangular coordinate system can be obtained. The principle of a stereo camera can also be utilized, and two MoPU 122s can be used to output vector information indicating the movement of the point representing the existence position of the object along each of the three coordinate axes (x-axis, y-axis, and z-axis) in the three-dimensional rectangular coordinate system. The z-axis is the axis along the depth direction (the driving direction of the vehicle).

[0204] In this embodiment, as sensor information, the surrounding conditions of the vehicle 12 are detected at a second cycle shorter than the first cycle of photographing the surrounding of the vehicle using a camera or the like. That is, in the above-mentioned Level 5, for example, the surrounding conditions can be detected twice by a camera or the like in 0.3 seconds, but in this embodiment, in Level 6, for example, the surrounding conditions can be detected 576 times. Then, an index value and a control variable can be calculated for each of the 576 detections of the surrounding conditions, and driving that is faster and safer than the autonomous driving performed by Level 5 can be executed, thereby enabling the behavior of the vehicle to be controlled.

[0205] [Avoidance (2)]

[0206] In addition, in order for the present vehicle 12A to drive while at least avoiding other vehicles 12D, there are sometimes multiple driving modes. Therefore, in this embodiment, the central brain 3120 can determine the optimal driving mode from multiple driving modes and set it as the driving strategy.

[0207] Figure 13 is a diagram schematically showing Figure 9 the driving modes applicable to the present vehicle 12A in the driving state showing the relationship between the present vehicle 12A and other vehicles 12B, 12C, 12D.

[0208] In Figure 13 the example shown, when adopting the driving mode 12Ax1 as the path of the present vehicle 12A, the present vehicle 12A will collide with other vehicles 12D. On the other hand, when adopting the driving modes 12Ax2 and 12Ax3, the present vehicle 12A can avoid collisions including contact with other vehicles 12D. Therefore, the central brain 3120 calculates the driving modes 12Ax2 and 12Ax3 respectively and sets any one of them as the driving mode. The setting of this driving mode sets the driving mode with the least risky relationship for the mutual relationship between the present vehicle 12A and other vehicles 12D. Specifically, as Figure 14 shown, the driving mode 12Ax2 with the largest distance between the present vehicle 12A and other vehicles 12D is set. That is, the driving mode 12Ax2 in which the size of the space between the vehicle and other vehicles as obstacles exceeds a predetermined value (for example, becomes the maximum value) among multiple different driving modes is set.

[0209] Figure 15 is a flowchart showing an example of the processing flow in the central brain 3120 in which the above-mentioned present vehicle 12A can drive while avoiding other vehicles 12D. In Figure 15 it, the processing of step S11 shown in Figure 3 is replaced with steps S3011A, 3011B, and 3011C and executed. The central brain 3120 can repeatedly execute Figure 15The processing shown is used instead of Figure 3 the processing shown.

[0210] In step S3011A, the central brain 3120 derives a driving strategy for the vehicle 12 based on sensor information. Then, in step S3011B, the optimal driving strategy (driving mode) is set from the derived driving strategies. Then, in step S3011C, a control variable corresponding to the driving strategy (driving mode) set to be able to drive while avoiding obstacles (such as other vehicles 12D) is calculated. The processing of steps S3011A to S3011C is an example of the function of the calculation unit, and the processing of steps S3011A and S3011B is an example of the function of the setting unit.

[0211] The above driving strategy (driving mode) can also change the driving mode according to the situation around the vehicle that changes all the time. For example, the driving strategy (driving mode) can include the timing that affects the control variable and changes vehicle behaviors such as the vehicle speed. In addition, the control variable can also include the above timing.

[0212] In addition, for the control variable related to the speed of the vehicle 12, 10 modes each of large, medium, and small (L, M, S) can be set for the acceleration side and a total of 30 modes can be applied, and 10 modes each of large, medium, and small (L, M, S) can be set for the deceleration side and a total of 30 modes can be applied for selection. In this case, the relationship between the host vehicle and other vehicles changes all the time. When the host vehicle approaches other vehicles, the distance changes in the approaching direction. Therefore, the options for selecting a mode become smaller all the time, the processing time for selection can be reduced, and further the adjustment of the increment difference can be reduced.

[0213] In addition, the above has described the case where the other vehicle 12D is the other vehicle relative to the host vehicle 12A, but it is also possible to calculate the control variable for reducing the collision risk with respect to at least one of the other vehicles 12B, 12C, 12D around the host vehicle 12A, and further all other vehicles.

[0214] Therefore, the central brain 3120 controls, based on the control variable calculated according to the prediction of a collision including contact, for example, the in-wheel motors 31 respectively mounted on the four wheels 30. Thus, by controlling the wheel speed and inclination of each of the four wheels 30 and the suspension 32 that supports each of the four wheels 30, it is possible to avoid a collision or reduce the damage to the vehicle during a collision and perform autonomous driving.

[0215] 〔Notification (2)〕

[0216] The central brain 3120A of the vehicle 12A repeatedly executes Figure 18CThe flowchart shown. Except for steps S3160, S3170, and S3180, Figure 18C the processing of the flowchart of Figure 15 can also be substantially the same as the processing of the flowchart of Figure 18A as shown in the example. The functional configuration of the central brain 3120 is

[0217] In step S3010, the central brain 3120A acquires sensor information detected by sensors. In step 11A, the central brain 3120A derives a driving strategy for the vehicle 12A based on the sensor information acquired in step S3010. In step S3011B, the central brain 3120A sets the best driving strategy (driving mode) from the derived driving strategies.

[0218] In step S3011C, the central brain 3120A calculates a total of 16 control variables corresponding to the driving strategy (driving mode) set to be able to drive while avoiding obstacles. In step S3160, the central brain 3120A determines whether the vehicle 12A has a possibility of colliding with other vehicles, such as vehicle 12D, based on the sensor information and / or the calculated control variables. If the determination in step S3160 is negative, the central brain 3120A proceeds to step S3012.

[0219] If the determination in step S3160 is affirmative, in step S3170, the central brain 3120A receives the sensor information of the other vehicle 12D. In step S3180, the central brain 3120A calculates the control variables of the other vehicle 12D based on the sensor information of the other vehicle 12D and the sensor information of the vehicle 12A, and sends the calculated control variables to the other vehicle 12D.

[0220] In step S3012, the central brain 3120A controls the autonomous driving according to the control variables calculated in step S3011, and ends the processing of this flowchart. The central brain 3120D of the vehicle 12D uses the control variables received from the vehicle 12A to control the autonomous driving of the vehicle 12D.

[0221] As described above, when the vehicle 12 has a possibility of colliding with other vehicles 12, it generates the control variables of the other vehicles 12 and sends the control variables of the other vehicles 12 to the other vehicles 12. Therefore, it is possible to control the autonomous driving that takes into account the movements of both the vehicle 12 and the other vehicles 12 in both the vehicle 12 and the other vehicles 12. Thus, it is possible to attempt to avoid a collision between the vehicle 12 and the other vehicles 12, and even if a collision occurs, the mutual damage can be reduced.

[0222] In addition, the above description is an example. For the sake of illustration, an example is given in which vehicle 12A sends the control variables of vehicle 12D to vehicle 12D. However, this embodiment is not limited to the above description. For example, vehicle 12A may also calculate the control variables of each of a plurality of other vehicles 12 that may collide with vehicle 12A and send them to each of the other vehicles 12.

[0223] [Fourth Embodiment]

[0224] During vehicle travel, there are cases where an obstacle approaches the vehicle. In such cases, as described above, it is preferable for the vehicle to change its behavior and travel to avoid contact or collision with the obstacle. As examples of the obstacle, as described above, other vehicles other than the own vehicle in travel, a wall surface, a guardrail, a curb, and other installations can be cited. In the following description, as an example of the obstacle, the case of applying another vehicle approaching vehicle 12 will be described.

[0225] In addition, vehicles generally have differences in various characteristics such as physical characteristics, response characteristics, and travel characteristics of parts related to travel behavior (hereinafter, simply referred to as "vehicle characteristics") among different vehicle models. In addition, even for vehicles of the same model, even if the vehicle characteristics are roughly the same when newly manufactured, as the travel distance increases, due to differences in usage environment, usage frequency, usage method, etc., the differences in vehicle characteristics generally gradually become larger.

[0226] Therefore, even for Figure 10 the various travel modes shown, the travel mode suitable as the control object for each vehicle is different.

[0227] Figure 19A is a schematic diagram of an example of the preferred travel mode of each vehicle when avoiding an obstacle. In addition, in Figure 19A the example shown, for each of the above travel mode PM and travel mode PS, an example of the travel mode of each of 3 different vehicles is shown. If the travel modes of different vehicles shown in Figure 19A are compared, when avoiding an obstacle, the offset amplitude in the left - right direction with respect to the traveling direction of the vehicle is different, or the timing of the change in the traveling direction of the vehicle is different.

[0228] Therefore, the central region 120, which functions as an example of the information processing apparatus according to the present embodiment, includes the function of a registration unit that registers a plurality of different driving modes in a memory, where the plurality of driving modes are the driving modes that a vehicle (hereinafter referred to as the "target vehicle") has performed in the past to avoid obstacles. In addition, the central brain 120 according to the present embodiment includes the function of an acquisition unit that acquires a plurality of pieces of information related to the target vehicle from a detection unit including sensors, where the sensors detect the surrounding conditions of the target vehicle including obstacles. Further, the central brain 120 according to the present embodiment includes a setting unit that selectively sets, from the registered plurality of driving modes, a driving mode for the target vehicle to avoid obstacles using the plurality of pieces of information acquired by the acquisition unit, and calculates a control variable for controlling the behavior of the target vehicle based on the plurality of pieces of information acquired by the acquisition unit and the set driving mode, so that the target vehicle travels according to the driving mode. And, the central brain 120 according to the present embodiment includes the function of a control unit that controls the behavior of the target vehicle based on the calculated control variable.

[0229] In the present embodiment, the registration unit registers the above-described driving modes in the memory for each vehicle to be controlled. In particular, in the present embodiment, in order to avoid obstacles when the target vehicle is traveling, the actual driving path when traveling in the driving mode selected and set by the setting unit is used as a candidate for the driving mode registered by the registration unit.

[0230] That is, as described above, there are differences in vehicle characteristics for each vehicle, and the follow-up performance of the driving path to the set driving mode varies for each vehicle. Therefore, the actual driving path does not necessarily coincide with the set driving mode, and the actual driving path reflects a lot of the vehicle characteristics of each vehicle. Thus, in the registration unit according to the present embodiment, the actual driving path when avoiding obstacles is registered as a driving mode dedicated to this vehicle corresponding to the vehicle characteristics.

[0231] However, it is not limited to this method. For example, when driving manually without autonomous driving in the target vehicle, in order to avoid obstacles, the actual driving path when changing the driving direction may be registered by the registration unit.

[0232] In addition, the registration unit according to the present embodiment updates any one of the driving modes including the previously registered modified examples to the newly obtained driving mode.

[0233] In addition, as described above, in the present embodiment, the driving patterns are registered in the memory for each vehicle to be controlled by the registration unit, but it is not limited thereto. For example, the driving patterns may be registered by the registration unit for each vehicle model of the vehicles to be controlled. According to this method, the generality of the driving patterns can be improved compared with the case where the driving patterns are registered for each vehicle.

[0234] Here, the calculation unit according to the present embodiment calculates a control variable so that the target vehicle travels according to the driving pattern set by the setting unit; the control variable includes the speed of the target vehicle and the timing at which the speed of the target vehicle changes; and the calculation unit calculates the control variable through multivariate analysis based on an integration method using deep learning, etc. These aspects are the same as those in the first embodiment described above.

[0235] Figure 19B It is a flowchart showing an example of the processing flow in the central brain 120 in which the own vehicle 12A according to the present embodiment can avoid other vehicles 12D and travel. In Figure 19B the processing shown, the processing of step S11 shown in Figure 15 is replaced with step S4011B1 and executed. The central brain 120 can repeatedly execute the processing shown in Figure 19B in place of the processing shown in Figure 15 For the steps that perform the same processing as the first embodiment, the same step numbers as those in the first embodiment are given, and the description thereof is omitted here.

[0236] In step S4011B1, the central brain 120 sets the optimal driving pattern from the registered driving patterns. In the present embodiment, the registered driving pattern refers to the driving pattern obtained from the driving paths that the own vehicle has traveled in the past and registered by the registration unit, as described above. The processing of steps S4011A to S4011C is an example of the function of the calculation unit, and the processing of steps S4011A and S4011B1 is an example of the function of the setting unit.

[0237] Figure 19C It is a flowchart showing an example of the processing flow in the central brain 120 when the registration unit registers the driving pattern related to the own vehicle according to the present embodiment. The central brain 120 immediately executes the processing shown in Figure 19C after the own vehicle avoids other vehicles and travels. In addition, hereinafter, in order to avoid complication, the case where a plurality of basic driving patterns such as driving patterns PL, PM, and PS and modified examples of the driving patterns are already registered in the memory will be described.

[0238] In step S4020, the central brain 120 obtains the driving route during the driving in which the host vehicle avoids other vehicles immediately before (hereinafter referred to as "target avoidance driving"). In addition, in the present embodiment, information indicating the driving route is sequentially stored in the memory, and the driving route is obtained by reading out the information from the memory. However, it is of course not limited to this method.

[0239] In step S4021, the central brain 120 determines a registered driving mode (hereinafter referred to as "corresponding driving mode") corresponding to the driving mode represented by the obtained driving route (hereinafter referred to as "obtained driving mode"). In addition, in the present embodiment, the corresponding driving mode is determined by determining the registered driving mode with the highest similarity to the obtained driving mode. However, it is not limited to this method. It may also be a method of determining the corresponding driving mode by determining the registered driving mode with the lowest non-similarity to the obtained driving mode. In addition, in the present embodiment, although the above similarity is derived using existing known template matching, it is not limited thereto. It may also be a method of applying other similarity derivation methods such as a method based on a cross-correlation function (CCF) or a method based on dynamic time warping (DTW).

[0240] In step S4022, the central brain 120 determines whether the obtained driving mode and the corresponding driving mode are inconsistent. If the determination is negative, this process ends. On the other hand, if the determination is positive, the process proceeds to step S4023. In the present embodiment, this determination is made as follows: It is determined whether the similarity at the time of determining the corresponding driving mode is below a predetermined threshold (for example, 0.9 when the maximum value of the similarity is 1). However, it is not limited to this. For example, when applying non-similarity at the time of determining the corresponding driving mode, it may be a method of making the above determination by determining whether the non-similarity is above a predetermined threshold (for example, 0.1 when the maximum value of the non-similarity is 1).

[0241] In step S4023, the central brain 120 updates the corresponding driving mode to the obtained driving mode (register the obtained driving mode) by replacing the corresponding driving mode with the obtained driving mode. Then, the central brain 120 ends the process of this flowchart. The processes of steps S4020 to S4023 are an example of the function of the registration unit.

[0242] In addition, of course, it is also possible to combine at least a part of the functions of the information processing device according to the first embodiment into the information processing device according to the present fourth embodiment to constitute the information processing device in the disclosed technology.

[0243] [Fifth Embodiment]

[0244] When the vehicle is in motion, there are situations where an obstacle approaches the vehicle. In such cases, as described above, it is preferable for the vehicle to change its behavior and travel to avoid contact or collision with the obstacle. As examples of obstacles, as described above, other vehicles, wall surfaces, guardrails, curbstones, and other installations other than the own vehicle in motion can be cited. In the following description, as an example of an obstacle, the case of another vehicle approaching the vehicle 12 will be described.

[0245] In addition, when the own vehicle travels using any one of the above-described multiple different driving modes, the size of the space between the own vehicle and another vehicle changes over time according to the movement of at least one of the own vehicle and the other vehicle.

[0246] Figure 20A schematically shows Figure 9 a diagram of the driving mode that can be applied to the own vehicle 12A and the predicted driving paths of the own vehicle 12A and the other vehicles 12D in the driving state showing the relationship between the own vehicle 12A and the other vehicles 12B, 12C, 12D shown. In the example of the figure, Figure 9 similar to

[0247] In Figure 20A the example shown, when the own vehicle 12A travels in the driving mode 12Ax2 while the other vehicle 12D travels in the driving mode 12Dx1, the space A1 between the own vehicle 12A and the other vehicle 12D narrows over time, and the possibility of a collision occurring finally increases significantly. On the other hand, even when the other vehicle 12D travels in the driving mode 12Dx1, when the own vehicle 12A travels in the driving mode 12Ax4, a distance for the vehicles to pass each other can be ensured for the space A2 between the own vehicle 12A and the other vehicle 12D. In addition, it is not necessary to apply the driving mode 12Ax4, and sometimes a collision including contact between the own vehicle 12A and the other vehicle 12D can be avoided by applying a modified example of the driving mode 12Ax2.

[0248] Therefore, the setting unit in the central brain 120 according to the present embodiment, as a driving strategy, applies a driving mode related to the driving path until reaching the destination of avoiding an obstacle (in the present embodiment, another vehicle), and sets the driving mode in which the size of the space that changes over time according to the movement of at least one of the own vehicle and the other vehicle between the own vehicle and the obstacle (in the present embodiment, another vehicle) among the multiple different driving modes to be the largest.

[0249] In addition, in Figure 20A the example shown, as an example of the space between this vehicle and other vehicles, a space that is circular when viewed from above is illustrated, but it is not limited thereto. For example, various spaces such as a space that is elliptical when viewed from above, a space that is triangular when viewed from above, a space that is rectangular when viewed from above, and a space that is a polygon with five or more sides when viewed from above can be applied as the space between this vehicle and other vehicles. In addition, in Figure 20A the example shown, the situation when viewed from above is illustrated, but it goes without saying that it can be applied as the above space not only in the horizontal direction but also in the height direction. That is, in the present embodiment, a driving strategy that uses not only a planar space but also a three-dimensional space is applied.

[0250] Figure 20B is a flowchart showing an example of the processing flow in the central brain 120 in which the above-described vehicle 12A according to the present embodiment can drive while avoiding other vehicles 12D. In Figure 20B it, the processing of step S11B shown in Figure 15 is replaced with steps S4011A and 11B2 and executed. The central brain 120 can repeatedly execute the processing shown in Figure 20B in place of the processing shown in Figure 15 For steps that perform the same processing as in the first embodiment, the same step numbers as in the first embodiment are given, and the description thereof is omitted here.

[0251] In step S4011B1, the central brain 120 derives the size of the above space for each of the driving modes (hereinafter referred to as "selected candidate driving modes") that can be selected as those capable of avoiding a collision between this vehicle and other vehicles, including modified examples. In addition, in the present embodiment, the derivation of the size of the above space is performed as follows: The position of this vehicle after a predetermined time (0.01 seconds in the present embodiment) when this vehicle travels in the driving mode as the object, and the position of other vehicles after this time are estimated, and the size of the above space is derived based on the estimated positions. However, it is not limited to this method. For example, it may be a method of using a conventionally known distance sensor such as a lidar (LiDAR) to successively derive the distance between this vehicle and other vehicles, and using the derived distance to derive the size of the above space.

[0252] In step S4011B2, the central brain 120 sets the selected candidate driving mode with the largest space size among the sizes of the spaces of each selected candidate driving mode derived as the driving mode to be actually applied.

[0253] The processes of steps S4011A to S4011C are an example of the functions of the calculation unit, and the processes of steps S4011A, S4011B1, and S4011B2 are an example of the functions of the setting unit.

[0254] In addition, of course, it is also possible to combine at least part of the functions of the information processing apparatus according to the first embodiment into the information processing apparatus according to the fifth embodiment, thereby constituting the information processing apparatus in the disclosed technology.

[0255] [Sixth Embodiment]

[0256] When the vehicle is traveling, there are cases where an obstacle approaches the vehicle. In such cases, as described above, it is preferable for the vehicle to change its behavior and travel to avoid contact or collision with the obstacle. As examples of the obstacle, as described above, other vehicles other than the own vehicle traveling, a wall surface, a guardrail, a curb, and other installations can be cited. In the following description, as an example of the obstacle, the case where another vehicle approaching the vehicle 12 is applied will be described.

[0257] In addition, even when it is predicted that the own vehicle can avoid a collision including contact with other vehicles by means of multiple driving modes among the above-described multiple different driving modes, the level of the risk of collision with other vehicles is different for each of the multiple driving modes.

[0258] Figure 21A is schematically shown in Figure 9 the driving state showing the relationship between the own vehicle 12A and other vehicles 12B, 12C, and 12D shown, the driving modes applicable to the own vehicle 12A, and the predicted driving paths of the own vehicle 12A and other vehicles 12D. In the example of the figure, similar to Figure 9 the same, other vehicle 12B is traveling behind the own vehicle 12A, other vehicle 12D is traveling ahead in the oncoming lane, and other vehicle 12C is traveling behind other vehicle 12D.

[0259] In Figure 21A the example shown, by adopting the driving mode 12Ax1 as the path of the own vehicle 12A, the own vehicle 12A will collide with other vehicle 12D. On the other hand, it is predicted that by adopting the driving modes 12Ax2, 12Ax3, and 12Ax4, the own vehicle 12A can avoid a collision including contact with other vehicle 12D. Therefore, the central brain 120 calculates the driving modes 12Ax2, 12Ax3, and 12Ax4 respectively and sets any one of them as the driving mode.

[0260] However, in this case, for example, the presence of pedestrians 5030 walking on the sidewalk, other vehicles 12B, and other vehicles 12C is not considered. For example, in Figure 21A in the case of the example shown, when applying driving mode 12Ax3 or driving mode 12Ax4, compared with other driving modes, it is closer to pedestrian 5030, and the overall risk is enlarged. In addition, among driving mode 12Ax3 and driving mode 12Ax4, the risk of driving mode 12Ax4 is also higher in this case. Additionally, in the case of applying driving mode 12Ax2, compared with other driving modes, it is closer to other vehicle 12C, and the risk is higher in this regard. In this case, since it is ultimately also close to other vehicle 12B, the risk is also higher in this sense.

[0261] Therefore, the setting unit in the central brain 120 according to the present embodiment applies, as a driving strategy, a driving mode related to the driving path to reach a destination that avoids other vehicles. Then, the setting unit sets, for a plurality of different driving modes, a low weighted value representing the collision risk including contact with other vehicles using the acquired plurality of pieces of information. Among the plurality of different driving modes, the larger the set weighted value, the more preferentially and selectively the driving mode is set. Additionally, in the present embodiment, a value in the range of 0 (zero) or more and 1 or less is used as the weighted value, but it goes without saying that it is not limited thereto.

[0262] Furthermore, the setting unit according to the present embodiment sets a smaller value as the weighted value when the present vehicle exceeds the driving lane compared to the case where it does not exceed the driving lane. Furthermore, the larger the size of the space between the present vehicle and other vehicles at the determination moment, the setting unit according to the present embodiment sets a larger value as the weighted value. Additionally, the method of setting the weighted value is not limited to the above method. For example, it may also be a method of setting a smaller weighted value in the case of exceeding the oncoming lane than in the case of exceeding the adjacent lane even when exceeding the driving lane.

[0263] Figure 21B is a flowchart showing an example of the processing flow in the central brain 120 according to the present embodiment, in which the present vehicle 12A can drive while avoiding other vehicle 12D. In Figure 21B it, the processing of step S11B shown in Figure 15 is replaced with steps S5011A and S5011B2 and executed. The central brain 120 may repeatedly execute the processing shown in Figure 21B instead of the processing shown in Figure 15 shown. Additionally, for steps that perform the same processing as in the first embodiment, the same step numbers as in the first embodiment are given, and the description thereof is omitted here.

[0264] In step S5011B1, the central brain 120 derives a weighted value indicating a low degree of the above risk for each driving mode (hereinafter referred to as "candidate driving mode for selection") that can be selected to avoid a collision between the present vehicle and other vehicles, including modified examples.

[0265] In addition, in the present embodiment, the derivation of the above weighted value is performed as follows: when the present vehicle travels in the driving mode as an object and the present vehicle exceeds the driving lane, a smaller value is derived compared to the case where it does not exceed the driving lane. Further, in the present embodiment, the derivation of the above weighted value is performed as follows: when the present vehicle travels in the driving mode as an object, the larger the size of the space between the present vehicle and other vehicles, the larger the value derived. In the present embodiment, among these two weighted values, the smaller one is applied, but it is not limited thereto. For example, it may also be a method of applying the average value of these two weighted values.

[0266] In addition, the method of deriving the weighted value is not limited to these methods. For example, it may also be as follows: when the present vehicle travels in the driving mode as an object, the closer the distance between the present vehicle and other obstacles such as other vehicles or pedestrians at the closest moment, the smaller the value derived, thereby deriving the above weighted value. In addition, for example, it may also be as follows: when the present vehicle travels in the driving mode as an object, compared with the case where the present vehicle goes straight, the more the number of approaching obstacles, the smaller the value derived, thereby deriving the above weighted value.

[0267] In step S5011B2, the central brain 120 sets the candidate driving mode for selection having the maximum weighted value among the weighted values of each candidate driving mode for selection derived as the driving mode to be actually applied.

[0268] The processing of steps S5011A to S5011C is an example of the function of the calculation unit, and the processing of steps S5011A, S5011B1, and S5011B2 is an example of the function of the setting unit.

[0269] In addition, of course, it may also be a method of combining at least a part of the functions of the information processing device according to the first embodiment into the information processing device according to the sixth embodiment to constitute the information processing device in the disclosed technology.

[0270] In addition, in the present embodiment, as the weighted value, a value indicating a low degree of risk is applied, but it is not limited thereto. For example, as the weighted value, it may also be a method of applying a value indicating a high degree of risk. In this case, in the candidate driving mode for selection, the smaller the weighted value set by the setting unit, the more preferentially the driving mode is selectively set.

[0271] [Seventh Embodiment]

[0272] Next, the seventh embodiment will be described. Since the seventh embodiment has the same structure as the first embodiment, the same parts are given the same reference numerals, and their detailed description is omitted. In addition, the parts different between the seventh embodiment and the first embodiment will be described.

[0273] In this embodiment, according to the current situation, a driving route for the vehicle to travel along the optimal route is set. Specifically, the case where the driving route set as the above driving strategy is set as the driving mode will be described.

[0274] The central brain 120 functions as a control unit that uses the information acquired by the acquisition unit to perform driving control of the vehicle to avoid obstacles such as other vehicles that pose an obstacle to the present vehicle. That is, information indicating the situation around the present vehicle detected by the above sensors is acquired, such as information about other vehicles around the present vehicle, information about obstacles such as other vehicles that pose an obstacle to the travel of the present vehicle, road information indicating the road conditions on which the vehicle 12 travels (for example, the material of the road, the up / down / horizontal / oblique inclination angle of the slope, the freezing mode of the road, and the moisture content of the road), etc., and thus driving control of the vehicle to avoid obstacles is performed.

[0275] In addition, the central brain 120 functions as a control unit that calculates control variables for controlling vehicle behaviors such as vehicle speed and controls the autonomous driving of the vehicle in units of one billionth of a second based on the control variables. Specifically, the central brain 120 controls the behavior of the vehicle according to the driving mode set by the following relational expression. Then, the vehicle travel that can avoid obstacles is achieved.

[0276] The relational expression represents the driving trajectory that a vehicle in motion can travel by changing the vehicle's behavior. That is, when the vehicle avoids an obstacle, it avoids the obstacle from the current position and travels to the destination (for example, the position of the current driving lane after a predetermined time) along a curved path (turn). The driving trajectory of this turn is represented by an expression. As an example of this relational expression, Equation (3) representing a quadratic curve as shown below can be applied.

[0277] [Expression 3]

[0278] y = ax 2 (3)

[0279] In the formula, a is a proportionality constant, which is an example of a control variable. As the proportionality constant changes, the radius of curvature of the curve also changes. In addition, the driving direction changes according to the sign (+ / -). Through this expression, a driving pattern that can avoid obstacles can be represented. Based on the above sensor information, the central brain 120 transmits, as the behavior of the vehicle corresponding to the proportionality constant derived through target search, to the steering control and then correctly transmits to the in-wheel motors and the four rotation angles respectively mounted on the four wheels 21, thereby achieving perfect speed control for acceleration and deceleration and perfect steering control based on the optimal driving direction.

[0280] Figure 22A is a conceptual diagram showing the driving pattern based on a representative proportionality constant. Figure 22B is a conceptual diagram showing the driving pattern based on a combination of multiple proportionality constants.

[0281] The proportionality constant a can be classified and applied as a plurality of representative values corresponding to representative driving patterns based on different radii of curvature. In the present embodiment, as Figure 22A shown, the representative proportionality constants can be set to four levels of proportionality constants including the proportionality constant corresponding to straight travel, 0, S, M, L (L corresponds to, for example, the driving pattern with the maximum radius of curvature). In this way, the proportionality constant performs acceleration and deceleration through the in-wheel motors respectively mounted on the four wheels 21, and performs steering control through the mechanism for controlling the vehicle steering angle. Then, for example, as Figure 22B shown, by combining multiple proportionality constants, the speed and driving direction of the vehicle can be variably driven.

[0282] Just setting the driving pattern of the above four levels of proportionality constants of 0, S (Small), M (Middle), and L (Large) as the driving pattern for the vehicle to avoid obstacles may be insufficient. Therefore, in the present embodiment, for each level of proportionality constant of S, M, and L, a predetermined number (for example, 10) of proportionality constants corresponding to each of the driving patterns of a size similar in shape to the driving pattern are preset. Each of the driving patterns of a size similar in shape can be determined according to the speed of the vehicle.

[0283] That is, 10 driving patterns of a size similar in shape to the basic S curve are preset. Similarly, for the M curve and the L curve, 10 driving patterns of a size similar in shape are also preset respectively. Moreover, it is only necessary to select one driving pattern from a total of 30 driving patterns that can avoid collision with the other vehicle 12D. Specifically, a proportionality constant that can avoid collision with the other vehicle 12D can be selected from the proportionality constants corresponding to each of the 30 driving patterns. The above basic S curve, M curve, and L curve correspond to Figure 22AThe driving modes shown are S, M, and L.

[0284] The above driving modes can also be determined based on whether the other vehicle 12D that is an obstacle to the vehicle 12A is in an accelerating driving state or a decelerating driving state. For example, 10 driving modes of a size similar to the S curve of the basic shape can be determined based on the accelerating driving state and decelerating driving state of the vehicle. The M curve and L curve can be determined in the same way. In this case, there are a total of 60 driving modes, but only one can be selected from 30 driving modes according to the accelerating and decelerating driving states.

[0285] In addition, when the vehicle 12A travels while avoiding the other vehicle 12D that is an obstacle, the central brain 120 can consider the subsequent vehicle of the other vehicle 12D, that is, the other vehicle 12C. Specifically, when traveling while avoiding the other vehicle 12D that is an obstacle, in order to avoid contact (secondary obstacle) with the other vehicle 12C that is subsequent, it is required that the vehicle 12A quickly return to the driving lane (driving road) during driving. In this case, it is only necessary to estimate the speed of the other vehicle 12C after the other vehicle 12D and the distance between the vehicle 12A and the other vehicle 12C. Then, the driving mode can be readjusted (selected) based on whether the other vehicle 12C after the other vehicle 12D is in an accelerating driving state or a decelerating driving state. In addition, the speed and acceleration / deceleration of the other vehicle 12D that is an obstacle can be estimated, and based on the speed of the other vehicle 12D, it can be estimated whether the subsequent other vehicle 12C transfers to a decelerating driving state, and the driving mode can be maximally adjusted (selected). That is, although the surrounding environment of the vehicle 12A changes all the time, the driving mode can be changed every one billionth of a second, and the behavior of the vehicle can be controlled according to the surrounding environment of the vehicle 12A to enable appropriate driving.

[0286] The driving mode selected according to the speed of the above other vehicle is not limited to the acceleration and deceleration of the vehicle, and can also include the driving state during constant speed driving, and a part of the acceleration and deceleration driving mode can also be used as the driving mode of the constant speed driving state.

[0287] In addition, the above relational expression is not limited to formula (3), and can also be a complex expression such as a multi-dimensional formula and a polynomial formula. In addition, the above representative proportional constants are not limited to 4 levels.

[0288] Next, the processing in the central brain 120 according to this embodiment will be described.

[0289] The central brain 120 acquires the sensor information detected by the sensor ( Figure 3 step S10), calculates the control variable ( Figure 3 step S11), and controls the autonomous driving ( Figure 3Step S12). Specifically, the central brain 120 derives a driving strategy for the vehicle 12 based on the sensor information ( Figure 15 Step S11A). The optimal driving strategy (driving mode) is set from the derived driving strategy ( Figure 15 Step S11B). That is, one driving mode (scaling factor) that can avoid obstacles (e.g., other vehicles 12D) is selected from a predetermined number (e.g., 30 types) of driving modes of the S-curve, M-curve, and L-curve. Next, the control variable corresponding to the set driving strategy (selected driving mode) is calculated ( Figure 15 Step S11C).

[0290] The above driving strategy (driving mode) can be changed according to the situation around the vehicle that changes moment by moment. Therefore, the central brain 120 controls the behavior of the vehicle, such as the steering and speed of the vehicle, based on the control variable calculated according to the prediction of collisions including the above contact, so as to avoid collisions including contact and reduce the damage to the vehicle during collisions for autonomous driving.

[0291] In the present embodiment, according to the situation around the own vehicle that changes moment by moment, for the driving state that the own vehicle should take, the driving path to be taken is represented by an expression indicating a predetermined curve (turn), and a path based on this expression is selected as the driving mode for autonomous driving. Therefore, the central brain 120 can change the driving mode, for example, every one billionth of a second, according to the situation around the vehicle that changes moment by moment, and can make decisions flexibly according to the situation around the vehicle to enable appropriate driving.

[0292] Figure 16 An example of the hardware configuration of the computer 1200 that functions as the central brains 120, 2120, and 3120 is schematically shown. The program installed in the computer 1200 enables the computer 1200 to function as one or more "parts" of the device according to the present embodiment, or enables the computer 1200 to perform operations associated with the device according to the present embodiment or the one or more "parts", and / or enables the computer 1200 to execute the process according to the present embodiment or a stage of the process. Such a program can be executed by the CPU 1212 to enable the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described in this specification.

[0293] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216 that are interconnected via a host controller 1210. The computer 1200 also includes an input / output unit such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive can be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 can be a hard disk drive, a solid state drive, etc. The computer 1200 also includes a ROM 1230 and a conventional input / output unit such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0294] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 from a frame buffer provided in the RAM 1214 or within itself, and causes the image data to be displayed on the display device 1218.

[0295] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 within the computer 1200. The DVD drive reads programs or data from a DVD-ROM, etc., and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card, and / or writes programs and data to the IC card.

[0296] The ROM 1230 stores therein a boot program executed by the computer 1200 at startup, etc., and / or a program dependent on the hardware of the computer 1200. The input / output chip 1240 can also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0297] The program is provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The program is read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the aforementioned various types of hardware resources. The apparatus or method can be configured by implementing operations or processing of information according to the use of the computer 1200.

[0298] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214, and based on the processes described in the communication program, command the communication interface 1222 to perform communication processing. Under the control of the CPU 1212, the communication interface 1222 reads the transmission data stored in the transmission buffer provided in a recording medium such as the RAM 1214, the storage device 1224, the DVD-ROM, or the IC card, and transmits the read transmission data to the network, or writes the received data received from the network into the reception buffer provided on the recording medium, etc.

[0299] In addition, the CPU 1212 may cause all or a necessary part of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive (DVD-ROM), the IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. Next, the CPU 1212 may write the processed data back to the external recording medium.

[0300] Various types of information such as various types of programs, data, tables, and databases may be stored in the recording medium to undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, and write the results back to the RAM 1214. The various types of processing include various types of operations, information processing, conditional judgment, conditional branch, unconditional branch, information retrieval / replacement, etc. described throughout this disclosure and specified by an instruction sequence of a program. In addition, the CPU 1212 may retrieve information in files, databases, etc. within the recording medium. For example, in a case where there are a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute stored in the recording medium, the CPU 1212 may retrieve an entry that matches the condition specifying the attribute value of the first attribute from the plurality of entries, and read the attribute value of the second attribute stored in the entry, thereby obtaining the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0301] The programs or software modules described above may be stored in a computer-readable storage medium on or near the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a computer-readable storage medium, thereby providing a program to the computer 1200 via the network.

[0302] The blocks in the flowcharts and block diagrams in this embodiment may represent stages of a process of performing operations or "parts" of a device having the function of performing operations. Specific stages and "parts" may be implemented by dedicated circuits, programmable circuits supplied together with computer-readable instructions stored on a computer-readable storage medium, and / or processors supplied together with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuits may include digital and / or analog hardware circuits, and may also include integrated circuits (ICs) and / or discrete circuits. The programmable circuits may include, for example, reconfigurable hardware circuits such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), and the reconfigurable hardware circuits include logical AND, logical OR, logical exclusive OR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, and storage elements.

[0303] A computer-readable storage medium may include any tangible device capable of storing instructions executable by an appropriate device. As a result, a computer-readable storage medium having instructions stored therein has a product including instructions that can be executed to generate units for performing the operations specified in the flowchart or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy (registered trademark) disks, magnetic disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disc read-only memories (CD-ROMs), digital versatile discs (DVDs), Blu-ray (registered trademark) disks, memory sticks, integrated circuit cards, etc.

[0304] Computer-readable instructions can include any one of assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code described in any combination of one or more programming languages, the one or more programming languages including object-oriented programming languages such as Smalltalk (registered trademark), JAVA (registered trademark), C++, etc. and traditional procedural programming languages such as the "C" programming language or similar programming languages.

[0305] The computer-readable instructions can be provided locally or via a local area network (LAN), a wide area network (WAN) such as the Internet, etc. to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, causing the processor or programmable circuit of the general-purpose computer, special-purpose computer, or other programmable data processing device to execute the computer-readable instructions to generate units for performing the operations specified in the flowchart or block diagram. As examples of the processor, include a computer processor, a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, etc.

[0306] As described above, the technology of the present disclosure has been described using embodiments, but the technical scope of the present disclosure is not limited to the scope described in the above embodiments. Those skilled in the art should be clear that various changes or improvements can be made to the above embodiments. As can be seen from the claims, embodiments with such changes or improvements can also be included in the technical scope of the present disclosure.

[0307] It should be noted that the execution order of each process such as actions, sequences, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is not particularly specified as "before...", "earlier than...", etc., or as long as the output of the previous process is not used in the subsequent process, it can be implemented in any order. Regarding the action flowcharts in the claims, the specification, and the drawings, even if they are described using "first", "then", etc. for convenience, it does not mean that they must be implemented in that order.

[0308] The disclosures of Japanese Patent Application No. 2022-170165, Japanese Patent Application No. 2022-182131, Japanese Patent Application No. 2022-186645, Japanese Patent Application No. 2023-000629, Japanese Patent Application No. 2023-000630, Japanese Patent Application No. 2023-002663, Japanese Patent Application No. 2023-004746, Japanese Patent Application No. 2023-008289, and Japanese Patent Application No. 2023-008290 are hereby incorporated by reference in their entirety into this specification.

[0309] All documents, patent applications, and technical standards cited in this specification are hereby incorporated by reference into this specification to the same extent as if each such document, patent application, and technical standard were specifically and individually incorporated by reference.

Claims

1. An information processing device, the information processing device comprising: An acquisition unit that, as a period for detecting the condition of the surroundings of the vehicle, acquires, from a detection unit including sensors, a plurality of pieces of information related to the vehicle at a second period shorter than a first period for photographing the surroundings of the vehicle, the sensors detecting the condition of the surroundings of the vehicle including obstacles; A calculation unit that includes a setting unit, the setting unit setting a driving strategy for the vehicle to avoid the obstacles based on the acquired plurality of pieces of information, and calculating a control variable for controlling the behavior of the vehicle based on the acquired plurality of pieces of information and the set driving strategy; And A control unit that controls the behavior of the vehicle based on the calculated control variable.

2. The information processing device according to claim 1, wherein the calculation unit calculates the control variable by multivariate analysis based on an integration method using deep learning.

3. The information processing device according to claim 1, wherein the acquisition unit acquires the plurality of pieces of information in units of one billionth of a second; the calculation unit calculates the control variable using the information acquired in units of one billionth of a second; the control unit controls the behavior of the vehicle in units of one billionth of a second for the control variable.

4. The information processing device according to claim 1, wherein the setting unit applies a driving mode related to a driving path to reach a destination for avoiding the obstacle as the driving strategy, and sets a driving mode among a plurality of different driving modes in which the size of the space between the vehicle and the obstacle exceeds a predetermined value determined in advance.

5. The information processing device according to claim 4, wherein the calculation unit calculates the control variable so that the vehicle travels according to the set driving mode.

6. The information processing device according to claim 5, wherein the control variable is the speed of the vehicle and the timing for changing the speed of the vehicle.

7. The information processing device according to claim 1, wherein the information processing device further includes a transmission unit, and in a case where it is determined based on the control variable calculated by the calculation unit that the vehicle performs an emergency behavior, the transmission unit transmits the control variable to other vehicles existing around the vehicle.

8. The information processing device according to claim 1, wherein the information processing device further includes a transmission unit, and in a case where it is determined based on the control variable calculated by the calculation unit that the vehicle performs an emergency behavior, the transmission unit transmits a signal indicating that the vehicle performs an emergency behavior to other vehicles existing around the vehicle.

9. The information processing device according to claim 1, wherein the calculation unit calculates the control variable by multivariate analysis based on an integration method using deep learning.

10. An information processing device, the information processing device comprising: An acquisition unit that, as a cycle for detecting the situation around the vehicle, uses a second cycle shorter than a first cycle for photographing the surroundings of the vehicle, and acquires a plurality of pieces of information related to the vehicle from a detection unit including sensors, where the sensors detect the situation around the vehicle including obstacles; A calculation unit that includes a setting unit. The setting unit sets a driving strategy for the vehicle to avoid the obstacle based on the acquired plurality of pieces of information, and calculates a control variable for controlling the behavior of the vehicle based on the acquired plurality of pieces of information and the set driving strategy; A control unit that controls the behavior of the vehicle based on the calculated control variable; A receiving unit that, when the control unit determines that there is a possibility of collision with the obstacle and the obstacle is another vehicle, receives a plurality of pieces of information related to the other vehicle, including the situation around the other vehicle, acquired by the other vehicle through sensors; And A transmitting unit that calculates, at least based on a plurality of pieces of information related to the other vehicle and a plurality of pieces of information related to the vehicle, a control variable for controlling the behavior of the other vehicle through the calculation unit, and transmits the calculated control variable for controlling the behavior of the other vehicle to the other vehicle.

11. The information processing device according to claim 10, wherein The control variable for controlling the behavior of the other vehicle is calculated based on at least a plurality of pieces of information related to the other vehicle, a plurality of pieces of information related to the vehicle, and a control variable for controlling the behavior of the vehicle.

12. The information processing device according to claim 10, wherein The calculation unit calculates the control variable through multivariate analysis based on an integration method using deep learning.

13. An information processing device, the information processing device includes: A registration unit that registers different multiple driving modes, where the multiple driving modes are driving modes that the vehicle has performed in the past to avoid obstacles; An acquisition unit that acquires a plurality of pieces of information related to the vehicle from a detection unit including sensors, where the sensors detect the situation around the vehicle including the obstacle; A calculation unit that includes a setting unit. The setting unit uses the acquired plurality of pieces of information to selectively set a driving mode for the vehicle to avoid the obstacle from the registered multiple driving modes, and calculates a control variable for controlling the behavior of the vehicle based on the acquired plurality of pieces of information and the set driving mode, so that the vehicle travels according to this driving mode; And A control unit that controls the behavior of the vehicle based on the calculated control variable.

14. The information processing device according to claim 13, wherein The registration unit registers the driving mode for each vehicle as the control object or for each vehicle model of the vehicle.

15. The information processing device according to claim 13, wherein The obstacle is another vehicle other than the vehicle.

16. An information processing device, the information processing device comprising: An acquisition unit that acquires a plurality of pieces of information related to a vehicle from a detection unit including a sensor, the sensor detecting the condition around the vehicle including an obstacle; A calculation unit that includes a setting unit, the setting unit sets a driving strategy for the vehicle to avoid the obstacle based on the acquired plurality of pieces of information, and calculates a control variable for controlling the behavior of the vehicle based on the acquired plurality of pieces of information and the set driving strategy ； And A control unit that controls the behavior of the vehicle based on the calculated control variable; The setting unit applies a driving mode related to a driving path to reach a destination for avoiding the obstacle as the driving strategy, and sets, among a plurality of different driving modes, the driving mode in which the size of the space that changes over time according to the movement of at least one of the vehicle and the obstacle between the vehicle and the obstacle is the largest.

17. The information processing device according to claim 16, Wherein, The obstacle is another vehicle other than the vehicle.

18. An information processing device, the information processing device comprising: An acquisition unit that acquires a plurality of pieces of information related to a vehicle from a detection unit including a sensor, the sensor detecting the condition around the vehicle including an obstacle; A calculation unit that includes a setting unit, the setting unit sets a driving strategy for the vehicle to avoid the obstacle based on the acquired plurality of pieces of information, and calculates a control variable for controlling the behavior of the vehicle based on the acquired plurality of pieces of information and the set driving strategy ； And A control unit that controls the behavior of the vehicle based on the calculated control variable; The setting unit applies a driving mode related to a driving path to reach a destination for avoiding the obstacle as the driving strategy, and for a plurality of different driving modes, uses the acquired plurality of pieces of information to set a weighting value corresponding to a collision risk including contact with the obstacle, and selectively sets a driving mode from the plurality of different driving modes according to the set weighting value.

19. The information processing device according to claim 16, Wherein, The weighting value is a value indicating a low degree of the risk or a value indicating a high degree of the risk.

20. The information processing device according to claim 18, Wherein, The obstacle is another vehicle other than the vehicle.

21. The information processing device according to claim 18, Wherein, When the vehicle exceeds the driving lane, the setting unit sets a higher value of the risk as the weighting value compared to the case where it does not exceed the driving lane.

22. The information processing device according to claim 18, Wherein, The larger the size of the space between the vehicle and the obstacle, the lower the value of the risk set by the setting unit as the weighting value.

23. A program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 22.

Images