Information processing equipment and vehicles

Abstract

To provide a vehicle having an automatic driving function which protects an occupant at the time of collision with another vehicle or an object and of secondary collision.SOLUTION: An information processor includes: an acquisition part for acquiring a plurality of information relating to a vehicle from a detection part including a sensor for detecting peripheral situations of the vehicle; a calculation part for calculating an index value relating to the peripheral situations of the vehicle from the plurality of acquired information, and calculating a control variable for controlling the behavior of the vehicle from the calculated index value; and a control part for controlling the behavior of the vehicle, on the basis of the calculated control variable. The calculation part predicts collision of an object with the vehicle, on the basis of the plurality of acquired information, and calculates such a control variable that a damage generated in the vehicle becomes a damage equal to or less than a predetermined threshold, in inevitable collision and re-collision colliding with another object after the collision, when the predicted prediction result indicates the inevitable collision.SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a vehicle.

Background Art

[0002] Patent Document 1 describes a vehicle having an automatic driving function.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to provide an information processing apparatus and a vehicle that protect an occupant in a vehicle having an automatic driving function during a collision with another vehicle or object and a secondary collision.

Means for Solving the Problems

[0005] A first aspect is an information processing apparatus including: an acquisition unit that acquires a plurality of pieces of information related to the vehicle from a detection unit including sensors that detect the situation around the vehicle; a calculation unit that calculates an index value related to the situation around the vehicle from the acquired plurality of pieces of information and calculates a control variable for controlling the behavior of the vehicle from the calculated index value; and a control unit that controls the behavior of the vehicle based on the calculated control variable, wherein the calculation unit predicts a collision of an object with respect to the vehicle based on the acquired plurality of pieces of information, and when the predicted prediction result indicates an inevitable collision, calculates a control variable such that damage occurring to the vehicle in the inevitable collision and a secondary collision that collides with another object after the collision becomes damage equal to or less than a predetermined threshold value.

[0006] In the second embodiment of the information processing device, the acquisition unit acquires information relating to the conditions around the vehicle from another vehicle which is the object to be hit.

[0007] The third embodiment of the information processing device is an information processing device of the first or second embodiment in which the acquisition unit acquires information relating to the conditions around the vehicle from an external device installed on the road on which the vehicle travels.

[0008] The fourth embodiment of the information processing device is an information processing device according to any one of the first to third embodiments, wherein the calculation unit calculates the control variable from the index value by multivariate analysis using an integral method with deep learning.

[0009] The fifth embodiment of the information processing device is an information processing device of any one of the first to fourth embodiments, wherein the damage occurring to the vehicle is at least one of the deformation position and deformation amount of the vehicle.

[0010] The sixth embodiment of the information processing device is an information processing device according to any one of the first to fifth embodiments, wherein the control variable corresponding to the damage occurring to the vehicle is at least one of the vehicle collision angle and vehicle speed.

[0011] The seventh embodiment is a vehicle comprising an information processing device of the third embodiment, a sensor connected to the information processing device for detecting the conditions around the vehicle, and a receiving unit for receiving the plurality of pieces of information from the external device.

[0012] It should be noted that the above summary of the invention does not list all the necessary features of the present invention. Furthermore, subcombinations of these features may also constitute an invention. [Brief explanation of the drawing]

[0013] [Figure 1] This diagram schematically illustrates the hazard prediction capability of the AI ​​for ultra-high-performance autonomous driving according to the first embodiment. [Figure 2]This diagram schematically shows an example of the network configuration inside a vehicle according to the first embodiment. [Figure 3] This is a flowchart executed by Central Brain according to the first embodiment. [Figure 4] This is the first explanatory diagram illustrating an example of autonomous driving control by Central Brain according to the first embodiment. [Figure 5] This is a second explanatory diagram illustrating an example of autonomous driving control by Central Brain according to the first embodiment. [Figure 6] This is a third explanatory diagram illustrating an example of autonomous driving control by Central Brain according to the first embodiment. [Figure 7] This is a fourth explanatory diagram illustrating an example of autonomous driving control by Central Brain according to the first embodiment. [Figure 8] This is a fifth explanatory diagram illustrating an example of autonomous driving control by Central Brain according to the first embodiment. [Figure 9] As a sixth explanatory diagram illustrating an example of autonomous driving control by Central Brain according to the first embodiment, this is a schematic diagram showing a state in which other vehicles are driving around the vehicle. [Figure 10] This is a block diagram showing an example of the configuration of an information processing device including a Central Brain according to the first embodiment. [Figure 11] This is a flowchart executed by Central Brain according to the first embodiment. [Figure 12] This is a diagram showing the configuration of a vehicle system according to the second embodiment. [Figure 13] This is a flowchart executed by Central Brain according to the second embodiment. [Figure 14] This diagram schematically shows an example of a computer hardware configuration that functions as a central brain. [Modes for carrying out the invention]

[0014] Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution means of the invention.

[0015] <First Embodiment> The information processing apparatus according to the first embodiment may obtain an index value necessary for driving control with high accuracy based on a lot of information related to vehicle control. Therefore, the information processing apparatus according to the present embodiment may be at least partially mounted on a vehicle and realize vehicle control.

[0016] Also, the information processing apparatus according to the present embodiment can be realized in real time based on data obtained by AI / multivariate analysis / goal seeking / strategy formulation / optimal probability solution / optimal speed solution / optimal course management / various sensor inputs at the edge for Autonomous Driving at Level 6, and can provide a driving system adjusted based on the delta optimal solution.

[0017] "Level 6" is a level representing autonomous driving and corresponds to a level higher than Level 5 representing fully autonomous driving. Although Level 5 represents fully autonomous driving, it is at the same level as human driving, and there is still a probability of accidents and the like occurring. Level 6 represents a level higher than Level 5 and corresponds to a level with a lower probability of accidents than Level 5.

[0018] The computing power at Level 6 is about 1000 times the computing power at Level 5. Therefore, high-performance driving control that could not be realized at Level 5 can be realized.

[0019] Figure 1 schematically shows the hazard prediction capability of the AI ​​for ultra-high-performance autonomous driving according to this embodiment. In this embodiment, multiple types of sensor information from multiple types of sensors acting as detection units is converted into AI data and stored in the cloud. The AI ​​predicts and determines the best mix of situations every nanosecond (one billionth of a second) and optimizes the operation of the vehicle 12.

[0020] Figure 2 is a schematic diagram showing an example of a vehicle 12 equipped with a Central Brain 120. As shown in Figure 2, the Central Brain 120 may have multiple Gateways connected in a communicative manner. The Central Brain 120 according to this embodiment can realize Level 6 autonomous driving based on multiple pieces of information acquired via the Gateways. The Central Brain 120 is an example of an information processing device.

[0021] As shown in Figure 2, multiple gateways are connected to the Central Brain 120 in a communicative manner. The Central Brain 120 is connected to an external cloud via the gateways. The Central Brain 120 is configured to be able to access the external cloud via the gateways. On the other hand, the presence of the gateways prevents direct access to the Central Brain 120 from the outside.

[0022] The Central Brain 120 outputs a request signal to the server at predetermined intervals. Specifically, the Central Brain 120 outputs a request signal representing a query to the server every one billionth of a second.

[0023] Examples of sensors provided in the vehicle 12 used in this embodiment include radar, LiDAR, high-resolution, telephoto, ultra-wide-angle, 360-degree, and high-performance cameras, vision recognition, subtle sound, ultrasound, vibration, infrared, ultraviolet, electromagnetic waves, temperature, humidity, spot AI weather forecasting, high-precision multi-channel GPS, low-altitude satellite information, and long-tail incident AI data. Long-tail incident AI data refers to trip data from a vehicle equipped with functions capable of achieving Level 5 autonomous driving.

[0024] The above sensor includes a sensor that detects the conditions around the vehicle. The sensor that detects the conditions around the vehicle detects the conditions around the vehicle with a second cycle that is shorter than the first cycle in which the area around the vehicle is photographed with a camera or the like.

[0025] Sensor information collected from multiple types of sensors includes weight center of gravity shift, road material detection, outside temperature detection, outside humidity detection, up-down, side-to-side, and diagonal inclination angle detection of slopes, road freezing conditions, moisture content detection, tire material, wear status, air pressure detection, road width, presence or absence of overtaking restrictions, oncoming vehicles, vehicle type information of vehicles in front and behind, cruising status of those vehicles, and surrounding conditions (birds, animals, soccer balls, accident vehicles, earthquakes, fires, wind, typhoons, heavy rain, light rain, blizzards, fog, etc.). In this embodiment, these detections are performed every 1 billionth of a second.

[0026] The Central Brain 120, which functions as an example of an information processing device according to this embodiment, includes at least the following functions: an acquisition unit capable of acquiring multiple pieces of information related to a vehicle; a calculation unit that calculates control variables from the multiple pieces of information acquired by the acquisition unit; and a control unit that performs vehicle driving control based on the control variables.

[0027] For example, the Central Brain 120 uses one or more sensor data detected by the above sensors to calculate the wheel speed and tilt of each of the vehicle's four wheels, as well as the wheel speed, tilt, and control variables for controlling each suspension supporting the wheels. Note that the wheel tilt includes both the tilt of the wheel relative to the axis horizontal to the road (i.e., steering angle) and the tilt of the wheel relative to the axis perpendicular to the road (i.e., camber angle).

[0028] Here, one or more sensor information can be applied from sensors that detect the conditions around the vehicle. Furthermore, when using multiple sensor information, a predetermined number of sensor information can be applied. For example, three sensor information points can be used. Based on these three sensor information points, index values ​​for controlling wheel speed, tilt (steering angle, camber angle), and suspension are calculated. The number of index values ​​calculated from the combination of three sensor information points is, for example, three. These index values ​​for controlling wheel speed, tilt (steering angle, camber angle), and suspension include, for example, index values ​​calculated from information regarding air resistance, index values ​​calculated from information regarding road resistance, and index values ​​calculated from information regarding the coefficient of slip.

[0029] Furthermore, the system aggregates index values ​​calculated for each combination of sensor information (either a single sensor or a combination of multiple sensor information items) to calculate control variables for controlling wheel speed, tilt (steering angle, camber angle), and suspension. For example, index values ​​are calculated from sensor information of a sensor that detects the surrounding conditions of the vehicle, and control variables are calculated from this index value. When using multiple sensor information, for example, multiple index values ​​are calculated for the combination of sensors 1, 2, and 3; multiple index values ​​are calculated for the combination of sensors 4, 5, and 6; and multiple index values ​​are calculated for the combination of sensors 1, 3, and 7. These index values ​​are then aggregated to calculate the control variables. In this way, a predetermined number of index values, for example 300, are calculated by changing the combination of sensor information, and the control variables are calculated. Specifically, the calculation unit may be capable of calculating control variables from sensor information using machine learning, more specifically, deep learning. In other words, the calculation unit that calculates index values ​​and control variables can be composed of AI (Artificial Intelligence).

[0030] The calculation unit uses Level 6 computing power to perform multivariate analysis using the integral method shown in equation (1) below (see, for example, equation (2)) on nanosecond-by-nanosecond data collected from a large group of sensors, thereby determining accurate control variables. More specifically, while calculating the integral values ​​of the delta values ​​of various Ultra High Resolution sensors using Level 6 computing power, it obtains indexed values ​​for each variable at the edge level and in real time, thereby obtaining the most probabilistic value for the result occurring in the next nanosecond.

number

number

[0031] Furthermore, while equations (1) and (2) above calculate the wheel speed (V), the tilt (steering angle, camber angle) and the control variables for controlling the suspension are calculated in the same manner.

[0032] In other words, the tilt (rudder angle R) can be accurately determined by performing multivariate analysis using the integral method shown in equation (3) below (see, for example, equation (4)).

[0033]

number

[0034]

number

[0035] Furthermore, the slope (camber angle C) can be accurately determined by performing multivariate analysis using the integral method shown in equation (5) below (see, for example, equation (6)).

[0036]

number

[0037]

number

[0038] Furthermore, the exact control variables for the suspension S can be determined by performing multivariate analysis using the integral method shown in equation (7) below (see, for example, equation (8)).

[0039]

number

[0040]

number

[0041] Specifically, the Central Brain 120 calculates the wheel speed of each of the four wheels, the inclination of each of the four wheels relative to an axis horizontal to the road (steering angle, camber angle), the inclination of each of the four wheels relative to an axis perpendicular to the road (steering angle, camber angle), and a total of 16 control variables for controlling the suspension supporting each of the four wheels. In this embodiment, the calculation of the above 16 control variables is performed every 1 billionth of a second.

[0042] Furthermore, the wheel speed of each of the four wheels mentioned above can also be described as "the number of spins (rotations) of the in-wheel motor mounted on each of the four wheels."

[0043] The inclination (steering angle) of each of the four wheels relative to the horizontal axis of the road can also be called the "horizontal angle of each of the four wheels."

[0044] The inclination (camber angle) of each of the four wheels relative to the axis perpendicular to the road can be called the "vertical angle of each of the four wheels."

[0045] The suspension (coil springs, shock absorbers) that determines the position of the wheels relative to the road can be described as "the amount of damping that absorbs the impact received from the road for each of the four wheels."

[0046] Furthermore, the control variables mentioned above are, for example, values ​​that enable optimal steering for a mountain road when the vehicle is driving on one, and values ​​that enable the vehicle to drive at the optimal angle for a parking lot when the vehicle is parking in a parking lot.

[0047] Furthermore, in this embodiment, the Central Brain 120 calculates the wheel speed of each of the four wheels, the tilt (steering angle) of each of the four wheels relative to an axis horizontal to the road, the tilt (camber angle) of each of the four wheels relative to an axis perpendicular to the road, and a total of 16 control variables for controlling the suspension supporting each of the four wheels. However, this calculation does not necessarily have to be performed by the Central Brain 120; a dedicated anchor chip for calculating the above control variables may be provided separately. In this case as well, DL in equation (2) indicates deep learning, and A, B, C, ..., N indicate index values ​​calculated from sensor information. If the number of indices to be aggregated is 300 as described above, the number of indices in the equation will also be 300.

[0048] Furthermore, in this embodiment, the Central Brain 120 functions as a control unit that controls automatic driving in units of one billionth of a second based on the control variables calculated above. Specifically, the Central Brain 120 controls the in-wheel motors mounted on each of the four wheels based on the 16 control variables above, thereby controlling the wheel speed, tilt (steering angle, camber angle) of each of the four wheels of the vehicle 12, and the suspension supporting each of the four wheels to perform automatic driving.

[0049] Furthermore, the index value calculated from sensor information may be calculated from the sensor information of a single sensor, for example, a sensor that detects the surrounding conditions of a vehicle, and the control variable may be calculated from the index value of that sensor.

[0050] The Central Brain 120 repeatedly executes the flowchart shown in Figure 3.

[0051] In step S10, the Central Brain 120 acquires sensor information, including road information detected by the sensors. Then, the Central Brain 120 proceeds to step S11. The processing in step S10 is an example of the function of the acquisition unit.

[0052] In step S11, the Central Brain 120 calculates the 16 control variables based on the sensor information acquired in step S10. Then, the Central Brain 120 proceeds to step S12. The process in step S11 is an example of the function of the calculation unit.

[0053] In step S12, the Central Brain 120 controls the automatic driving based on the control variables calculated in step S11. Then, the Central Brain 120 terminates the processing of the flowchart. The processing in step S12 is an example of the function of the control unit.

[0054] Figures 4 to 8 are explanatory diagrams illustrating examples of autonomous driving control by Central Brain 120. Figures 4 to 6 are explanatory diagrams showing the vehicle 12 from a frontal view, while Figures 7 and 8 are explanatory diagrams showing the vehicle 12 from a downward view.

[0055] Figure 4 shows the vehicle 12 traveling on a flat road R1. Based on the 16 control variables calculated according to the road R1, the Central Brain 120 controls the in-wheel motors 31 mounted on each of the four wheels 30, thereby controlling the wheel speed, tilt (steering angle, camber angle) of each of the four wheels 30, and the suspension 32 supporting each of the four wheels 30 to perform autonomous driving.

[0056] Figure 5 shows the vehicle 12 traveling on a mountain road R2. Based on the 16 control variables calculated according to the mountain road R2, the Central Brain 120 controls the in-wheel motors 31 mounted on each of the four wheels 30, thereby controlling the wheel speed, tilt (steering angle, camber angle) of each of the four wheels 30, and the suspension 32 supporting each of the four wheels 30 to perform autonomous driving.

[0057] Figure 6 shows the vehicle 12 driving through a puddle R3. Based on the 16 control variables calculated according to the puddle R3, the Central Brain 120 controls the in-wheel motors 31 mounted on each of the four wheels 30, thereby controlling the wheel speed, tilt (steering angle, camber angle) of each of the four wheels 30, and the suspension 32 supporting each of the four wheels 30 to perform automatic driving.

[0058] Figure 7 shows the case where the vehicle 12 curves in the direction indicated by arrow A1. Based on the 16 control variables calculated according to the curved road being entered, the Central Brain 120 controls the in-wheel motors 31 mounted on each of the four wheels 30, thereby controlling the wheel speed, tilt (steering angle, camber angle) of each of the four wheels 30, and the suspension 32 (not shown) supporting each of the four wheels 30 to perform automatic driving.

[0059] Figure 8 shows the case where the vehicle 12 moves in parallel in the direction indicated by arrow A2. Based on the 16 control variables calculated in accordance with the parallel movement in the direction indicated by arrow A2, the Central Brain 120 controls the in-wheel motors 31 mounted on each of the four wheels 30, thereby controlling the wheel speed, tilt (steering angle, camber angle) of each of the four wheels 30, and the suspension 32 (not shown) supporting each of the four wheels 30 to perform automatic driving.

[0060] It should be noted that the states (tilts) of the wheels 30 and suspension 32 shown in Figures 4 to 8 are merely examples, and it goes without saying that different states of the wheels 30 and suspension 32 may occur.

[0061] While conventional in-wheel motors in vehicles can independently control each drive wheel, this particular vehicle was unable to analyze road conditions and control the in-wheel motors accordingly. Therefore, for example, when driving on mountain roads or through puddles, this vehicle could not perform appropriate automated driving based on road conditions. However, according to the vehicle 12 of this embodiment, based on the configuration described above, it is possible to perform automated driving in which the speed, steering, etc., are controlled in a manner suitable for the environment, such as road conditions.

[0062] Incidentally, while a vehicle is in motion, obstacles may approach it. In this case, it is preferable for the vehicle to change its behavior to avoid contact or collision with the obstacle. Examples of obstacles include other vehicles other than the vehicle itself, walls, guardrails, curbs, and other installed objects. In the following explanation, we will describe the case where another vehicle approaches vehicle 12 as an example of an obstacle. Note that the obstacle is just one example of an object.

[0063] Figure 9 schematically shows a situation where vehicle 12 is traveling on a two-lane road with opposing traffic, with other vehicles 12B, 12C, and 12D traveling around it. In the example shown in the figure, vehicle 12B is following vehicle 12A, vehicle 12D is traveling ahead in the opposing lane, and vehicle 12C is following behind.

[0064] The Central Brain 120 of the vehicle 12A performs autonomous driving by controlling the in-wheel motors 31 mounted on each of the four wheels 30, based on the control variables described above which are calculated according to the constantly changing conditions of driving on the road. This controls the wheel speed, tilt (steering angle, camber angle) of each of the four wheels 30, as well as the suspension 32 that supports each of the four wheels 30. The Central Brain 120 also detects the behavior of other vehicles 12B, 12C, and 12D around the vehicle 12A using sensors and acquires this sensor information.

[0065] The Central Brain 120 has a function to predict collisions, including contact, with obstacles relative to the vehicle, based on acquired sensor information, in order to at least avoid obstacles while driving. As shown in Figure 9, if the Central Brain 120 predicts from the sensor information that another vehicle 12D will enter in front of the vehicle 12A, it calculates control variables that indicate the behavior of the vehicle 12A that will at least avoid contact with the other vehicle 12D.

[0066] For example, in the example shown in Figure 9, the current route 12Ax1 taken by vehicle 12A will result in a collision with other vehicle 12D. Therefore, Central Brain 120 calculates alternative routes such as 12Ax2 and 12Ax3 that avoid the collision with other vehicle 12D, selects one of them, and calculates the control variables. Route selection is simply a matter of choosing the route in which the load on vehicle 12A due to the control variables is lower than a predetermined value (for example, the minimum value).

[0067] The Central Brain 120, which enables the autonomous driving described above, will be explained further. The Central Brain 120 is configured as the information processing device 10 shown in Figure 10. Note that the Central Brain 120 described above is a broad-sense processing device that functions as an information processing device including a gateway, while the Central Brain 125, which will be described later, is a narrow-sense processing device when the functions are classified by processor.

[0068] Figure 10 is a block diagram showing an example of the configuration of an information processing device 10 including a Central Brain according to an embodiment. The information processing device 10 includes an IPU (Image Processing Unit) 121, a MoPU (Motion Processing Unit) 122, a Central Brain 125, and memory 126. The Central Brain 125 is configured to include a GNPU (Graphics Neural Network Processing Unit) 123 and a CPU (Central Processing Unit) 124.

[0069] The IPU121 can be integrated into an ultra-high-resolution camera (not shown) installed in the vehicle. The IPU121 performs predetermined image processing, such as Bayer transformation, demosaicing, noise reduction, and sharpening, on images of objects surrounding the vehicle, and outputs the processed images of the objects at a frame rate of, for example, 10 frames per second and a resolution of 12 million pixels. The images output from the IPU121 are supplied to the Central Brain125 and memory126.

[0070] The MoPU122 can be integrated into a low-resolution camera separate from the ultra-high-resolution camera installed on the vehicle. The MoPU122 outputs motion information indicating the movement of the captured object at a frame rate of, for example, 1920 frames per second. That is, the frame rate of the MoPU122 output is 100 times that of the IPU121 output. The MoPU122 outputs motion information that is vector information of the movement of a point indicating the location of the object along predetermined coordinate axes. In other words, the motion information output from the MoPU122 does not contain information necessary to identify what the captured object is (e.g., whether it is a person or an obstacle), but only information indicating the movement (direction of movement and speed of movement) on the coordinate axes (x axis, y axis, z axis) of the object's center point (or center of gravity). The image output from the MoPU122 is supplied to the Central Brain125 and memory126. By not including image information in the motion information, the amount of information transferred to the Central Brain125 and memory126 can be reduced.

[0071] This embodiment includes a first processor that outputs an image of a captured object at a first frame rate, and a second processor that outputs motion information indicating the movement of the captured object at a second frame rate higher than the first frame rate. That is, the detection unit that captures images of the area around the vehicle in a first cycle as the period for detecting the conditions around the vehicle in this embodiment is an example of the "first processor," and IPU121 is an example of the "first processor." Furthermore, the detection unit that includes sensors that detect the conditions around the front and rear in a second cycle shorter than the first cycle in this embodiment is an example of the "second processor," and MoPu122 is an example of the "second processor."

[0072] The Central Brain 125 performs vehicle driving control based on images output from the IPU 121 and motion information output from the MoPU 122. For example, the Central Brain 125 recognizes objects (people, animals, roads, traffic lights, signs, crosswalks, obstacles, buildings, etc.) around the vehicle based on images output from the IPU 121. The Central Brain 125 also recognizes the movement of the recognized objects around the vehicle based on motion information output from the MoPU 122. Based on the recognized information, the Central Brain 125 performs actions such as controlling the motors that drive the wheels (speed control), brake control, and steering control. In the Central Brain 125, the GNPU 123 may be responsible for image recognition processing, and the CPU 124 may be responsible for vehicle control processing.

[0073] Generally, ultra-high resolution cameras are used for image recognition in autonomous driving. It is possible to recognize what objects are contained in an image captured by a high-resolution camera. However, this alone is insufficient for autonomous driving in the Level 6 era. In the Level 6 era, it is also necessary to recognize the movement of objects. By recognizing the movement of objects, for example, it becomes possible for autonomous vehicles to perform evasive maneuvers to avoid obstacles with greater precision. However, high-resolution cameras can only acquire images at a rate of about 10 frames per second, making it difficult to analyze the movement of objects. On the other hand, cameras equipped with MoPU122, although lower resolution, can output at a high frame rate of, for example, 1920 frames per second.

[0074] Therefore, the technology of this embodiment uses two independent processors, IPU121 and MoPU122. The high-resolution camera (IPU121) is assigned the role of acquiring image information necessary to recognize what the captured object is, and the MoPU122 is assigned the role of detecting the movement of the object. The MoPU122 perceives the object as a point and analyzes in which direction and at what speed the coordinates of that point move along the x, y, and z axes. Since the overall contour of the object and the detection of what the object is can be done from the image from the high-resolution camera, if the MoPU122 can determine how the center point of the object moves, it can determine how the entire object behaves.

[0075] By analyzing only the movement and velocity of the object's center point, the amount of information transferred to the Central Brain 125 can be significantly reduced compared to determining how the entire image of the object moves, thereby drastically reducing the computational load on the Central Brain 125. For example, when sending a 1000x1000 pixel image to the Central Brain 15 at a frame rate of 1920 frames / second, including color information, 4 billion bits / second of data would be sent to the Central Brain 125. By having the MoPU 122 transmit only motion information indicating the movement of the object's center point, the amount of data transferred to the Central Brain 125 can be compressed to 20,000 bits / second. In other words, the amount of data transferred to the Central Brain 125 is compressed to 1 / 200,000th of the original amount.

[0076] In this way, by combining the low-frame-rate, high-resolution images output from IPU121 with the high-frame-rate, lightweight motion information output from MoPU122, it becomes possible to perform object recognition, including object motion, with a small amount of data.

[0077] Furthermore, when using one MoPU122, it is possible to obtain vector information of the movement of a point indicating the object's location along each of the two coordinate axes (x-axis and y-axis) in a three-dimensional Cartesian coordinate system. Alternatively, using the principle of a stereo camera, two MoPU122s can be used to output vector information of the movement of a point indicating the object's location along each of the three coordinate axes (x-axis, y-axis, and z-axis) in a three-dimensional Cartesian coordinate system. The z-axis is the axis along the depth direction (vehicle movement).

[0078] In this embodiment, as sensor information, it is possible to detect the conditions around the vehicle 12 at a shorter interval than the first interval for capturing images of the vehicle's surroundings with a camera, etc. That is, while Level 5 as described above can detect the surrounding conditions twice every 0.3 seconds using a camera, etc., in this embodiment Level 6 can detect the surrounding conditions 576 times. Then, an index value and control variables can be calculated for each of the 576 detections of the surrounding conditions, and the vehicle's behavior can be controlled to enable faster and safer driving than the autonomous driving performed at Level 5.

[0079] The above describes a case where a collision is avoided by selecting a path, but it is not limited to this. For example, if it is difficult for the vehicle 12A to avoid a collision with another vehicle, it is possible to calculate control variables that reduce the damage to the vehicle in an unavoidable collision. That is, the Central Brain 120 may calculate control variables that correspond to damage to the vehicle in an unavoidable collision that is below a predetermined threshold. Therefore, the calculation unit described above includes predicting a collision of an object with the vehicle 12 based on acquired sensor information, and if the predicted result indicates an unavoidable collision, calculating control variables that correspond to damage to the vehicle in an unavoidable collision that is below a predetermined threshold. At least one of the vehicle's deformation position and deformation amount can be applied to the damage to the vehicle. At least one of the vehicle's collision angle and vehicle speed can be applied to the control variables corresponding to the damage to the vehicle.

[0080] (Measures to mitigate damage) Figure 11 is a flowchart showing an example of the processing flow in the Central Brain 120 that enables the reduction of damage to the vehicle 12 in an unavoidable collision. In Figure 11, the process of step S11 shown in Figure 3 is performed in place of steps S11A to S11D. The Central Brain 120 can repeatedly perform the process shown in Figure 11 in place of the process shown in Figure 3.

[0081] In step S11A, the Central Brain 120 calculates the relationship between its own vehicle and other vehicles. Specifically, in step S11A, based on sensor information, it predicts a collision between the vehicle 12 and an object (for example, another vehicle), and determines whether the prediction result indicates an unavoidable collision. Then, in step S11B, it determines whether the judgment result from the prediction indicates an unavoidable collision. If the judgment in step S11B is positive, the process moves to step S11D; if the judgment is negative, control variables are calculated in step S11C, similar to step S11 in Figure 3 described above. In step S11D, as described above, control variables that reduce the damage to the vehicle in an unavoidable collision are calculated, and the process proceeds to step S12.

[0082] The processing in steps S11A, S11B, S11C, and S11D is not limited to the order described above. For example, steps S11B and S11D may be performed after step S11C. Specifically, first, in step S11C, a collision of vehicle 12 with another vehicle is predicted, and control variables are calculated that represent the relationship between the vehicle and the other vehicle that minimizes the risk. The relationship that minimizes the risk includes, for example, relationships that include contact or collision, and relationships that avoid such relationships, where the likelihood of a contact or collision is low. In other words, control variables with the lowest collision risk are calculated. Next, in step S11B, the behavior of vehicle 12 based on the calculated control variables is used to predict a collision of vehicle 12 with another vehicle, and it is determined whether the collision is unavoidable based on the prediction result. In this case, if the judgment in step S11B is negative, the process can proceed directly to step S12. On the other hand, if the judgment is positive, in step S11D, the control variables calculated in step S11C are used to calculate control variables that can further reduce the collision risk. Specifically, it calculates at least one of the vehicle collision angle and vehicle speed to reduce the damage to the vehicle caused by contact or collision between the vehicle and another vehicle.

[0083] Furthermore, the control variables for the vehicle 12's speed may be set to 10 types each for large, medium, and small (L, M, S) on the acceleration side, resulting in a total of 30 possible patterns, and 10 types each for large, medium, and small (L, M, S) on the deceleration side, resulting in a total of 30 possible patterns, which can then be selected. In this case, the relationship between the vehicle and other vehicles changes moment by moment, and when the vehicle and other vehicles are approaching each other, the distance decreases in the direction of decreasing proximity. As a result, the number of options for selecting a pattern decreases moment by moment, reducing the processing time for selection and further decreasing the adjustment of the delta difference.

[0084] Furthermore, although the above describes the case where vehicle 12D is applied as another vehicle to vehicle 12A, control variables that reduce the collision risk to the other vehicles may be calculated by targeting at least one of the other vehicles 12B, 12C, and 12D surrounding vehicle 12A, and all of the other vehicles mentioned above.

[0085] Therefore, the Central Brain 120 can avoid collisions or reduce damage to the vehicle in the event of a collision by controlling, for example, the in-wheel motors 31 mounted on each of the four wheels 30 based on control variables calculated in accordance with the prediction of a collision, including contact. This allows for autonomous driving by controlling the wheel speed, tilt (steering angle, camber angle) of each of the four wheels 30, and the suspension 32 supporting each of the four wheels 30.

[0086] <Second Embodiment> In the first embodiment, the damage mitigation process described above included an example where the vehicle 12A inevitably collides with other surrounding vehicles (for example, other vehicles 12B, 12C, 12D). In contrast, the second embodiment includes a process to mitigate damage not only to other vehicles but also to collisions with other objects after a collision with another vehicle. Here, re-collisions include secondary collisions, tertiary collisions, etc. The differences from the first embodiment will be described below. The same components will be denoted by the same reference numerals, and detailed explanations will be omitted.

[0087] Figure 12 shows a configuration diagram of the vehicle system 1 of this embodiment. As shown in Figure 12, the vehicle system 1 of this embodiment is composed of a plurality of vehicles 12, an information providing device 14, and a server 16. The vehicles 12 include the vehicle 12A and other vehicles 12B, 12C, and 12D that travel around the vehicle 12A. In this embodiment, each of the other vehicles 12B, 12C, and 12D is equipped with a Central Brain 120, but this is not limited to this. The information providing device 14 and the server 16 are examples of external devices.

[0088] The information providing device 14 is a monitoring unit equipped with a camera, radar, etc. The information providing device 14 is installed around the road on which the vehicle 12 travels, for example, on top of traffic lights, on lampposts, on gates supporting signs, etc.

[0089] Server 16 includes, for example, a map information server 16A and a weather information server 16B. Map information server 16A has a map information database and can provide detailed map information to the vehicle 12A as map information. A detailed map is a map that includes information on the terrain and structures around the road. Weather information server 16B can provide weather information for the present and the near future (for example, one hour ahead) to the vehicle 12A. Map information server 16A and weather information server 16B are connected to the vehicle 12A via a public network N. In Figure 12, map information server 16A and weather information server 16B are connected only to the vehicle 12A, but they are not limited to this and may also be connected to other vehicles 12B, 12C, and 12D.

[0090] (Measures to mitigate damage) Figure 13 is a flowchart illustrating an example of the processing flow in the Central Brain 120 that enables the reduction of damage to the vehicle 12A in the event of an unavoidable collision. Steps S20 and S21 are examples of the functions of the acquisition unit. Steps S22 to S27 are examples of the functions of the calculation unit. Furthermore, step S28 is an example of the functions of the control unit.

[0091] In step S20 of Figure 13, the Central Brain 120 acquires sensor information, including road information detected by the sensors.

[0092] In step S21, the Central Brain 120 acquires external information. Specifically, the Central Brain 120 acquires captured images of the surroundings of its own vehicle 12A, vehicle and object detection data, and information such as traffic signals from the information providing device 14. The Central Brain 120 also acquires map information from the map information server 16A and weather information from the weather information server 16B.

[0093] In step S22, the Central Brain 120 calculates the relationship between its own vehicle 12A and other vehicles and objects. Specifically, the Central Brain 120 calculates the position and speed of other vehicles relative to its own vehicle 12A, such as the type of vehicle (motorcycle, large vehicle, etc., vehicle size, and whether they are communication-capable vehicles (other vehicles 12B, 12C, 12D), etc.). The Central Brain 120 also calculates the type of objects on the road (people, animals, fallen objects, etc.), their position, and their speed relative to its own vehicle 12A.

[0094] In step S23, the Central Brain 120 determines whether its own vehicle 12A has already made contact with or collided with another vehicle or object. If the Central Brain 120 determines that its own vehicle 12A has already made contact with or collided with another vehicle or object (Y in step S23), it proceeds to step S27. On the other hand, if the Central Brain 120 determines that its own vehicle 12A has not already made contact with or collided with another vehicle or object, that is, it has not made contact or collided with any vehicle or object (N in step S23), it proceeds to step S24.

[0095] In step S24, the Central Brain 120 determines whether its own vehicle 12A is inevitably going to collide with another vehicle or object. If the Central Brain 120 determines that its own vehicle 12A is inevitably going to collide with another vehicle or object (Y in step S24), it proceeds to step S26. On the other hand, if the Central Brain 120 determines that its own vehicle 12A is not inevitably going to collide with another vehicle or object (N in step S24), it proceeds to step S25.

[0096] In step S25, the Central Brain 120 calculates the 16 control variables (Vn, Rn, Cn, Sn) based on the sensor information acquired in step S20 and the external information acquired in step S21. Then, the process proceeds to step S28.

[0097] In step S26, the Central Brain 120 calculates control variables (Vn, Rn, Cn, Sn) to reduce the damage to the vehicle 12A in an unavoidable collision, based on the sensor information acquired in step S20 and the external information acquired in step S21, as a damage mitigation mode. Then, the process proceeds to step S28.

[0098] If it is determined in step S23 that the vehicle 12A has already made contact or collided, step S27 is executed. In step S27, the Central Brain 120 calculates control variables (Vn, Rn, Cn, Sn) to reduce the damage to the vehicle 12A during a re-collision, based on the sensor information acquired in step S20 and the external information acquired in step S21, as a re-collision mode. Then, the process proceeds to step S28. In step S28, the Central Brain 120 controls the autonomous driving of its own vehicle 12A based on the calculated control variables. Then, the process returns to step S20.

[0099] According to this embodiment, similar to the first embodiment, if a collision is unavoidable for the vehicle 12A, control variables are calculated based on the relationship between the vehicle 12A and other vehicles or objects to minimize risk. Furthermore, even if the vehicle 12A has already made contact with or collided with other vehicles or objects, control variables are calculated based on the relationship between the vehicle 12A and other vehicles or objects to minimize risk. For example, if the vehicle 12A veers into the oncoming lane due to the recoil from a collision with another vehicle, control variables are calculated to minimize damage if the vehicle 12A collides with another vehicle again. Also, for example, if there is a cliff outside the road, control variables are calculated to keep the vehicle 12A on the road so that it does not fall off the cliff due to the recoil from a collision with another vehicle.

[0100] As described above, the information processing device 10 of this embodiment can precisely calculate the angle of incidence to the other object, the angle of reflection after the collision, the rebound velocity, the distance traveled, and the direction of movement when the vehicle 12A collides with the other object, thereby preventing secondary damage from subsequent collisions such as secondary and tertiary collisions. Depending on the object to be hit, the time until the collision, the surrounding environment, etc., it may be decided whether to limit the collision to a primary collision, a secondary collision, or allow a tertiary collision or beyond.

[0101] Furthermore, according to this embodiment, the vehicle takes into consideration other vehicles 12 (e.g., following vehicles) and other objects (e.g., pedestrians, animals, etc.) surrounding the vehicle 12A and the object it has collided with, and re-collides with the object. This ensures consideration for surrounding vehicles.

[0102] Furthermore, according to this embodiment, preparations for a re-collision can be made by considering the surrounding conditions of the vehicle 12A (for example, the presence of cliffs or guardrails). For example, the Central Brain 120 considers the surrounding conditions at the time of collision (whether it is a cliff, a guardrail, or a sidewalk) and calculates where the vehicle 12A and the other object will bounce and move after the collision. In particular, if the surroundings are cliffs or sidewalks with people, it calculates how to re-collide in a way that prevents the vehicle 12A and the other object from moving in that direction after the collision. If the surroundings are sidewalks or street trees without people, or guardrails, it calculates how to allow for some re-collision in that direction. After these calculations, the Central Brain 120 calculates control variables (Vn, Rn, Cn, Sn).

[0103] <Supplement> Figure 14 schematically shows an example of the hardware configuration of a computer 1200 that functions as a Central Brain 120. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the apparatus according to this embodiment, or to cause the computer 1200 to execute operations associated with the apparatus according to this embodiment or such one or more "parts", and / or to cause the computer 1200 to execute a process or a stage of such process according to this embodiment. Such a program may be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0104] The computer 1200 according to this embodiment includes a CPU 1212, RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive and a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive and a solid-state drive, etc. The computer 1200 also includes legacy input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0105] The CPU 1212 operates according to the programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires the image data generated by the CPU 1212 and stores it in the frame buffer provided in RAM 1214 or within itself, so that the image data is displayed on the display device 1218.

[0106] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0107] The ROM 1230 stores boot programs and / or hardware-dependent programs of the computer 1200, which are executed by the computer 1200 upon activation. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

[0108] The program is provided on a computer-readable storage medium such as a DVD-ROM or IC card. The program is read from the computer-readable storage medium and installed on a storage device 1224, RAM 1214, or ROM 1230, which are examples of computer-readable storage media, and executed by the CPU 1212. The information processing described within these programs is read by the computer 1200, resulting in coordination between the program and the various types of hardware resources described above. The apparatus or method may be configured to realize the operation or processing of information in accordance with the use of the computer 1200.

[0109] For example, when communication is performed between a computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into RAM 1214 and, based on the processing described in the communication program, instruct the communication interface 1222 to perform communication processing. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as RAM 1214, storage device 1224, DVD-ROM, or IC card, transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area provided on the recording medium.

[0110] Furthermore, the CPU 1212 may read all or necessary parts of a file or database stored on an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), or an IC card into the RAM 1214, and perform various types of processing on the data in the RAM 1214. The CPU 1212 may then write the processed data back to the external recording medium.

[0111] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., as described throughout each embodiment and specified by the program instruction sequence, and write the results back to the RAM 1214. The CPU 1212 may also search for information in files, databases, etc., within the recording medium. For example, if multiple entries are stored in the recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 may search among the multiple entries for an entry that matches the specified condition for the attribute value of the first attribute, read the attribute value of the second attribute stored in that entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies the predetermined condition.

[0112] The program or software module described above may be stored on or near the computer 1200 in a computer-readable storage medium. Alternatively, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0113] In this embodiment, blocks in the flowchart and block diagram may represent a stage in a process in which an operation is performed or a "part" of a device that has the role of performing an operation. A particular stage and "part" may be implemented by a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include reconfigurable hardware circuits, such as field-programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), which include logical AND, logical OR, exclusive OR, negated AND, negated OR, and other logical operations, flip-flops, registers, and memory elements.

[0114] A computer-readable storage medium may include any tangible device capable of storing instructions to be executed by a suitable device, and as a result, a computer-readable storage medium having instructions stored therein will comprise a product that includes instructions that can be executed to create means for performing operations specified in a flowchart or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital multipurpose disc (DVD), Blu-ray® disc, memory stick, integrated circuit card, etc.

[0115] Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, Java®, C++, and traditional procedural programming languages ​​such as the C programming language or similar programming languages.

[0116] Computer-readable instructions may be provided to a general-purpose computer, a special-purpose computer, or a programmable circuit, either locally or via a wide area network (WAN) such as a local area network (LAN) or the internet, so that the computer-readable instructions may be executed by the processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, in order to generate means for performing operations specified in a flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

[0117] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0118] It should be noted that the execution order of operations, procedures, steps, and stages in the apparatus, systems, programs, and methods shown in the claims, specifications, and drawings is not explicitly stated as "before" or "prior to," and that these can be implemented in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," and "next," for convenience, this does not mean that it is essential to perform the operations in that order. [Explanation of Symbols]

[0119] 12 vehicles 12A My vehicle 12B, 12C, 12D and other vehicles 14. Information Processing Device (External Device) 16. Server (External Device) 120 Central Brain (Information Processing System)

Claims

1. An acquisition unit that acquires multiple pieces of information related to the vehicle from a detection unit which includes a sensor that detects the conditions around the vehicle, A calculation unit calculates an index value relating to the conditions around the vehicle from the acquired plurality of pieces of information, and calculates control variables for controlling the behavior of the vehicle from the calculated index value. The system includes a control unit that controls the behavior of the vehicle based on the calculated control variables, The calculation unit predicts a collision of an object with the vehicle based on the acquired plurality of pieces of information, and if the predicted result indicates an unavoidable collision, it calculates control variables such that the damage to the vehicle in the unavoidable collision and in a subsequent collision with another object is less than or equal to a predetermined threshold.

2. The acquisition unit acquires information about the surrounding conditions of the vehicle from another vehicle that is the object to be hit. The information processing apparatus according to claim 1.

3. The acquisition unit acquires information relating to the conditions around the vehicle from an external device installed on the road on which the vehicle travels. The information processing apparatus according to claim 1.

4. The calculation unit calculates the control variables from the index values ​​by multivariate analysis using an integral method based on deep learning. The information processing apparatus according to claim 1.

5. The damage to the vehicle is at least one of the deformation location and the amount of deformation of the vehicle. The information processing apparatus according to claim 1.

6. The control variable corresponding to the damage to the vehicle is at least one of the vehicle's collision angle and vehicle speed. The information processing apparatus according to claim 1.

7. The information processing apparatus according to claim 3, The sensor connected to the information processing device detects the conditions around the vehicle, A receiving unit that receives the plurality of pieces of information from the external device, A vehicle equipped with the following features.

Images