Information processing system, information processing method, and information processing device

Abstract

The present disclosure relates to an information processing system, an information processing method, and an information processing device, with which deterioration of a sensor can be diagnosed. In the present disclosure a recognition result that is for a prescribed subject to be recognized and that is based on a sensing result of a plurality of sensors mounted in a vehicle is acquired, the acquired recognition result is compared with another recognition result different from the acquired recognition result, a comparison result is output, and deterioration of individual sensors is evaluated on the basis of the comparison result. The present disclosure is applicable to mobile bodies.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and an information processing device, and in particular to an information processing system, an information processing method, and an information processing device that are capable of diagnosing deterioration of sensors required for autonomous driving. [Background technology]

[0002] Various technologies have been proposed to realize autonomous driving, but as autonomous driving becomes more widespread, sensor information from cameras, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), and other sensors will play a very important role in ensuring safety while driving.

[0003] However, when the sensing results of a plurality of sensors are integrated and subjected to recognition processing, it becomes difficult to determine the deterioration of each individual sensor.

[0004] In other words, even if one of the sensors is degraded, the sensing results of multiple sensors are integrated, and the sensing results of the sensor that is not degraded are used preferentially, so there is no significant decrease in recognition accuracy, making it difficult for the user to determine whether there is degradation.

[0005] However, even if the deterioration of an individual sensor does not cause any problems during normal driving, it may have a significant impact on safe driving in situations where the sensing results of the deteriorated sensor are important.

[0006] For example, when driving and recognizing obstacles using images captured by a camera and 3D point clouds sensed by LiDAR, overlooking LiDAR degradation could affect safe driving in situations such as backlight.

[0007] In other words, when backlighting occurs, the accuracy of obstacle recognition using images captured by the camera decreases, so the system has to rely on the obstacle recognition results based on the LiDAR sensing results.

[0008] However, due to deterioration of the LiDAR, sensing results that are not sufficiently accurate are used to recognize obstacles, and as a result, various decisions are made based on the recognition results based on the LiDAR sensing results, even though the accuracy of obstacle recognition has decreased, which could result in the risk of driving safety not being fully ensured.

[0009] Therefore, it is conceivable to apply the technology disclosed in Patent Document 1 for detecting whether or not a sensor has failed, and when a failure is detected, not use the failed sensor, thereby avoiding the use of the sensing results of sensors that cannot ensure sufficient accuracy as described above, and thereby preventing them from affecting various judgments (see Patent Document 1). [Prior art documents] [Patent documents]

[0010] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-321350 Summary of the Invention [Problem to be solved by the invention]

[0011] However, the technology disclosed in Patent Document 1 can detect sensor failures but cannot detect deterioration. Therefore, if there is significant deterioration that has not yet resulted in a failure, the sensor will not be recognized as having a failure. As a result, various processes will be carried out using significantly deteriorated sensing results, which could result in insufficient safety being ensured.

[0012] The present disclosure has been made in view of such circumstances, and in particular, makes it possible to diagnose deterioration of a sensor. [Means for solving the problem]

[0013] An information processing system and an information processing device according to one aspect of the present disclosure are a system and an information processing device that include a recognition result acquisition unit that acquires recognition results of a predetermined recognition object based on sensing results of a plurality of sensors mounted on a vehicle, a comparison unit that compares the acquired recognition result with other recognition results different from the acquired recognition result and outputs the comparison result, and a degradation evaluation unit that evaluates individual performance degradation of the sensor based on the comparison result by the comparison unit.

[0014] An information processing method according to one aspect of the present disclosure is an information processing method including the steps of acquiring a recognition result of a predetermined recognition object based on sensing results of a plurality of sensors mounted on a vehicle, comparing the acquired recognition result with another recognition result different from the acquired recognition result, outputting the comparison result, and evaluating individual performance degradation of the sensor based on the comparison result.

[0015] In one aspect of the present disclosure, a recognition result of a predetermined recognition object is obtained based on sensing results of a plurality of sensors mounted on a vehicle, the obtained recognition result is compared with another recognition result different from the obtained recognition result, and the comparison result is output, and performance degradation of each of the sensors is evaluated based on the comparison result. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram illustrating an overview of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an overview of the present disclosure. [Figure 3] 1 is a diagram illustrating a configuration example of a degradation diagnosis system according to the present disclosure. [Figure 4] FIG. 1 is a diagram illustrating an example configuration of a vehicle control system according to the present disclosure. [Figure 5] 1 is a block diagram illustrating a configuration example of a degradation diagnosis unit according to a first embodiment of the present disclosure. [Figure 6]10A and 10B are diagrams illustrating examples of recognition results of object recognition processing. [Figure 7] 10A and 10B are diagrams illustrating examples of recognition results of object recognition processing. [Figure 8] 10A and 10B are diagrams illustrating an example of a method for determining deterioration (outlier detection) based on a recognition result. [Figure 9] 10A and 10B are diagrams illustrating an example of a degradation determination method (change point detection) based on a recognition result. [Figure 10] 10A and 10B are diagrams illustrating an example of a method for determining deterioration (detecting an abnormal portion) based on a recognition result. [Figure 11] 10A and 10B are diagrams illustrating an example of a method for determining deterioration (detecting an abnormal portion) based on a recognition result. [Figure 12] 4 is a flowchart illustrating a degradation diagnosis process according to the first embodiment. [Figure 13] 4 is a flowchart illustrating a degradation diagnosis process according to the first embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of an image requesting approval for insurance premium renewal. [Figure 15] FIG. 10 is a diagram illustrating an example of an image requesting approval for insurance premium renewal. [Figure 16] FIG. 10 is a diagram illustrating an example of an image requesting approval for insurance premium renewal. [Figure 17] FIG. 10 is a diagram illustrating an example of an image requesting approval for insurance premium renewal. [Figure 18] FIG. 10 is a diagram illustrating an example of an image requesting approval for insurance premium renewal. [Figure 19] FIG. 10 is a block diagram illustrating a configuration example of a modification of the first embodiment of the degradation diagnostic unit of the present disclosure. [Figure 20] FIG. 10 is a block diagram illustrating a configuration example of a second embodiment of a degradation diagnosis unit of the present disclosure. [Figure 21] 10 is a flowchart illustrating a degradation diagnosis process according to a second embodiment. [Figure 22] 10 is a flowchart illustrating a degradation diagnosis process according to a second embodiment. [Figure 23] 10A and 10B are diagrams illustrating application examples of the degradation diagnosis system of the present disclosure. [Figure 24] FIG. 1 is a diagram illustrating an example of the configuration of a general-purpose computer. DETAILED DESCRIPTION OF THE INVENTION

[0017] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0018] Hereinafter, embodiments of the present technology will be described in the following order. 1. Overview of this Disclosure 2. First Embodiment 3. Modification of the First Embodiment 4. Second Embodiment 5. Application Examples 6. Software implementation example

[0019] <<1. Overview of this Disclosure>> The present disclosure allows for the diagnosis of degradation of multiple sensors.

[0020] First, an overview of the present disclosure will be described with reference to FIG.

[0021] In the deterioration diagnosis system disclosed herein, a testing site TR is provided for diagnosing the deterioration of multiple sensors, as shown in Figure 1. Within the testing site TR, test processing is performed using various sensors of a vehicle V, the test results are accumulated, and the deterioration is diagnosed by comparing them with the accumulated test results.

[0022] More specifically, it is desirable that the testing site TR be located at the entrances to various areas where safety is required and where there is a high possibility of contact with people, for example, at the entrance to a residential area.

[0023] The vehicle V, which will be driven autonomously, enters a testing site TR located just before entering an area where high safety is required, where it undergoes a test to diagnose sensor deterioration.If, for example, the deterioration is so severe that sufficient safety cannot be ensured, autonomous driving is discontinued.

[0024] The test site TR is configured as an indoor space to realize test processing without being affected by the environment such as light, wind, etc. Furthermore, a stopping position P is set in the test site TR where a vehicle V to be tested is stopped, and a recognition target H is set at a position a predetermined distance D from the stopping position P.

[0025] In FIG. 1, a humanoid doll is set as the recognition target object H.

[0026] The vehicle V is equipped with, for example, a camera C as a sensor, and is the sensor to be tested.

[0027] When the vehicle V enters the testing site TR and stops at the stopping position P, the test begins and the recognition target C is imaged by the camera C. Based on the image, object recognition processing is performed and the result R is output.

[0028] Result R in Fig. 1 indicates that, as an object recognition result, a recognition target object H is detected at the position indicated by the frame in the image captured by camera C, the object name as a recognition result is a person (Person), and the reliability is 95%. The reliability here refers to the reliability of the object recognition result, which combines, for example, the position where the recognition target object H is detected and the object name.

[0029] The positional relationship between the stopping position P at the test site TR and the recognition object H installed at a predetermined position a distance D away is configured in a common manner, and the vehicle V accumulates information on the results R at various test sites TR.

[0030] More specifically, the vehicle V accumulates the reliability information included in the result R as time-series information as shown in FIG. 2, for example.

[0031] In FIG. 2, the horizontal axis represents the elapsed time t, and the vertical axis represents the reliability.

[0032] As mentioned above, at the test site TR, the positional relationship between the vehicle V and the recognition target H is constant, and the environment is not affected by external factors such as light and wind, so theoretically, the reliability is almost constant.

[0033] However, camera C deteriorates over time, so its reliability gradually decreases.

[0034] If the accumulated results of this time series fall below a predetermined threshold th, the vehicle V is deemed to be significantly affected by deterioration and to be dangerous to continue autonomous driving based on the recognition results of the object recognition process, and the vehicle V is deemed to have failed, and autonomous driving of the vehicle V is discontinued.

[0035] The above describes an example of determining degradation due to a decrease in reliability, but the same determination can be made if the object name in the recognition result is no longer a human, or if the position indicated by the frame is shifted.

[0036] In this way, the test is repeated at the test site TR where the same environment is prepared, and the information of the results R is sequentially accumulated and compared with the threshold value to determine the deterioration of the sensor.

[0037] As a result, it becomes possible to diagnose the deterioration that occurs up until a complete failure occurs.

[0038] <<2. First Embodiment>> Next, a configuration example of the first embodiment of the present disclosure will be described with reference to FIG.

[0039] The deterioration diagnosis system 31 of the present disclosure is composed of a test site 51 and a vehicle 91 to be tested.

[0040] Test site 51 is provided at the entrances of various areas where safety is required and where there is a high possibility of contact with people, for example, at the entrance to a residential area. Furthermore, as long as a deterioration diagnosis test can be performed, test site 51 may be provided anywhere where vehicle 91 can be stopped or in the vicinity, for example, at a gas station or parking lot.

[0041] The vehicle 91 is equipped with, for example, a camera 91a, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) 91b, and a millimeter wave radar 91c as sensors for realizing autonomous driving.

[0042] The vehicle 91 performs object recognition processing by integrating the sensing results of the camera 91a, LiDAR 91b, and millimeter-wave radar 91c, or by using them separately depending on the conditions, and performs autonomous driving based on the recognition results.

[0043] When the vehicle 91 enters an area where high safety is required, it enters a testing site TR located near the entrance, where a test to diagnose deterioration of the sensor is carried out, and if the deterioration is significant, the automatic driving is stopped.

[0044] The test site 51 is configured as an indoor space so that the test process can be carried out without being affected by the environment, such as light or wind. A stopping position P is set where the vehicle 91 to be tested is stopped, and furthermore, a recognition object 61 is set at a position a predetermined distance D from the stopping position P.

[0045] In FIG. 3, a humanoid doll is set as the recognition object 61.

[0046] When the vehicle 91 enters the testing site 51 and stops at the stopping position P, it starts a test related to deterioration diagnosis, senses the recognition target 61 using the camera 91a, LiDAR 91b, and millimeter wave radar 91c, and performs each object recognition process based on each sensing result.

[0047] The vehicle 91 stores and accumulates the recognition results obtained by the object recognition process for each sensor individually, and evaluates the deterioration based on the information of the accumulated recognition results.

[0048] The deterioration is evaluated based on the position, object name, and reliability of each recognition result from each sensor.

[0049] More specifically, the position is evaluated based on the amount of deviation of the position resulting from the recognition from the correct position.

[0050] Furthermore, the object name is evaluated based on whether the object name resulting from the recognition is the same as at least one of the correct object name and an object name similar to the correct answer that is acceptable.

[0051] The confidence level is calculated based on the location and object name and evaluated against a predetermined threshold value that defines the acceptable range.

[0052] Depending on the deterioration assessment, vehicle 91 can continue to drive autonomously using either sensor, but if the risk of an accident due to autonomous driving increases, the insurance premium will be recalculated in accordance with the risk, and autonomous driving will continue if confirmation is obtained from the user or owner of vehicle 91.

[0053] Furthermore, if the vehicle 91 is unable to drive automatically based on the evaluation of the deterioration, the vehicle 91 will stop driving automatically and, if necessary, arrange for a replacement vehicle, for example.

[0054] <Example of vehicle control system configuration> FIG. 4 is a block diagram showing an example of a schematic functional configuration of a vehicle control system 100 of a vehicle 91, which is an example of a vehicle moving body control system to which the present technology can be applied.

[0055] 1. In the following description, when the vehicle in which the vehicle control system 100 is installed is to be distinguished from other vehicles, it will be referred to as the host vehicle or the host vehicle. Note that the vehicle 91 in FIG. 4 corresponds to the vehicle C in FIG.

[0056] The vehicle control system 100 includes an input unit 101, a data acquisition unit 102, a communication unit 103, in-vehicle devices 104, an output control unit 105, an output unit 106, a drivetrain control unit 107, a drivetrain system 108, a body system control unit 109, a body system 110, a memory unit 111, and an automatic driving control unit 112. The input unit 101, the data acquisition unit 102, the communication unit 103, the output control unit 105, the drivetrain control unit 107, the body system control unit 109, the memory unit 111, and the automatic driving control unit 112 are connected to each other via a communication network 121. The communication network 121 is formed of an in-vehicle communication network or bus conforming to any standard such as a Controller Area Network (CAN), a Local Interconnect Network (LIN), a Local Area Network (LAN), or FlexRay (registered trademark). Note that the components of the vehicle control system 100 may be directly connected to each other without using the communication network 121.

[0057] In the following, when each unit of the vehicle control system 100 communicates via the communication network 121, the description of the communication network 121 will be omitted. For example, when the input unit 101 and the automatic driving control unit 112 communicate via the communication network 121, it will simply be described as the input unit 101 and the automatic driving control unit 112 communicating with each other.

[0058] The input unit 101 includes a device used by the passenger to input various data, instructions, etc. For example, the input unit 101 includes operation devices such as a touch panel, buttons, a microphone, switches, and levers, as well as operation devices that allow input by voice, gestures, or other means other than manual operation. Furthermore, for example, the input unit 101 may be a remote control device that uses infrared or other radio waves, or an externally connected device such as a mobile device or wearable device that supports operation of the vehicle control system 100. The input unit 101 generates an input signal based on the data, instructions, etc. input by the passenger, and supplies the input signal to each component of the vehicle control system 100.

[0059] The data acquisition unit 102 includes various sensors and the like that acquire data used for processing by the vehicle control system 100 , and supplies the acquired data to each unit of the vehicle control system 100 .

[0060] For example, the data acquisition unit 102 includes various sensors for detecting the state of the vehicle, etc. Specifically, for example, the data acquisition unit 102 includes a gyro sensor, an acceleration sensor, an inertial measurement unit (IMU), and sensors for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine rotation speed, the motor rotation speed, or the rotation speed of the wheels, etc.

[0061] Furthermore, for example, the data acquisition unit 102 includes various sensors for detecting information outside the vehicle. Specifically, for example, the data acquisition unit 102 includes imaging devices such as a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. Furthermore, for example, the data acquisition unit 102 includes an environmental sensor for detecting weather or climate, and a surrounding information detection sensor for detecting objects around the vehicle. The environmental sensor includes, for example, a raindrop sensor, a fog sensor, a sunlight sensor, a snow sensor, etc. The surrounding information detection sensor includes, for example, an ultrasonic sensor, radar, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), sonar, etc.

[0062] Furthermore, for example, the data acquisition unit 102 includes various sensors for detecting the current position of the vehicle. Specifically, for example, the data acquisition unit 102 includes a GNSS (Global Navigation Satellite System) receiver that receives GNSS signals from GNSS satellites.

[0063] Furthermore, for example, the data acquisition unit 102 includes various sensors for detecting information inside the vehicle. Specifically, for example, the data acquisition unit 102 includes an imaging device for capturing an image of the driver, a biosensor for detecting biometric information of the driver, and a microphone for collecting sound inside the vehicle. The biosensor is provided, for example, on the seat or steering wheel, and detects biometric information of a passenger sitting in the seat or a driver gripping the steering wheel.

[0064] The communication unit 103 communicates with the in-vehicle device 104 as well as various devices, servers, base stations, etc. outside the vehicle, transmits data supplied from each part of the vehicle control system 100, and supplies received data to each part of the vehicle control system 100. Note that the communication protocol supported by the communication unit 103 is not particularly limited, and the communication unit 103 can also support multiple types of communication protocols.

[0065] For example, the communication unit 103 performs wireless communication with the in-vehicle device 104 using wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), WUSB (Wireless USB), etc. Also, for example, the communication unit 103 performs wired communication with the in-vehicle device 104 using USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), MHL (Mobile High-Definition Link), etc. via a connection terminal (and a cable, if necessary) not shown.

[0066] Furthermore, for example, the communication unit 103 communicates with devices (e.g., application servers or control servers) present on an external network (e.g., the Internet, a cloud network, or a network specific to a carrier) via a base station or an access point. Furthermore, for example, the communication unit 103 communicates with terminals present near the vehicle (e.g., terminals of pedestrians or stores, or MTC (Machine Type Communication) terminals) using P2P (Peer To Peer) technology. Furthermore, for example, the communication unit 103 performs V2X communication such as vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication. Furthermore, for example, the communication unit 103 includes a beacon receiver that receives radio waves or electromagnetic waves transmitted from radio stations or the like installed on roads, and acquires information such as the current location, congestion, traffic restrictions, and required travel time.

[0067] The in-vehicle devices 104 include, for example, mobile devices or wearable devices owned by the passengers, information devices carried into or attached to the vehicle, and navigation devices that search for routes to any destination.

[0068] The output control unit 105 controls the output of various types of information to passengers in the vehicle or to the outside of the vehicle. For example, the output control unit 105 generates an output signal including at least one of visual information (e.g., image data) and auditory information (e.g., audio data) and supplies the output signal to the output unit 106, thereby controlling the output of the visual information and audio information from the output unit 106. Specifically, for example, the output control unit 105 synthesizes image data captured by different imaging devices in the data acquisition unit 102 to generate an overhead image, a panoramic image, or the like, and supplies an output signal including the generated image to the output unit 106. Furthermore, for example, the output control unit 105 generates audio data including a warning sound or a warning message against danger such as a collision, contact, or entry into a dangerous area, and supplies an output signal including the generated audio data to the output unit 106.

[0069] The output unit 106 includes a device capable of outputting visual or auditory information to the passengers of the vehicle or to the outside of the vehicle. For example, the output unit 106 includes a display device, an instrument panel, audio speakers, headphones, a wearable device such as an eyeglass-type display worn by the passenger, a projector, a lamp, etc. The display device included in the output unit 106 may be a device having a normal display, or may be a device that displays visual information within the driver's field of view, such as a head-up display, a see-through display, or a device having an AR (Augmented Reality) display function.

[0070] The drivetrain control unit 107 generates various control signals and supplies them to the drivetrain system 108, thereby controlling the drivetrain system 108. In addition, the drivetrain control unit 107 supplies control signals to each unit other than the drivetrain system 108 as necessary, and notifies the control status of the drivetrain system 108, etc.

[0071] The drivetrain system 108 includes various devices related to the drivetrain of the vehicle, such as a drive force generating device for generating drive force from an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle, a braking device for generating braking force, an ABS (Antilock Brake System), an ESC (Electronic Stability Control), and an electric power steering device.

[0072] The body system control unit 109 generates various control signals and supplies them to the body system 110, thereby controlling the body system 110. Furthermore, the body system control unit 109 supplies control signals to each unit other than the body system 110 as necessary, and notifies the control status of the body system 110, etc.

[0073] The body system 110 includes various body system devices mounted on the vehicle body, such as a keyless entry system, a smart key system, a power window device, a power seat, a steering wheel, an air conditioning system, and various lamps (for example, head lamps, backup lamps, brake lamps, blinkers, fog lamps, etc.).

[0074] The storage unit 111 includes, for example, a magnetic storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), or a HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage unit 111 stores various programs, data, and the like used by each unit of the vehicle control system 100. For example, the storage unit 111 stores map data such as a three-dimensional high-precision map such as a dynamic map, a global map that is less accurate than a high-precision map and covers a wide area, and a local map that includes information about the surroundings of the vehicle.

[0075] The autonomous driving control unit 112 performs control related to autonomous driving, such as autonomous driving or driving assistance. Specifically, for example, the autonomous driving control unit 112 performs cooperative control aimed at realizing functions of an Advanced Driver Assistance System (ADAS), including collision avoidance or impact mitigation for the host vehicle, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, collision warning for the host vehicle, or lane departure warning for the host vehicle. Furthermore, for example, the autonomous driving control unit 112 performs cooperative control aimed at autonomous driving, which travels autonomously without relying on driver operation. The autonomous driving control unit 112 includes a detection unit 131, a self-position estimation unit 132, a situation analysis unit 133, a planning unit 134, and an operation control unit 135.

[0076] The detection unit 131 detects various types of information required for controlling autonomous driving. The detection unit 131 includes an outside-vehicle information detection unit 141, an inside-vehicle information detection unit 142, and a vehicle state detection unit 143.

[0077] The outside-vehicle information detection unit 141 performs a process of detecting information outside the vehicle based on data or signals from each unit of the vehicle control system 100. For example, the outside-vehicle information detection unit 141 performs a process of detecting, recognizing, and tracking objects around the vehicle, as well as a process of detecting the distance to the objects. Objects to be detected include, for example, vehicles, people, obstacles, structures, roads, traffic lights, traffic signs, road markings, etc. Furthermore, for example, the outside-vehicle information detection unit 141 performs a process of detecting the environment around the vehicle. The surrounding environment to be detected includes, for example, weather, temperature, humidity, brightness, and road surface conditions. The outside-vehicle information detection unit 141 supplies data indicating the results of the detection process to the self-position estimation unit 132, the map analysis unit 151, traffic rule recognition unit 152, and situation recognition unit 153 of the situation analysis unit 133, the emergency situation avoidance unit 171 of the operation control unit 135, etc.

[0078] The interior information detection unit 142 performs a process of detecting information about the interior of the vehicle based on data or signals from each unit of the vehicle control system 100. For example, the interior information detection unit 142 performs a process of authenticating and recognizing the driver, a process of detecting the driver's state, a process of detecting passengers, and a process of detecting the interior environment of the vehicle. The driver's state to be detected includes, for example, physical condition, alertness, concentration, fatigue, and gaze direction. The interior environment to be detected includes, for example, temperature, humidity, brightness, and odor. The interior information detection unit 142 supplies data indicating the results of the detection process to the situation recognition unit 153 of the situation analysis unit 133 and the emergency situation avoidance unit 171 of the operation control unit 135.

[0079] The vehicle state detection unit 143 performs a process of detecting the state of the vehicle based on data or signals from each unit of the vehicle control system 100. The vehicle state to be detected includes, for example, the speed, acceleration, steering angle, the presence or absence and content of an abnormality, the state of driving operation, the position and tilt of the power seat, the state of door locks, and the state of other in-vehicle devices. The vehicle state detection unit 143 supplies data indicating the results of the detection process to the situation recognition unit 153 of the situation analysis unit 133 and the emergency situation avoidance unit 171 of the operation control unit 135, etc.

[0080] The vehicle state detection unit 143 also includes a recognition unit 181 and a deterioration diagnosis unit 182 .

[0081] The recognition unit 181 executes object recognition processing based on data or signals from various sensors in the data acquisition unit 102 of the vehicle control system 100 , and outputs the recognition results to the degradation diagnosis unit 182 .

[0082] The deterioration diagnosis unit 182 diagnoses deterioration of the sensor based on the recognition result and reliability of the recognition unit 181, and evaluates the deterioration.

[0083] More specifically, the recognition unit 181 performs object recognition processing based on data or signals from various sensors in the data acquisition unit 102 of the vehicle control system 100 acquired by testing at the proving ground 51, and outputs the recognition results to the deterioration diagnosis unit 182.

[0084] The deterioration diagnosis unit 182 accumulates the recognition results supplied from the recognition unit 181 in chronological order, and evaluates the deterioration of each sensor by comparing the accumulated results with the current recognition results.

[0085] At this time, the deterioration diagnosis unit 182 stops the automatic driving in accordance with the evaluation of the deterioration in the deterioration diagnosis.

[0086] Furthermore, the deterioration diagnosis unit 182 determines that autonomous driving is possible according to the deterioration evaluation in the deterioration diagnosis, but if the risk of an accident or the like increases, it recalculates a new insurance premium for the automobile according to the risk, requests approval from the user or owner of the vehicle 91 for renewal of the insurance contract with the new insurance premium, and allows autonomous driving to resume once approval is received.

[0087] The detailed configurations of recognition section 181 and degradation diagnosis section 182 will be described later with reference to FIG.

[0088] The self-position estimation unit 132 performs estimation processing of the position, attitude, etc. of the vehicle based on data or signals from each unit of the vehicle control system 100, such as the outside-vehicle information detection unit 141 and the situation recognition unit 153 of the situation analysis unit 133. Furthermore, the self-position estimation unit 132 generates a local map (hereinafter referred to as a self-position estimation map) used for estimating the self-position as necessary. The self-position estimation map is a high-precision map using technology such as SLAM (Simultaneous Localization and Mapping). The self-position estimation unit 132 supplies data indicating the result of the estimation processing to the map analysis unit 151, the traffic rule recognition unit 152, the situation recognition unit 153, etc. of the situation analysis unit 133. Furthermore, the self-position estimation unit 132 stores the self-position estimation map in the storage unit 111.

[0089] The situation analysis unit 133 performs an analysis process of the situation of the vehicle and its surroundings. The situation analysis unit 133 includes a map analysis unit 151, a traffic rule recognition unit 152, a situation recognition unit 153, and a situation prediction unit 154.

[0090] The map analysis unit 151 analyzes various maps stored in the storage unit 111, using data or signals from each unit of the vehicle control system 100, such as the self-position estimation unit 132 and the outside-vehicle information detection unit 141, as needed, to construct a map including information necessary for autonomous driving processing. The map analysis unit 151 supplies the constructed map to the traffic rule recognition unit 152, the situation recognition unit 153, the situation prediction unit 154, and the route planning unit 161, the behavior planning unit 162, the operation planning unit 163, etc. of the planner 134.

[0091] The traffic rule recognition unit 152 performs a recognition process of traffic rules around the vehicle based on data or signals from each unit of the vehicle control system 100, such as the self-position estimation unit 132, the outside vehicle information detection unit 141, and the map analysis unit 151. This recognition process recognizes, for example, the positions and states of traffic signals around the vehicle, the details of traffic regulations around the vehicle, and available lanes. The traffic rule recognition unit 152 supplies data indicating the results of the recognition process to the situation prediction unit 154 and the like.

[0092] The situation recognition unit 153 performs recognition processing of the situation related to the vehicle based on data or signals from each unit of the vehicle control system 100, such as the self-position estimation unit 132, the vehicle outside information detection unit 141, the vehicle inside information detection unit 142, the vehicle state detection unit 143, and the map analysis unit 151. For example, the situation recognition unit 153 performs recognition processing of the situation of the vehicle, the situation around the vehicle, the situation of the driver of the vehicle, etc. Furthermore, the situation recognition unit 153 generates a local map (hereinafter referred to as a situation recognition map) used to recognize the situation around the vehicle, as needed. The situation recognition map is, for example, an occupancy grid map.

[0093] The vehicle's conditions to be recognized include, for example, the vehicle's position, posture, and movement (e.g., speed, acceleration, direction of movement, etc.), as well as the presence or absence of abnormalities and their details. The conditions around the vehicle to be recognized include, for example, the type and position of surrounding stationary objects, the type, position, and movement (e.g., speed, acceleration, direction of movement, etc.) of surrounding moving objects, the configuration and condition of the surrounding road surface, and the surrounding weather, temperature, humidity, brightness, etc. The driver's conditions to be recognized include, for example, physical condition, alertness, concentration, fatigue, eye movement, driving operation, etc.

[0094] The situation recognition unit 153 supplies data indicating the result of the recognition process (including a situation recognition map, if necessary) to the self-position estimation unit 132, the situation prediction unit 154, etc. Furthermore, the situation recognition unit 153 stores the situation recognition map in the storage unit 111.

[0095] The situation prediction unit 154 performs prediction processing of the situation regarding the vehicle based on data or signals from each unit of the vehicle control system 100, such as the map analysis unit 151, the traffic rule recognition unit 152, and the situation recognition unit 153. For example, the situation prediction unit 154 performs prediction processing of the situation of the vehicle, the situation around the vehicle, the situation of the driver, and the like.

[0096] The vehicle conditions to be predicted include, for example, the vehicle's behavior, the occurrence of an abnormality, and the driving distance. The conditions around the vehicle to be predicted include, for example, the behavior of moving objects around the vehicle, changes in traffic lights, and changes in the environment such as the weather. The driver conditions to be predicted include, for example, the driver's behavior and physical condition.

[0097] The situation prediction unit 154 supplies data indicating the results of the prediction process to the route planning unit 161, behavior planning unit 162, and operation planning unit 163 of the planning unit 134, together with data from the traffic rule recognition unit 152 and the situation recognition unit 153.

[0098] The route planning unit 161 plans a route to the destination based on data or signals from each unit of the vehicle control system 100, such as the map analysis unit 151 and the situation prediction unit 154. For example, the route planning unit 161 sets a route from the current location to a specified destination based on a global map. Furthermore, for example, the route planning unit 161 changes the route as appropriate based on conditions such as congestion, accidents, traffic restrictions, and construction, as well as the driver's physical condition. The route planning unit 161 supplies data indicating the planned route to the action planning unit 162 and the like.

[0099] The behavior planning unit 162 plans the behavior of the vehicle to travel safely along the route planned by the route planning unit 161 within the planned time based on data or signals from each unit of the vehicle control system 100, such as the map analysis unit 151 and the situation prediction unit 154. For example, the behavior planning unit 162 plans starting, stopping, traveling direction (for example, moving forward, backward, turning left, turning right, changing direction, etc.), traveling lane, traveling speed, overtaking, etc. The behavior planning unit 162 supplies data indicating the planned behavior of the vehicle to the operation planning unit 163, etc.

[0100] The action planning unit 163 plans the action of the host vehicle to realize the action planned by the action planning unit 162, based on data or signals from each unit of the vehicle control system 100, such as the map analysis unit 151 and the situation prediction unit 154. For example, the action planning unit 163 plans acceleration, deceleration, a driving trajectory, etc. The action planning unit 163 supplies data indicating the planned action of the host vehicle to the acceleration / deceleration control unit 172, the direction control unit 173, etc. of the action control unit 135.

[0101] The operation control unit 135 controls the operation of the host vehicle. The operation control unit 135 includes an emergency situation avoidance unit 171, an acceleration / deceleration control unit 172, and a direction control unit 173.

[0102] The emergency situation avoidance unit 171 performs detection processing for an emergency situation such as a collision, contact, entry into a dangerous area, driver abnormality, or vehicle abnormality based on the detection results of the vehicle exterior information detection unit 141, the vehicle interior information detection unit 142, and the vehicle state detection unit 143. When the emergency situation avoidance unit 171 detects the occurrence of an emergency situation, it plans an operation of the vehicle to avoid the emergency situation such as a sudden stop or a sharp turn. The emergency situation avoidance unit 171 supplies data indicating the planned operation of the vehicle to the acceleration / deceleration control unit 172, the direction control unit 173, etc.

[0103] The acceleration / deceleration control unit 172 performs acceleration / deceleration control to realize the operation of the host vehicle planned by the operation planning unit 163 or the emergency situation avoidance unit 171. For example, the acceleration / deceleration control unit 172 calculates a control target value of a driving force generation device or a braking device to realize the planned acceleration, deceleration, or sudden stop, and supplies a control command indicating the calculated control target value to the drive train control unit 107.

[0104] The direction control unit 173 performs direction control to realize the operation of the host vehicle planned by the action planning unit 163 or the emergency situation avoidance unit 171. For example, the direction control unit 173 calculates a control target value of the steering mechanism to realize the traveling trajectory or sharp turn planned by the action planning unit 163 or the emergency situation avoidance unit 171, and supplies a control command indicating the calculated control target value to the drive train control unit 107.

[0105] <First Configuration Example of Degradation Diagnosis Unit> Next, a first configuration example of recognition section 181 and degradation diagnosis section 182 will be described with reference to FIG.

[0106] The recognition unit 181 performs object recognition processing based on data and signals supplied from various sensors of the data acquisition unit 102 , and outputs the recognition results for each sensor to the recognition result acquisition unit 221 .

[0107] More specifically, the recognition unit 181 includes a camera recognition unit 241 , a LiDAR recognition unit 242 , a millimeter wave radar recognition unit 243 , an other sensor recognition unit 244 , and a position information acquisition unit 245 .

[0108] The camera recognition unit 241 performs object recognition processing based on an image captured by a camera consisting of an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) among the sensors of the data acquisition unit 102, and outputs a recognition result consisting of the object's position and object name to the recognition result acquisition unit 221.

[0109] The LiDAR recognition unit 242 performs object recognition processing based on a point cloud, which is the sensing result sensed by the LiDAR, one of the sensors of the data acquisition unit 102, and outputs the recognition result consisting of the object position and object name to the recognition result acquisition unit 221.

[0110] The millimeter wave radar recognition unit 243 performs object recognition processing based on the sensing results obtained by the millimeter wave radar among the sensors of the data acquisition unit 102, and outputs the recognition results consisting of the object position and object name to the recognition result acquisition unit 221.

[0111] The other sensor recognition unit 244 performs object recognition processing based on sensing results of sensors other than the above-mentioned camera, LiDAR, and millimeter wave radar, and outputs a recognition result consisting of the object position and object name to the recognition result acquisition unit 221. For specific examples of other sensors, please refer to the description of the data acquisition unit 102 described with reference to FIG. 4.

[0112] The location information acquisition unit 245 acquires information about a location on the earth acquired from the GPS of the data acquisition unit 102, and supplies this information to the recognition result acquisition unit 221 together with time information at the time of acquisition.

[0113] The deterioration diagnosis unit 182 includes a recognition result acquisition unit 221 , a reliability calculation unit 222 , a comparison unit 223 , a deterioration determination unit 224 , an insurance premium determination unit 225 , and a UI display control unit 226 .

[0114] The recognition result acquisition unit 221 acquires the recognition results of various sensors supplied from the camera recognition unit 241, LiDAR recognition unit 242, millimeter wave radar recognition unit 243, and other sensor recognition unit 244 of the recognition unit 181, as well as the location information and time information supplied from the location information acquisition unit 245, associates them, stores them in the memory unit 111, and supplies them to the reliability calculation unit 222.

[0115] The reliability calculation unit 222 calculates the reliability of each recognition result based on the recognition results of various sensors supplied from the camera recognition unit 241, LiDAR recognition unit 242, millimeter wave radar recognition unit 243, and other sensor recognition unit 244 of the recognition unit 181 by the recognition result acquisition unit 221, and stores the reliability of each recognition result in the memory unit 111 in association with each other, and also outputs it to the comparison unit 223.

[0116] The comparison unit 223 compares the reliability of the current recognition result supplied from the reliability calculation unit 222 with the reliability accumulated in the past and stored in the storage unit 111 .

[0117] Furthermore, the comparison unit 223 compares the position as the recognition result with the correct position and object name (object name similar to the correct answer).

[0118] Then, the comparison unit 223 supplies to the deterioration determination unit 224 the comparison result between the position that is the recognition result and the correct position and object name for the object name (object name similar to the correct answer), the reliability of the current recognition result supplied from the reliability calculation unit 222, and the comparison result with the past accumulated reliability stored in the memory unit 111.

[0119] The deterioration determination unit 224 evaluates the deterioration of each sensor based on the position and object name in the recognition result, and the comparison result of the reliability.

[0120] More specifically, the deterioration determination unit 224 evaluates the deterioration of each sensor based on the comparison result of whether or not there is a deviation of the position information from the correct position that is equal to or greater than a predetermined value that is within an allowable range.

[0121] Furthermore, the deterioration determination unit 224 evaluates the deterioration of each sensor based on the comparison result as to whether the object name is the correct object name or whether the object name is similar to the correct object name.

[0122] Furthermore, the deterioration determination unit 224 evaluates the deterioration based on the comparison result by the comparison unit 223 between the reliability of the current recognition result supplied from the reliability calculation unit 222 and the past accumulated reliability stored in the memory unit 111.

[0123] The deterioration determination unit 224 evaluates the deterioration of each sensor based on the comparison results of the reliability and the comparison results of the location information and object name, which are the recognition results, and outputs the deterioration evaluation to the insurance premium determination unit 225, the UI display control unit 226, and the situation recognition unit 153.

[0124] Based on the deterioration assessment obtained from the deterioration judgment unit 224, if deterioration is observed but automatic driving is possible, the insurance premium determination unit 225 recalculates the insurance premium corresponding to the risk depending on the degree of deterioration and outputs it to the UI display control unit 226.

[0125] Based on the recalculated insurance premium information supplied by the insurance premium determination unit 225, the UI display control unit 226 displays an image to the user or owner of the vehicle 91 requesting approval for an update of the insurance premium in accordance with the risk associated with deterioration, and inquires whether or not to approve the change in insurance premium.

[0126] If the change in the recalculated insurance premium is approved, the insurance premium determination unit 225 controls the communication unit 103 to access the server managed and operated by the insurance company and request renewal of the contract with the new insurance premium, and if the request is approved, notifies the deterioration judgment unit 224 of this fact.

[0127] When it is confirmed that a contract with a new insurance premium has been made, the deterioration determination unit 224 determines that autonomous driving is possible and notifies the UI display control unit 226 and the situation recognition unit 153 of this.

[0128] Furthermore, when the insurance premium determination unit 225 receives a notification from the user or owner of the vehicle 91 that the change to the recalculated insurance premium is not accepted, it notifies the UI display control unit 226 and the situation recognition unit 153 that automatic driving will be stopped.

[0129] The UI display control unit 226 displays various information on the display or the like of the output unit 106 based on the deterioration assessment, and notifies the user or owner whether or not autonomous driving can be continued.

[0130] In addition, when a request for insurance premium renewal is made as described above, the UI display control unit 226 displays an image requesting insurance premium renewal to the user or owner of the vehicle 91, accepts a response, and returns it to the insurance premium determination unit 225.

[0131] The situation recognition unit 153 receives a notification regarding whether or not autonomous driving is possible based on the deterioration assessment by the deterioration determination unit 224 and the insurance premium update, and recognizes whether or not autonomous driving of the vehicle 91 will continue or be discontinued in accordance with the notification, and executes or discontinues autonomous driving of the vehicle 91.

[0132] <Example of recognition results from image-based object recognition processing> (Part 1) Next, examples of recognition results obtained by image-based object recognition processing will be described with reference to FIGS.

[0133] For example, when an image such as that shown in Fig. 6 is supplied from a camera (e.g., camera 91a in Fig. 3) of the data acquisition unit 102, the camera recognition unit 241 outputs a recognition result such as that shown in frames F1 to F4 in Fig. 6. Furthermore, the reliability calculation unit 222 calculates the reliability based on the position where the object, which is the recognition result, is detected and the name of the object.

[0134] That is, in the image of FIG. 6, a person H1 is walking on the left side, a vehicle V1 is traveling in the center, and a person H2 riding a bicycle B1 is traveling on the right side.

[0135] In this case, the camera recognition unit 241 outputs a recognition result as shown in frame F1 for the person H1 on the right.

[0136] In frame F1, it is shown that an object exists at the position of frame F1, and "person" is written in the upper left corner, indicating that the object name has been recognized as "human" through object recognition processing by the camera recognition unit 241, and "43.6" is written in the upper right corner, indicating that the reliability has been calculated as 43.6% by the reliability calculation unit 222 based on the recognition result.

[0137] That is, it is shown that the object recognition process has recognized the presence of a human being at the position of frame F1 with a reliability of 43.6%.

[0138] Furthermore, the camera recognition unit 241 outputs a recognition result for the central vehicle V1 as shown in frame F2.

[0139] In frame F2, it is shown that an object exists at the position of frame F2, and "car" is written in the upper left corner, indicating that the object name has been recognized as a "vehicle" through object recognition processing by the camera recognition unit 241, and "51.2" is written in the upper right corner, indicating that the reliability calculation unit 222 has calculated the reliability as 51.2% based on the recognition result.

[0140] That is, it is shown that the object recognition process has recognized the presence of a vehicle at the position of frame F2 with a reliability of 51.2%.

[0141] Furthermore, the camera recognition unit 241 outputs the recognition results shown in frames F3 and F4 for the person H2 riding the bicycle B1 on the right side.

[0142] In frame F3, it is shown that an object exists at the position of frame F3, and "person" is written in the upper left corner, indicating that the object name has been recognized as "human" through object recognition processing by the camera recognition unit 241, and "46.6" is written in the upper right corner, indicating that the reliability has been calculated as 46.6% by the reliability calculation unit 222 based on the recognition result.

[0143] Furthermore, in frame F4, it is shown that an object exists at the position of frame F4, and "bicycle" is written in the upper left corner, indicating that the object name has been recognized as "bicycle" through object recognition processing by the camera recognition unit 241, and "51.7" is written in the upper right corner, indicating that the reliability calculation unit 222 has calculated the reliability as 51.7% based on the recognition result.

[0144] That is, the object recognition process has recognized that a bicycle is located at the position of frame F3 with a reliability of 46.6%, and that a person is located at the position of frame F4 with a reliability of 51.7%.

[0145] In this way, the camera recognition unit 241 outputs a recognition result consisting of the position and name of an object in an image through object recognition processing. Then, the reliability calculation unit 222 calculates the reliability based on the recognition result, that is, the position and name of the object.

[0146] (Part 2) 7 is supplied from the camera of the data acquisition unit 102, the camera recognition unit 241 outputs a recognition result as shown in frames F11 to F16 in Fig. 7. The reliability calculation unit 222 calculates the reliability based on this recognition result.

[0147] That is, the image in FIG. 7 shows five vehicles V11 to V15 traveling.

[0148] In such a case, the camera recognition unit 241 outputs the recognition results shown in frames F1 to F14 and F16 for each of the vehicles V11 to V15.

[0149] That is, frames F1 to F14 and F16 indicate that an object exists at each position of frames F1 to F14 and F16.

[0150] Furthermore, "car" is written in the upper left corner of each of the frames F1 to F14 and F16, indicating that the object name has been recognized as a "vehicle" through object recognition processing by the camera recognition unit 241.

[0151] Furthermore, the upper right corner of frames F1 to F14 and F16 are marked with "84", "82", "64", "83", and "67", respectively, indicating that the reliability calculation unit 222 calculated the reliability as 84%, 82%, 64%, 83%, and 67% based on the recognition results.

[0152] That is, the object recognition process shows as a recognition result that vehicles are present at positions F1 to F14 and F16 with reliabilities of 84%, 82%, 64%, 83%, and 67%, respectively.

[0153] Furthermore, the camera recognition unit 241 outputs a recognition result for the central vehicle V14 as shown in frame F14.

[0154] In frame F14, "person" is written in the lower right corner, indicating that the object name has been recognized as "human" through object recognition processing by the camera recognition unit 241, and "65" is written further to the right, indicating that the reliability calculation unit 222 has calculated the reliability as 65% based on the recognition result.

[0155] That is, the object recognition process shows that a human being is present at the position of frame F14 with a reliability of 65%.

[0156] In this way, the camera recognition unit 241 outputs the position and name of an object in an image as a recognition result through object recognition processing. Then, the reliability calculation unit 222 calculates the reliability based on the recognition result.

[0157] As described above, the camera recognition unit 241 obtains the position and name of an object as a recognition result of the object recognition process, as well as the reliability.

[0158] Note that the results obtained by object recognition processing based on sensing results from other sensors other than cameras, such as LiDAR and millimeter wave radar, will not be described here, but will be explained assuming that they are basically the same.

[0159] <Deterioration evaluation method> (Outlier detection) The comparison unit 223 compares, for each type of sensor, the reliability of the past object recognition results stored in the memory unit 111 with the reliability calculated by the reliability calculation unit 222, and outputs the comparison result to the deterioration determination unit 224.

[0160] The deterioration determining unit 224 evaluates (determines) the deterioration of the sensor based on the comparison result of the comparing unit 223.

[0161] At this time, the comparison unit 223 may determine, as a comparison result, whether or not the current reliability calculated by the reliability calculation unit 222 is an outlier in comparison with the reliability of past object recognition results.

[0162] Here, an outlier is a reliability where, when the average value of the local density of each reliability and a predetermined number k of neighboring reliability is calculated, the difference from the average value of the local density is greater than a predetermined value.

[0163] More specifically, for example, when the reliability levels Pa to Pd are as shown in Fig. 8, the reciprocals of the average values of the distances from the data group of k neighboring points are defined as the local densities Da to Dd. Note that the reliability level Pa is the calculated reliability level, and the reliability levels Pb to Pd are the past accumulated reliability levels.

[0164] In the case of Figure 8, if k = 3, the average distance to the data group of k (= 3) points in the vicinity of reliability Pa is distance ra, the average distance to the data group of k points in the vicinity of reliability Pb is distance rb, the average distance to the data group of k points in the vicinity of reliability Pc is distance rc, and the average distance to the data group of k points in the vicinity of reliability Pd is distance rd.

[0165] Here, as shown in FIG. 8, the distances ra to rd are ra>rb, ra>rc, and ra>rd.

[0166] Therefore, the local density Da (∝1 / ra) of the reliability Pa is a sufficiently small value compared with the local densities Db to Dd of the reliability Pb to Pd.

[0167] Therefore, the reliability Pa is considered an outlier compared to the reliability Pb to Pd.

[0168] Therefore, the deterioration determination unit 224 may determine (evaluate) that deterioration has occurred when the reliability Pa, which is the current calculation result, is deemed to be an outlier based on the comparison result of the comparison unit 223.

[0169] However, it is known that the reliability of the recognition results in object recognition processing varies, and outliers occur with a certain probability even if the sensor is not deteriorated.

[0170] Therefore, if the reliability Pa, which is the current actual measurement value, is deemed to be an outlier and it is assumed that degradation has occurred, there is a risk that the sensor will be deemed to have deteriorated due to an outlier that occurs by chance, even though there is no deterioration in the sensor.

[0171] Therefore, the deterioration determination unit 224 may determine that deterioration has occurred based on the comparison result of the comparison unit 223, for example, when outliers occur consecutively a predetermined number of times, or when the cumulative number of outliers exceeds a predetermined number even if the occurrence of outliers is discontinuous.

[0172] Furthermore, when the cumulative number of outliers is greater than a first predetermined number and less than a second predetermined number that is greater than the first predetermined number, the deterioration determination unit 224 may determine that autonomous driving is possible, although the risk increases in accordance with the number of outliers, and may determine that autonomous driving is not possible when the cumulative number of outliers exceeds the second predetermined number.

[0173] (change point detection) The above has described an example of determining deterioration based on outliers, but the comparison unit 223 may also determine as a comparison result whether the difference between the acquired current reliability and a predicted value predicted from the reliability of past object recognition results is greater than a predetermined value.

[0174] For example, consider the case shown in FIG. 9, in which the vertical axis indicates reliability and the horizontal axis indicates time.

[0175] In Figure 9, from time t0 to around t60, the reliability value fluctuates near the predicted value as shown by line L1, but after time t60, the value fluctuates near the predicted value as shown by line L2.

[0176] That is, when the transition of reliability as shown in FIG. 9 is observed, it can be considered that a large change point in reliability as indicated by the difference Δ occurs in the vicinity of time t60.

[0177] The comparison unit 223 obtains a predicted value based on information on past reliability, and, for example, defines the square of the difference from the current reliability as the abnormality level, and outputs the result of the comparison to the deterioration determination unit 224.

[0178] The deterioration determining unit 224 may compare the abnormality level with a predetermined threshold value, and may determine (evaluate) that deterioration has occurred if the abnormality level is higher than the predetermined threshold value, for example.

[0179] However, as described above, the reliability of the recognition results in the object recognition process varies, and the degree of abnormality increases with a certain degree of probability even if the sensor is not deteriorated.

[0180] Therefore, the deterioration determination unit 224 may compare the degree of abnormality with a predetermined threshold value, and determine (evaluate) that deterioration has occurred, for example, if the state where the degree of abnormality is higher than the predetermined threshold value continues for a predetermined period of time.

[0181] In addition, the deterioration judgment unit 224 may be configured to determine that when the degree of abnormality is higher than a first threshold and lower than a second threshold that is higher than the first threshold, the risk increases corresponding to the degree of abnormality but autonomous driving is possible, and when the second threshold is exceeded, autonomous driving is not possible.

[0182] (Detection of abnormal areas) In the above, an example of determining deterioration based on change point detection has been explained. However, for example, in a case where the recognition object 61 does not change its position but moves around a predetermined vertical axis fixed at the same position, rotating 360 degrees while changing direction in a time series, the reliability theoretically changes periodically, so deterioration may be determined based on whether the periodic change continues or not.

[0183] By rotating the recognition target object 61, for example, the reliability as shown in FIG. 10 is detected in time series.

[0184] In FIG. 10, the vertical axis represents reliability and the horizontal axis represents time.

[0185] That is, theoretically, as the recognition target object 61 rotates, a waveform Wr of the reliability of the time series with a time width T as shown in FIG. 10 is repeated, but for some reason, a waveform We of the reliability of the time series as shown near the center of the figure may be detected instead of the waveform Wr.

[0186] Therefore, the comparison unit 223 calculates the average time series reliability waveform Wr based on the similarity of waveform changes from the time series changes in past reliability of the time series, and compares it with the time series reliability waveforms detected periodically in sequence.

[0187] For example, as shown in the lower part of FIG. 11, waveforms W2 and W4 are almost identical to waveform W1 of the average time series reliability, but waveform W3 is different.

[0188] Therefore, in order to detect the occurrence of such a waveform, the comparison unit 223 may output the area differences between the average waveform W1 of the reliability of the past time series and each of the waveforms W2 to W4 as the comparison result to the deterioration determination unit 224.

[0189] By this processing, the waveforms W2 and W4 are almost identical to the waveform W1, and therefore the area difference becomes almost zero.

[0190] However, for the waveform W3, the area difference between the waveform W1 and the waveform W3 is found as shown in the lower part of FIG.

[0191] For this reason, the comparison unit 223 outputs the magnitude of the area difference with the reference waveform W1 to the deterioration determination unit 224 as the comparison result.

[0192] The deterioration determination unit 224 may determine (evaluate) the presence or absence of an abnormal portion, which is a partial time series where the reliability of the time series deteriorates or an abnormality occurs, by comparing the magnitude of the area difference for each waveform of the reliability of the time series with a predetermined threshold.

[0193] Furthermore, as described above, it is known that the reliability of the recognition results in object recognition processing varies, and there is a certain probability that the reliability will vary even if the sensor is not deteriorated.

[0194] Therefore, the deterioration determination unit 224 may compare the magnitude of the area difference with a predetermined threshold value, and, for example, if the area difference is larger than the predetermined threshold value, determine (evaluate) that an abnormal area has occurred and deterioration has occurred.

[0195] In addition, when the area difference is greater than a first threshold and less than a second threshold greater than the first period, the deterioration determination unit 224 may determine that autonomous driving is possible, although the risk increases in accordance with the size of the area difference, and when the area difference is greater than the second threshold, it may determine that autonomous driving is not possible.

[0196] <Deterioration diagnosis process according to the first embodiment> Next, the degradation diagnosis process of the first embodiment will be described with reference to the flowcharts of FIGS.

[0197] Here, we will explain the processing when the camera recognition unit 241 realizes object recognition processing based on the captured image, which is the sensing result of the camera in the data acquisition unit 102, but since the object recognition processing is basically the same for other sensors based on the sensing results, we will omit the explanation. Furthermore, for multiple sensors, parallel processing is performed and deterioration diagnosis processing is performed in the same way.

[0198] In step S11 (FIG. 12), the deterioration judgment unit 224 of the deterioration diagnosis unit 182 determines whether or not an instruction to start deterioration diagnosis has been issued based on whether or not to pass through a test site 51 located near the entrance when entering a residential area during automatic driving.

[0199] In step S11, if passage through the test site 51 is not detected and the start of deterioration diagnosis is not instructed, the same processing is repeated.

[0200] In step S11, if an instruction to start a deterioration diagnosis is given upon passing through a testing station 51 provided at the entrance to a residential area, the process proceeds to step S12.

[0201] In step S12, the deterioration determining unit 224 notifies the situation recognizing unit 153 that a deterioration diagnosis has been instructed.

[0202] Accordingly, the situation recognition unit 153 instructs the planning unit 134 to move the vehicle 91 to a predetermined stopping position P within the test site 51, and the planning unit 134 creates a corresponding movement plan and controls the operation control unit 135 to move the vehicle 91 so that it can stop at the stopping position P.

[0203] In step S13, the deterioration determination unit 224 determines whether or not the vehicle 91 has stopped at a predetermined stopping position P, and if the vehicle 91 has not stopped, the process returns to step S12.

[0204] The processing of steps S12 and S13 is repeated until the vehicle 91 stops at the predetermined stopping position P, and the vehicle continues to move so as to be able to stop at the stopping position P.

[0205] Then, in step S13, if the vehicle 91 stops at a predetermined stopping position P, the process proceeds to step S14.

[0206] In step S14, a camera (for example, camera 91a) of the data acquisition unit 102 captures an image including a recognition target 61 installed in a predetermined direction and at a predetermined distance while the vehicle 91 is parked at a predetermined position P, and outputs the captured image to the camera recognition unit 241. At this time, the position information acquisition unit 245 acquires position information and time information of the vehicle from the GPS of the data acquisition unit 102, and supplies them to the recognition result acquisition unit 221.

[0207] Furthermore, when the vehicle 91 stops at a predetermined stopping position P, the recognition object 61 is installed in a predetermined direction at a predetermined distance, i.e., the vehicle 91 and the recognition object 61 maintain a predetermined positional relationship.

[0208] In step S15, the camera recognition unit 241 performs object recognition processing based on the image captured by the camera 91a, and outputs the position of the recognized object relative to the vehicle 91 and information on the name of the recognized object as the recognition result to the recognition result acquisition unit 221.

[0209] In step S16, the recognition result acquisition unit 221 outputs the recognition result to the reliability calculation unit 222, which calculates and stores the reliability of the recognition result.

[0210] In step S17, the recognition result acquisition unit 221 determines whether or not the recognition target object 61 has been recognized a predetermined number of times.

[0211] If it is determined in step S17 that the recognition target object 61 has not been recognized the predetermined number of times, the process returns to step S14.

[0212] That is, the processes of steps S14 to S17 are repeated until the images of the recognition target object 61 captured by the camera 91a or the like are subjected to object recognition processing a predetermined number of times.

[0213] Then, in step S17, if it is determined that the images of the recognition target object 61 captured by the camera 91a or the like have been subjected to object recognition processing a predetermined number of times, the process proceeds to step S18.

[0214] In step S18, the recognition result acquisition unit 221 averages the recognition result consisting of the position of the recognition target object 61, the object name, and the reliability information for a predetermined number of times, and stores the averaged result in the memory unit 111 in association with the position information and time information of the vehicle supplied from the position information acquisition unit 245.

[0215] Here, averaging the reliability information for a predetermined number of times may be, for example, the average value of the reliability calculated repeatedly a predetermined number of times, or the average value of the reliability calculated repeatedly a predetermined number of times excluding values that are significantly outliers. Here, for example, when the overall average of reliability is 80%, a value that is less than about 60% of the average, i.e., a reliability of 48% or less, may be set as an outlier.

[0216] In step S19 (FIG. 13), the comparison unit 223 compares the object name based on the current recognition result stored in the storage unit 111 with the correct object name, and supplies the comparison result to the deterioration determination unit 224. Based on the comparison result, the deterioration determination unit 224 determines whether the object name of the object recognition result is correct or whether the object name is similar to the correct object name.

[0217] If it is determined in step S19 that the object name in the object recognition result is correct or similar to the correct object name, the process proceeds to step S20.

[0218] In step S20, the comparison unit 223 compares the position as the recognition result based on the current recognition result stored in the storage unit 111 with the correct position, and supplies the comparison result to the deterioration determination unit 224. Based on the comparison result, the deterioration determination unit 224 determines whether the position as the recognition result is within an allowable range with respect to the correct position.

[0219] Here, whether the position as the recognition result is within an acceptable range of the correct position means whether the distance between the position as the recognition result and the correct position is within a specified distance and whether it can be recognized with a specified accuracy.

[0220] If it is determined in step S20 that the position as the recognition result is within the allowable range of the correct position, the process proceeds to step S21.

[0221] In step S21, the comparison unit 223 compares the calculated current reliability with the accumulated reliability stored in the storage unit 111, and outputs the comparison result to the deterioration determination unit 224. Based on the comparison result, the deterioration determination unit 224 determines whether the deterioration determined from the current reliability has fallen below a threshold value that requires a change in insurance premium.

[0222] The determination here may be made by, for example, at least one of outlier detection, change point detection, and abnormal portion detection described with reference to FIG. 8, or a combination thereof.

[0223] If it is determined in step S21 that the deterioration determined from the current reliability has not fallen below the threshold value that requires a change in the insurance premium, the process proceeds to step S22.

[0224] In step S22, the deterioration determination unit 224 determines that there is no effect due to deterioration, determines that autonomous driving is possible, and notifies the situation recognition unit 153. Automated driving by the vehicle 91 is then continued.

[0225] On the other hand, if it is determined in step S21 that the deterioration determined from the current reliability is lower than the threshold value that requires a change in the insurance premium, the process proceeds to step S23.

[0226] In step S23, the deterioration determination unit 224 determines, based on the comparison result, whether the deterioration determined from the current reliability is lower than a threshold value at which the vehicle is deemed undriveable.

[0227] In step S23, if it is determined based on the comparison result that the deterioration determined from the current reliability has not fallen below the threshold value at which the vehicle is deemed undriveable, the process proceeds to step S24.

[0228] In step S24, the deterioration determination unit 224 causes the insurance premium determination unit 225 to calculate an insurance premium according to the current degree of deterioration.

[0229] In step S25, the deterioration determination unit 224 determines whether the insurance premium calculated by the insurance premium determination unit 225 according to the deterioration is higher than a predetermined amount, and renewal approval is required from the user or owner.

[0230] In step S25, if the insurance premium calculated by the insurance premium determination unit 225 according to the deterioration is higher than a predetermined amount, which requires the user or owner to approve the renewal, the process proceeds to step S26.

[0231] In step S26, the deterioration judgment unit 224 controls the UI display control unit 226 based on the insurance premium information corresponding to deterioration calculated by the insurance premium determination unit 225 to display an image requesting the user or owner of the vehicle 91 to approve the insurance premium update.

[0232] For example, when requesting approval from the user by displaying an image on the display device 251 in the output unit 106 inside the vehicle, as shown in FIG. 14, the UI display control unit 226 may display an image requesting approval for updating the insurance premium, as shown in image P51.

[0233] Image P51 displays the message "The image sensor has deteriorated. This will affect safe driving, so your insurance premium will be XXX. Is this OK?" and displays a button B51 labeled "Yes" that is pressed to approve the request, and a button B52 labeled "No" that is pressed to deny the request.

[0234] 15, the UI display control unit 226 may display an image P51 on a projection image 261 on the windshield by a projection device in the output unit 106 inside the vehicle, to request approval from the user. Note that the projection image may be projected and displayed on, for example, a head-mounted display.

[0235] Furthermore, the UI display control unit 226 may control the communication unit 103 to display an image P51 on the display unit 281 of the terminal device 271 carried by the user, as shown in FIG. 16, to request approval from the user.

[0236] In addition, the UI display control unit 226 may control the communication unit 103 to notify the owner of the vehicle 91 who is not in the vehicle 91, for example, by email to the owner's personal computer 291, as shown in Figure 17, and cause the display unit 292 to display an image requesting approval for the insurance premium renewal, such as that shown in image P71.

[0237] Image P71 displays the message "The image sensor of self-driving vehicle ID XX has deteriorated. As this will affect safe driving, the insurance premium will be YYY. Is this OK?" and displays a button B71 labeled "Yes" that is pressed to approve the request, and a button B72 labeled "No" that is pressed to deny the request.

[0238] In addition, the UI display control unit 226 may control the communication unit 103 to display an image requesting approval for insurance premium renewal, such as that shown in image P71, via email or the like on the display unit 341 of the owner's terminal device 331 when the owner is not in the vehicle 91, as shown in Figure 18.

[0239] In step S27, the deterioration determination unit 224 determines whether or not the renewal of the insurance premium has been approved in response to the display of the image requesting approval for renewal of the insurance premium, which has been described with reference to FIGS.

[0240] In step S27, if the insurance premium renewal is deemed to have been approved by operating button B51 in FIGS. 14 to 16 or button B71 in FIGS. 17 and 18, the process proceeds to step S28.

[0241] In step S28, the deterioration determination unit 224 controls the communication unit 103 to access the server of the insurance company (not shown), renew the contract with the approved insurance premium, and the process proceeds to step S222.

[0242] In other words, in this case, while acknowledging the occurrence of deterioration, the insurance premium is recalculated to respond to the risk according to the level of deterioration, and when approval is requested by the user or owner, the insurance contract is renewed at the approved premium, and autonomous driving is continued.

[0243] Furthermore, in step S19, if it is determined that the object name in the object recognition result is neither correct nor similar to the correct object name, if it is determined in step S23 that the deterioration judged is lower than the threshold value at which the vehicle is deemed undriveable, or if in step S27, it is deemed that the insurance premium renewal is not approved because button B52 in Figures 14 to 16 or button B72 in Figures 17 and 18 is operated, processing proceeds to step S29 in all cases.

[0244] In step S29, the deterioration determination unit 224 determines that automatic driving is not possible due to the influence of the deterioration, and notifies the situation recognition unit 153 to that effect. Then, automatic driving by the vehicle 91 is stopped.

[0245] In step S30, the deterioration determination unit 224 controls the communication unit 103 to request a substitute vehicle for the vehicle 91 capable of automatic driving, if necessary.

[0246] Furthermore, in step S25, if the insurance premium calculated by the insurance premium determination unit 225 according to the deterioration is not higher than a predetermined amount that requires approval for renewal from the user or owner, the processing of steps S26 and S27 is skipped.

[0247] In other words, if the recalculated insurance premium is small enough that approval for renewal by the user or owner is not required, the insurance contract is automatically renewed at a recalculated insurance premium that acknowledges the occurrence of deterioration and corresponds to the risk according to the level of deterioration, without requesting approval from the user or owner, and autonomous driving is continued.

[0248] Through the above series of processes, if the object name in the object recognition result is correct, the location recognition result is within a specified range, and the deterioration in reliability is not so severe that a change in insurance premium is necessary, autonomous driving will continue.

[0249] Furthermore, if the object name in the object recognition result is correct, the location recognition result is within a specified range, and the deterioration in reliability requires a change in the insurance premium but does not make autonomous driving impossible, the insurance premium is recalculated according to the deterioration, and if the amount is lower than the specified amount, the insurance contract is automatically renewed with the recalculated insurance premium and autonomous driving continues, and if the amount is higher than the specified amount, approval is requested for renewal of the contract with the recalculated insurance premium, and if approval is given in response to the request, the contract is renewed with the recalculated insurance premium and autonomous driving continues.

[0250] Furthermore, if the object name in the object recognition result is incorrect, if the location recognition result is not within a specified range, or if the reliability has deteriorated to the point where a change in insurance premium is necessary and the insurance premium renewal is not approved, or if the reliability has deteriorated to the point where autonomous driving is no longer possible, autonomous driving will be stopped.

[0251] Furthermore, the above description of the process is an example of deterioration diagnosis using a camera among the sensors, but similar deterioration diagnosis processing is performed for all sensors included in the data acquisition unit 102.

[0252] This makes it possible to determine whether autonomous driving is possible or not depending on the degree of deterioration when deterioration of any of the sensors is detected.

[0253] As a result, it is possible to determine whether autonomous driving is possible before the sensor fails completely, thereby reducing the risk of an accident occurring due to failure.

[0254] In addition, when calculating the insurance premium, if deterioration of multiple sensors is recognized, the amount may be calculated based on the number of sensors deemed to be deteriorated in addition to the degree of deterioration of the sensors.

[0255] <<3. Modification of the First Embodiment>> In the above, we have described an example in which the recognition result acquisition unit 221 associates the recognition result consisting of the position and object name with the reliability and stores it in the memory unit 111, the comparison unit 223 compares the reliability accumulated in the past with the calculated reliability and outputs the comparison result to the deterioration determination unit 224, and the deterioration determination unit 224 determines deterioration based on the comparison result.

[0256] However, the memory unit 111 and the insurance premium determination unit 225 may be provided on a cloud server, and may communicate via the communication unit 103, so that the recognition result and the reliability are associated and stored, or the insurance premium is calculated and determined.

[0257] That is, FIG. 19 shows an example of the configuration of the deterioration diagnosis unit 182 in which the storage unit 111 and the insurance premium determination unit 225 are provided on a cloud server.

[0258] 19, components having the same functions as those in degradation diagnosis section 182 in FIG. 5 are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0259] That is, the deterioration diagnosis unit 182 in Figure 19 differs from the deterioration diagnosis unit 182 in Figure 5 in that the insurance premium determination unit 225 is omitted, and a memory unit 311 that functions in the same way as the memory unit 111 and an insurance premium determination unit 312 are provided in a cloud server 301 that can communicate via a network or the like using the communication unit 103.

[0260] With this configuration, even if storage unit 111 breaks down due to aging, for example, it is possible to restore the recognition results and reliability levels backed up in storage unit 311 in cloud server 301.

[0261] Furthermore, the storage unit 111 may be abolished from storing the association between the recognition result and the reliability, and the result may be stored only in the storage unit 311, so that the result can be exchanged via the communication unit 103 as necessary.

[0262] Furthermore, the insurance premium determination unit 312 may be provided, for example, on a cloud server 301 managed by the insurance company, so that the insurance premium can be calculated and the contract renewal can be smoothly carried out using the calculated new insurance premium.

[0263] Furthermore, the reliability calculation unit 222 , the comparison unit 223 , and the deterioration determination unit 224 may also be provided on the cloud server 301 .

[0264] The degradation diagnosis processing by degradation diagnosis section 182 in FIG. 19 is basically the same as the processing by degradation diagnosis section 182 in FIG. 5, and therefore a description thereof will be omitted.

[0265] <<4. Second Embodiment>> The above has described an example in which an automatically driven vehicle 91 performs a deterioration diagnosis process independently at a proving ground 51, but it is also possible for multiple vehicles 91 to upload their recognition results and reliability to a cloud server, and perform a deterioration diagnosis based on the recognition results and reliability of the other vehicles 91.

[0266] At this time, deterioration diagnosis may be performed taking into consideration the type of sensor, operation time, etc.

[0267] FIG. 20 shows an example configuration of a deterioration diagnosis unit 182 in which multiple vehicles 91 upload their recognition results and reliability to a cloud server, and deterioration diagnosis is performed based on the recognition results and reliability of other vehicles 91.

[0268] The functions of each component of the degradation diagnosis unit 182 in Figure 20 are basically the same as the components in Figure 5, but the recognition result acquisition unit 221 stores the recognition result and reliability in the memory unit 111, and controls the communication unit 103 to transmit the recognition result and reliability stored in the memory unit 111 to the memory unit 361 of the cloud server 351 and store them.

[0269] The cloud server 351 sequentially stores in a storage unit 361 the recognition results and the reliability levels supplied from the plurality of vehicles 91-1 to 91-n.

[0270] In addition, the comparison unit 223 in Figure 20 controls the communication unit 103 to compare past recognition results and reliability of other vehicles 91 stored in the cloud server 351 based on information on the recognition results and reliability of sensors that are the same or similar type as the sensors of the vehicle 91 of the subject vehicle and have similar operating times, and outputs the comparison result to the deterioration determination unit 224.

[0271] That is, the comparison unit 223 compares the recognition results and reliability of the other vehicle's sensors, for example, the average values of the recognition results and reliability of sensors that are the same or similar type as the vehicle's sensor and have similar operating times, with the recognition results and reliability of the vehicle's sensor, and outputs the comparison results to the deterioration determination unit 224.

[0272] Deterioration diagnosis using such comparison results enables comparison according to the characteristics specific to the sensor type, and makes it possible to realize deterioration diagnosis with higher accuracy.

[0273] In order to enable the recognition result acquisition unit 221 to select the recognition result and reliability of a sensor that is the same or similar type as the sensor of the vehicle itself and has a similar operating time, when storing the recognition result and reliability in the memory unit 361, the recognition result acquisition unit 221 stores information that identifies the vehicle 91 itself (vehicle ID), location information of the vehicle 91 (measurement location, measurement location ID, etc.) and time information (date and time), and information on the sensor model (sensor name) and operating time (date of start of use, usage time, etc.) in association with each other.

[0274] This makes it possible to perform deterioration diagnosis processing based on the results of comparison with a sensor that is identical to or similar to the sensor used in another vehicle 91 and has a similar operating time, in other words, a sensor that has experienced approximately the same deterioration over time.

[0275] As a result, it is possible to realize deterioration diagnosis processing that takes into account the characteristics of each sensor model and aging deterioration.

[0276] <Deterioration diagnosis process according to the second embodiment> Next, the degradation diagnosis process of the second embodiment will be described with reference to the flowcharts of Figures 21 and 22. Note that the processes of steps S41 to S48 in Figure 21 are the same as the processes of steps S11 to S18 in Figure 12, and therefore description thereof will be omitted.

[0277] That is, in step S48, the recognition result acquisition unit 221 stores the recognition result consisting of the position and object name of the recognition target object 61, as well as reliability information, in the memory unit 111 in association with the position information and time information of the vehicle supplied from the position information acquisition unit 245, and then the processing proceeds to step S49.

[0278] In step S49, the recognition result acquisition unit 221 transmits the recognition result including the position and object name of the recognition target object 61, and reliability information to the cloud server 351. At this time, the recognition result acquisition unit 221 averages the reliability information and transmits it to the cloud server 351.

[0279] At this time, the recognition result acquisition unit 221 transmits the recognition result and reliability information to the cloud server 351 in association with information identifying the vehicle 91 (vehicle ID), the location information (measurement location, measurement location ID, etc.) supplied by the location information acquisition unit 245, time information (date and time), as well as information on the sensor model (sensor name) and sensor operating time (date of start of use, usage time, etc.).

[0280] In step S91, the cloud server (controller not shown) 351 determines whether or not the recognition result and reliability information have been transmitted from the vehicle 91.

[0281] In step S91, if the recognition result and reliability information are transmitted, the process proceeds to step S92.

[0282] In step S92, the cloud server 351 receives the transmitted recognition result and reliability information.

[0283] In step S93, the cloud server 351 stores the received recognition result and reliability information in the storage unit 361.

[0284] At this time, the cloud server 351 stores the received recognition results and reliability information in association with information identifying the vehicle 91 (vehicle ID), location information of the vehicle 91 (measurement location, measurement location ID, etc.), time information (date and time), sensor model (sensor name), and sensor operating time (start date of use, usage time, etc.).

[0285] Through this series of processes, the recognition results and reliability information supplied from vehicles 91-1 to 91-n are stored in memory unit 361 in association with information identifying vehicle 91 (vehicle ID), location information of vehicle 91 (measurement location, measurement location ID, etc.), time information (date and time), sensor model (sensor name), and sensor operating time (start date of use, usage time, etc.).

[0286] In step S91, if the recognition result and reliability information are not transmitted, the processes of steps S92 and S93 are skipped.

[0287] Here, we return to the description of the processing by the deterioration diagnosis unit 182 of the vehicle 91.

[0288] When the recognition results and information averaging the reliability a predetermined number of times are stored in the storage unit 111 and the storage unit 361 of the cloud server 351 by the processes of steps S44 to S49, the process proceeds to step S50.

[0289] In step S50, comparison unit 223 controls communication unit 103 to request the accumulated sensor recognition results and reliability from cloud server 351 on the network.

[0290] At this time, the comparison unit 223 also transmits information identifying its own vehicle 91 (vehicle ID), location information of the vehicle 91 (measurement location, measurement location ID, etc.), time information (date and time), sensor model (sensor name), and sensor operating time (start date of use, usage time, etc.), and requests the recognition results and reliability of sensors of the same or similar model as that of its own vehicle and with similar operating time from the recognition results and reliability stored in the memory unit 361 on the cloud server 351.

[0291] In step S94, the cloud server (controller not shown) 351 determines whether or not the accumulated recognition results and reliability have been requested from any vehicle 91.

[0292] If the accumulated recognition results and confidence levels are requested in step S94, the process proceeds to step S95.

[0293] In step S95, the cloud server 351 searches and extracts the accumulated recognition results and reliability of sensors that are the same or similar type as the sensor of the requested vehicle 91 and have similar operating times from the recognition results and reliability stored in the memory unit 361, and transmits them to the requested vehicle 91.

[0294] If the accumulated recognition results and confidence levels are not required in step S94, the process of step S95 is skipped.

[0295] In step S96, the cloud server 351 determines whether or not an instruction to end the process has been given. If an instruction to end the process has not been given, the process returns to step S91, and the subsequent processes are repeated.

[0296] Then, in step S96, the process ends.

[0297] Meanwhile, in vehicle 91, in step S51, comparison unit 223 controls communication unit 103 to receive accumulated recognition results and reliability of sensors that are the same as or similar in type to the sensor of the vehicle itself and have similar operating times, transmitted from cloud server 351.

[0298] In step S52 (FIG. 22), the comparison unit 223 compares the object name based on the current recognition result stored in the storage unit 111 with the correct object name, and supplies the comparison result to the deterioration determination unit 224. Based on the comparison result, the deterioration determination unit 224 determines whether the object name of the object recognition result is correct or whether the object name is similar to the correct object name. Note that the object name similar to the correct object name may be the object name of the recognition result of the same or similar sensor acquired in the processing of step S51 and having a similar operating time.

[0299] If it is determined in step S52 that the object name in the object recognition result is correct or similar to the correct object name, the process proceeds to step S53.

[0300] In step S53, the comparison unit 223 compares the position based on the current recognition result with the correct position, and supplies the comparison result to the deterioration determination unit 224. Based on the comparison result, the deterioration determination unit 224 determines whether the position of the recognition result is within an allowable range with respect to the correct position.

[0301] Here, whether the position recognition result is within an acceptable range relative to the correct position refers to, for example, the average value of the distance from the correct position based on the position of the recognition result of the same or similar sensor obtained in the processing of step S51 and with a similar operating time, and may also refer to whether the recognition is performed with sufficient accuracy in comparison with sensors of other vehicles.

[0302] If it is determined in step S53 that the position of the recognition result is within the allowable range of the correct position, the process proceeds to step S54.

[0303] In step S54, the comparison unit 223 compares the current reliability based on the sensing results of the sensor of the vehicle with a threshold value (for example, an average value) determined from the reliability of the other vehicle acquired in the processing of step S51, and outputs the comparison result to the deterioration determination unit 224. Based on the comparison result, the deterioration determination unit 224 determines whether the deterioration determined from the current reliability has fallen below a threshold value that requires a change in insurance premium.

[0304] The determination here may be made, for example, based on the reliability of the other vehicle, by at least one of outlier detection, change point detection, and abnormal part detection described with reference to Figures 8 to 12, or a combination thereof.

[0305] If it is determined in step S54 that the deterioration determined from the currently calculated reliability has not fallen below the threshold value that requires a change in the insurance premium, the process proceeds to step S55.

[0306] In step S55, the deterioration determination unit 224 determines that there is no effect of deterioration, and that automatic driving is possible, and notifies the situation recognition unit 153 of this fact. Then, automatic driving by the vehicle 91 continues.

[0307] On the other hand, if it is determined in step S54 that the deterioration determined from the current reliability is lower than the threshold value that requires a change in the insurance premium, the process proceeds to step S56.

[0308] In step S56, the deterioration determination unit 224 determines, based on the comparison result, whether the deterioration determined from the current reliability has decreased below a threshold value at which the vehicle is deemed undriveable.

[0309] In step S56, if it is determined based on the comparison result that the deterioration determined from the current reliability has not fallen below the threshold value at which the vehicle is deemed undriveable, the process proceeds to step S57.

[0310] In step S57, the deterioration determination unit 224 causes the insurance premium determination unit 225 to calculate an insurance premium according to the current degree of deterioration.

[0311] In step S58, the deterioration determination unit 224 determines whether the insurance premium calculated by the insurance premium determination unit 225 according to the deterioration is higher than a predetermined amount, which requires the user or owner to approve the renewal.

[0312] In step S58, if the insurance premium calculated by the insurance premium determination unit 225 according to the deterioration is higher than a predetermined amount, which requires the user or owner to approve the renewal, the process proceeds to step S59.

[0313] In step S59, the deterioration judgment unit 224 controls the UI display control unit 226 based on the insurance premium information corresponding to deterioration calculated by the insurance premium determination unit 225 to display the image described with reference to Figures 14 to 18, requesting the user or owner of the vehicle 91 to approve the insurance premium update.

[0314] In step S60, the deterioration determination unit 224 determines whether or not the renewal of the insurance premium has been approved in response to the display of the image requesting approval for renewal of the insurance premium, which has been described with reference to FIGS.

[0315] In step S60, if the insurance premium renewal is deemed to have been approved by operating button B51 in FIGS. 14 to 16 or button B71 in FIGS. 17 and 18, the process proceeds to step S61.

[0316] In step S61, the deterioration determination unit 224 controls the communication unit 103 to access the server of the insurance company (not shown), renew the contract with the approved insurance premium, and the process proceeds to step S55.

[0317] In other words, in this case, by comparing the recognition results and reliability information of the same or similar sensors in other vehicles that have similar operating times with the recognition results and reliability information of the vehicle itself, the occurrence of deterioration is acknowledged, and in order to respond to risks according to the level of deterioration, the insurance premium is recalculated, and when approval is requested by the user or owner, the insurance contract is renewed at the approved premium, and autonomous driving is continued.

[0318] Furthermore, if it is determined in step S52 that the object name in the object recognition result is neither correct nor similar to the correct object name, if it is determined in step S56 that the deterioration is lower than the threshold value at which the vehicle is deemed undriveable, or if it is determined in step S60 that the insurance premium renewal is not approved because button B52 in Figures 14 to 16 or button B72 in Figures 17 and 18 is operated, processing proceeds to step S62 in all cases.

[0319] In step S62, the deterioration determination unit 224 determines that autonomous driving is not possible due to the influence of the deterioration, and notifies the situation recognition unit 153 to that effect. As a result, autonomous driving by the vehicle 91 is stopped.

[0320] In step S63, the deterioration determination unit 224 controls the communication unit 103 to request a substitute vehicle 91 capable of automatic driving, if necessary.

[0321] Furthermore, in step S58, if the insurance premium calculated by the insurance premium determination unit 225 according to the deterioration is not higher than a predetermined amount that requires approval for renewal from the user or owner, the processing of steps S59 and S60 is skipped.

[0322] In other words, by comparing the recognition results and reliability information of the same or similar sensors in other vehicles that have similar operating times with the recognition results and reliability information of the vehicle itself, the system acknowledges the occurrence of deterioration, recalculates the insurance premium to correspond to the risk according to the level of deterioration, and automatically renews the insurance contract without requesting approval from the user or owner, thereby allowing autonomous driving to continue.

[0323] Through the above series of processes, based on the recognition results and reliability information of sensors of other vehicles that are the same or similar type as the vehicle's sensor and have similar operating times, if the object name in the object recognition results is correct, the position recognition results are within a specified range, and the deterioration in reliability is not so severe that a change in insurance premium is necessary, autonomous driving will continue.

[0324] Furthermore, if the object name in the object recognition result is correct, the location recognition result is within a specified range, and the deterioration in reliability requires a change in the insurance premium but does not make autonomous driving impossible, the insurance premium is recalculated according to the deterioration, and if the amount is lower than the specified amount, the insurance contract is automatically renewed with the recalculated insurance premium and autonomous driving continues, and if the amount is higher than the specified amount, approval is requested for renewal of the contract with the recalculated insurance premium, and if approval is given in response to the request, the contract is renewed with the recalculated insurance premium and autonomous driving continues.

[0325] Furthermore, if the object name in the object recognition result is incorrect, if the location recognition result is not within a specified range, or if the reliability has deteriorated to the point where a change in insurance premium is necessary and the insurance premium renewal is not approved, or if the reliability has deteriorated to the point where autonomous driving is no longer possible, autonomous driving will be stopped.

[0326] Furthermore, the above description of the process is an example of deterioration diagnosis using a camera among the sensors, but similar deterioration diagnosis processing is performed for all sensors included in the data acquisition unit 102.

[0327] This makes it possible to determine whether autonomous driving is possible or not depending on the degree of deterioration when deterioration of any of the sensors is detected.

[0328] This also makes it possible to perform deterioration diagnosis processing based on the results of comparison with a sensor that is the same as or similar to the sensor used in the other vehicle 91 and has a similar operating time, in other words, a sensor that has experienced approximately the same deterioration over time.

[0329] As a result, by performing a deterioration diagnosis process that takes into account the characteristics of each sensor model and deterioration over time, it is possible to accurately determine whether autonomous driving is possible before the sensor reaches a state where it completely fails, thereby reducing the risk of an accident occurring due to failure.

[0330] <<5. Application Examples>> The above has described a deterioration diagnosis system 31 in which deterioration diagnosis processing is performed on a vehicle 91 at a test site 51 set up at the entrance of various areas where high safety is required, such as residential areas, but it is also possible to configure a deterioration diagnosis system 31 in which deterioration diagnosis processing is performed at a specific outdoor location, for example, when the vehicle is stopped at a traffic light.

[0331] That is, for example, as shown in the deterioration diagnosis system 31 of FIG. 23, when a vehicle 91 traveling on a road 383 stops at a stop position P111 that is a predetermined distance D11 from a traffic light 381 because the traffic light 381 is red, deterioration diagnosis processing may be performed using a sign 382 or the like that is located a predetermined distance D12 from the stop position P111 as a recognition object.

[0332] In this way, a configuration such as that of the test site 51 is not required, and the deterioration diagnosis process can be easily realized.

[0333] In addition, in the deterioration diagnosis system 31 of Figure 23, since the deterioration diagnosis process is performed outdoors without using the test site 51, it is necessary to evaluate the object recognition results taking into account the influence of the outdoor environment, such as brightness and wind, depending on the weather and time of day.

[0334] For example, by executing a degradation diagnosis process using the recognition results and reliability of other vehicles under the same environmental conditions, it is possible to realize a degradation diagnosis process that takes into account the influence of the environment.

[0335] <<6. Example of execution by software>> The above-described series of processes can be executed by hardware, but can also be executed by software. When the series of processes are executed by software, the programs constituting the software are installed from a recording medium into a computer incorporated in dedicated hardware, or into, for example, a general-purpose computer that can execute various functions by installing various programs.

[0336] 24 shows an example of the configuration of a general-purpose computer. This personal computer has a built-in CPU (Central Processing Unit) 1001. An input / output interface 1005 is connected to the CPU 1001 via a bus 1004. A ROM (Read Only Memory) 1002 and a RAM (Random Access Memory) 1003 are connected to the bus 1004.

[0337] Connected to the input / output interface 1005 are an input unit 1006 including input devices such as a keyboard and a mouse through which a user inputs operation commands, an output unit 1007 that outputs a processing operation screen and images of processing results to a display device, a storage unit 1008 including a hard disk drive or the like that stores programs and various data, and a communication unit 1009 including a LAN (Local Area Network) adapter or the like that executes communication processing via a network typified by the Internet. Also connected to the input / output interface 1005 is a drive 1010 that reads and writes data from / to removable storage media 1011 such as a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (Compact Disc-Read Only Memory) and a DVD (Digital Versatile Disc)), a magneto-optical disk (including an MD (Mini Disc)), or a semiconductor memory.

[0338] The CPU 1001 executes various processes in accordance with a program stored in a ROM 1002 or a program read from a removable storage medium 1011 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, installed in a storage unit 1008, and loaded from the storage unit 1008 into a RAM 1003. The RAM 1003 also stores data necessary for the CPU 1001 to execute various processes as appropriate.

[0339] In a computer configured as described above, the CPU 1001 performs the above-described series of processes by, for example, loading a program stored in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing it.

[0340] The program executed by the computer (CPU 1001) can be provided by being recorded on a removable storage medium 1011 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0341] In a computer, a program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting a removable storage medium 1011 into the drive 1010. The program can also be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. Alternatively, the program can be installed in the ROM 1002 or the storage unit 1008 in advance.

[0342] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0343] 24 realizes the functions of the degradation diagnosis unit 182 in FIGS. 5, 19, and 20. In FIG.

[0344] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0345] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0346] For example, the present disclosure can be configured as a cloud computing system in which a single function is shared and processed collaboratively by multiple devices via a network.

[0347] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0348] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0349] Furthermore, although the above-described embodiment is directed to executing a deterioration diagnosis process for a sensor, it is also possible to perform a safety performance evaluation of a sensor using a similar method.

[0350] The present disclosure can also be configured as follows.

[0351] <1> a recognition result acquisition unit that acquires a recognition result of a predetermined recognition target based on sensing results of a plurality of sensors mounted on the vehicle; a comparison unit that compares the acquired recognition result with another recognition result different from the acquired recognition result and outputs the result as a comparison result; a degradation evaluation unit that evaluates performance degradation of each of the sensors based on the comparison result by the comparison unit; An information processing system comprising: <2> The other recognition results are past recognition results of the own vehicle. <1> An information processing system according to claim 1. <3> further comprising a storage unit that stores the recognition result acquired by the recognition result acquisition unit; The comparison unit compares the recognition result stored in the storage unit with the acquired recognition result as the other recognition result. <2> An information processing system according to claim 1. <4> the comparison unit calculates outliers based on the acquired recognition result and the other recognition results, and compares the number of the outliers with a predetermined threshold; The degradation assessment unit assesses performance degradation of each of the sensors based on a comparison result between the individual outlier value of each of the sensors and the predetermined threshold value. <1> An information processing system according to claim 1. <5> the comparison unit detects a change point based on the acquired recognition result and the other recognition result, and compares an amount of change at the change point with a predetermined threshold; The deterioration evaluation unit evaluates the performance deterioration of each of the sensors based on a comparison result between the amount of change at each of the change points of the sensors and the predetermined threshold value, made by the comparison unit. <1> An information processing system according to claim 1. <6> The other recognition result is a recognition result of another vehicle different from the own vehicle. <1> An information processing system according to claim 1. <7> The other recognition result is a recognition result based on a sensing result of a sensor of the other vehicle that is the same as or similar to the sensor of the host vehicle. <6> An information processing system according to claim 1. <8> The other recognition result is a recognition result based on a sensing result of a sensor of the other vehicle that has the same operating time as or a similar operating time to the sensor of the host vehicle. <6> An information processing system according to claim 1. <9> further comprising a storage unit that stores the recognition result acquired by the recognition result acquisition unit; the storage unit is provided in a cloud server, The comparison unit compares the recognition result stored in the storage unit of the cloud server with the acquired recognition result as the other recognition result. <6> An information processing system according to claim 1. <10> The vehicle further includes a situation recognition unit that stops automatic driving of the vehicle when the degradation evaluation unit evaluates that the performance degradation of the sensor is greater than a predetermined threshold. <1> An information processing system according to claim 1. <11> an insurance premium determination unit that determines an insurance premium according to the deterioration when the deterioration evaluation unit evaluates that the performance deterioration of the sensor is deteriorated below a predetermined threshold; When the renewal of the insurance contract with the insurance premium according to the deterioration is approved, the situation recognition unit continues the automatic driving of the vehicle. <10> An information processing system according to claim 1. <12> When the deterioration evaluation unit evaluates that the performance deterioration of the sensor is deteriorated below a predetermined threshold, the insurance premium determination unit determines an insurance premium according to the deterioration, and notifies the user or owner of the vehicle of information requesting approval for renewal of the insurance contract with an insurance premium according to the deterioration, and when renewal of the insurance contract with an insurance premium according to the deterioration is approved in response to the notification, the situation recognition unit continues automatic driving of the vehicle. <11> An information processing system according to claim 1. <13> When the deterioration evaluation unit evaluates that the performance deterioration of the sensor is deteriorated below a predetermined threshold, the insurance premium determination unit determines an insurance premium according to the deterioration, and when renewal of the insurance contract with the insurance premium according to the deterioration is not approved, the situation recognition unit stops automatic driving of the vehicle. <11> An information processing system according to claim 1. <14> The comparison unit and the deterioration assessment unit are provided in a cloud server. <1> An information processing system according to claim 1. <15> The recognition result acquisition unit acquires a recognition result of the predetermined object to be recognized based on sensing results of a plurality of sensors mounted on the vehicle in a space in which a positional relationship between the vehicle and the predetermined object to be recognized is known. <1> An information processing system according to claim 1. <16> The space in which the positional relationship between the vehicle and the predetermined recognition object is known is a space that is not affected by the environment. <14> An information processing system according to claim 1. <17> a space in which the vehicle and the predetermined recognition object have a known positional relationship is a space that is affected by the environment; The deterioration assessment unit assesses performance deterioration of each of the sensors based on the individual comparison results of the sensors by the comparison unit, taking into account the influence of the environment. <16> An information processing system according to claim 1. <18> the comparison unit compares the reliability based on the recognition result with the reliability based on the other recognition result, The degradation assessment unit assesses performance degradation of the sensor based on a comparison result between the reliability of the recognition result and the reliability of the other recognition result. <16> An information processing system according to claim 1. <19> The object to be recognized further has a known object name, the recognition result is a detected position and an object name of the object to be recognized; The apparatus further includes a reliability calculation unit that calculates a reliability of the recognition result based on the recognition result. <17> An information processing system according to claim 1. <20> Obtaining a recognition result of a predetermined recognition target based on sensing results of a plurality of sensors mounted on the vehicle; comparing the acquired recognition result with another recognition result different from the acquired recognition result, and outputting the result as a comparison result; Based on the comparison results, the performance degradation of each of the sensors is evaluated. An information processing method comprising the steps. <21> a recognition result acquisition unit that acquires a recognition result of a predetermined recognition target based on sensing results of a plurality of sensors mounted on the vehicle; a comparison unit that compares the acquired recognition result with another recognition result different from the acquired recognition result and outputs the result as a comparison result; a degradation evaluation unit that evaluates performance degradation of each of the sensors based on the comparison result by the comparison unit; An information processing device comprising: [Explanation of symbols]

[0352] 31 Deterioration diagnosis system, 51 Proving ground, 61 Recognition object, 91, 91-1 to 91-n Vehicle, 91a Camera, 91b LiDAR, 91c Millimeter wave radar, 103 Communication unit, 111 Memory unit, 143 Vehicle state detection unit, 181 Deterioration diagnosis unit, 220 Recognition unit, 221 Recognition result acquisition unit, 222 Reliability calculation unit, 223 Comparison unit, 224 Deterioration judgment unit, 225 Insurance premium determination unit, 226 UI display control unit, 241 Camera recognition unit, 242 LiDAR recognition unit, 243 Millimeter wave radar recognition unit, 244 Other sensor recognition unit, 225 Location information acquisition unit, 301 Cloud server, 311 Memory unit, 312 Insurance premium determination unit, 351 Cloud server 361 memory unit, 223 third feature amount calculation unit, 271 second feature amount calculation unit, 272 third feature amount calculation unit

Claims

1. a recognition result acquisition unit that acquires a recognition result of a predetermined recognition target based on sensing results of a plurality of sensors mounted on the vehicle; a comparison unit that compares the acquired recognition result with another recognition result different from the acquired recognition result and outputs the result as a comparison result; a degradation evaluation unit that evaluates performance degradation of each of the sensors based on the comparison result by the comparison unit; an insurance premium determination unit that determines an insurance premium according to the deterioration when the deterioration evaluation unit evaluates that the performance deterioration of the sensor is deteriorated below a predetermined threshold; a situation recognition unit that determines whether the vehicle is capable of autonomous driving based on the evaluation result of the deterioration evaluation unit, When the degradation evaluation unit evaluates that the performance degradation of the sensor is more than a predetermined threshold, When the renewal of the insurance contract with the insurance premium determined by the insurance premium determination unit according to the deterioration is approved, the situation recognition unit determines that autonomous driving of the vehicle is possible, When the renewal of the insurance contract with the insurance premium determined by the insurance premium determination unit according to the deterioration is not approved, the situation recognition unit determines that the vehicle is not capable of autonomous driving. Information processing system.

2. The other recognition results are past recognition results of the own vehicle. The information processing system according to claim 1 .

3. further comprising a storage unit that stores the recognition result acquired by the recognition result acquisition unit; The comparison unit compares the recognition result stored in the storage unit with the acquired recognition result as the other recognition result. The information processing system according to claim 2 .

4. the comparison unit calculates outliers based on the acquired recognition result and the other recognition results, and compares the number of the outliers with a predetermined threshold; The degradation assessment unit assesses performance degradation of each of the sensors based on a comparison result between the individual outlier value of each of the sensors and the predetermined threshold value. The information processing system according to claim 1 .

5. the comparison unit detects a change point based on the acquired recognition result and the other recognition result, and compares an amount of change at the change point with a predetermined threshold; The deterioration evaluation unit evaluates the performance deterioration of each of the sensors based on a comparison result between the amount of change at each of the change points of the sensors and the predetermined threshold value, made by the comparison unit. The information processing system according to claim 1 .

6. The other recognition result is a recognition result of another vehicle different from the own vehicle. The information processing system according to claim 1 .

7. The other recognition result is a recognition result based on a sensing result of a sensor of the other vehicle that is the same as or similar to the sensor of the host vehicle. The information processing system according to claim 6.

8. The other recognition result is a recognition result based on a sensing result of a sensor of the other vehicle that has the same operating time as or a similar operating time to the sensor of the host vehicle. The information processing system according to claim 6.

9. further comprising a storage unit that stores the recognition result acquired by the recognition result acquisition unit; the storage unit is provided in a cloud server, The comparison unit compares the recognition result stored in the storage unit of the cloud server with the acquired recognition result as the other recognition result. The information processing system according to claim 6.

10. When the situation recognition unit determines that the vehicle is not capable of automatic driving, the situation recognition unit stops the automatic driving of the vehicle. The information processing system according to claim 1 .

11. When the situation recognition unit determines that autonomous driving of the vehicle is possible, the situation recognition unit continues autonomous driving of the vehicle. The information processing system according to claim 1 .

12. When the deterioration evaluation unit evaluates that the performance deterioration of the sensor is deteriorated below a predetermined threshold, the insurance premium determination unit determines an insurance premium according to the deterioration, and notifies the user or owner of the vehicle of information requesting approval for renewal of the insurance contract with an insurance premium according to the deterioration, and when renewal of the insurance contract with an insurance premium according to the deterioration is approved in response to the notification, the situation recognition unit determines that autonomous driving of the vehicle is possible and allows the vehicle to continue autonomous driving. The information processing system according to claim 11.

13. When the deterioration evaluation unit evaluates that the performance deterioration of the sensor is deteriorated below a predetermined threshold, the insurance premium determination unit determines an insurance premium according to the deterioration, and when renewal of the insurance contract with the insurance premium according to the deterioration is not approved, the situation recognition unit determines that autonomous driving of the vehicle is impossible and stops autonomous driving of the vehicle. The information processing system according to claim 11.

14. The comparison unit and the deterioration assessment unit are provided in a cloud server. The information processing system according to claim 1 .

15. The recognition result acquisition unit acquires a recognition result of the predetermined object to be recognized based on sensing results of a plurality of sensors mounted on the vehicle in a space in which a positional relationship between the vehicle and the predetermined object to be recognized is known. The information processing system according to claim 1 .

16. The space in which the positional relationship between the vehicle and the predetermined recognition object is known is a space that is not affected by the environment. The information processing system according to claim 14.

17. a space in which the vehicle and the predetermined recognition object have a known positional relationship is a space that is affected by the environment; The deterioration assessment unit assesses performance deterioration of each of the sensors based on the individual comparison results of the sensors by the comparison unit, taking into account the influence of the environment.

17. The information processing system according to claim 16.

18. the comparison unit compares the reliability based on the recognition result with the reliability based on the other recognition result, The degradation assessment unit assesses performance degradation of the sensor based on a comparison result between the reliability of the recognition result and the reliability of the other recognition result.

17. The information processing system according to claim 16.

19. The object to be recognized further has a known object name, the recognition result is a detected position and an object name of the object to be recognized; The apparatus further includes a reliability calculation unit that calculates a reliability of the recognition result based on the recognition result.

18. The information processing system according to claim 17.

20. The automated driving includes autonomous driving or driver assistance. The information processing system according to claim 1 .

21. acquiring a recognition result of a predetermined recognition target based on sensing results of a plurality of sensors mounted on the vehicle; a comparison process for comparing the acquired recognition result with another recognition result different from the acquired recognition result and outputting the comparison result; performing a degradation evaluation process for evaluating individual performance degradation of the sensor based on the comparison result; performing a situation recognition process for determining whether the vehicle is capable of automatic driving based on an evaluation result of the deterioration evaluation process; and when the degradation evaluation process evaluates that the performance degradation of the sensor is more than a predetermined threshold, performing an insurance premium determination process to determine an insurance premium according to the degradation, When the degradation evaluation process evaluates that the performance degradation of the sensor is lower than a predetermined threshold, When the renewal of the insurance contract with the insurance premium determined by the insurance premium determination process according to the deterioration is approved, the situation recognition process determines that autonomous driving of the vehicle is possible, An information processing method in which, when renewal of the insurance contract at the insurance premium determined by the insurance premium determination process according to the deterioration is not approved, the situation recognition process determines that autonomous driving of the vehicle is not possible.

22. a recognition result acquisition unit that acquires a recognition result of a predetermined recognition target based on sensing results of a plurality of sensors mounted on the vehicle; a comparison unit that compares the acquired recognition result with another recognition result different from the acquired recognition result and outputs the result as a comparison result; a degradation evaluation unit that evaluates performance degradation of each of the sensors based on the comparison result by the comparison unit; an insurance premium determination unit that determines an insurance premium according to the deterioration when the deterioration evaluation unit evaluates that the performance deterioration of the sensor is deteriorated below a predetermined threshold; a situation recognition unit that determines whether the vehicle is capable of autonomous driving based on the evaluation result of the deterioration evaluation unit, When the degradation evaluation unit evaluates that the performance degradation of the sensor is more than a predetermined threshold, When the renewal of the insurance contract with the insurance premium determined by the insurance premium determination unit according to the deterioration is approved, the situation recognition unit determines that autonomous driving of the vehicle is possible, When the renewal of the insurance contract with the insurance premium determined by the insurance premium determination unit according to the deterioration is not approved, the situation recognition unit determines that the vehicle is not capable of autonomous driving. Information processing device.

Images