Sensor control device, information processing device, method and program for controlling sensor control device, and method and program for controlling information processing device

Abstract

The present invention enables optimization of power consumption for performing information processing for acquiring information necessary for automatic driving control. A sensor control device (30) includes: a level setting unit (332) for acquiring an automatic driving level of a vehicle (V) capable of switching the automatic driving level; and an accuracy setting unit (333) for setting the sensing accuracy of a sensor mounted on the vehicle on the basis of the acquired automatic driving level.

Description

Sensor control apparatus, information processing apparatus, control method and program for sensor control apparatus, and control method and program for information processing apparatus

[0001] The present disclosure relates to a sensor control apparatus, an information processing apparatus, a control method and program for a sensor control apparatus, and a control method and program for an information processing apparatus.

[0002] In a vehicle capable of both autonomous driving and manual driving, there is known a technology that proposes switching between autonomous driving and manual driving of the vehicle and controls the operation of the autonomous driving function based on an instruction from a user. Further, there is known a technology for stopping autonomous driving in accordance with the detection capability of a sensor mounted on the vehicle while autonomous driving is being performed.

[0003] Japanese Patent Application Laid-Open No. 2016-095831

[0004] However, in such services, a large amount of electric power is required to operate the sensors that enable autonomous driving. Furthermore, a large amount of electric power is also required for information processing for using information acquired from sensors in autonomous driving, and there is room for improvement in terms of suppressing such power consumption.

[0005] The present disclosure has been made in view of the above, and an object of the present disclosure is to provide a sensor control apparatus, an information processing apparatus, a control method and program for a sensor control apparatus, and a control method and program for an information processing apparatus, which are capable of optimizing power consumption for performing information processing for acquiring information necessary for autonomous driving control.

[0006] The sensor control apparatus according to the present embodiment includes: a level setting unit that acquires the autonomous driving level of a vehicle capable of switching autonomous driving levels; and an accuracy setting unit that sets the sensing accuracy of a sensor mounted on the vehicle based on the acquired autonomous driving level.

[0007] A control method for a sensor control apparatus according to the present embodiment includes the steps of: acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; and setting sensing accuracy of a sensor mounted on the vehicle based on the acquired autonomous driving level.

[0008] The program according to this embodiment causes a computer to perform the steps of acquiring the autonomous driving level of a vehicle whose autonomous driving level can be switched, and setting the sensing accuracy of sensors mounted on the vehicle based on the acquired autonomous driving level.

[0009] The information processing device according to this embodiment includes a level setting unit that acquires the autonomous driving level of a vehicle capable of switching autonomous driving levels, a data acquisition unit that acquires sensor data which is information acquired by sensors mounted on the vehicle, a model setting unit that sets a learned model based on the acquired autonomous driving level, and an object detection unit that inputs the acquired sensor data into the set learned model and acquires information including objects around the vehicle.

[0010] The control method for the information processing device according to this embodiment includes the steps of: acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; acquiring sensor data which is information acquired by sensors mounted on the vehicle; setting a trained model based on the acquired autonomous driving level; and inputting the acquired sensor data into the set trained model to acquire information including objects around the vehicle.

[0011] The program according to this embodiment causes a computer to perform the following steps: acquire the autonomous driving level of a vehicle whose autonomous driving level can be switched; acquire sensor data which is information acquired by sensors mounted on the vehicle; set a trained model based on the acquired autonomous driving level; and input the acquired sensor data into the set trained model to acquire information including objects around the vehicle.

[0012] The information processing device according to this embodiment includes a level setting unit that acquires the autonomous driving level of a vehicle capable of switching autonomous driving levels, a data acquisition unit that acquires sensor data which is information acquired by sensors mounted on the vehicle, an accuracy setting unit that sets the data accuracy of the acquired sensor data based on the acquired autonomous driving level, and an object detection unit that acquires information including objects around the vehicle based on the sensor data with the set data accuracy.

[0013] The control method for the sensor control device according to this embodiment includes the steps of: acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; acquiring sensor data, which is information acquired by sensors mounted on the vehicle; setting the data accuracy of the acquired sensor data based on the acquired autonomous driving level; and acquiring information including objects around the vehicle based on the sensor data with the set data accuracy.

[0014] The program according to this embodiment causes a computer to perform the following steps: acquire the autonomous driving level of a vehicle whose autonomous driving level can be switched; acquire sensor data which is information acquired by sensors mounted on the vehicle; set the data accuracy of the acquired sensor data based on the acquired autonomous driving level; and acquire information including objects around the vehicle based on the sensor data with the set data accuracy.

[0015] According to this disclosure, it is possible to provide a sensor control device, an information processing device, a control method and program for the sensor control device, and a control method and program for the information processing device, which can optimize the power consumption for information processing to acquire information necessary for autonomous driving control.

[0016] Figure 1 is a schematic diagram of a vehicle system according to the first embodiment. Figure 2 is a schematic block diagram of a sensor control device according to the first embodiment. Figure 3 is a table showing example 1 of control factors for the sensing accuracy of each sensor according to the autonomous driving level. Figure 4 is a table showing example 2 of control factors for the sensing accuracy of each sensor according to the autonomous driving level. Figure 5A is a diagram showing example 1 of a control method using chirp signals. Figure 5B is a table summarizing each control factor in example 1 of the control method using chirp signals. Figure 6A is a diagram showing example 2 of a control method using chirp signals. Figure 6B is a table summarizing each control factor in example 2 of the control method using chirp signals. Figure 7A is a diagram showing example 3 of a control method using chirp signals. Figure 7B is a table summarizing each control factor in example 3 of the control method using chirp signals. Figure 8 is a table showing example 3 of control factors for the sensing accuracy of each sensor according to the autonomous driving level. Figure 9 is a table showing an example of control factors for sensing accuracy in a GNSS sensor. Figure 10 is a table showing example 4 of control factors for the sensing accuracy of each sensor according to the autonomous driving level. Figure 11 is a table showing example 5 of control factors for the sensing accuracy of each sensor according to the autonomous driving level. Figure 12 is a flowchart illustrating the processing flow of the sensor control device according to the first embodiment. Figure 13 is an example of a correspondence table showing the correspondence between the autonomous driving level and the sensors used when acquiring information. Figure 14 is a schematic diagram of the vehicle system according to the second embodiment. Figure 15 is a schematic block diagram of the information processing device according to the second embodiment. Figure 16 is a table showing example control factors for the data accuracy of the millimeter-wave radar according to the autonomous driving level. Figure 17 is a table showing example control factors for the data accuracy of the stereo camera according to the autonomous driving level. Figure 18A is a diagram showing example 1 of the settings of a trained model according to the second embodiment. Figure 18B is a diagram showing example 2 of the settings of a trained model according to the second embodiment. Figure 19 is a flowchart illustrating the processing flow of the information processing device according to the second embodiment. Figure 20 is a schematic block diagram of the information processing device according to the third embodiment. Figure 21 is a table showing examples of control factors for the processing accuracy of a trained model tailored to the autonomous driving level. Figure 22 is a diagram showing example 1 of the settings for a trained model according to the third embodiment.Figure 23 shows an example of setting up a trained model according to the third embodiment (Example 2). Figure 24 shows an example of setting up a trained model according to the third embodiment (Example 3). Figure 25 shows an example of setting up a trained model according to the third embodiment (Example 4). Figure 26 is a flowchart illustrating the processing flow of the information processing device according to the third embodiment.

[0017] Preferred embodiments of the present disclosure will be described in detail below with reference to the attached drawings. However, this disclosure is not limited to these embodiments, and if there are multiple embodiments, they may be combinations of these embodiments.

[0018] (First Embodiment) The configuration of the vehicle system 1 will be described using Figure 1. Figure 1 is a schematic diagram of the vehicle system according to the first embodiment.

[0019] (Overview of the Vehicle System) As shown in Figure 1, the vehicle system 1 according to this embodiment is a system for controlling a vehicle V. The vehicle system 1 according to this embodiment, as shown in Figure 1, includes a vehicle V and a sensor control device 30. The vehicle V includes a user interface 10, a sensor 20, and a power supply 40. The sensor control device 30 is a device used to control the sensor 20 mounted on the vehicle V. The sensor control device 30, the user interface 10, the sensor 20, and the power supply 40 communicate with each other via a wire harness W. The sensor 20 and the power supply 40 are connected by the wire harness W, and power is supplied to the sensor 20 from the power supply 40. The sensor control device 30 according to this embodiment controls the sensing accuracy of each sensor 20 based on the autonomous driving level of the vehicle V and optimizes the power consumption of each sensor 20. Note that the sensor control device 30, the user interface 10, the sensor 20, and the power supply 40 are not limited to the wire harness W, but may be connected by other conductive members.

[0020] (Vehicle) Vehicle V is, for example, an automobile that travels on a road surface. Vehicle V is equipped with a drive mechanism and the like (not shown).

[0021] (Automated Driving Level) Vehicle V is a vehicle capable of automated driving, which automatically performs at least some of the driving controls. Furthermore, vehicle V can switch between automated driving levels. The automated driving level is an indicator of the level of automation of vehicle V, and different automated driving levels correspond to different types of automated driving controls. In this embodiment, the higher the numerical value of the automated driving level, the higher the degree of automation. Preferably, vehicle V can switch between manual driving, in which all driving controls are performed manually by user U, and automated driving.

[0022] Autonomous driving levels may be classified into six stages as defined by the SAE (Society of Automotive Engineers).

[0023] Automated driving level 0 indicates manual driving. Automated driving levels 1 and 2 monitor safe driving, and the driver is the primary responder for driving control. In automated driving level 1, the automated driving system performs subtasks of vehicle motion control in either the longitudinal direction (acceleration / deceleration control) or the lateral direction (steering control) within a limited area. In automated driving level 2, the automated driving system performs subtasks of vehicle motion control in both the longitudinal and lateral directions within a limited area.

[0024] In autonomous driving levels 3, 4, and 5, the autonomous driving system is primarily responsible for monitoring and responding to safe driving. In autonomous driving level 3, if it becomes difficult to continue autonomous driving control of the vehicle V, the driver monitors and takes over the driving control. Autonomous driving in autonomous driving level 3 is conditional. More specifically, in autonomous driving level 3, the autonomous driving system performs all driving-related controls. In autonomous driving level 3, if the autonomous driving system requests intervention from the driver (for example, if it determines that an emergency response is necessary), the driver is required to take control of the vehicle V. In autonomous driving level 4, the autonomous driving system performs all dynamic driving tasks and responses to situations where continued operation is difficult within a limited area. For example, in autonomous driving level 4, in specific locations or on specific roads, the autonomous driving system will perform all vehicle control tasks, including emergency responses. In autonomous driving level 5, the system performs all dynamic driving tasks and responses to situations where continued operation is difficult without limitation (i.e., in all locations and on all roads, not within a limited area).

[0025] As described above, in autonomous driving levels 4 and 5, the autonomous driving system performs all dynamic driving tasks and responds to situations where continued operation becomes difficult, therefore, being seated in the driver's seat is not a mandatory requirement. Only in autonomous driving levels 0 to 3 is being seated in the driver's seat a mandatory requirement. Furthermore, considering that autonomous driving level 4 is set only in specific locations or on specific roads, being seated in the driver's seat is not a mandatory requirement, similar to autonomous driving level 5.

[0026] (User Interface) The user interface 10 is a device that receives input from user U. User interface 10 is, for example, an input device used by user U when setting the autonomous driving level. User interface 10 may also have the function of a display device that shows the autonomous driving level options that user U can select based on the location of vehicle V on a map, the location of the road being traveled, and the remaining power of the power supply 40 described later. User interface 10 is, for example, a liquid crystal display, a touch panel, etc. In the example shown in Figure 1, it is shown that user interface 10 is a touch panel mounted on vehicle V. User interface 10 may also be something other than a touch panel. For example, it may include a voice recognition device, physical buttons, a joystick, a steering wheel controller, a gesture recognition device, or other input devices.

[0027] (Power Supply) The power supply 40 supplies power to drive the devices mounted on the vehicle V. The power supply 40 is a power supply that supplies power to the sensors 20 and outputs a predetermined output voltage to the sensors 20 mounted on the vehicle V. The output voltage output by the power supply 40 is, for example, DC 12V, but is not limited to this. The power supply 40 is connected to the sensors 20 by a wire harness W and supplies power to each sensor 20. In this embodiment, the power supply 40 also supplies power to other devices mounted on the vehicle V, such as a drive unit, in addition to the sensors 20. The vehicle V may also have multiple power supplies 40. For example, this may include a power supply for the sensors 20, a power supply for the drive unit, a power supply for the communication device, etc. Each may also be connected by a wire harness W, and power may be supplied to the power supply that is running low on power as appropriate.

[0028] (Sensor) Sensor 20 is at least one sensor mounted on vehicle V to enable autonomous driving of vehicle V. Vehicle V may be equipped with any sensor used for autonomous driving as sensor 20, but it is preferable to include a sensor that detects the surroundings of vehicle V. In the example in Figure 1, the sensor 20 that detects the surroundings of vehicle V is equipped with a millimeter-wave radar 20A, front and rear side millimeter-wave radars 20B, a monocular camera 20C, a stereo camera 20D, and an ultrasonic sensor 20E, but only some of these may be equipped. Vehicle V may also be equipped with a GNSS sensor 20F that detects the position information of vehicle V as sensor 20. Note that in Figure 1, the X1 direction is the front of the vehicle V.

[0029] (Millimeter-wave radar, front and rear side millimeter-wave radar) The millimeter-wave radar 20A and the front and rear side millimeter-wave radar 20B are sensors that irradiate an object with electromagnetic waves having a wavelength of 1 mm to 10 mm and a frequency of 30 GHz to 300 GHz, and use the reflected waves to measure the distance to the object, the angle, the velocity of the object, etc. In this embodiment, the millimeter-wave radar 20A is mounted on the front side of the vehicle V, around the center of the front end of the vehicle body. The front and rear side millimeter-wave radar 20B in this embodiment is mounted on the front side of the vehicle V, on the left and right sides of the front end of the vehicle body, and on the rear side of the vehicle V, on the left and right sides of the front end of the vehicle body. The sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radar 20B depends on the transmission power to the millimeter-wave radar 20A and the front and rear side millimeter-wave radar 20B; higher transmission power results in higher sensing accuracy. However, if the transmission power is high, the power consumption of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B will also increase.

[0030] (Monocular Camera, Stereo Camera) The monocular camera 20C acquires images using a single camera and is positioned to recognize white lines for lane keeping, recognize traffic signs such as speed limits, and compensate for the driver's blind spots. The stereo camera 20D acquires images using two cameras and is positioned to measure the three-dimensional shape of an object and its three-dimensional position by generating a stereoscopic image based on the acquired images. In this embodiment, the monocular camera 20C is mounted on the rear side of the vehicle V, around the upper center of the glass. The stereo camera 20D in this embodiment is mounted on the front side of the vehicle V, around the upper center of the glass. The sensing accuracy of the monocular camera 20C and the stereo camera 20D depends on the frame frequency (image acquisition interval), and the higher the frame frequency (shorter image acquisition interval), the higher the sensing accuracy. However, a higher frame frequency also increases power consumption. Here, the three-dimensional shape of the object and its three-dimensional position are measured using a stereo camera 20D, but measurements may also be taken using a monocular camera 20C with AI or the like. Specifically, a two-dimensional image acquired by the monocular camera 20C is used as input, and a depth estimation model (e.g., DepthNet or Monodepth) is used to estimate the depth information of each pixel. Alternatively, a technique that analyzes consecutive image frames accompanying the movement of the vehicle V and estimates the three-dimensional structure using parallax information (e.g., Structure from Motion (SfM)) may be used.

[0031] (Ultrasonic Sensor) The ultrasonic sensor 20E is a sensor that measures the distance to an object by irradiating it with ultrasonic waves with a frequency of 20 kHz or higher and using the reflected waves. The measurement distance is short, about 10 m, compared to the measurement distance of millimeter-wave radar 20A, which is tens to hundreds of meters. Therefore, it is mounted as a sonar sensor (surrounding object detection sensor) at the rear of the vehicle or in front of the vehicle V, or as a peripheral distance sensor in low-speed ranges such as automatic parking. In this embodiment, the ultrasonic sensor 20E is mounted around the left and right front ends of the vehicle V, and around the rear of the vehicle V and the center of the front end of the vehicle body. The sensing accuracy of the ultrasonic sensor 20E depends on the transmission power to the ultrasonic sensor 20E, and the higher the transmission power, the higher the sensing accuracy. However, if the transmission power is high, the power consumption of the ultrasonic sensor 20E will also be high.

[0032] (GNSS Sensor) The GNSS (Global Navigation Satellite System) sensor 20F is a sensor that acquires the position information of the vehicle V. The GNSS sensor 20F receives GNSS signals from GNSS satellites and acquires the position information of the vehicle V based on the received GNSS signals.

[0033] In the example shown in Figure 1, the sensor 20 includes a millimeter-wave radar 20A, front and rear side millimeter-wave radars 20B, a monocular camera 20C, a stereo camera 20D, an ultrasonic sensor 20E, and a GNSS sensor 20F. However, the number and types of sensors 20 are not limited to this example.

[0034] (Sensor control device) The sensor control device 30 controls the sensing accuracy of at least one sensor 20 based on the autonomous driving level of the vehicle V. The sensor control device 30 may be implemented by one or more information processing terminals, such as a PC (Personal Computer), WS (Work Station), or a computer equipped with server functions. In addition to being mounted on the vehicle V, the sensor control device 30 may also be a portable device that can be used in the vehicle V.

[0035] Figure 2 is a schematic block diagram of a sensor control device according to the first embodiment. As shown in Figure 2, the sensor control device 30 according to this embodiment includes an input / output unit 31, a storage unit 32, and a control unit 33. The input / output unit 31 is an interface for exchanging data between an external device (user interface 10, sensor 20, power supply 40) and the sensor control device, and is, for example, an input / output terminal.

[0036] (Storage Unit) The storage unit 32 is a storage device that stores various types of information. The storage unit 32 comprises a main memory and an auxiliary storage device. The main memory may be implemented by semiconductor memory elements such as RAM (Random Access Memory), ROM (Read Only Memory), or flash memory. The auxiliary storage device may be implemented by a hard disk, SSD (Solid State Drive), or optical disc. The program for the control unit 33 stored in the storage unit 32 may be stored on a recording medium that can be read by the sensor control device 30.

[0037] (Control Unit) The control unit 33 is a controller that manages and controls the sensor control device 30. The control unit 33 is realized by executing various programs stored in the memory unit 32 using RAM as the working area, using a CPU (Central Processing Unit) or MPU (Micro Processing Unit), etc. Alternatively, the control unit 33 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0038] As shown in Figure 2, the control unit 33 includes a condition acquisition unit 331, a level setting unit 332, an accuracy setting unit 333, and an output control unit 334. The control unit 33 implements these functions and performs these processes by reading and executing a program (software) from the storage unit 32. These functions of the control unit 33 may also be implemented by electronic circuits. Furthermore, the control unit 33 may execute these processes using a single CPU, or it may have multiple CPUs and execute these processes in parallel using multiple CPUs.

[0039] (Processing of the sensor control device) The processing contents of the sensor control device 30 are described below.

[0040] (Acquisition of location information) The condition acquisition unit 331 acquires location information, which is information about the location where the vehicle V is traveling. The method for acquiring location information can be any method, but in this embodiment, the condition acquisition unit 331 causes the GNSS sensor 20F to acquire the location information of the vehicle V, and based on the location information of the vehicle V acquired by the GNSS sensor 20F and the map information including the road network stored in the storage unit 32, it acquires location information, which is information about the location (road) where the vehicle V is traveling.

[0041] (Setting of Autonomous Driving Level Options) The level setting unit 332 sets the autonomous driving level options. The autonomous driving level options are information indicating candidate autonomous driving levels that can be selected by user U. The autonomous driving level options may be presented to user U from the user interface 10.

[0042] The method for setting the autonomous driving level options can be any method, but in this embodiment, the level setting unit 332 sets the autonomous driving level options based on location information acquired by the condition acquisition unit 331. More specifically, the level setting unit 332 sets the autonomous driving level options based on location information and a driving plan entered by user U. Here, the driving plan entered by user U is information that includes the destination and the route to the destination. In other words, the level setting unit 332 acquires the current position of the vehicle V on the map using location information, acquires the route from the current position to the destination from the driving plan entered by user U, and then sets the autonomous driving level options. For example, if it is determined that the route from the current position to the destination is a specific road where autonomous driving level 4 can be set, the autonomous driving level options that allow selection of autonomous driving levels 0 to 4 may be set. Alternatively, if it is determined that the route from the current position to the destination is not a specific road where autonomous driving level 4 can be set, autonomous driving levels 4 or 5 may be excluded from the options, and autonomous driving level options that allow selection of only autonomous driving levels 0 to 3 may be set. For example, if the system determines that the route from the current location to the destination is a specific road that can only be driven on at autonomous driving level 4, it may set up an option to indicate that autonomous driving level 4 has been set, preventing user U from selecting an autonomous driving level.

[0043] In this embodiment, the level setting unit 332 sets the autonomous driving level options based on location information, but the autonomous driving level options may also be set based on other conditions. For example, the level setting unit 332 may set the autonomous driving level options according to the road congestion. Specifically, even in situations where autonomous driving level 3 can be included in the autonomous driving level options, such as on a specific highway, if there is congestion on the route from the current location to the destination, the level setting unit 332 may exclude autonomous driving levels 3 to 5 from the options and set an autonomous driving level option where only autonomous driving levels 0 to 2 can be selected. In low-speed autonomous driving during congestion on a specific highway (equivalent to autonomous driving level 3), the autonomous driving system controls the vehicle, but when the congestion is cleared and the surrounding vehicle speed increases, it may be necessary to hand over the acceleration control of the vehicle V to match the increased surrounding vehicle speed from the autonomous driving system to the user U (change to equivalent to autonomous driving level 2). Therefore, by setting the autonomous driving level options according to the road congestion on the route from the current location to the destination, a safer autonomous driving level option can be set for the user U.

[0044] Alternatively, the level setting unit 332 may set the autonomous driving level options based on weather conditions. Specifically, even when it is possible to include autonomous driving levels 4 or 5 in the autonomous driving level options, if bad weather (heavy rain, snow, fog, etc.) is expected on the route from the current location to the destination, the level setting unit 332 may exclude autonomous driving levels 4 or 5 from the options and set the autonomous driving level options so that only autonomous driving levels 0 to 3 can be selected. In autonomous driving (equivalent to autonomous driving level 4) in a specific urban area and on a specific highway, the autonomous driving system controls the vehicle, but if the sensors cannot perform sensing with the expected accuracy in bad weather, it may be necessary to take over vehicle control from the autonomous driving system to user U control (change to equivalent to autonomous driving level 3). Therefore, by setting the autonomous driving level options based on the weather conditions on the route from the current location to the destination, a safer autonomous driving level option can be set for user U.

[0045] (Output of automatic driving level options) The output control unit 334 outputs (displays) the automatic driving level options set by the level setting unit 332 to the user interface 10.

[0046] (Acquisition of automatic driving level) The level setting unit 332 acquires the automatic driving level of the vehicle V. In the present embodiment, the level setting unit 332 acquires selection information indicating the automatic driving level specified by the user U, and acquires the automatic driving level indicated in the selection information. In this case, for example, the user interface 10 displays the set automatic driving level options, and accepts an input of the automatic driving level selected from the automatic driving level options by the user U. The level setting unit 332 acquires information on the automatic driving level input to the user interface 10 as the selection information.

[0047] However, the method for acquiring the automatic driving level by the level setting unit 332 is not limited to the above. For example, the setting and output of automatic driving level options are not essential, and the level setting unit 332 may acquire an automatic driving level freely selected by the user U as selection information. Further, the automatic driving level is not limited to being set according to the selection by the user U, and may be automatically set by the level setting unit 332. The automatic driving level setting method in this case is the same as the automatic driving level option setting method described above, so detailed description thereof will be omitted. Note that when a plurality of automatic driving levels are selected from the automatic driving level options, the level setting unit 332 may select one automatic driving level from among the plurality of automatic driving levels according to any arbitrary criteria.

[0048] (Setting of sensing accuracy) The accuracy setting unit 333 sets the sensing accuracy of each sensor 20 mounted on the vehicle V based on the automatic driving level acquired by the level setting unit 332. Sensing accuracy refers to detection accuracy by the sensor 20. Therefore, the higher the sensing accuracy, the higher the detection performance. On the other hand, the higher the sensing accuracy, the greater the power consumption of the sensor 20.

[0049] The sensing accuracy can be set by any method, but the accuracy setting unit 333 in this embodiment is set so that the lower the acquired automated driving level (i.e., the lower the degree of automation), the lower the sensing accuracy of the sensor 20.

[0050] Furthermore, the accuracy setting unit 333 may control the sensing accuracy of the sensor 20 in any way, but in this embodiment, the accuracy setting unit 333 controls the sensing accuracy of the millimeter-wave radar 20A and the front-rear and side millimeter-wave radars 20B by the transmission power to each sensor. In other words, the accuracy setting unit 333 sets the transmission power to the millimeter-wave radar 20A and the front-rear and side millimeter-wave radars 20B as a control factor for controlling the sensing accuracy of the millimeter-wave radar 20A and the front-rear and side millimeter-wave radars 20B. In addition, in this embodiment, the accuracy setting unit 333 controls the sensing accuracy of the monocular camera 20C and the stereo camera 20D by their respective frame frequencies (image acquisition intervals). In other words, the accuracy setting unit 333 sets the frame frequencies (image acquisition intervals) of the monocular camera 20C and the stereo camera 20D as control factors for controlling the sensing accuracy of the monocular camera 20C and the stereo camera 20D. Furthermore, the accuracy setting unit 333 according to this embodiment controls the sensing accuracy of the ultrasonic sensor 20E by the power transmitted to the ultrasonic sensor 20E. In other words, the accuracy setting unit 333 sets the power transmitted to the ultrasonic sensor 20E as a control factor for controlling the sensing accuracy of the ultrasonic sensor 20E. Here, for example, the millimeter-wave radar 20A is described as a single sensor that can set multiple sensing accuracy levels, but the millimeter-wave radar 20A may be composed of multiple sensors with different sensing accuracy levels (power consumption), and these may be switched between and used. The same applies to other sensors.

[0051] Specifically, the accuracy setting unit 333 controls the transmission power (control factor) to the millimeter-wave radar 20A and the front / rear / side millimeter-wave radars 20B to decrease as the acquired autonomous driving level decreases. The accuracy setting unit 333 also controls the frame frequency (control factor) of the monocular camera 20C and the stereo camera 20D to decrease (the image acquisition interval becomes longer) as the acquired autonomous driving level decreases. Furthermore, the accuracy setting unit 333 controls the transmission power (control factor) to the ultrasonic sensor 20E to decrease as the acquired autonomous driving level decreases. In other words, if the transmission power to the millimeter-wave radar 20A, the front / rear / side millimeter-wave radars 20B, and the ultrasonic sensor 20E decreases, the power consumption of these sensors will also decrease. Similarly, if the frame frequency of the monocular camera 20C and the stereo camera 20D decreases (the image acquisition interval becomes longer), the number of images acquired by the monocular camera 20C and the stereo camera 20D will also decrease, and the power consumption of these cameras will also decrease. Here, we have explained that as the level of autonomous driving decreases, the control factor is controlled to decrease as the frame frequency decreases. However, the control factor can also be controlled to decrease as the bit depth, or as the spatial resolution.

[0052] In this way, by controlling the sensing accuracy of each sensor 20 to decrease as the acquired autonomous driving level decreases, the power consumption of each sensor 20 can also be reduced. That is, when the autonomous driving level is low, the degree to which the user U controls the driving is high, so even if the accuracy of the sensor 20 is reduced to some extent, the driving control can be performed appropriately. Therefore, as in this embodiment, by reducing the sensing accuracy as the autonomous driving level decreases, power consumption can be reduced while maintaining appropriate driving control. On the other hand, when the autonomous driving level is high, the accuracy of the sensor 20 can be kept to a certain extent so that automatic driving control can be performed appropriately.

[0053] (Execution of sensing based on sensing accuracy) The output control unit 334 in the sensor control device 30 causes each sensor 20 to perform sensing processing based on the sensing accuracy set by the accuracy setting unit 333. That is, the sensor 20 performs sensing with the control factor indicated by the sensing accuracy.

[0054] (Example where the control factors are transmission power and frequency) Figure 3 is a table showing example 1 of the control factors for the sensing accuracy of each sensor according to the autonomous driving level. In the example shown in Figure 3, the accuracy setting unit 333 sets the sensing accuracy to two levels according to the autonomous driving level, and the control factors for each sensor 20 corresponding to each sensing accuracy level are listed in the table. Specifically, the accuracy setting unit 333 sets sensing accuracy level 1 for autonomous driving levels 0 to 2, and sensing accuracy level 2 for autonomous driving levels 3 to 5, and each sensor 20A to 20E is assigned a control factor corresponding to sensing accuracy level 1 or sensing accuracy level 2, respectively. As described above, the control factor for the millimeter-wave radar 20A, the front and rear side millimeter-wave radars 20B and the ultrasonic sensor 20E is the transmission power to each sensor, and the control factor for the monocular camera 20C and the stereo camera 20D is the frame frequency (image acquisition interval) in each camera. As a result, when comparing autonomous driving levels 0-2 (lower level) with autonomous driving levels 3-5 (higher level), the sensing accuracy can be set lower in autonomous driving levels 0-2 due to the burden of vehicle control by the user U, and the power consumption of each sensor 20 can be set lower.

[0055] Figure 4 is a table showing example 2 of the control factors for the sensing accuracy of each sensor according to the autonomous driving level. In the example shown in Figure 4, the accuracy setting unit 333 sets the sensing accuracy to four levels according to the autonomous driving level, and the control factors for each sensor 20 corresponding to each sensing accuracy level are listed in the table. Here, the sensing accuracy at autonomous driving level 0, which corresponds to fully manual driving, is set to sensing accuracy level 0. To explain in more detail, the accuracy setting unit 333 sets sensing accuracy level 1 for autonomous driving level 1, sensing accuracy level 2 for autonomous driving level 2, sensing accuracy level 3 for autonomous driving level 3, and sensing accuracy level 4 for autonomous driving levels 4 and 5. The control factors for each sensor 20A to 20D are the same as in the example in Figure 3, so their explanation is omitted. As a result, the power consumption of each sensor 20 can be optimized by finely setting the sensing accuracy level according to the autonomous driving level.

[0056] (Example where the control factor is the waveform element of the chirp signal) Up to this point, we have described an example in which the accuracy setting unit 333 sets the transmission power to each sensor 20 as the control factor for the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B. However, the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B may also be controlled using other control factors. For example, when the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B use the FMCW (Frequency Modulated Continuous Wave) method, the accuracy setting unit 333 may control the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B using the waveform element of the chirp signal used in the FMCW method as the control factor. Here, a chirp signal is a signal whose frequency increases or decreases with time, and is expressed by the starting frequency (fc1), duration (Tc), and bandwidth (B). Furthermore, a waveform element is an element that determines the waveform of a chirp signal, and refers to at least one of the following: start frequency (fc1), duration (Tc), and bandwidth (B). In other words, the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B may be controlled by controlling one of the start frequency (fc1), duration (Tc), or bandwidth (B) of the chirp signal in the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B as a control factor.

[0057] Figure 5A shows an example 1 of a control method using a chirp signal. In the example in Figure 5A, the accuracy setting unit 333 controls a chirp signal with a starting frequency of fc1, a duration of Tc, and a bandwidth of B1, by changing only the bandwidth from B1 to B2, while keeping the starting frequency and duration unchanged. More specifically, by changing the bandwidth, the wavelength (intensity) of the electromagnetic waves irradiated at once can be changed. In other words, by setting the bandwidth as a control factor, the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B can be changed. For example, in the case of autonomous driving level 5, widening the bandwidth increases the power consumption due to sensing, but enables more accurate sensing. Also, for example, in the case of autonomous driving level 0, narrowing the bandwidth lowers the sensing accuracy and reduces the power consumption due to sensing. In short, by changing the bandwidth (control factor), the power consumption of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B can be changed.

[0058] Figure 5B is a table summarizing the control factors in Example 1 of the control method using chirp signals. In the example of Figure 5B, the accuracy setting unit 333 sets the sensing accuracy to four levels according to the automatic driving level, and the control factors for each sensor 20 corresponding to each sensing accuracy level are listed in the table. To explain in detail, in the example of Figure 5B, the control factors for the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B are the bandwidths of the chirp signals of each sensor, and the accuracy setting unit 333 is set so that the bandwidths of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B become smaller as the automatic driving level decreases. The control factors for the other sensors 20 other than the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B are the same as in Figure 4, so their explanation is omitted. As a result, the precision setting unit 333 can optimize the power consumption of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B by finely setting the bandwidth of the chirp signals of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B in accordance with the level of autonomous driving.

[0059] Figure 6A shows an example 2 of a control method using a chirp signal. In the example in Figure 6A, the accuracy setting unit 333 changes the transmission frequency of the chirp signal by setting a chirp signal transmission stop time Tr between the first chirp signal and the second chirp signal, without changing the start frequency, duration, or bandwidth of the chirp signal, which has a start frequency of fc1, a duration of Tc, and a bandwidth of B1. More specifically, by setting the chirp signal transmission stop time Tr, the irradiation interval of the irradiated electromagnetic waves can be changed. In other words, by setting the transmission frequency of the chirp signal as a control factor, the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B can be changed. For example, in the case of autonomous driving level 5, increasing the transmission frequency increases the power consumption due to sensing, but it becomes possible to acquire more real-time, high-precision information. Also, for example, in the case of autonomous driving level 0, it is possible to reduce the sensing accuracy and power consumption due to sensing by lowering the transmission frequency. In other words, by changing the transmission frequency (control factor) of the chirp signal, the power consumption of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B can be changed.

[0060] Figure 6B is a table summarizing the control factors in Example 2 of the control method using chirp signals. In the example of Figure 6B, the accuracy setting unit 333 sets the sensing accuracy to four levels according to the automatic driving level, and the control factors for each sensor 20 corresponding to each sensing accuracy level are listed in the table. To explain in detail, in the example of Figure 6B, the control factor for the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B is the transmission interval of the chirp signal of each sensor (chirp signal transmission stop time Tr). The accuracy setting unit 333 sets the chirp signal transmission stop time Tr of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B to be equal to the duration of the chirp signal Tc in automatic driving levels 1 to 3, which are low automatic driving levels, and sets the chirp signal transmission stop time Tr of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B to be 0 in automatic driving levels 4 or 5, which are high automatic driving levels. The control factors for the other sensors 20, other than the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B, are the same as in Figure 4, so their explanation is omitted. As a result, the accuracy setting unit 333 can optimize the power consumption of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B by setting the chirp signal transmission stop time Tr for the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B in accordance with the level of automatic driving.

[0061] Figure 7A shows an example 3 of a control method using a chirp signal. In the example in Figure 7A, the accuracy setting unit 333 controls a chirp signal with a starting frequency of fc1, a duration of Tc, and a bandwidth of B1, by changing only the starting frequency from fc1 to fc2, while keeping the duration and bandwidth unchanged. More specifically, by changing the starting frequency, the starting frequency of the electromagnetic wave that affects the directivity of the sensor can be changed. In other words, by changing the starting frequency, the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B can be changed. Here, the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B consume more power when irradiating with high-frequency electromagnetic waves compared to when irradiating with low-frequency electromagnetic waves. For example, in the case of autonomous driving level 5, increasing the starting frequency increases the power consumption due to sensing, but enables more accurate sensing. Also, for example, in the case of autonomous driving level 0, lowering the starting frequency reduces the sensing accuracy and thus the power consumption due to sensing. In other words, by changing the starting frequency, the power consumption of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B can be changed.

[0062] Figure 7B is a table summarizing the control factors in Example 3 of the control method using chirp signals. In the example of Figure 7B, the accuracy setting unit 333 sets the sensing accuracy to four levels according to the automatic driving level, and the control factors for each sensor 20 corresponding to each sensing accuracy level are listed in the table. To explain in more detail, in the example of Figure 7B, the accuracy setting unit 333 sets the control factors for the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B to the starting frequency in the chirp signal of each sensor, and sets the starting frequencies of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B to be smaller as the automatic driving level decreases. The control factors for the other sensors 20 other than the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B are the same as in Figure 4, so their explanation is omitted. As a result, the precision setting unit 333 can optimize the power consumption of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B by precisely setting the starting frequency (control factor) in the chirp signals of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B in accordance with the level of autonomous driving.

[0063] (Examples of controlling other sensors) Up to this point, we have described the control of the sensing accuracy of sensors 20 that detect the area around the vehicle V, such as the millimeter-wave radar 20A, front and rear side millimeter-wave radars 20B, monocular camera 20C, stereo camera 20D, and ultrasonic sensor 20E. However, the sensors 20 controlled by the sensor control device 30 are not limited to sensors that detect the area around the vehicle V, and the sensing accuracy of other sensors 20 may also be controlled in accordance with the autonomous driving level. Other sensors that do not detect the area around the vehicle V include, for example, the GNSS sensor 20F that acquires position information of the vehicle V, the tire pressure sensor 20G that acquires information on the tire pressure of the vehicle V, the acceleration sensor 20H that acquires the acceleration of the vehicle V, and the gyro sensor 20J that acquires the angular velocity of the vehicle V, and their sensing accuracy may also be controlled in accordance with the autonomous driving level.

[0064] Here, the sensing accuracy of each sensor 20F to 20J may be controlled by setting an arbitrary control factor, but the accuracy setting unit 333 in the sensor control device 30 may control the sensing accuracy of the GNSS sensor 20F and the tire pressure sensor 20G by setting the data acquisition interval for each sensor as the control factor. Alternatively, the accuracy setting unit 333 may control the sensing accuracy of the acceleration sensor 20H and the gyro sensor 20J by setting the ON or OFF of the power supply of each sensor as the control factor. Specifically, the accuracy setting unit 333 controls the data acquisition interval (control factor) of the GNSS sensor 20F and the tire pressure sensor 20G to become longer as the acquired autonomous driving level decreases. Also, the accuracy setting unit 333 controls the power supply of the acceleration sensor 20H and the gyro sensor 20J to be turned ON when the sensing accuracy level is 1 or higher. In other words, as the data acquisition interval for the GNSS sensor 20F and tire pressure sensor 20G lengthens, the number of data points acquired by the GNSS sensor 20F and tire pressure sensor 20G also decreases, resulting in lower power consumption for these sensors. In other words, by controlling the sensing accuracy of the GNSS sensor 20F and tire pressure sensor 20G to decrease as the acquired autonomous driving level decreases, the power consumption of the GNSS sensor 20F and tire pressure sensor 20G can also be reduced. Furthermore, the accuracy setting unit 333 controls the power to turn off the acceleration sensor 20H and gyro sensor 20J when the sensing accuracy level is 0 (in the case of fully manual driving), thereby reducing the power consumption of the acceleration sensor 20H and gyro sensor 20J as well.

[0065] (Example where the control factors are the data acquisition interval and the ON / OFF state of the power supply) Figure 8 is a table showing example 3 of the control factors for the sensing accuracy of each sensor according to the autonomous driving level. In the example shown in Figure 8, the accuracy setting unit 333 sets the sensing accuracy to three levels according to the autonomous driving level, and the control factors for each sensor 20 corresponding to each sensing accuracy level are listed in the table. Specifically, the accuracy setting unit 333 sets sensing accuracy level 1 for autonomous driving level 1, sensing accuracy level 2 for autonomous driving level 2, and sensing accuracy level 3 for autonomous driving levels 3 to 5, and each sensor 20F to 20J is assigned a control factor corresponding to sensing accuracy levels 1 to 3. As described above, in this example, the control factors for the GNSS sensor 20F and the tire pressure sensor 20G are the data acquisition interval for each sensor, and the control factors for the acceleration sensor 20H and the gyro sensor 20J are the ON or OFF state of the power supply for each sensor. As a result, the accuracy setting unit 333 can optimize the power consumption of each sensor 20F to 20J by finely setting the sensing accuracy level in accordance with the level of automatic operation.

[0066] (Example of using other control factors) In addition to setting the data acquisition interval as a control factor, other control factors may also be set for the sensing accuracy of the GNSS sensor 20F. For example, the accuracy setting unit 333 may set the number of GNSS satellites used and the number of antennas used as control factors to control the sensing accuracy of the GNSS sensor 20F. Specifically, the accuracy setting unit 333 controls the number of satellites (control factor) and the number of antennas (control factor) used by the GNSS sensor 20F to acquire position information to decrease as the acquired autonomous driving level decreases. In other words, if the number of satellites and antennas used by the GNSS sensor 20F decreases, the amount of data acquired from GNSS satellites decreases. This means that the power consumption required for data acquisition from GNSS satellites by the GNSS sensor 20F decreases. Therefore, the accuracy setting unit 333 controls the GNSS sensor 20F so that the lower the acquired autonomous driving level, the lower the sensing accuracy of the GNSS sensor 20F, thereby also lowering the power consumption of the GNSS sensor 20F.

[0067] Figure 9 is a table showing an example of control factors for sensing accuracy in a GNSS sensor. In the example shown in Figure 9, the accuracy setting unit 333 sets the sensing accuracy of the GNSS sensor 20F to two levels depending on the automatic driving level, and the control factors for the GNSS sensor 20F corresponding to each sensing accuracy level are listed in the table. Specifically, the accuracy setting unit 333 sets sensing accuracy level 1 for automatic driving levels 0 to 2, and sensing accuracy level 2 for automatic driving levels 3 to 5, and the GNSS sensor 20F is assigned a control factor corresponding to either sensing accuracy level 1 or sensing accuracy level 2, respectively. As described above, the control factors for the GNSS sensor 20F in this example are the number of satellites used to acquire position information and the number of antennas used. As a result, when the accuracy setting unit 333 compares the lower levels of autonomous driving (levels 0-2) with the higher levels of autonomous driving (levels 3-5), it can set the sensing accuracy lower for the lower levels of autonomous driving (levels 0-2) due to the burden of vehicle control by the user U, and thus set the power consumption of the GNSS sensor 20F lower.

[0068] (Example of control using information other than the autonomous driving level) Up to this point, we have described an example in which the accuracy setting unit 333 in the sensor control device 30 sets the sensing accuracy of each sensor 20 mounted on the vehicle V based on the autonomous driving level acquired by the level setting unit 332. However, it is not limited to this, and the accuracy setting unit 333 may set the sensing accuracy based on other information in addition to the autonomous driving level.

[0069] For example, the accuracy setting unit 333 may set the sensing accuracy based on the autonomous driving level and the location information of the vehicle V acquired by the condition acquisition unit 331. In other words, depending on the location where the vehicle V is traveling, it may be preferable to have a higher sensing accuracy, or it may be possible to control the vehicle appropriately even with a lower sensing accuracy. In such cases, by setting the sensing accuracy using location information as well, the sensor 20 can be controlled more appropriately according to the location where the vehicle is traveling.

[0070] The method for setting the sensing accuracy using location information is arbitrary, but for example, a correspondence between location attributes (e.g., district or road type) and sensing accuracy may be set, similar to the correspondence between the autonomous driving level and sensing accuracy described above. In this case, the accuracy setting unit 333 may set the sensing accuracy to be adopted based on the sensing accuracy associated with the acquired autonomous driving level and the sensing accuracy associated with the location attributes shown in the location information. In this case, for example, the higher accuracy of the sensing accuracy based on the autonomous driving level and the sensing accuracy based on location may be selected, or the lower accuracy may be selected.

[0071] Furthermore, if the location information includes information about the number of roadside devices along the travel route to the destination that vehicle V is traveling to, the sensing accuracy may be set based on the number of roadside devices. Here, roadside devices along the travel route refer to V2X (Vehicle to X) devices, that is, devices from which vehicle V can acquire information through mutual communication. In other words, when vehicle V travels on a road with a large number of roadside devices along the travel route (a road where the number of roadside devices required for information acquisition is greater than or equal to a specified number), vehicle V can acquire a lot of information used for autonomous driving from the roadside devices along the travel route, and it may be possible to lower the sensing accuracy in vehicle V. For example, the accuracy setting unit 333 may lower the sensing accuracy as the number of roadside devices at the location indicated in the location information increases.

[0072] Furthermore, the accuracy setting unit 333 may set the sensing accuracy of the sensor 20 based on the automatic operation level and the remaining charge of the power supply 40 (hereinafter referred to as battery charge). For example, in this case, the condition acquisition unit 331 acquires battery information, which is information regarding the battery charge, from the power supply 40. The accuracy setting unit 333 sets the sensing accuracy of the sensor 20 based on the automatic operation level and the battery information. Specifically, the accuracy setting unit 333 may set the sensing accuracy level to be lower when the automatic operation level is low and the battery charge is low. That is, for example, the accuracy setting unit 333 may lower the sensing accuracy as the battery charge decreases. By setting the sensing accuracy level to be lower when the battery charge is low, the power consumption of the sensor 20 can be reduced, leading to a faster recovery of the battery charge.

[0073] Furthermore, the accuracy setting unit 333 may set the sensing accuracy of the sensor 20 based on the autonomous driving level and the biometric information of the user U driving the vehicle V. For example, the accuracy setting unit 333 may lower the sensing accuracy as the activity level of user U, as indicated in the user U's biometric information, increases. That is, for example, the condition acquisition unit 331 may use a biosensor (not shown, such as an electroencephalogram sensor) to acquire the user U's biometric information (e.g., electroencephalogram), and calculate the user U's concentration level from the acquired biometric information of user U.

[0074] In addition to electroencephalograms (EEGs), other biometric information may include heart rate and heart rate variability, respiratory rate, respiratory rate variability, breathing depth, blood pressure, cerebral blood flow, pupil diameter, etc.

[0075] Furthermore, the biosensor that acquires biological information may be a sensor capable of acquiring multi-channel measurement data, such as an electroencephalogram (EEG) or electrocardiogram (ECG). For example, the biosensor may be an electroencephalograph or an electrocardiogram measuring device. The EEG or ECG signals acquired as biological information may be single-channel signals or multi-channel signals. The biosensor may also continuously measure the user U's biological information at regular intervals, or it may constantly monitor the user U's biological information. The biosensor may also be, for example, an external wearable sensor attached to the user U. The biosensor may also be a camera that generates images of the user U. The detection information obtained from the captured images may include the direction of gaze, gaze movement, face orientation, face movement, etc.

[0076] Furthermore, the method for calculating the concentration level of user U can be any method. For example, the condition acquisition unit 331 may include an estimation unit (not shown), and the estimation unit may input the biometric information acquired by the biosensor as input data to a trained model that has been trained to output the concentration level when biometric information is input, and obtain the concentration level as the output result of the trained model.

[0077] For example, it is known that when user U's level of concentration increases, the sympathetic nervous system becomes dominant over the parasympathetic nervous system, and user U's heart rate increases. Therefore, the estimation unit may calculate the level of concentration based on the change in user U's heart rate.

[0078] For example, when user U lacks concentration, they tend to stop moving or move slowly. Using this tendency, the estimation unit may determine that user U's level of concentration has decreased if there is no operation on the control unit after a predetermined amount of time has elapsed.

[0079] Alternatively, the estimation unit may calculate the user U's level of concentration based on gaze detection. Specifically, the estimation unit acquires gaze data such as the position, movement speed, and fixed duration of the user U's gaze from a camera or a dedicated gaze tracking device. The estimation unit can determine that user U is concentrating on a specific object if the user U's gaze remains fixed for a long time, and that the level of concentration may be decreasing if the gaze moves frequently.

[0080] Alternatively, the estimation unit may analyze whether user U's gaze is concentrated in a specific area and calculate user U's concentration level based on the analysis results.

[0081] Alternatively, the estimation unit may integrate gaze data and heart rate information to comprehensively evaluate user U's level of concentration. For example, the estimation unit can determine that user U has a high level of concentration if their gaze is fixed for a long time and their heart rate is elevated. The concentration level may be quantified based on gaze data and heart rate information, and a model may be constructed that predicts the level of concentration from changes in gaze patterns and heart rate using a machine learning algorithm.

[0082] The accuracy setting unit 333 may set the sensing accuracy of each sensor 20 mounted on the vehicle V based on the autonomous driving level and the concentration level of the user U calculated by the condition acquisition unit 331. In other words, when the autonomous driving level is low and the concentration level of the user U calculated by the condition acquisition unit 331 is high, the burden of vehicle control by the user U can be safely managed, so the sensing accuracy can be set low, and the power consumption of each sensor 20 can be set low. That is, for example, the accuracy setting unit 333 may lower the sensing accuracy as the concentration level of the user U increases.

[0083] Furthermore, when the condition acquisition unit 331 acquires location information and battery information, the accuracy setting unit 333 may set the sensing accuracy of each sensor 20 based on at least one of the location information and battery information in addition to the automatic driving level. Alternatively, when the condition acquisition unit 331 acquires location information, battery information, and biometric information, the accuracy setting unit 333 may set the sensing accuracy of each sensor 20 based on at least one of the location information, battery information, and biometric information in addition to the automatic driving level, or it may set the sensing accuracy of each sensor 20 based on all of the automatic driving level, location information, battery information, and biometric information.

[0084] (Example of control using location information and biometric information) Figure 10 is a table showing example 4 of control factors for the sensing accuracy of each sensor according to the autonomous driving level. In the example shown in Figure 10, the accuracy setting unit 333 sets the sensing accuracy of each sensor 20 to two levels based on the autonomous driving level, biometric information, and location information, and the control factors for each sensor 20 corresponding to each sensing accuracy level are listed in the table. Specifically, the accuracy setting unit 333 sets the sensing accuracy of each sensor 20 to two levels based on the autonomous driving level, the level of concentration of the user U, and whether or not the number of roadside devices along the route is greater than or equal to a specified number. To explain in more detail, if the number of roadside devices along the route is less than or equal to a specified number, the sensing accuracy level is set to sensing accuracy level 2, regardless of the autonomous driving level or the level of concentration of the user U. Also, if the number of roadside devices along the route is greater than or equal to a specified number, and the autonomous driving level is 0 to 2, the sensing accuracy level is set to sensing accuracy level 1, regardless of the level of concentration of the user U. This is because, in autonomous driving levels 0-2, the burden of vehicle control by the user U is greater compared to autonomous driving levels 3-5, and there are fewer functions that can be controlled by the autonomous driving system, thus reducing the amount of information required by the sensor 20. Furthermore, even if the sensing accuracy of each sensor 20 mounted on the vehicle V is reduced, the information necessary for control can be supplemented by information acquired from roadside equipment along the route. Therefore, the sensing accuracy level can be set to sensing accuracy level 1, and the power consumption of the sensor 20 can be reduced. The control factors for each sensor 20A-20D are the same as in the example in Figure 3, so their explanation is omitted.

[0085] (Example of control using battery level) Figure 11 is a table showing example 5 of control factors for the sensing accuracy of each sensor according to the autonomous driving level. In the example shown in Figure 11, the accuracy setting unit 333 sets the sensing accuracy of each sensor 20 to five levels based on the autonomous driving level and battery information, and the control factors for each sensor 20 corresponding to each sensing accuracy level are listed in the table. Specifically, the accuracy setting unit 333 sets the sensing accuracy of each sensor 20 to five levels based on the autonomous driving level and the level of battery level. To explain in more detail, when the battery level is low, the user U cannot set the advanced autonomous driving levels 3 to 5, which consume a lot of power. Even in the autonomous driving levels 0 to 2, which the user U can set, the accuracy setting unit 333 sets the sensing accuracy level to sensing accuracy level 1 in order to reduce power consumption. Also, even when the battery level is high, if the autonomous driving level is 0, advanced sensing accuracy is not required, so the accuracy setting unit 333 sets the sensing accuracy level to level 1. Thereafter, if the battery level is high, the accuracy setting unit 333 sets the sensing accuracy level to 2 for automatic driving level 1, to sensing accuracy level 3 for automatic driving level 2, to sensing accuracy level 4 for automatic driving level 3, and to sensing accuracy level 5 for automatic driving level 4 or 5. The control factors for each sensor 20A to 20D are the same as in the example in Figure 7B, so their explanation is omitted. In this way, the power consumption of each sensor 20 can be optimized by finely setting the sensing accuracy level based on the automatic driving level and the remaining battery level. The method for calculating the remaining battery level can be any method, for example, a voltage measurement method or a current measurement method (Coulomb counter method) may be used. In the case of the voltage measurement method, the battery voltage is measured and the remaining battery level is calculated from that value. In the case of the current measurement method, the battery charge and discharge current is accumulated over time and the remaining battery level is calculated from the change. Alternatively, a Battery Management System (BMS) may be used to calculate the remaining battery level in real time.In that case, the BMS collects data such as battery voltage, current, and temperature, and calculates the remaining battery level based on that data.

[0086] (Processing Flow) Next, the processing flow of this embodiment will be described. Figure 12 is a flowchart illustrating the processing flow of the sensor control device according to this embodiment. As shown in Figure 12, the condition acquisition unit 331 in the sensor control device 30 causes the GNSS sensor 20F to acquire location information of the vehicle V, and acquires location information, which is information about the place (road) where the vehicle V is traveling, based on the location information of the vehicle V acquired by the GNSS sensor 20F and the map information including the road network stored in the storage unit 32 (step S10). The level setting unit 332 sets an autonomous driving level selection list, which shows candidate autonomous driving levels that the user U can select, based on the location information acquired by the condition acquisition unit 331, and outputs it to the user U from the user interface 10. Then, the level setting unit 332 acquires selection information from the user interface 10, which is information indicating the autonomous driving level selected by the user U, from the autonomous driving level selection list, and acquires an autonomous driving level based on the selection information (step S12). The precision setting unit 333 sets the sensing precision of each sensor 20 mounted on the vehicle V based on the automatic driving level acquired by the level setting unit 332 (step S14). The output control unit 334 causes each sensor 20 to perform sensing processing based on the sensing precision set by the precision setting unit 333 (step S16), and then terminates this process.

[0087] (Effects) As described above, the sensor control device 30 according to this embodiment sets an autonomous driving level selection list that indicates a candidate for an autonomous driving level that can be selected by a user U, based on location information, which is information about the location (road) where the vehicle V capable of switching autonomous driving levels is traveling. From the autonomous driving level selection list, the sensor control device 30 acquires an autonomous driving level based on selection information, which is information indicating the autonomous driving level selected by the user U. Then, based on the acquired autonomous driving level, the sensing accuracy of each sensor 20 mounted on the vehicle V is set. According to this embodiment, the sensors 20 mounted on the vehicle V capable of switching autonomous driving levels can be controlled and the power consumption of the sensors 20 can be optimized.

[0088] Furthermore, the accuracy setting unit 333 according to this embodiment sets the sensing accuracy to four levels depending on the level of automatic operation. This allows for fine-tuning of the sensing accuracy level according to the level of automatic operation, thereby optimizing the power consumption of each sensor 20.

[0089] Furthermore, the accuracy setting unit 333 according to this embodiment controls the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B based on the chirp signals used by the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B. This allows for more precise setting of the sensing accuracy of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B, and optimizes the power consumption of the millimeter-wave radar 20A and the front and rear side millimeter-wave radars 20B.

[0090] Furthermore, the accuracy setting unit 333 according to this embodiment sets the sensing accuracy based on the autonomous driving level and location information. This makes it possible to set the sensing accuracy according to the number of roadside devices along the travel path included in the location information, thereby optimizing the power consumption of the sensor 20.

[0091] Furthermore, the accuracy setting unit 333 according to this embodiment sets the sensing accuracy based on the autonomous driving level and biological information. This makes it possible to set the sensing accuracy considering the concentration level, which is the biological information of user U, and optimize the power consumption of the sensor 20.

[0092] Furthermore, the accuracy setting unit 333 according to this embodiment sets the sensing accuracy based on the automatic driving level and battery information. This makes it possible to set the sensing accuracy while taking into account the remaining battery level, and optimize the power consumption of the sensor 20.

[0093] (Modified Version of the First Embodiment) In the first embodiment, the accuracy setting unit 333 in the sensor control device 30 controlled the sensing accuracy of each sensor 20 by setting a control factor for each sensor 20 based on the acquired automatic driving level. However, in the modified version of the first embodiment, the accuracy setting unit 333 controls the sensing accuracy of each sensor 20 by setting which sensors 20 to use for information acquisition (selecting the number of sensors 20 to use for detection) based on the acquired automatic driving level. In other words, in the modified version of the first embodiment, the number of sensors 20 used for detection can be said to be the control factor. It should be noted that the more sensors 20 used, the higher the overall sensing accuracy of the vehicle V, so the number of sensors 20 used for detection can also be said to correspond to the sensing accuracy.

[0094] To explain in more detail, the accuracy setting unit 333 in this modified example sets, depending on the autonomous driving level, which of the sensors 20 mounted on the vehicle V will be used to acquire the information necessary for autonomous driving, or which sensors 20 will not be used when acquiring information. As a result, the accuracy setting unit 333 reduces the number of sensors 20 used when acquiring the information necessary for autonomous driving, and reduces the power consumption required for data acquisition by the sensors 20. Here, the method for setting whether or not to use each sensor for information acquisition can be any method, but the accuracy setting unit 333 in this modified example obtains a correspondence table from the storage unit 32 that summarizes the correspondence between the autonomous driving level and the sensors used when acquiring information, and sets the sensors to be used based on the obtained correspondence table.

[0095] Figure 13 is an example of a correspondence table showing the relationship between the autonomous driving level and the sensors used for information acquisition. In the example shown in Figure 13, the accuracy setting unit 333 sets the sensing accuracy to three levels depending on the autonomous driving level, and the table indicates whether or not to use each sensor 20 corresponding to each sensing accuracy level. A circle in the table indicates that the sensor is used. Here, the sensing accuracy at autonomous driving level 0, which corresponds to fully manual driving, is set to sensing accuracy level 0. Specifically, the accuracy setting unit 333 sets sensing accuracy level 1 for autonomous driving level 1, sensing accuracy level 2 for autonomous driving level 2, and sensing accuracy level 3 for autonomous driving levels 3 to 5. Each sensor 20A to 20E is assigned whether or not to use it, corresponding to sensing accuracy levels 1 to 3. As a result, when comparing autonomous driving level 1, which has a low level of autonomous driving, with autonomous driving levels 3 to 5, the sensing accuracy can be set lower in autonomous driving level 1 due to the burden of vehicle control by the user U. In other words, the number of sensors required to acquire information can be reduced, and therefore the power consumption of sensor 20 can be set lower. Here, the stereo camera 20D is not used in sensing accuracy levels 1 and 2, but in this case, the 3D shape of the object and the 3D position to the object may be measured complementaryly using AI or the like with a monocular camera 20C.

[0096] (Effects) As described above, the sensor control device 30 according to this modified example sets, depending on the level of autonomous driving, which of the sensors 20 mounted on the vehicle V will be used to acquire the information necessary for autonomous driving, or which sensors 20 will not be used when acquiring the information. As a result, the accuracy setting unit 333 can reduce the number of sensors 20 used when acquiring the information necessary for autonomous driving, and reduce the power consumption required for data acquisition by the sensors 20.

[0097] (Second Embodiment) Next, the configuration of the vehicle system 1A will be described using Figure 14. Figure 14 is a schematic diagram of the vehicle system according to the second embodiment. In the following description, components common to the first embodiment will be denoted by the same reference numerals and detailed explanations will be omitted.

[0098] (Overview of the Vehicle System) As shown in Figure 14, the vehicle system 1A according to this embodiment is a system for controlling a vehicle V1. The vehicle system 1A according to this embodiment, as shown in Figure 14, includes a vehicle V1 and an information processing device 30A. The vehicle V1 includes a user interface 10 and a sensor 21. The information processing device 30A is a device used to control the sensor 21 mounted on the vehicle V1. The information processing device 30A, the user interface 10, and the sensor 21 communicate with each other via a wire harness W. The information processing device 30A according to this embodiment causes each sensor 21 to perform sensing and acquire sensor data, controls the data accuracy of the acquired sensor data based on the autonomous driving level of the vehicle V1, and optimizes the power consumption for information processing to acquire information necessary for autonomous driving control, including information about objects around the vehicle V1, from the sensor data. Note that the information processing device 30A, the user interface 10, and the sensor 21 are not limited to the wire harness W, but may be connected by other conductive members.

[0099] (Vehicle) Vehicle V1 is, for example, an automobile that travels on a road surface. Vehicle V1 is equipped with a drive mechanism (not shown) and a vehicle control unit that performs automatic driving control, which will be described later.

[0100] (Sensor) Sensor 21 is a sensor mounted on vehicle V to enable autonomous driving of vehicle V. Vehicle V1 may be equipped with any sensor used for autonomous driving as sensor 21, but it is preferable to include a sensor that detects the surroundings of vehicle V1. In the example in Figure 14, a millimeter-wave radar 21A, a stereo camera 21B, and an ultrasonic sensor 21C are mounted as sensors 21 that detect the surroundings of vehicle V1, but only some of these may be mounted. Vehicle V1 may also be equipped with an ITS sensor 21D that acquires information on the amount of traffic around vehicle V, and a GNSS sensor 21E that detects the position information of vehicle V1 as sensor 21. Note that in Figure 14, the X1 direction is the front of the vehicle V1.

[0101] (Millimeter-wave radar) In this embodiment, the millimeter-wave radar 21A is mounted on the front side of the vehicle V1, around the center of the front end of the vehicle body. The data accuracy of the millimeter-wave radar 21A depends on the amount of data, which is determined by the sampling rate of the millimeter-wave radar 21A (the ratio of data samples used for information processing out of the acquired sensor data) and the number of bits (the amount of information in the acquired data). If the amount of data is large (high sampling rate or high number of bits), the data accuracy will also be high. However, if the data accuracy is high, that is, if the ratio of data samples used for information processing out of the acquired sensor data is high, or if the amount of information in the acquired data is large, the power consumption required for information processing to acquire information necessary for automatic driving control from the acquired sensor data will also be high.

[0102] (Stereo Camera) In this embodiment, the stereo camera 21B is mounted on the front side of the vehicle V1, around the upper center of the glass. The data accuracy of the stereo camera 21B also depends on the amount of data, which is determined by the frame rate of the stereo camera 21B (number of images used for information processing), the color difference format of the acquired images, the resolution, and the number of bits (amount of information in the acquired images). When the amount of data is large (high frame rate, large color difference format of the acquired images, high resolution of the acquired images, or high number of bits in the acquired images), the data accuracy also increases. However, when the data accuracy is high, that is, when the amount of data is large, the power consumption required for information processing to acquire information necessary for autonomous driving control from the acquired sensor data also increases.

[0103] (Ultrasonic Sensor) In this embodiment, the ultrasonic sensor 21C is mounted around the left and right front ends of the vehicle V1, and around the rear of the vehicle V1 and the center of the front end of the vehicle body. The data accuracy of the ultrasonic sensor 21C depends on the amount of data acquired from the ultrasonic sensor 21C, and the amount of data is determined by the sampling rate of the ultrasonic sensor 21C (the ratio of the number of data samples used for information processing out of the acquired sensor data) and the number of bits (the amount of information in the acquired data). If the amount of data is large (high sampling rate or high number of bits), the data accuracy will also be high. However, if the data accuracy is high, that is, if the ratio of the number of data samples used for information processing out of the acquired sensor data is high, or if the amount of information in the acquired data is large, the power consumption for information processing to acquire information necessary for automatic driving control from the acquired sensor data will also be high.

[0104] (ITS Sensor) The ITS (Intelligent Transportation Systems) sensor 21D is a sensor that acquires information about the traffic volume around vehicle V1 that can be used by the automated driving system for automated driving (such as surrounding traffic volume and accident information) from, for example, V2X (Vehicle to X) equipment placed on a road that can communicate with vehicle V1, or from vehicles other than vehicle V1 that can communicate with vehicle V1.

[0105] (GNSS Sensor) In this embodiment, the GNSS sensor 21F is a sensor that acquires the position information of the vehicle V1. The GNSS sensor 21E receives GNSS signals from GNSS satellites and acquires the position information of the vehicle V1 based on the received GNSS signals.

[0106] In the example shown in Figure 14, the sensor 21 includes a millimeter-wave radar 21A, a stereo camera 21B, an ultrasonic sensor 21C, an ITS sensor 21D, and a GNSS sensor 21E. However, the number and types of sensors 21 are not limited to this example.

[0107] (Information Processing Device) The information processing device 30A controls the data accuracy of sensor data, which is information acquired by at least one sensor 21, based on the autonomous driving level of the vehicle V1. The information processing device 30A may be implemented by an information processing terminal such as a computer equipped with the functions of a PC, workstation, or server. In addition to being mounted on the vehicle V1, the information processing device 30A may also be a portable device that can be used in the vehicle V1.

[0108] Figure 15 is a schematic block diagram of an information processing device according to the second embodiment. As shown in Figure 15, the information processing device 30A according to this embodiment includes an input / output unit 31A, a storage unit 32A, and a control unit 33A. The input / output unit 31A is an interface for exchanging data between an external device (user interface 10, sensor 21) and the information processing device, and is, for example, an input / output terminal.

[0109] (Storage Unit) The storage unit 32A is a storage device that stores various types of information. The storage unit 32A comprises a main memory and an auxiliary storage device. The main memory may be implemented by semiconductor memory elements such as RAM, ROM, or flash memory. The auxiliary storage device may be implemented by a hard disk, SSD, or optical disc. The program for the control unit 33A stored in the storage unit 32A may be stored on a recording medium that can be read by the information processing device 30A.

[0110] (Control Unit) The control unit 33A is a controller that manages and controls the information processing device 30A. The control unit 33A is realized by a CPU, MPU, etc., executing various programs stored in the memory unit 32A using RAM as the working area. Alternatively, the control unit 33A may be realized by an integrated circuit such as an ASIC or FPGA.

[0111] As shown in Figure 15, the control unit 33A includes a condition acquisition unit 331A, a level setting unit 332A, a data acquisition unit 333A, an accuracy setting unit 334A, an object detection unit 335A, and an output control unit 336A. The control unit 33A realizes these functions and performs these processes by reading and executing a program (software) from the storage unit 32A. Note that these functions of the control unit 33 may be realized by electronic circuits. Furthermore, the control unit 33A may execute these processes with a single CPU, or it may have multiple CPUs and execute these processes in parallel with the multiple CPUs.

[0112] (Processing by the Information Processing Device) The processing details of the information processing device 30A are described below.

[0113] (Acquisition of location information) The condition acquisition unit 331A acquires location information, which is information about the location where the vehicle V1 is traveling. The method for acquiring location information can be any method, but in this embodiment, the condition acquisition unit 331A causes the GNSS sensor 21E to acquire the location information of the vehicle V1, and based on the location information of the vehicle V1 acquired by the GNSS sensor 21E and the map information including the road network stored in the storage unit 32A, it acquires location information, which is information about the location (road) where the vehicle V1 is traveling.

[0114] (Setting of Autonomous Driving Level Options) The level setting unit 332A sets the autonomous driving level options. The autonomous driving level options are information indicating candidate autonomous driving levels that can be selected by user U. The autonomous driving level options may be presented to user U from the user interface 10.

[0115] The method for setting the autonomous driving level options can be any method, but in this embodiment, the level setting unit 332A sets the autonomous driving level options based on location information acquired by the condition acquisition unit 331A. More specifically, the level setting unit 332A sets the autonomous driving level options based on location information and a driving plan entered by the user U. Here, the driving plan entered by the user U is information that includes the destination and the route to the destination. In other words, the level setting unit 332A acquires the current position of the vehicle V1 on the map using location information, acquires the route from the current position to the destination from the driving plan entered by the user U, and then sets the autonomous driving level options. For example, if it is determined that the route from the current position to the destination is a specific road where autonomous driving level 4 can be set, the autonomous driving level options that allow selection of autonomous driving levels 0 to 4 may be set. Alternatively, if it is determined that the route from the current position to the destination is not a specific road where autonomous driving level 4 can be set, autonomous driving levels 4 or 5 may be excluded from the options, and autonomous driving level options that allow selection of only autonomous driving levels 0 to 3 may be set. For example, if the system determines that the route from the current location to the destination is a specific road that can only be driven on at autonomous driving level 4, it may set up an option to indicate that autonomous driving level 4 has been set, preventing user U from selecting an autonomous driving level.

[0116] In this embodiment, the level setting unit 332A sets the autonomous driving level options based on location information, but the autonomous driving level options may also be set based on other conditions. For example, the level setting unit 332A may set the autonomous driving level options according to the road congestion. Specifically, even in situations where autonomous driving level 3 can be included in the autonomous driving level options, such as on a specific highway, if there is congestion on the route from the current location to the destination, the level setting unit 332A may exclude autonomous driving level 3 from the options and set the autonomous driving level options so that only autonomous driving levels 0 to 2 can be selected. In low-speed autonomous driving during congestion on a specific highway (equivalent to autonomous driving level 3), the autonomous driving system controls the vehicle, but when the congestion is cleared and the surrounding vehicle speed increases, it may be necessary to hand over the acceleration control of the vehicle V1 to match the increased surrounding vehicle speed from the autonomous driving system to the user U (change to equivalent to autonomous driving level 2). Therefore, by setting the autonomous driving level options according to the road congestion on the route from the current location to the destination, a safer autonomous driving level option can be set for the user U.

[0117] Alternatively, the level setting unit 332A may set the autonomous driving level options based on weather conditions. Specifically, even when the level setting unit 332A can include autonomous driving levels 4 or 5 in the autonomous driving level options, if bad weather (heavy rain, snow, fog, etc.) is expected on the route from the current location to the destination, it may exclude autonomous driving level 4 from the options and set the autonomous driving level options so that only autonomous driving levels 0 to 3 can be selected. In autonomous driving (equivalent to autonomous driving level 4) in a specific urban area and on a specific highway, the autonomous driving system controls the vehicle, but if the sensors cannot provide the expected data accuracy in bad weather, it may be necessary to take over vehicle control from the autonomous driving system to user U control (change to equivalent to autonomous driving level 3). Therefore, by setting the autonomous driving level options based on the weather conditions on the route from the current location to the destination, a safer autonomous driving level option can be set for user U.

[0118] (Output of Automated Driving Level Selection) The output control unit 336A outputs (displays) the automated driving level selection set by the level setting unit 332A to the user interface 10.

[0119] (Acquisition of Autonomous Driving Level) The level setting unit 332A acquires the autonomous driving level of the vehicle V1. In this embodiment, the level setting unit 332A acquires selection information indicating the autonomous driving level specified by the user U, and acquires the autonomous driving level indicated in the selection information. In this case, for example, the user interface 10 displays the set autonomous driving level options and accepts input from the user U of the autonomous driving level selected from the autonomous driving level options. The level setting unit 332A acquires the autonomous driving level information input to the user interface 10 as selection information.

[0120] However, the method for acquiring the automatic driving level by the level setting unit 332A is not limited to the above. For example, setting and outputting the automatic driving level selection is not mandatory, and the level setting unit 332A may acquire the automatic driving level freely selected by the user U as selection information. Also, the automatic driving level is not limited to being set by the user U's selection, but may be set automatically by the level setting unit 332A. In this case, the method for setting the automatic driving level is the same as the method for setting the automatic driving level selection described above, so a detailed explanation is omitted. If multiple automatic driving levels are selected in the automatic driving level selection, the level setting unit 332A may select one automatic driving level from among the multiple automatic driving levels based on any criterion.

[0121] (Data acquisition) The data acquisition unit 333A causes the sensor 21 to perform sensing processing. The data acquisition unit 333A acquires the data acquired by the sensor 21 through sensing processing as sensor data.

[0122] (Setting Data Accuracy) The accuracy setting unit 334A sets the data accuracy of the sensor data of the acquired sensor 21 based on the automatic driving level acquired by the level setting unit 332A. Data accuracy refers to the degree of the amount of data in the sensor data of each sensor 21. Therefore, the higher the data accuracy, the larger the amount of sensor data and the higher the detection accuracy. On the other hand, the higher the data accuracy, the greater the power consumption for information processing to acquire information including objects around the vehicle V1 from the sensor data.

[0123] The accuracy setting unit 334A may set the data accuracy in any way, but in this embodiment, the accuracy setting unit 334A is set so that the lower the acquired automated driving level (i.e., the lower the degree of automation), the lower the data accuracy of the sensor data acquired from each sensor 21.

[0124] In other words, for example, the accuracy setting unit 334A sets the data accuracy of the sensor data when a certain level of automatic driving is set to be lower than the data accuracy of the sensor data when a higher level of automatic driving is set. Also, for example, the accuracy setting unit 334A causes the sensor 21 to detect sensor data when the data accuracy is set to the maximum accuracy value, which represents the highest possible data accuracy. Then, when the automatic driving level is at its maximum (the highest degree of automation), that is, in the case of automatic driving level 5 in this embodiment, the accuracy setting unit 334A may leave the data accuracy of the sensor data at the maximum accuracy value. On the other hand, when the automatic driving level is lower than the maximum, the accuracy setting unit 334A may change the data accuracy of the sensor data to be lower than the maximum accuracy value.

[0125] Furthermore, the accuracy setting unit 334A may use any parameter corresponding to the amount of data as the data accuracy (amount of data) to be controlled. However, in this embodiment, the accuracy setting unit 334A controls either the sampling rate of the sensor 21 or the number of bits of the acquired data (preferably both) as the data accuracy to be controlled. In other words, the accuracy setting unit 334A sets either the sampling rate of the millimeter-wave radar 21A and the ultrasonic sensor 21C or the number of bits of the acquired data (preferably both) as control factors for controlling the data accuracy of the millimeter-wave radar 21A and the ultrasonic sensor 21C. In addition, the accuracy setting unit 334A in this embodiment controls the data accuracy of the stereo camera 21B by at least one (preferably all) of the frame rate, the color difference format of the acquired image, the resolution of the acquired image, and the number of bits of the acquired image. In other words, the precision setting unit 334A sets at least one (preferably all) of the frame rate of the stereo camera 21B, the color difference format of the acquired image, the resolution of the acquired image, and the number of bits of the acquired image as control factors for controlling the data precision of the stereo camera 21B.

[0126] Specifically, the accuracy setting unit 334A controls the sampling rate of the millimeter-wave radar 21A and the ultrasonic sensor 21C, and the number of bits in the acquired data (preferably both), to decrease (the control factor decreases) as the acquired autonomous driving level decreases. The accuracy setting unit 334A also controls the frame rate of the stereo camera 21B, the color difference format of the acquired image, the resolution of the acquired image, and the number of bits in the acquired image (preferably all of them), to decrease (the control factor decreases) as the acquired autonomous driving level decreases. In other words, if the sampling rate of the millimeter-wave radar 21A and the ultrasonic sensor 21C, and the number of bits in the acquired data (preferably both), decreases, the power consumption for information processing to acquire information including objects around the vehicle V1 from these sensor data also decreases. Similarly, if the frame rate of the stereo camera 21B, the color difference format of the acquired images, the resolution of the acquired images, and the number of bits in the acquired images are reduced, the power consumption for information processing to acquire information including objects around the vehicle V1 from the acquired sensor data will also be reduced.

[0127] In this way, by controlling the data accuracy of each sensor 21 to decrease as the acquired autonomous driving level decreases, the power consumption for information processing to acquire information including objects around the vehicle V1 from the acquired sensor data can be reduced. That is, when the autonomous driving level is low, the degree to which the user U controls the driving is high, so even if the data accuracy of the sensor 21 is reduced to some extent, the driving control can be performed appropriately. Therefore, as in this embodiment, by reducing the data accuracy as the autonomous driving level decreases, power consumption can be reduced while performing driving control appropriately. On the other hand, when the autonomous driving level is high, the data accuracy of the sensor 21 can be kept to a certain extent so that automatic driving control can be performed appropriately.

[0128] (Example where the control factors are the sampling rate and the number of bits of data to be acquired) Figure 16 is a table showing an example of control factors for the data accuracy of a millimeter-wave radar according to the level of autonomous driving. In the example shown in Figure 16, the accuracy setting unit 334A sets the data accuracy to four levels according to the level of autonomous driving, and the sampling rate and the number of bits of data to be acquired (control factors) of the millimeter-wave radar 21A corresponding to each data accuracy level are listed in the table. Here, the data accuracy for autonomous driving level 0, which corresponds to fully manual driving, is set to data accuracy level 0. Specifically, the accuracy setting unit 334A sets data accuracy level 0 for autonomous driving level 0, data accuracy level 1 for autonomous driving levels 1 and 2, data accuracy level 2 for autonomous driving level 3, and data accuracy level 3 for autonomous driving levels 4 and 5, and the sampling rate and the number of bits of data to be acquired are assigned to each of the data accuracy levels 0 to 3. This allows for fine-tuning of data accuracy levels according to the autonomous driving level, thereby optimizing power consumption for information processing that acquires information including objects around the vehicle V1 from the sensor data of the millimeter-wave radar 21A.

[0129] (Example where the control factors are frame rate, color difference format of the acquired image, resolution of the acquired image, and bit depth of the acquired image) Figure 17 is a table showing an example of control factors for the data accuracy of a stereo camera according to the autonomous driving level. In the example shown in Figure 17, the accuracy setting unit 334A sets the data accuracy of the stereo camera 21B into six levels according to the autonomous driving level, and the control factors for the stereo camera 21B corresponding to each data accuracy level (frame rate, color difference format of the acquired image, resolution of the acquired image, and bit depth of the acquired image) are listed in the table. More specifically, the accuracy setting unit 334A sets separate data accuracy levels for each of the autonomous driving levels 0 to 5. By finely adjusting the data accuracy level according to the autonomous driving level, it is possible to optimize the power consumption for information processing that acquires information including objects around the vehicle V1 from the sensor data of the stereo camera 21B.

[0130] (Acquisition of data to be processed) The object detection unit 335A acquires data to be processed from the sensor data acquired from each sensor 21, which will be used for information processing to acquire information including objects around the vehicle V1. More specifically, the object detection unit 335A converts the sensor data acquired from each sensor 21 to match the data accuracy (data amount) set by the accuracy setting unit 334A, and acquires the converted data as data to be processed.

[0131] (Information processing to obtain information including objects around the vehicle V1 from the data to be processed) The object detection unit 335A obtains information including objects around the vehicle V1 from the acquired data to be processed. Specifically, the object detection unit 335A performs information processing on the acquired data to be processed to obtain information including objects around the vehicle V1. The information processing to obtain information including objects around the vehicle V1 from the data to be processed can be performed in any way, but in this embodiment, the object detection unit 335A obtains information including objects around the vehicle V1 from the data to be processed using a trained model.

[0132] (Trained Model) The trained model according to this embodiment is an AI model that has learned the correspondence between sensor data and information about objects surrounding the vehicle V1 through machine learning. The trained model may be a neural network model such as a DNN (Deep Neural Network), CNN (Convolutional Neural Network), Transformer, or fully connected layer (linear layer), or a neural network model that combines these as appropriate. The trained model is configured to output information about objects (for example, the position of objects around the vehicle V, their distance from the vehicle V, and what those objects are) in response to the input of processing data, which is sensor data. This trained model is a pattern matching model trained using training data consisting of pairs of input data corresponding to sensor data and correct data (output data) corresponding to object information. Note that the trained model is not limited to a pattern matching model, and a pattern matching model using multiple rule bases may also be used.

[0133] (Example of setting a trained model 1) In this embodiment, it is preferable for the object detection unit 335A to use different trained models to detect object information for processing data with different data accuracy. That is, it is preferable to prepare a trained model for each data accuracy. In this case, the object detection unit 335A sets the trained model to be used for information processing based on the data accuracy of the processing data. Figure 18A is a diagram showing example 1 of setting a trained model according to the second embodiment. In the example in Figure 18A, the object detection unit 335A sets two trained models to be used for information processing depending on the data accuracy of the processing data. More specifically, the object detection unit 335A sets the trained models to be used for information processing separately for data accuracy levels 0 to 2 and data accuracy levels 3 to 5 of the processing data. This makes it possible to set trained models that can handle the amount of input data.

[0134] (Example of setting a trained model 2) Figure 18B shows an example of setting a trained model 2 according to the second embodiment. In the example in Figure 18B, the object detection unit 335A sets up six trained models to be used for information processing, depending on the data accuracy of the data to be processed. More specifically, the object detection unit 335A sets up a separate trained model to be used for information processing for each of the data accuracy levels 0 to 5 in the data to be processed. This makes it possible to set up trained models that can correspond to the amount of data in the input data in more detail.

[0135] (Output of information including objects around the vehicle V1) The output control unit 336A in the information processing device 30A outputs the information including objects around the vehicle V1 acquired by the object detection unit 335A to a vehicle control unit (not shown). The vehicle control unit uses the information including objects around the vehicle V1 to perform automatic driving control of the vehicle V1.

[0136] (Processing Flow) Next, the processing flow of this embodiment will be described. Figure 19 is a flowchart illustrating the processing flow of the information processing device according to this embodiment. As shown in Figure 19, the condition acquisition unit 331A in the information processing device 30A causes the GNSS sensor 21E to acquire location information of the vehicle V1, and acquires location information, which is information about the place (road) where the vehicle V1 is traveling, based on the location information of the vehicle V1 acquired by the GNSS sensor 21E and the map information including the road network stored in the storage unit 32A (step S20). The level setting unit 332A sets an autonomous driving level selection list, which shows candidate autonomous driving levels that the user U can select, based on the location information acquired by the condition acquisition unit 331A, and outputs it to the user U from the user interface 10. Then, the level setting unit 332A acquires selection information from the user interface 10, which is information indicating the autonomous driving level selected by the user U, from the autonomous driving level selection list, and acquires an autonomous driving level based on the selection information (step S22). The data acquisition unit 333A causes each sensor 21 to perform sensing processing and acquires sensor data (step S24). The accuracy setting unit 334A sets the data accuracy of each sensor 21 mounted on the vehicle V1 based on the automatic driving level acquired by the level setting unit 332A (step S26). The object detection unit 335A converts the sensor data acquired from the sensors 21 to match the data accuracy set by the accuracy setting unit 334A, and acquires the converted data as data to be processed. Then, the object detection unit 335A acquires information including objects around the vehicle V1 from the acquired data to be processed (step S28), and terminates this process.

[0137] (Effects) As described above, the information processing device 30A according to this embodiment sets an autonomous driving level selection list that indicates a candidate for an autonomous driving level that can be selected by a user U, based on location information, which is information about the location (road) where the vehicle V1 capable of switching autonomous driving levels is traveling. From the autonomous driving level selection list, it acquires an autonomous driving level based on selection information, which is information indicating the autonomous driving level selected by the user U. Based on the acquired autonomous driving level, it sets the data accuracy of each sensor 21 mounted on the vehicle V1. Then, based on the sensor data with the set data accuracy, it acquires information including objects around the vehicle V1. According to this embodiment, it is possible to control the data accuracy of the sensor data acquired from the sensors 21 mounted on the vehicle V1 capable of switching autonomous driving levels, and optimize the power consumption for information processing that acquires information including objects around the vehicle V1 from the sensor data.

[0138] (Third Embodiment) Up to this point in the second embodiment, the accuracy setting unit 334A in the information processing device 30A controlled the data accuracy (data amount) of the sensor data acquired from the sensor 21 based on the acquired automatic driving level, and optimized the power consumption related to the information processing of that data. In the third embodiment, however, the data accuracy of the sensor data acquired from the sensor 21 is not controlled. In the third embodiment, the model setting unit 337A, described later, sets a trained model for information processing based on the automatic driving level acquired by the accuracy setting unit 334A, thereby optimizing the power consumption related to the information processing of the data. The processing of the third embodiment may be performed in conjunction with the processing of the second embodiment, or the processing of the third embodiment may be performed without performing the processing of the second embodiment. That is, the information processing device 30A may change the data accuracy of the sensor data based on the automatic driving level as in the second embodiment, while performing the processing of the third embodiment described later, or it may perform the processing of the third embodiment described later without changing the data accuracy of the sensor data based on the automatic driving level as in the second embodiment (i.e., while keeping the data accuracy constant).

[0139] Figure 20 is a schematic block diagram of the information processing device according to the third embodiment. As shown in Figure 20, the information processing device 30A according to this embodiment comprises an input / output unit 31A, a storage unit 32A, and a control unit 33A. The input / output unit 31A and the storage unit 32A are the same as in the second embodiment, so their description is omitted.

[0140] As shown in Figure 20, the control unit 33A includes a condition acquisition unit 331A, a level setting unit 332A, a data acquisition unit 333A, an accuracy setting unit 334A, an object detection unit 335A, an output control unit 336A, and a model setting unit 337A. The control unit 33A realizes these functions and performs these processes by reading and executing a program (software) from the storage unit 32A. Note that these functions of the control unit 33A may be realized by electronic circuits. Furthermore, the control unit 33A may execute these processes with a single CPU, or it may have multiple CPUs and execute these processes in parallel with the multiple CPUs.

[0141] (Processing of the Information Processing Device) The processing contents of the information processing device 30A according to this embodiment will be described below. However, the same processing contents as in the second embodiment will be omitted from the explanation.

[0142] (Setting the trained model) The model setting unit 337A sets a trained model to be used to perform information processing to acquire information including objects around the vehicle V1 from the sensor data of the sensor 21 acquired by the method of the second embodiment, based on the autonomous driving level acquired by the level setting unit 332A by the method of the second embodiment. More specifically, the model setting unit 337A in this embodiment sets the processing accuracy of the trained model used to perform information processing to acquire information including objects around the vehicle V1 from the acquired sensor data of the sensor 21, based on the acquired autonomous driving level. The trained model used in this embodiment may be the one described in the second embodiment. Here, the processing accuracy of the trained model refers to the computational complexity of the trained model. Therefore, the higher the processing accuracy, the greater the computational complexity of the trained model, and the more complex the calculation of events becomes possible. On the other hand, the greater the computational complexity of the trained model, the greater the power consumption for the information processing to acquire information including objects around the vehicle V1 from the sensor data. The computational complexity of the trained model is specifically determined by at least one of the number of weight parameters, the number of layers, the number of fully connected layers, and the resolution. The more weight parameters there are, the more computational complexity the model requires, and the better the accuracy. The more layers there are, the more computational complexity the trained model requires, and the more complex the events it can handle. The more fully connected layers there are, the more computational complexity the trained model requires, and the better the accuracy. Higher resolution increases the computational complexity of the model, and the better the accuracy.

[0143] The model setting unit 337A may set the processing accuracy of the trained model in any way it chooses, but in this embodiment, the model setting unit 337A sets the processing accuracy of the trained model used for information processing to be lower the lower the acquired autonomous driving level (i.e., the lower the degree of automation). That is, for example, the model setting unit 337A sets the processing accuracy of the trained model used when a certain autonomous driving level is set to be lower than the processing accuracy of the trained model used when an autonomous driving level higher than that level is set to.

[0144] Furthermore, the model setting unit 337A may use any parameter corresponding to the processing accuracy as the processing accuracy (computational complexity) of the trained model to be controlled. However, in this embodiment, the model setting unit 337A controls at least one (preferably all) of the number of weight parameters, the number of layers, the number of fully connected layers, and the resolution of the trained model as the processing accuracy to be controlled. In other words, the model setting unit 337A sets at least one (preferably all) of the number of weight parameters, the number of layers, the number of fully connected layers, and the resolution of the trained model as control factors for controlling the processing accuracy of the trained model. Here, the number of weight parameters refers to the number of variables that the trained model needs to optimize during training. These weights are adjusted when the model is trained and determine the output for the input data. The more weight parameters there are, the more information the model learns and the better its ability to recognize complex patterns. However, as the number of weight parameters increases, the computational complexity also increases proportionally. For example, in an image recognition model, an increase in the number of weight parameters increases the computation required to analyze the information of each pixel in detail. For example, in an autonomous driving model using a LiDAR sensor, if the number of weight parameters is 1 million, the model will perform 1 million calculations to analyze the distance information of each point in detail. This improves the accuracy of the model, but also increases the computational load. The number of layers refers to the total number of input, hidden, and output layers in a neural network. The more layers there are, the better the model can extract more features and analyze complex phenomena. However, the computational load increases proportionally with the number of layers. For example, in an autonomous driving model using a camera sensor, if the number of layers is 10, each layer will perform 10 calculations to extract image features. Increasing the number of layers improves the ability to capture subtle changes in the image, but also increases the computational load. The number of fully connected layers refers to the number of layers in a neural network where all neurons are tightly connected to the preceding and succeeding layers. For example, in an autonomous driving model using a radar sensor, if the number of fully connected layers is 5, each layer will perform 5 calculations to analyze the velocity and direction of an object. Increasing the number of fully connected layers improves the prediction accuracy of the model, but it also increases the computational cost.Furthermore, resolution refers to the resolution of the image output by the trained model. The higher the resolution, the more detailed the model's ability to extract features and process information with higher accuracy improves. However, as the resolution increases, the computational load also increases proportionally. For example, in an autonomous driving model using a camera sensor, if the resolution is 1920 x 1080 pixels, 1920 x 1080 calculations are performed to analyze the information of each pixel. Higher resolution improves the model's ability to extract more detailed features, but it also increases the computational load. In the case of pattern matching, the model setting unit 337A controls the trained model so that the number of rules decreases (control factors decrease) as the acquired autonomous driving level decreases, thereby reducing the power consumption for information processing by the trained model that acquires information including objects around the vehicle V1.

[0145] Specifically, the model setting unit 337A controls the trained model so that the lower the acquired autonomous driving level, the lower at least one of the number of weight parameters, the number of layers, the number of fully connected layers, and the resolution (the lower the control factor). In other words, if at least one of the number of weight parameters, the number of layers, the number of fully connected layers, and the resolution in the trained model is lower, the power consumption for information processing by the trained model, which acquires information including objects around the vehicle V from the acquired sensor data, will also decrease.

[0146] In this way, by controlling the system so that the processing accuracy of the trained model used for information processing decreases as the acquired autonomous driving level decreases, the power consumption for information processing that acquires information including objects around the vehicle V1 from the acquired sensor data can be reduced. That is, when the autonomous driving level is low, the degree to which the user U controls the driving is high, so even if the processing accuracy of the trained model is reduced to some extent, the driving control can be performed appropriately. Therefore, as in this embodiment, by reducing the processing accuracy of the trained model as the autonomous driving level decreases, power consumption can be reduced while maintaining appropriate driving control. On the other hand, when the autonomous driving level is high, appropriate automatic driving control can be performed by keeping the processing accuracy of the trained model relatively high.

[0147] The model setting unit 337A may set trained models with different processing accuracies in any way. For example, the model setting unit 337A may store multiple trained models with different processing accuracies in the memory unit in advance, associating them with the autonomous driving level, and then read out and use the trained model that corresponds to the current autonomous driving level. Alternatively, the model setting unit 337A may store a reference trained model (reference trained model) in the memory unit in advance, read out the reference trained model, change the processing accuracy of the reference trained model according to the current autonomous driving level, and then use it.

[0148] (Example of preparing multiple pre-trained models with different processing accuracies in advance) Figure 21 is a table showing an example of control factors for the processing accuracy of pre-trained models tailored to the autonomous driving level. In the example shown in Figure 21, the model setting unit 337A sets the processing accuracy of the pre-trained models in six levels according to the autonomous driving level, and the control factors (number of weight parameters, number of DNN layers, number of fully connected layers, resolution) corresponding to each processing accuracy level are listed in the table. More specifically, the model setting unit 337A sets the processing accuracy level of the pre-trained models separately for each of the autonomous driving levels 0 to 5. By finely adjusting the processing accuracy level of the pre-trained models according to the autonomous driving level, it is possible to optimize the power consumption for information processing that acquires information including objects around the vehicle V1 from sensor data.

[0149] (Example 1 of changing the processing accuracy of the reference trained model) For example, the model setting unit 337A may set a trained model with lower processing accuracy based on the reference trained model, which is a trained model with high processing accuracy. More specifically, the model setting unit 337A may distill the reference trained model to set multiple trained models with different processing accuracies and lower processing accuracy than the reference trained model. Distillation is a technique that reduces the size and complexity of a model while retaining the knowledge of the reference trained model. Specifically, it is a technique that saves computational resources by using the output of the reference trained model as training data to train a smaller model. Figure 22 is a diagram showing example 1 of setting a trained model according to this embodiment. As shown in Figure 22, the model setting unit 337A may distill the trained model with the highest processing accuracy level 5 (reference trained model), which corresponds to autonomous driving level 5, and set trained models for each processing accuracy level. The processing accuracy of the reference trained model is, for example, 95% accuracy. By distilling this baseline pre-trained model, multiple pre-trained models with different accuracy levels, such as 85% and 75%, are created. Specific algorithms used for distillation include knowledge distillation and hard-label distillation. One step in the distillation process is to first obtain the output of the baseline pre-trained model and use it as training data to train a smaller model. This method allows for the reduction of model size and complexity while retaining knowledge of the baseline pre-trained model. Specifically, the output of the baseline pre-trained model is obtained. This output is the prediction result that the baseline pre-trained model generates for various input data. Next, this output is used as training data to train a smaller model. Specifically, the output of the baseline pre-trained model is supplied to the smaller model along with the input data, and the model is trained to reproduce the prediction result of the baseline pre-trained model. This training process reduces model size and complexity while retaining knowledge of the baseline pre-trained model. Here, a "smaller model" refers to a model whose size and complexity have been reduced compared to the baseline pre-trained model. Specifically, it is a model with fewer parameters and less computational resource consumption.Here, we assumed that all trained models with low processing accuracy levels (trained models from processing accuracy levels 4 to 0) are distilled from a single trained model with a high processing accuracy level of 5. However, for example, trained models with processing accuracy levels 4 and 3 are distilled from a trained model with processing accuracy level 5, and trained models with processing accuracy levels 1 and 0 are distilled from a trained model with processing accuracy level 2, thereby setting up multiple trained models with high processing accuracy levels and distilling them.

[0150] (Example 2 for changing the processing accuracy of the reference trained model) Furthermore, the model setting unit 337A is not limited to setting multiple trained models with lower processing accuracy than the reference trained model from the reference trained model, but may also set the trained model to be used using a trained model that has higher processing accuracy than the trained model but has a similar level of processing accuracy. In more detail, the model setting unit 337A may distill trained models with similar processing accuracy levels and set a trained model with lower processing accuracy. That is, for example, the model setting unit 337A may use the reference trained model to generate a first trained model with lower processing accuracy than the reference trained model, and use the first trained model to generate a second trained model with lower processing accuracy than the first trained model. Figure 23 is a diagram showing example 2 of setting trained models according to this embodiment. As shown in Figure 23, the model setting unit 337A may distill a trained model with one higher processing accuracy level and set a trained model with one lower processing accuracy level.

[0151] (Example 3 for changing the processing accuracy of the reference trained model) The model setting unit 337A may also set the trained model based on multiple trained models with higher processing accuracy. More specifically, the model setting unit 337A may set the trained model by distilling multiple trained models. That is, for example, the model setting unit 337A may use the reference trained model to generate a first trained model with lower processing accuracy than the reference trained model, and then use the reference trained model and the first trained model to generate a second trained model with lower processing accuracy than the first trained model. Figure 24 shows an example 3 of setting a trained model according to this embodiment. In the example in Figure 24, the trained model with processing accuracy level 4 is set by distilling the trained model with processing accuracy level 5. The trained model with processing accuracy level 3 is set by distilling the trained models with processing accuracy levels 5 and 4. The trained model with processing accuracy level 2 is set by distilling the trained models with processing accuracy levels 5 to 3. Furthermore, a trained model with processing accuracy level 1 is created by distilling trained models with processing accuracy levels 5 to 2. Similarly, a trained model with processing accuracy level 0 is created by distilling trained models with processing accuracy levels 5 to 1. Here, lower-level (lower processing accuracy) trained models are distilled by weighting and averaging them by the number of weight parameters of higher-level (higher processing accuracy) trained models. For example, a trained model with processing accuracy level 3 is created by weighting and averaging the outputs of trained models with processing accuracy levels 5 and 4 by the number of weight parameters of the trained models. Alternatively, distillation may be performed by weighting and averaging not only by the number of weight parameters of the trained models, but also by the number of layers or the input resolution.

[0152] (Example 4 of changing the processing accuracy of the reference trained model) The model setting unit 337A may also set a trained model with lower processing accuracy by pruning a trained model with high processing accuracy (for example, a reference trained model). Pruning is a technique that reduces the size of the model and improves computational efficiency by reducing unnecessary nodes and neurons in the trained model. Figure 25 shows an example 4 of setting a trained model according to this embodiment. In the example shown in Figure 25, the model setting unit 337A sets a trained model with processing accuracy level 4 by pruning the trained model with the highest processing accuracy level 5 (reference trained model) corresponding to autonomous driving level 5, and reducing the number of nodes and the number of neurons connected to the nodes (pruning). It is desirable to calculate in advance the nodes to be reduced and the number of neurons to be reduced, corresponding to the processing accuracy level, when pruning trained models. For example, the degree of influence on the output of each node is calculated in order to identify nodes and neurons of low importance. Nodes and neurons with low influence are selected as targets for pruning. Next, the number of nodes or neurons to be reduced is determined according to the processing accuracy level. For example, to correspond to processing accuracy level 4, specific reduction rates are set, such as reducing the number of nodes in the reference trained model by 20% and the number of neurons by 30%. In this way, the nodes and neurons to be pruned can be calculated in advance, and the processing accuracy of the trained model can be adjusted efficiently. Specifically, if the number of nodes in the reference trained model is reduced by 20%, for example, if the reference trained model has 1000 nodes, 200 nodes are reduced, resulting in 800 nodes. Also, if the number of neurons in the reference trained model is reduced by 30%, for example, if the reference trained model has 5000 neurons, 1500 neurons are reduced, resulting in 3500 neurons. To calculate the impact of each node on the output, the impact of the node's output on the overall model performance is evaluated, and nodes or neurons with a low impact are selected as targets for pruning.Next, the number of nodes and neurons to be reduced is determined based on a reduction rate corresponding to the processing accuracy level, and the model size is adjusted by reducing the selected nodes and neurons. To evaluate the degree of impact, the following procedure is used to assess the influence of each node's output on the overall model performance. First, to calculate the impact on each node's output, the impact of the node's output on the overall model performance is evaluated. Specifically, the impact of each node's output on the model's prediction accuracy and error is calculated, and nodes and neurons with low impact are selected for pruning. For example, to calculate the impact of a node's output on the model's prediction accuracy, the output of each node is temporarily disabled, and the resulting change in the model's prediction accuracy is evaluated. Based on this evaluation, nodes and neurons with low impact are identified and selected for pruning. Next, the number of nodes and neurons to be reduced is determined based on a reduction rate corresponding to the processing accuracy level, and the model size is adjusted by reducing the selected nodes and neurons. Here, the lower-level trained model is trained with the number of neurons to be reduced for each layer set in advance compared to the higher-level trained model. For example, if the number of neurons in a lower-level trained model is 90% of the number of neurons in a higher-level trained model, you could set it up so that each layer of the lower-level trained model is reduced by 90%, or you could set it up so that one or more specific layers are reduced intensively.

[0153] (Information processing using a set pre-trained model) The object detection unit 335A inputs the sensor data acquired by the method described in the second embodiment into the pre-trained model set by the model setting unit 337A, and acquires information including objects around the vehicle V1.

[0154] (Processing Flow) Next, the processing flow of this embodiment will be described. Figure 26 is a flowchart illustrating the processing flow of the information processing device according to this embodiment. As shown in Figure 26, the condition acquisition unit 331A in the information processing device 30A causes the GNSS sensor 21E to acquire location information of the vehicle V1, and acquires location information, which is information about the place (road) where the vehicle V1 is traveling, based on the location information of the vehicle V1 acquired by the GNSS sensor 21E and the map information including the road network stored in the storage unit 32A (step S30). The level setting unit 332A sets an autonomous driving level selection list, which shows candidate autonomous driving levels that the user U can select, based on the location information acquired by the condition acquisition unit 331A, and outputs it to the user U from the user interface 10. Then, the level setting unit 332A acquires selection information from the user interface 10, which is information indicating the autonomous driving level selected by the user U, from the autonomous driving level selection list, and acquires an autonomous driving level based on the selection information (step S32). The data acquisition unit 333A causes each sensor 21 to perform sensing processing and acquires sensor data (step S34). The model setting unit 337A sets a trained model (processing accuracy of the trained model) to be used for information processing based on the autonomous driving level acquired by the level setting unit 332A (step S36). The object detection unit 335A inputs the sensor data acquired from the sensors 21 into the trained model set by the model setting unit 337A, acquires information including objects around the vehicle V1 (step S38), and terminates this process.

[0155] (Effects) As described above, the information processing device 30A according to this embodiment sets an autonomous driving level selection list that indicates a candidate for an autonomous driving level that can be selected by a user U, based on location information, which is information about the location (road) where a vehicle V1 capable of switching autonomous driving levels is traveling. From the autonomous driving level selection list, it acquires an autonomous driving level based on selection information, which is information indicating the autonomous driving level selected by the user U. Based on the acquired autonomous driving level, it sets a trained model (processing accuracy of the trained model) to be used for information processing. Then, by inputting the acquired sensor data into the set trained model, it acquires information including objects around the vehicle V1. According to this embodiment, the processing accuracy of the trained model used for information processing can be controlled based on the autonomous driving level, and the power consumption for information processing that acquires information including objects around the vehicle V1 from sensor data can be optimized.

[0156] (Effects of the Disclosure) As described above, the sensor control device 30 of the Disclosure includes a level setting unit 332 that acquires the autonomous driving level of a vehicle V capable of switching autonomous driving levels, and an accuracy setting unit 333 that sets the sensing accuracy of the sensor 20 mounted on the vehicle V based on the acquired autonomous driving level. According to the Disclosure, it is possible to control the sensor 20 mounted on a vehicle V capable of switching autonomous driving levels and optimize the power consumption of the sensor 20.

[0157] Furthermore, according to this disclosure, the accuracy setting unit 333 is set so that the lower the acquired autonomous driving level, the lower the sensing accuracy of the sensor 20. According to this disclosure, the power consumption of the sensor 20 can be reduced at autonomous driving levels where the proportion of vehicle control by the user U is large and high sensing accuracy is not required.

[0158] Furthermore, according to this disclosure, the system further includes a condition acquisition unit 331 that acquires location information, which is information about the location where the vehicle V is traveling, and battery information, which is information about the remaining battery level of the vehicle V. The accuracy setting unit 333 sets the sensing accuracy based on at least one of the location information and the battery information. According to this disclosure, it becomes possible to set a finer sensing accuracy based on at least one of the number of roadside devices in the travel route included in the location information and the remaining battery level included in the battery information, thereby optimizing the power consumption of the sensor 20.

[0159] The information processing device 30A of this disclosure includes a level setting unit 332A that acquires the autonomous driving level of a vehicle V1 whose autonomous driving level can be switched, a data acquisition unit 333A that acquires sensor data which is information acquired by a sensor 21 mounted on the vehicle V1, a model setting unit 337A that sets a trained model based on the acquired autonomous driving level, and an object detection unit 335A that inputs the acquired sensor data into the set trained model and acquires information including objects around the vehicle V. According to this disclosure, the processing accuracy of the trained model used for information processing can be controlled based on the autonomous driving level, and the power consumption for information processing that acquires information including objects around the vehicle V1 from sensor data can be optimized.

[0160] Furthermore, according to this disclosure, the model setting unit 337A sets a trained model whose processing accuracy decreases as the autonomous driving level decreases. According to this disclosure, at autonomous driving levels where the proportion of vehicle control by the user U is large and high processing accuracy of the trained model is not required, it is possible to reduce the power consumption for information processing that acquires information including objects around the vehicle V from sensor data.

[0161] Furthermore, according to this disclosure, the model setting unit 337A sets the trained model to be used when the autonomous driving level is low based on the trained model to be used when the autonomous driving level is higher than that trained model. According to this disclosure, a trained model with reduced processing accuracy can be efficiently set based on the trained model to be used when the autonomous driving level is high.

[0162] The information processing device 30A of this disclosure includes a level setting unit 332A that acquires the autonomous driving level of a vehicle V1 whose autonomous driving level can be switched, a data acquisition unit 333A that acquires sensor data which is information acquired by a sensor 21 mounted on the vehicle V1, an accuracy setting unit 334A that sets the data accuracy of the acquired sensor data based on the acquired autonomous driving level, and an object detection unit 335A that acquires information including objects around the vehicle V1 based on the sensor data with the set data accuracy. According to this disclosure, it is possible to control the data accuracy of the sensor data acquired from a sensor 21 mounted on a vehicle V1 whose autonomous driving level can be switched, and optimize the power consumption for information processing that acquires information including objects around the vehicle V from the sensor data.

[0163] Furthermore, according to this disclosure, the accuracy setting unit 334A is set so that the lower the autonomous driving level, the lower the data accuracy. According to this disclosure, in autonomous driving levels where the proportion of vehicle control by the user U is large and high data accuracy is not required, the power consumption for information processing that acquires information including objects around the vehicle V1 from sensor data can be reduced.

[0164] Furthermore, according to this disclosure, the object detection unit 335A inputs sensor data with a set data accuracy into a trained model corresponding to the autonomous driving level to acquire information including objects around the vehicle V1. According to this disclosure, by performing information processing using a trained model corresponding to the autonomous driving level, it is possible to perform information processing using a trained model that corresponds to the data accuracy (data amount) of the data to be processed, and to acquire highly accurate information including objects around the vehicle V1.

[0165] The control method for the sensor control device 30 according to this disclosure comprises the steps of acquiring the autonomous driving level of a vehicle V capable of switching autonomous driving levels, and setting the sensing accuracy of the sensor 20 mounted on the vehicle V based on the acquired autonomous driving level. According to this disclosure, it is possible to control the sensor 20 mounted on a vehicle V capable of switching autonomous driving levels and optimize the power consumption of the sensor 20.

[0166] The control method for the information processing device 30A according to this disclosure includes the steps of: acquiring the autonomous driving level of a vehicle V1 whose autonomous driving level can be switched; acquiring sensor data, which is information acquired by a sensor 21 mounted on the vehicle V1; setting a trained model based on the acquired autonomous driving level; and inputting the acquired sensor data into the set trained model to acquire information including objects around the vehicle V1. According to this disclosure, the processing accuracy of the trained model used for information processing can be controlled based on the autonomous driving level, and the power consumption for information processing that acquires information including objects around the vehicle V1 from the sensor data can be optimized.

[0167] The control method for the information processing device 30A according to this disclosure includes the steps of: acquiring the autonomous driving level of a vehicle V whose autonomous driving level can be switched; acquiring sensor data, which is information acquired by a sensor 21 mounted on the vehicle V1; setting the data accuracy of the acquired sensor data based on the acquired autonomous driving level; and acquiring information including objects around the vehicle V1 based on the sensor data with the set data accuracy. According to this disclosure, it is possible to control the data accuracy of sensor data acquired from a sensor 21 mounted on a vehicle V whose autonomous driving level can be switched, and optimize the power consumption for information processing that acquires information including objects around the vehicle V1 from the sensor data.

[0168] The program relating to this disclosure causes a computer to perform the steps of acquiring the autonomous driving level of a vehicle V capable of switching autonomous driving levels, and setting the sensing accuracy of a sensor 20 mounted on the vehicle V based on the acquired autonomous driving level. According to this disclosure, it is possible to control the sensor 20 mounted on a vehicle V capable of switching autonomous driving levels and optimize the power consumption of the sensor 20.

[0169] The program relating to this disclosure causes a computer to perform the following steps: acquire the autonomous driving level of a vehicle V1 that can switch between autonomous driving levels; acquire sensor data, which is information acquired by a sensor 21 mounted on the vehicle V1; set a trained model based on the acquired autonomous driving level; and input the acquired sensor data into the set trained model to acquire information including objects around the vehicle V1. According to this disclosure, the processing accuracy of the trained model used for information processing can be controlled based on the autonomous driving level, and the power consumption for information processing that acquires information including objects around the vehicle V1 from the sensor data can be optimized.

[0170] The program relating to this disclosure causes a computer to perform the following steps: acquire the autonomous driving level of a vehicle V1 capable of switching autonomous driving levels; acquire sensor data, which is information acquired by a sensor 21 mounted on the vehicle V1; set the data accuracy of the acquired sensor data based on the acquired autonomous driving level; and acquire information including objects around the vehicle V1 based on the sensor data with the set data accuracy. According to this disclosure, it is possible to control the data accuracy of the sensor data acquired from a sensor 21 mounted on a vehicle V1 capable of switching autonomous driving levels, and optimize the power consumption for information processing that acquires information including objects around the vehicle V1 from the sensor data.

[0171] Although embodiments of the present invention have been described above, the embodiments are not limited to those described herein. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the embodiments described above.

[0172] The sensor control device, information processing device, control method and program for the sensor control device, and control method and program for the information processing device of this embodiment can be used, for example, when performing autonomous driving.

[0173] 1 Vehicle System 1A Vehicle System U User 10 User Interface 20 Sensor 21 Sensor 30 Sensor Control Device 30A Information Processing Device 40 Power Supply W Wire Harness

Claims

1. A sensor control device comprising: a level setting unit for acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; and an accuracy setting unit for setting the sensing accuracy of sensors mounted on the vehicle based on the acquired autonomous driving level.

2. The sensor control device according to claim 1, wherein the accuracy setting unit is set such that the lower the automatic driving level, the lower the sensing accuracy of the sensor.

3. The sensor control device according to claim 1 or 2, further comprising a condition acquisition unit that acquires location information, which is information about the location where the vehicle is traveling, and battery information, which is information about the remaining battery level of the vehicle, wherein the accuracy setting unit sets the sensing accuracy based on at least one of the location information and the battery information.

4. An information processing device comprising: a level setting unit for acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; a data acquisition unit for acquiring sensor data, which is information acquired by sensors mounted on the vehicle; a model setting unit for setting a learned model based on the acquired autonomous driving level; and an object detection unit for inputting the acquired sensor data into the set learned model and acquiring information including objects around the vehicle.

5. The information processing apparatus according to claim 4, wherein the model setting unit sets the learned model, the lower the processing accuracy of which the autonomous driving level is.

6. The information processing apparatus according to claim 4 or 5, wherein the model setting unit sets the learned model to be used when the autonomous driving level is low based on the learned model to be used when the autonomous driving level is higher than that learned model.

7. An information processing device comprising: a level setting unit for acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; a data acquisition unit for acquiring sensor data which is information acquired by sensors mounted on the vehicle; an accuracy setting unit for setting the data accuracy of the acquired sensor data based on the acquired autonomous driving level; and an object detection unit for acquiring information including objects around the vehicle based on the sensor data with the set data accuracy.

8. The information processing apparatus according to claim 7, wherein the accuracy setting unit is set such that the lower the automatic driving level, the lower the data accuracy.

9. The information processing apparatus according to claim 7 or 8, wherein the object detection unit inputs the sensor data with the set data accuracy into a learned model corresponding to the autonomous driving level to acquire information including objects around the vehicle.

10. A control method for a sensor control device comprising: a step of acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; and a step of setting the sensing accuracy of a sensor mounted on the vehicle based on the acquired autonomous driving level.

11. A control method for an information processing device comprising: a step of acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; a step of acquiring sensor data which is information acquired by sensors mounted on the vehicle; a step of setting a trained model based on the acquired autonomous driving level; and a step of inputting the acquired sensor data into the set trained model and acquiring information including objects around the vehicle.

12. A control method for an information processing device comprising: a step of acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; a step of acquiring sensor data which is information acquired by sensors mounted on the vehicle; a step of setting the data accuracy of the acquired sensor data based on the acquired autonomous driving level; and a step of acquiring information including objects around the vehicle based on the sensor data with the set data accuracy.

13. A program that causes a computer to perform the steps of: acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; and setting the sensing accuracy of sensors mounted on the vehicle based on the acquired autonomous driving level.

14. A program that causes a computer to perform the following steps: acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; acquiring sensor data which is information acquired by sensors mounted on the vehicle; setting a trained model based on the acquired autonomous driving level; and inputting the acquired sensor data into the set trained model to acquire information including objects around the vehicle.

15. A program that causes a computer to perform the following steps: acquiring the autonomous driving level of a vehicle capable of switching autonomous driving levels; acquiring sensor data which is information acquired by sensors mounted on the vehicle; setting the data accuracy of the acquired sensor data based on the acquired autonomous driving level; and acquiring information including objects around the vehicle based on the sensor data with the set data accuracy.

Images