Control device, control method, and computer program

The control device supports vehicle behavior determination in complex environments by limiting speed, improving sensor accuracy, and executing specific models to enhance situational awareness and safety.

JP2026182162APending Publication Date: 2026-11-19TOYOTA JIDOSHA KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2025078132
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2025-05-08
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Conventional in-vehicle devices fail to support vehicle behavior determination in complex environments such as construction sites, accident sites, and urban areas with complicated road traffic, due to insufficient data amount, calculation time, or difficulty in determining vehicle behavior.

Method used

A control device with a support execution unit that performs predetermined support processing, including limiting vehicle speed, enhancing sensor recognition accuracy, executing specific application models, and providing explanations or dialogues to support vehicle behavior determination in specific environments.

Benefits of technology

Enables accurate vehicle behavior determination and situational awareness in complex environments by increasing data acquisition, computation time, and providing information to occupants and traffic participants, enhancing safety and decision-making.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2026182162000001_ABST
    Figure 2026182162000001_ABST
Patent Text Reader

Abstract

To enable support for determining vehicle behavior under specific environmental conditions. [Solution] The control device 5 of the vehicle 100 includes a support execution unit 63 that, when the surrounding environment of the vehicle is a predetermined specific environment, performs predetermined support processing according to the specific environment to support the determination of the vehicle's behavior in parallel with the execution of an inference model that determines the vehicle's behavior.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a control device, a control method, and a computer program.

Background Art

[0002] Patent Document 1 discloses, as a conventional in-vehicle device, an in-vehicle device that notifies a user of the time during which communication with an access point is possible while a vehicle is traveling. With this in-vehicle device, the user can grasp the time during which communication with the access point is possible even while traveling, and can efficiently download data. The in-vehicle device has a function of calculating a required communication time based on the amount of data to be downloaded and notifying whether downloading is possible, and a function of calculating a change in the communicable time corresponding to a change in vehicle speed and notifying the vehicle speed corresponding to a predetermined communicable time.

Prior Art Literature

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In specific environments where complex judgments are expected to be required, such as construction sites, accident sites, and urban areas with complicated road traffic environments, there is a risk that the amount of data and calculation time required to determine vehicle behavior will be insufficient. However, the conventional in-vehicle device described above cannot perform support for determining vehicle behavior by judging whether the environment is such a specific environment.

[0005] The present invention has been made in view of such problems, and an object thereof is to enable implementation of support for determining vehicle behavior under a specific environment.

Means for Solving the Problems

[0006] The gist of this disclosure is as follows:

[0007] (1) A control device comprising a support execution unit that, when the surrounding environment of the vehicle is a predetermined specific environment, performs predetermined support processing according to the specific environment to support the determination of the vehicle's behavior in parallel with the execution of an inference model that determines the vehicle's behavior.

[0008] (2) The support execution unit shall The control device described in (1) above, which performs, as a prescribed support process, a process to limit the vehicle speed of the vehicle to a predetermined speed or less, a process to improve the recognition accuracy of the surrounding sensors that recognize the surrounding environment of the vehicle compared to when the surrounding environment of the vehicle is not a specific environment, a process to execute a model for a specific application according to the specific environment, a process to provide at least one of the following to the occupants of the vehicle and at least one of the traffic participants around the vehicle: an explanation of the specific environment and an explanation of the content of the vehicle behavior according to the specific environment, and a process to engage in dialogue with the occupants of the vehicle in order to determine the vehicle behavior.

[0009] (3) The control device described in (1) or (2) above, wherein the specific environment is an environment in which the inference model has difficulty or is impossible to determine the vehicle behavior of the vehicle, or in which there is insufficient computation time to determine the vehicle behavior.

[0010] (4) A control method in which, when the surrounding environment of the vehicle is a predetermined specific environment, the control device performs predetermined support processing according to the specific environment to support the determination of the vehicle's behavior in parallel with the execution of an inference model that determines the vehicle's behavior.

[0011] (5) A computer program that causes a computer to perform predetermined support processing according to a specific environment in parallel with the execution of an inference model that determines the vehicle behavior of the vehicle, when the surrounding environment of the vehicle is a predetermined specific environment. [Effects of the Invention]

[0012] According to these aspects of the present invention, it is possible to provide support for determining vehicle behavior under specific environmental conditions. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of a vehicle according to one embodiment of the present invention. [Figure 2] This flowchart illustrates the support process for determining vehicle behavior using one embodiment of the present invention. [Modes for carrying out the invention]

[0014] Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, similar components will be given the same reference numerals.

[0015] Figure 1 is a schematic diagram of a vehicle 100 according to one embodiment of the present invention.

[0016] Vehicle 100 includes a surrounding sensor 1, a vehicle sensor 2, an HMI (Human Machine Interface) 3, an actuator 4, and a control device 5. The surrounding sensor 1, vehicle sensor 2, HMI 3, actuator 4, and control device 5 are each connected to communicate via an in-vehicle network 9 that conforms to standards such as a controller area network.

[0017] The peripheral sensor 1 is a sensor for generating peripheral data to recognize the surrounding environment of the vehicle 100. In this embodiment, the vehicle 100 is equipped with one or more external cameras 11 as the peripheral sensor 1 for capturing images of the area around the vehicle 100. The external cameras 11 capture images of the area around the vehicle 100 at a predetermined frame rate (for example, 10 Hz to 40 Hz) and generate peripheral images of the area around the vehicle 100. Each time the external cameras 11 generate a peripheral image, they transmit the generated peripheral image to the control device 5 as peripheral data.

[0018] In addition to the external camera 11, the vehicle 100 may also be equipped with one or more distance measuring sensors as surrounding sensors 1 to measure the distance to targets and features in its vicinity. Examples of distance measuring sensors include LiDAR (Light Detection and Ranging), which measures distance based on the reflected light emitted by radar, and millimeter-wave radar sensors, which measure distance based on the reflected waves emitted by radio waves.

[0019] Vehicle sensor 2 is a sensor for acquiring vehicle data representing the state of vehicle 100. Vehicle 100 according to this embodiment includes, as vehicle sensor 2, a speed sensor 21 that acquires speed data indicating the vehicle's speed, a positioning sensor 22 that acquires current position data indicating the vehicle's current position such as latitude and longitude, a steering sensor 23 that acquires data related to steering operations such as steering grip, steering torque, and steering angle (steering operation amount), an accelerator sensor 24 that acquires data related to accelerator operations such as accelerator pedal depression amount (accelerator operation amount), and a brake sensor 25 that acquires data related to brake operations such as detection of brake pedal input and depression amount (brake operation amount). However, vehicle sensor 2 is not limited to these sensors. The data acquired by each sensor 21 to 25 is transmitted to the control device 5 as vehicle data.

[0020] HMI3 is a user interface for exchanging information between the vehicle 100 and its occupants. HMI3 includes output devices 31 for notifying the vehicle occupants through their bodily senses (e.g., sight, hearing, and touch), and input devices 32 for the vehicle occupants to perform input and response operations. Output devices 31 include, for example, displays (e.g., meter displays, center displays, head-up displays, etc.) and speakers. Input devices 32 include, for example, touch panels and microphones.

[0021] HMI 3 notifies the vehicle occupant of information corresponding to the output signal received from the control device 5 via the output device 31, and transmits data input by the vehicle occupant via the input device 32 to the control device 5.

[0022] The HMI 3 may be pre-installed in the vehicle 100, or may be a terminal such as a smartphone owned by a vehicle occupant (driver and fellow passenger). In the latter case, for example, information may be exchanged by performing short-range wireless communication between the vehicle 100 and the vehicle occupant's terminal, or communication may be performed between the vehicle occupant's terminal and an external server (not shown) to exchange information indirectly via the server.

[0023] The actuator 4 is a device used for travel control of the vehicle 100. The vehicle 100 according to the present embodiment includes, as the actuator 4, an acceleration actuator 41 (for example, at least one of an engine and a motor) that performs acceleration control of the vehicle 100, a brake actuator 42 (for example, a hydraulic actuator) that performs brake control of the vehicle 100, and a steering actuator 43 (for example, a steering motor) that performs steering control of the vehicle 100.

[0024] The control device 5 is an ECU (Electronic Control Unit) including a communication unit 51, a storage unit 52, and a processing unit 53.

[0025] The communication unit 51 includes an interface circuit for connecting the control device 5 to the in-vehicle network 9. The communication unit 51 supplies various types of data received from the outside to the processing unit 53. The communication unit 51 also outputs various signals output from the processing unit 53 to the outside.

[0026] The storage unit 52 has a storage medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a semiconductor memory, and stores various data such as various computer programs used for processing in the processing unit 53 and high-precision map data.

[0027] The processing unit 53 has one or more CPUs (Central Processing Units) and their peripheral circuits, and executes various computer programs stored in the memory unit 52. The processing unit 53 is, for example, a processor. The processing unit 53 may further have other arithmetic circuits such as a logical operation unit, a numerical operation unit, or a graphics processing unit. By executing processing according to the computer program, the processing unit 53 functions as a vehicle behavior control unit 61, a determination unit 62, and a support execution unit 63, and operates as a functional unit (module) that realizes a predetermined function. In the following description, when describing the processing with each functional unit 61 to 63 as the subject, it means that the processing unit 53 is executing the program that realizes each functional unit 61 to 63.

[0028] The following describes the specific processing performed by the control device 5. Specifically, it describes the contents of each functional unit 61 to 63, which are realized by the processing unit 53 executing processing according to the computer program.

[0029] The vehicle behavior control unit 61 inputs necessary parameters such as surrounding data, vehicle data, and high-precision map data into an inference model for determining vehicle behavior, calculates target amounts for accelerator operation, brake operation, and steering operation, and controls the acceleration actuator 41, brake actuator 42, and steering actuator 43 so that each operation amount becomes the target amount, thereby controlling the vehicle behavior.

[0030] The inference model consists of multiple models, such as a detection model that detects other vehicles, people, traffic lights, signs, lanes, and other landmarks and features around vehicle 100, and a prediction model that predicts the movement of other vehicles, people, etc. As the detection model, for example, a deep neural network (DNN) with a convolutional neural network (CNN) type architecture such as Single Shot MultiBox Detector, Faster R-CNN, or You Only Look Once (YOLO) can be used. As the prediction model, for example, a neural network with a recurrent structure such as Recurrent Neural Network (RNN), Long Short Term Memory (LSTM), or Gated Recurrent Unit (GRU) can be used.

[0031] The determination unit 62 determines whether the surrounding environment is a specific environment. The surrounding environment can be determined, for example, based on targets and features detected by the detection model. In this embodiment, a first list of normal environments in which the inference model can determine vehicle behavior and a second list of predetermined specific environments different from the normal environment are pre-stored in the storage unit 52. The determination unit 62 determines that the surrounding environment is a specific environment if the surrounding environment determined based on the detection results of the detection model is registered in the second list.

[0032] A specific environment is one in which determining vehicle behavior is difficult or impossible with the inference model used under normal conditions, or in which the computation time (time required for inference) for determining vehicle behavior is insufficient. In this embodiment, construction sites, accident sites, and urban areas with complex road traffic environments are registered in the second list.

[0033] If the surrounding environment is a specific environment, the support execution unit 63 performs predetermined support processing according to the specific environment in parallel with the execution of the inference model used in the normal environment, in order to assist in determining the vehicle behavior.

[0034] For example, the support execution unit 63 can perform a process to limit the vehicle speed of the vehicle 100 to a predetermined speed or less, as well as a process to improve the recognition accuracy of the surrounding sensor 1 compared to when the surrounding environment is not a specific environment. An example of a process to improve the recognition accuracy of the surrounding sensor 1 is a process to increase the resolution of the surrounding image generated by the external camera 11.

[0035] By limiting vehicle speed in this way, the amount of data acquired per unit distance traveled can be increased, allowing for more accurate recognition of detailed road conditions in the surrounding area, as well as the surrounding environment such as the positions of traffic controllers and workers, and the placement of cones, in the case of construction sites or accident sites. Furthermore, limiting vehicle speed also increases the time available for inference (determining vehicle behavior) per unit distance traveled, thus accommodating the increased amount of data acquired and the increased data volume due to improved recognition accuracy of the surrounding sensor 1. By improving the recognition accuracy of the surrounding sensor 1, the surrounding environment can be recognized even more accurately, enabling appropriate situational judgments according to specific environments and supporting the determination of vehicle behavior under specific conditions.

[0036] Furthermore, for example, the support execution unit 63 can perform a process to limit the vehicle speed of the vehicle 100 to a predetermined speed or less, as well as a separate process to execute a specific application model tailored to a specific environment in order to optimize the vehicle behavior of the vehicle 100 in that specific environment.

[0037] A model designed for a specific application could, for example, identify the gestures of traffic controllers or temporary traffic signals in a construction site environment. By running such a model separately, it becomes possible to interpret the meaning of traffic controllers' gestures and temporary traffic signals more accurately.

[0038] Furthermore, a model designed for a specific application can, for example, identify elements specific to an accident site, such as falling objects, flames, and smoke, if the surrounding environment is an accident site. By running such a model separately, it becomes possible to appropriately avoid elements specific to an accident site, such as falling objects, flames, and smoke.

[0039] Furthermore, a model designed for specific applications can, for example, in urban areas with complex road traffic environments, recognize signals in intersecting directions, pedestrian signals, and signs indicating "time-delayed" signals, and predict the timing of signal changes. It can also be used as a model to recognize infants, shopping carts, and other objects.

[0040] In this way, by running a separate, application-specific model tailored to a specific environment in parallel with the inference model, the application-specific model can perform appropriate situational judgments according to the specific environment, thereby supporting the determination of vehicle behavior under specific conditions.

[0041] Furthermore, for example, the support execution unit 63 can perform a process as a predetermined support process that limits the vehicle speed of the vehicle 100 to a predetermined speed or less, and also performs a process that provides at least one of the following to at least one of the occupants of the vehicle and the traffic participants around the vehicle: an explanation of a specific environment and an explanation of the vehicle's behavior in accordance with that specific environment.

[0042] Explanations to the occupants of the vehicle can be provided via HMI3. Explanations to surrounding traffic can be provided using speakers, displays, etc., if the vehicle is equipped with them, or, if communication with surrounding traffic participants' terminals is possible, by communicating with those terminals and providing explanations through those terminals.

[0043] In this way, by using the surplus computational resources resulting from limiting vehicle speed to provide information and explain specific environments and vehicle behavior, it is possible to give a sense of security to the occupants of the vehicle and other traffic participants around the vehicle when driving in specific environments.

[0044] Furthermore, the support execution unit 63 can, for example, perform a process to limit the vehicle speed of the vehicle 100 to a predetermined speed or less, as well as a process to communicate with the occupants of the vehicle in order to determine the vehicle's behavior under specific conditions, as a predetermined support process.

[0045] In this way, by using the surplus computing resources resulting from limiting the vehicle speed for information provision processing and interacting with the vehicle's occupants, it becomes possible to have the vehicle's occupants make decisions about the vehicle's behavior under specific conditions.

[0046] Figure 2 is a flowchart illustrating the support process for determining vehicle behavior according to this embodiment, which is performed by the support execution unit 63 and, consequently, the control device 5. The control device 5 repeatedly executes this routine at predetermined calculation cycles.

[0047] In step S1, the control device 5 recognizes the surrounding environment. In this embodiment, the control device 5 recognizes the surrounding environment based on the targets and features detected by the detection model.

[0048] In step S2, the control device 5 determines whether the surrounding environment is a specific environment. In this embodiment, if the surrounding environment determined based on the detection results of the detection model is registered in the second list, the control device 5 determines that the surrounding environment is a specific environment and proceeds to the process in step S3. On the other hand, if the surrounding environment is not a specific environment, the control device 5 terminates the current process without performing any support processing.

[0049] In step S3, the control device 5 performs a predetermined support process according to the specific environment to assist in determining the vehicle behavior. As a support process, the control device 5 performs a process to limit the vehicle speed to a predetermined speed or less, along with at least one of the following four processes: 1. A process to improve the recognition accuracy of the surrounding sensor 1 compared to when the surrounding environment is not the specific environment; 2. A process to execute a specific application model according to the specific environment to optimize the vehicle behavior of the vehicle 100 in the specific environment, in parallel with the execution of an inference model for a normal environment; 3. A process to provide at least one of the following to the occupants of the vehicle and at least one of the traffic participants around the vehicle: an explanation of the specific environment and an explanation of the content of the vehicle behavior of the vehicle according to the specific environment; and 4. A process to engage in dialogue with the occupants of the vehicle in order to determine the vehicle behavior under the specific environment.

[0050] For example, if the surrounding environment is a construction site, which is an example of a specific environment, the control device 5 will limit the vehicle speed to a predetermined speed or less, and in parallel with running the inference model used in normal environments, it will run a special-purpose model that identifies the gestures of traffic controllers and temporary traffic signals, and further, based on the identification results, will provide the vehicle occupants with an explanation of the vehicle's behavior, such as "Proceed according to the traffic controllers."

[0051] Furthermore, if the surrounding environment is, for example, an accident scene which is an example of a specific environment, the control device 5 will limit the vehicle speed to below a predetermined speed and increase the resolution of the surrounding image generated by the external camera 11 to accurately grasp the detailed situation of the accident scene. In addition, in parallel with executing the inference model used in normal environments, the control device 5 will execute a specific-purpose model that identifies things unique to accident scenes, such as falling objects, flames, and smoke, to improve the accuracy of obstacle avoidance, and will engage in dialogue with the vehicle occupants to determine the vehicle's behavior, for example, by asking "Is it safe to proceed?"

[0052] Furthermore, for example, if the surrounding environment is a complex urban area with a road traffic environment, which is an example of a specific environment, the control device 5 will limit the vehicle speed to a predetermined speed or less, and in parallel with the execution of the inference model used in normal environments, it will execute multiple application-specific models, including a model that recognizes signals in the direction of intersection, pedestrian signals, and signs (signs) that say "time-delayed" for time-delayed signals, and a model that predicts the timing of signal changes based on the recognition results of the said model, so that it can make appropriate situational judgments even in a complex road traffic environment.

[0053] The control device 5 according to this embodiment, as described above, includes a support execution unit 63 that, when the surrounding environment of the vehicle is a predetermined specific environment, performs predetermined support processing according to the specific environment to support the determination of the vehicle's behavior in parallel with the execution of an inference model that determines the vehicle's behavior.

[0054] This makes it possible to provide support for determining vehicle behavior in specific environments where the inference model has difficulty or is impossible to determine the vehicle behavior, or where there is insufficient computation time to determine the vehicle behavior.

[0055] Specifically, the support execution unit 63 performs at least one of the following processes as a predetermined support process: a process to limit the vehicle speed of its own vehicle to a predetermined speed or less; a process to improve the recognition accuracy of the surrounding sensor 1 that recognizes the surrounding environment of the vehicle compared to when the surrounding environment of the vehicle is not a specific environment; a process to execute a model for a specific application according to the specific environment; a process to provide at least one of the following to the occupants of the vehicle and at least one of the traffic participants around the vehicle: an explanation of the specific environment and an explanation of the content of the vehicle behavior according to the specific environment; and a process to engage in dialogue with the occupants of the vehicle in order to determine the vehicle behavior.

[0056] By limiting vehicle speed in this way, the amount of data acquired per unit distance traveled can be increased, allowing for a more accurate recognition of detailed road conditions in the surrounding area, as well as the surrounding environment such as the positions of traffic controllers and workers, and the placement of cones, in the case of construction sites or accident sites. Furthermore, limiting vehicle speed also increases the time available for inference (determining vehicle behavior) per unit distance traveled, thus accommodating the increased amount of data acquired and the increased data volume due to improved recognition accuracy of the surrounding sensor 1. By improving the recognition accuracy of the surrounding sensor 1, the surrounding environment can be recognized even more accurately, enabling appropriate situational judgments to be made according to the specific environment when determining vehicle behavior under specific conditions.

[0057] Furthermore, by running a separate, application-specific model tailored to a specific environment in parallel with the inference model, the application-specific model can perform appropriate situational judgments according to that specific environment, thereby supporting the determination of vehicle behavior under specific conditions.

[0058] Furthermore, by using the surplus computing resources resulting from limiting vehicle speed to provide information and explain specific environments and vehicle behavior, a sense of security can be given to the occupants of the vehicle and other traffic participants around it. Additionally, by engaging in dialogue with the occupants of the vehicle, decisions regarding vehicle behavior under specific conditions can be made by the occupants themselves.

[0059] Although embodiments of the present invention have been described above, these embodiments only represent a part of the application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

[0060] For example, in the above embodiment, the computer program executed in the control device 5 may be provided in the form of a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium, or it may be provided as a computer program product. [Explanation of Symbols]

[0061] 5 Control device 100 vehicles

Claims

1. When the surrounding environment of the vehicle is a predetermined specific environment, the system includes a support execution unit that performs predetermined support processing according to the predetermined specific environment to support the determination of the vehicle's behavior, in parallel with the execution of an inference model that determines the vehicle's behavior. Control device.

2. The aforementioned support execution unit, As the predetermined support processing, the vehicle performs at least one of the following processes: a process to limit the vehicle speed to a predetermined speed or less; a process to improve the recognition accuracy of the surrounding sensors that recognize the surrounding environment of the vehicle compared to a case where the surrounding environment of the vehicle is not the specified environment; a process to execute a model for a specific application corresponding to the specified environment; a process to provide at least one of the following to the occupants of the vehicle and at least one of the traffic participants around the vehicle: an explanation of the specified environment and an explanation of the content of the vehicle behavior corresponding to the specified environment; and a process to engage in dialogue with the occupants of the vehicle in order to determine the vehicle behavior. The control device according to claim 1.

3. The aforementioned specific environment is one in which, when determining the vehicle behavior of the vehicle itself, the inference model finds it difficult or impossible to determine the vehicle behavior, or there is insufficient computation time to determine the vehicle behavior. The control device according to claim 1 or claim 2.

4. When the surrounding environment of the vehicle is a predetermined specific environment, the control device performs predetermined support processing corresponding to the specific environment to assist in determining the vehicle's behavior, in parallel with the execution of the inference model that determines the vehicle's behavior. Control method.

5. A computer program that causes a computer to perform predetermined support processing corresponding to a predetermined specific environment in parallel with the execution of an inference model that determines the vehicle's behavior, when the surrounding environment of the vehicle is a predetermined specific environment.

Citation Information

Patent Citations

  • Onboard apparatus

    JP2004140526A