Vehicle control device, vehicle control method, and computer program for vehicle control
The vehicle control system uses neural networks to detect and assess bicycle wobbling likelihood, adjusting lateral offset distances to ensure safety by anticipating potential wobbling, addressing the inadequacies of existing systems in setting distances for bicycles.
Patent Information
- Application Number
- JP2026199313
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-09-02
- Publication Date
- 2026-11-19
AI Technical Summary
Existing vehicle control systems fail to appropriately set a lateral offset distance for bicycles that may suddenly weave into their lane, as they rely solely on lateral position variation, neglecting the potential for sudden wobbling behavior.
A vehicle control system utilizing a camera and ECU with neural networks to detect bicycles and assess their likelihood of wobbling, adjusting the lateral offset distance based on this assessment to maintain a safe distance, even when the bicycle is not currently weaving.
The system effectively sets an appropriate lateral offset distance for bicycles, ensuring safety by anticipating potential wobbling behavior and maintaining a safe distance, even when the bicycle is not actively weaving.
Smart Images

Figure 2026182959000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a vehicle control device, a vehicle control method, and a computer program for vehicle control. Background Art
[0002] Technologies for controlling the driving of the host vehicle based on the recognition result of a two-wheeled vehicle traveling ahead of the host vehicle have been studied (see Patent Document 1).
[0003] The vehicle control device described in Patent Document 1 determines that there is a high probability that the two-wheeled vehicle is weaving when the maximum value of the lateral position variation amount of the two-wheeled vehicle over a predetermined distance is equal to or greater than a threshold value. The vehicle control device then sets a large offset distance from the dedicated two-wheeled vehicle lane in which the two-wheeled vehicle is traveling, so as to move the host vehicle away from the two-wheeled vehicle. Prior Art Documents Patent Documents
[0004] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2019-185113 Summary of Invention Problem to be Solved by the Invention
[0005] In the above technology, since the offset distance is adjusted based on the lateral position variation amount of the two-wheeled vehicle, the offset distance is not set to a large value for a two-wheeled vehicle that has not been particularly weaving up to that point. However, depending on the rider of a two-wheeled vehicle such as a bicycle, the two-wheeled vehicle may suddenly weave toward the lane side in which the vehicle is traveling.
[0006] Accordingly, an object of the present invention is to provide a vehicle control device that can appropriately set a lateral offset distance for a bicycle traveling around the vehicle. Means for Solving the Problem
[0007] According to one embodiment, a vehicle control device is provided. This vehicle control device includes a camera that generates images representing the surroundings of the vehicle, and an ECU that controls the vehicle's movement, comprising at least one functional module composed of a neural network. The ECU determines whether or not the bicycle is likely to wobble based on the images acquired from the camera, and further sets a predetermined offset distance based on whether or not the bicycle is likely to wobble in order to output a control signal that causes the vehicle to move at a predetermined offset distance or more away from the bicycle.
[0008] In one embodiment, the ECU is configured to determine whether or not the bicycle is likely to exhibit wobbly behavior based on the object region in which the bicycle is detected in the image.
[0009] In one embodiment, the ECU is configured to set a predetermined offset distance for the bicycle in a direction perpendicular to the direction of extension of the road on which the vehicle is traveling to a value greater than the offset distance at which it is determined that the bicycle is unlikely to exhibit wobbling behavior.
[0010] In one embodiment, the ECU detects an object region representing a bicycle in an image by inputting the image into a first classifier that has been pre-trained to detect a bicycle from the image, and determines whether or not the bicycle is likely to exhibit a wobbling behavior by inputting the object region into a second classifier that has been pre-trained to determine the likelihood of the bicycle exhibiting a wobbling behavior.
[0011] In one embodiment, the ECU inputs information representing the terrain of the road on which the vehicle is traveling, along with the object region, to a second classifier to determine whether or not the bicycle is likely to exhibit wobbly behavior.
[0012] In one embodiment, the ECU inputs an image to a third classifier that has been pre-trained to detect a bicycle represented in the image and determine whether or not the bicycle is likely to exhibit a wobbling behavior, thereby detecting the object region in the image in which the bicycle is represented and determining whether or not the bicycle is likely to exhibit a wobbling behavior.
[0013] In one embodiment, the ECU adjusts the offset distance based on the terrain of the road the vehicle is traveling on.
[0014] In one embodiment, the ECU corrects the offset distance to be longer by a predetermined distance if the road the vehicle is traveling on is uphill or curved.
[0015] Another embodiment provides a vehicle control method. This vehicle control method includes at least one functional module composed of a neural network, wherein the ECU that controls the vehicle's movement determines whether or not the bicycle is likely to wobble based on an image acquired from a camera that generates an image representing the area around the vehicle, and sets a predetermined offset distance based on whether or not the bicycle is likely to wobble in order to output a control signal that causes the vehicle to move at a predetermined offset distance or more away from the bicycle.
[0016] In yet another embodiment, a computer program for vehicle control is provided. This computer program for vehicle control includes instructions to cause an ECU that controls the vehicle's movement to perform the following actions: determine whether or not the bicycle is likely to wobble based on an image acquired from a camera that generates an image representing the area around the vehicle, and set a predetermined offset distance based on whether or not the bicycle is likely to wobble, in order to output a control signal that causes the vehicle to move at a distance greater than or equal to a predetermined offset distance from the bicycle. [Effects of the Invention]
[0017] The vehicle control device according to the present disclosure achieves an effect that the lateral offset distance can be appropriately set for a bicycle traveling around the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] [Figure 1] 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. [Figure 2] 1 is a hardware configuration diagram of an electronic control device that is an example of a vehicle control device. [Figure 3] 1 is a functional block diagram of a processor of an electronic control device related to vehicle control processing. [Figure 4] (a) and (b) are diagrams showing an example of setting of an offset distance. [Figure 5] 1 is an operation flowchart of vehicle control processing. DETAILED DESCRIPTION OF EMBODIMENTS
[0019] Hereinafter, a vehicle control device, a vehicle control method executed in the vehicle control device, and a computer program for vehicle control will be described with reference to the drawings. This vehicle control device detects a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determines whether or not the bicycle represented in the image may exhibit a wobbling behavior. Then, this vehicle control device sets the offset distance to the bicycle in the direction orthogonal to the extending direction of the road on which the vehicle is traveling (hereinafter referred to as the lateral direction) when it is determined that the bicycle may exhibit a wobbling behavior, to a value larger than the offset distance when it is determined that the bicycle has no possibility of exhibiting a wobbling behavior. Thereby, this vehicle control device appropriately sets the lateral offset distance even for a bicycle that does not actually exhibit a wobbling behavior.
[0020] In the present embodiment, the bicycle is not limited to a two-wheeled vehicle, and may be a bicycle with training wheels for children, or a tricycle for seniors or children. Further, the bicycle may be a bicycle with auxiliary power obtained from a motor or the like.
[0021] Figure 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. Figure 2 is a hardware configuration diagram of an electronic control device which is one embodiment of the vehicle control device. In the present embodiment, a vehicle control system 1 that is mounted on a vehicle 10 and controls the vehicle 10 includes a camera 2, a GPS receiver 3, a storage device 4, and an electronic control unit (ECU) 5 which is an example of the vehicle control device. The camera 2, the GPS receiver 3, the storage device 4, and the ECU 5 are communicably connected via an in-vehicle network conforming to a standard such as a Controller Area Network. The vehicle 10 is an example of the host vehicle. In addition, the vehicle control system 1 may further include a distance measurement sensor (not shown), such as LiDAR or radar, that measures the distance from the vehicle 10 to an object present around the vehicle 10.
[0022] The camera 2 generates an image representing the surroundings of the vehicle 10. The camera 2 is installed in the cabin of the vehicle 10 so as to face a predetermined direction, for example, the front of the vehicle 10. The camera 2 captures an area around the vehicle 10, for example, a front area of the vehicle 10 at every predetermined imaging cycle (for example, 1 / 30 second to 1 / 10 second), and generates an image representing the front area. The image obtained by the camera 2 may be a color image or a grayscale image. The vehicle 10 may be provided with a plurality of cameras having different imaging directions or focal lengths.
[0023] Each time the camera 2 generates an image, the camera 2 outputs the generated image to the ECU 5 via the in-vehicle network.
[0024] The GPS receiver 3 receives GPS signals from GPS satellites at predetermined intervals and determines the vehicle's position based on the received GPS signals. At predetermined intervals, the GPS receiver 3 outputs positioning information representing the vehicle's position based on the GPS signals to the ECU 5 via the in-vehicle network. The vehicle 10 may also have a receiver compliant with a satellite positioning system other than the GPS receiver 3. In this case, that receiver is responsible for determining the vehicle's position.
[0025] The storage device 4 is an example of a storage unit and includes, for example, a hard disk drive, a non-volatile semiconductor memory, or an optical recording medium and its access device. The storage device 4 stores map information. The map information includes, for example, information representing road markings such as lane markings or stop lines, information representing road signs, and information representing features around the road for individual road sections included in a predetermined area represented in the map information. Furthermore, the map information may also include information representing the curvature and gradient of individual road sections.
[0026] Furthermore, the storage device 4 may have a processor for performing tasks such as updating map information and processing requests for reading map information from the ECU 5. For example, each time the vehicle 10 moves a predetermined distance, the storage device 4 sends a request to acquire map information, along with the vehicle 10's current location, to the map server via a wireless communication terminal (not shown) mounted on the vehicle 10. The storage device 4 then receives map information from the map server via the wireless communication terminal for a predetermined area around the vehicle 10's current location. When the storage device 4 receives a request to read map information from the ECU 5, it extracts a range from the stored map information that includes the vehicle 10's current location and is relatively smaller than the predetermined area, and outputs it to the ECU 5 via the in-vehicle network.
[0027] The ECU 5 controls the driving of the vehicle 10 according to a predetermined level of autonomous driving. The predetermined level of autonomous driving can be any level of autonomous driving defined by the Society of Automotive Engineers (SAE), level 1 or higher.
[0028] As shown in Figure 2, the ECU 5 includes a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may each be configured as separate circuits, or they may be integrated as a single integrated circuit.
[0029] The communication interface 21 has an interface circuit for connecting the ECU 5 to the in-vehicle network. Whenever the communication interface 21 receives an image from the camera 2, it passes the received image to the processor 23. Also, whenever the communication interface 21 receives positioning information from the GPS receiver 3, it passes that positioning information to the processor 23. Furthermore, the communication interface 21 passes map information read from the storage device 4 to the processor 23.
[0030] Memory 22 is another example of a storage unit and includes, for example, volatile semiconductor memory and non-volatile semiconductor memory. Memory 22 stores various data used in vehicle control processing executed by the processor 23 of the ECU 5. For example, memory 22 stores parameters of camera 2 such as focal length, field of view, shooting direction and mounting position, map information, and parameter sets for identifying various classifiers used for detecting bicycles traveling around vehicle 10 and determining wobbling. Furthermore, memory 22 temporarily stores images generated by camera 2 and the positioning results of the GPS receiver 3. In addition, memory 22 temporarily stores various data generated during vehicle control processing.
[0031] The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may further have other arithmetic circuits such as a logic unit, a numerical unit, or a graphics processing unit. The processor 23 then performs vehicle control processing for the vehicle 10.
[0032] Figure 3 is a functional block diagram of the processor 23 related to vehicle control processing. The processor 23 includes a determination unit 31, an offset setting unit 32, and a vehicle control unit 33. Each of these parts of the processor 23 is, for example, a functional module realized by a computer program running on the processor 23. Alternatively, each of these parts of the processor 23 may be a dedicated arithmetic circuit provided on the processor 23.
[0033] The determination unit 31 detects bicycles traveling around the vehicle 10 based on the image generated by the camera 2 that represents the area around the vehicle 10. The determination unit 31 then determines whether or not the detected bicycles shown in the image are likely to exhibit swaying behavior.
[0034] In this embodiment, the determination unit 31 detects bicycles traveling around the vehicle 10 by inputting the image to the classifier each time the ECU 5 acquires an image from the camera 2. The classifier for bicycle detection is an example of a first classifier. The classifier for bicycle detection can be, for example, a deep neural network (DNN) with a convolutional neural network (CNN) type architecture, such as a Single Shot MultiBox Detector (SSD) or Faster R-CNN. Alternatively, the classifier for bicycle detection may be a DNN with an attention mechanism such as a Vision Transformer. Or, the classifier for bicycle detection may be a classifier based on other machine learning algorithms other than DNNs, such as a support vector machine or an adaBoost classifier. Such a classifier is pre-trained according to a predetermined learning algorithm, such as backpropagation, using a large number of training images that include images representing bicycles, in order to detect bicycles from images. The classifier outputs information that identifies the object region representing the detected bicycle on the input image. The object region is, for example, the bounding rectangle of the bicycle represented on the image.
[0035] When the determination unit 31 detects a bicycle from the image, it inputs the object region representing the bicycle to a classifier for determining whether or not the bicycle may exhibit a wobbling behavior (hereinafter sometimes simply referred to as wobbling). The determination unit 31 may also upsample or downsample the object region representing the bicycle so that it becomes a predetermined size before inputting it to the classifier for wobbling determination. This simplifies the configuration of the classifier for wobbling determination. When the classifier for wobbling determination receives the object region representing the bicycle as input, it outputs a confidence score indicating that the bicycle represented in that object region exhibits a wobbling behavior. If the confidence score is higher than a predetermined threshold (for example, 0.7 to 0.9), the determination unit 31 determines that there is a possibility that the bicycle may exhibit a wobbling behavior, and if the confidence score is below the threshold, it determines that there is no possibility that the bicycle may exhibit a wobbling behavior.
[0036] A classifier for wavering detection is an example of a second classifier, and can be a CNN-type DNN having, in order from the input side, one or more convolutional layers, one or more fully connected layers, and an output layer that performs a softmax or sigmoid operation. Alternatively, the classifier for wavering detection may be a DNN with an attention mechanism, or a classifier based on other machine learning algorithms.
[0037] The classifier for detecting wobbling is pre-trained according to a predetermined learning algorithm using a large number of training images representing various bicycles, so as to output a confidence level that the bicycle will exhibit wobbling behavior based on the characteristics of the bicycle or the bicycle rider represented in the object domain. For example, if the bicycle rider is elderly or a child, the rider may not be able to operate the bicycle without wobbling. Therefore, the classifier for detecting wobbling is pre-trained to have a high confidence level that bicycles will exhibit wobbling behavior when the rider is elderly or a child. Also, if the bicycle is a child's bicycle, it may exhibit wobbling behavior because the rider is a child. Furthermore, bicycles with child seats attached may exhibit wobbling behavior due to the difficulty of handling the handlebars. Therefore, the classifier for detecting wobbling is pre-trained to have a high confidence level that bicycles for children or bicycles with child seats attached will exhibit wobbling behavior. On the other hand, bicycles that are ridden by adults who are not elderly and do not have child seats are less likely to exhibit wobbling behavior. Therefore, the classifier used for detecting wobbling is pre-trained to have a low confidence level in indicating wobbling behavior for such bicycles.
[0038] If multiple bicycles are detected from a single image, the determination unit 31 inputs the object region representing each detected bicycle into the wobble detection classifier to determine whether each detected bicycle is likely to exhibit wobble behavior. Alternatively, the wobble detection classifier may be configured to accept multiple object regions as input simultaneously. For example, the wobble detection classifier may be configured to accept different object regions as input for each channel. In this case, when multiple object regions representing each detected bicycle are input to the wobble detection classifier simultaneously, the wobble detection classifier outputs a confidence level indicating that at least one of the bicycles exhibits wobble behavior. In this case, the wobble detection classifier may be pre-trained to more easily determine that at least one of two or more bicycles riding together as a parent and child is likely to exhibit wobble behavior.
[0039] In a modified version, a single classifier may be configured to detect a bicycle from an image and calculate the confidence level that the detected bicycle exhibits swaying behavior. This modified classifier is an example of a third classifier and can be a CNN type, a DNN with an attention mechanism, or a classifier based on other machine learning algorithms. In this case, the classifier outputs the object region representing the detected bicycle and the confidence level that indicates swaying behavior for that object region. In this case as well, the determination unit 31 only needs to determine whether or not the bicycle represented in the object region is likely to exhibit swaying behavior by comparing the confidence level for that object region with a threshold.
[0040] The determination unit 31 notifies the offset setting unit 32 of the determination result regarding the detected bicycle, which is whether or not the bicycle is likely to exhibit wobbly behavior, and the position of the object region representing the bicycle on the image.
[0041] The offset setting unit 32 sets the lateral offset distance relative to the detected bicycle. In this embodiment, the offset setting unit 32 sets the offset distance when it is determined that the detected bicycle may exhibit wobbly behavior to a larger value than the offset distance when it is determined that the detected bicycle is not likely to exhibit wobbly behavior.
[0042] For example, if a dedicated bicycle lane exists on the road that vehicle 10 is traveling on, it is expected that bicycles will, in principle, travel within the dedicated bicycle lane. In this case, the offset setting unit 32 sets a limit position for approaching bicycles at a position offset distance from the bicycle's position towards the vehicle's own lane. The offset setting unit 32 refers to the latest position of vehicle 10, determined by the GPS receiver 3, and map information to determine whether or not a dedicated bicycle lane exists on the road that vehicle 10 is traveling on. Alternatively, the offset setting unit 32 may determine whether or not a dedicated bicycle lane exists on the road that vehicle 10 is traveling on by inputting the image generated by the camera 2 into a classifier that has been pre-trained to detect dedicated bicycle lanes. In this case, the offset setting unit 32 can use a semantic segmentation DNN, such as a Fully Convolutional Network (FCN) or U-Net, as the classifier.
[0043] Furthermore, the offset setting unit 32 estimates the position of the detected bicycle. Here, each pixel on the image generated by camera 2 corresponds one-to-one with the orientation from camera 2. Also, the lower end of the object region representing the bicycle is estimated to represent the position where the bicycle is in contact with the road surface. Therefore, the offset setting unit 32 can estimate the distance and direction to the bicycle relative to the position of camera 2 at the time of image generation, based on the position of the lower end of the object region representing the bicycle on the image and the parameters of camera 2, such as the installation height, focal length, and shooting direction of camera 2. If the vehicle 10 is equipped with a distance measuring sensor, the offset setting unit 32 may estimate the distance measured by the distance measuring sensor at the orientation corresponding to the object region representing the bicycle as the distance to the bicycle. Furthermore, the offset setting unit 32 can estimate the position of the bicycle in the world coordinate system based on the distance and direction to the bicycle relative to the position of camera 2 and the position and direction of travel of vehicle 10 at the time of image generation. Therefore, in order to detect the accurate position and direction of travel of vehicle 10, the offset setting unit 32 compares the image generated by camera 2 with map information. For example, the offset setting unit 32 assumes the position and orientation of the vehicle 10 and projects road features detected from the image onto the map information, or projects road features around the vehicle 10 as shown in the map information onto the image. Road features can be, for example, road markings such as lane markings or stop lines, curbs, or various road signs. The offset setting unit 32 then detects the position and orientation of the vehicle 10 at which the features detected from the image and the features shown in the map information best match, and uses this as the vehicle 10's accurate position and direction of travel. Furthermore, the offset setting unit 32 detects the lane containing the vehicle 10's position as the vehicle 10's own lane.
[0044] The offset setting unit 32 determines the position on the map or image where the features are projected, using the assumed position and orientation of the vehicle 10 and parameters of the camera 2, such as focal length, installation height, and shooting direction. The offset setting unit 32 then calculates the degree of agreement between the features on or around the road detected from the image and the corresponding features represented on the map (for example, the reciprocal of the sum of the squares of the distances between the corresponding features).
[0045] The offset setting unit 32 repeats the above process while changing the assumed position and orientation of the vehicle 10. The offset setting unit 32 then detects the assumed position and orientation when the degree of agreement is maximized as the accurate position and direction of travel of the vehicle 10.
[0046] The offset setting unit 32 can detect a feature by inputting an image into a classifier that has been pre-trained to detect the feature from the image. The offset setting unit 32 can use a classifier similar to the one used for detecting bicycles or bicycle lanes as such a classifier.
[0047] Once the position of the bicycle is estimated, the offset setting unit 32 sets a position that is offset by an offset distance laterally toward the center of the lane, relative to the estimated position, as the limit position for approaching the bicycle. As described above, the offset distance when it is determined that the bicycle may exhibit wobbly behavior is set to a larger value (e.g., 1.5m to 2m) than the offset distance when it is determined that the bicycle may not exhibit wobbly behavior (e.g., 1m to 1.5m).
[0048] Furthermore, even if there is no dedicated bicycle lane on the road the vehicle 10 is traveling on, the offset setting unit 32 may set the approach limit position at a location offset distance away from the vehicle's own lane by an offset distance relative to the bicycle's position. However, if a dedicated bicycle lane is provided within a predetermined distance from the bicycle's current position along the road, or if the dedicated bicycle lane disappears within a predetermined distance from the bicycle's current position along the road, the bicycle's position in the lateral direction may change. Therefore, if a point of change where the presence or absence of a dedicated bicycle lane changes exists within a predetermined distance from the bicycle's current position, the offset setting unit 32 may set the approach limit position at a location offset distance plus a predetermined correction distance (for example, 0.5 m) relative to the vehicle's own lane, based on the lane markings that demarcate the vehicle's own lane located on the bicycle's side. The location of the lane markings on the road the vehicle 10 is traveling on can be determined by referring to the location of the vehicle 10 shown in the map information and the latest positioning information.
[0049] Furthermore, if no bicycle is detected around the vehicle 10, the offset setting unit 32 may set the approach limit position to the position of the lane markings that define the vehicle's lane, or a predetermined distance (for example, 0.1 to 0.5 m) away from the lane markings toward the center of the vehicle's lane, so that the vehicle 10 can travel along the center of its own lane.
[0050] The offset setting unit 32 notifies the vehicle control unit 33 of the set approach limit position.
[0051] The vehicle control unit 33 controls the vehicle 10's movement so that it moves away from the bicycle by a set offset distance or more. To this end, the vehicle control unit 33 sets a planned trajectory along its own lane so that the vehicle 10 moves away from the bicycle by a distance greater than the approach limit position notified by the offset setting unit 32 in a predetermined section before and after the bicycle's position in the direction of extension of the road while the vehicle 10 is traveling. The predetermined section is set so that it is greater than or equal to the distance required for the vehicle 10 to overtake the bicycle, depending on the relative speed and distance between the bicycle and the vehicle 10. At this time, the vehicle control unit 33 can determine the relative speed between the bicycle and the vehicle 10 based on the change in the distance between the camera 2 and the bicycle at the time of generation of each of the multiple images by the camera 2. The distance between the camera 2 and the bicycle is determined by the same method as described in the offset setting unit 32. The vehicle control unit 33 then controls each part of the vehicle 10 so that the vehicle 10 travels along the planned trajectory.
[0052] If a bicycle is detected, the vehicle control unit 33 sets a planned driving trajectory according to a trajectory generation model obtained by learning the trajectory of vehicle 10 when it overtakes a bicycle during past manual driving by the driver. The trajectory generation model is generated, for example, as the average of multiple trajectories from the point when the distance to the bicycle reaches a predetermined distance until vehicle 10 overtakes the bicycle and travels a predetermined distance. Alternatively, the trajectory generation model may consist of a DNN that has been pre-trained to output a planned driving trajectory. In this case, the vehicle control unit 33 generates a planned driving trajectory by inputting the distance between vehicle 10 and the bicycle, and the positions of vehicle 10 and the bicycle into the trajectory generation model. Then, the vehicle control unit 33 corrects the generated planned driving trajectory so that vehicle 10 moves further away from the bicycle than the approach limit position in the predetermined section.
[0053] When a planned travel route is set, the vehicle control unit 33 controls various parts of the vehicle 10 so that the vehicle 10 travels along that planned route. To this end, the vehicle control unit 33 determines the steering angle of the vehicle 10 necessary for the vehicle 10 to travel along the planned route based on the planned route and the current position of the vehicle 10, and outputs a control signal corresponding to that steering angle to an actuator (not shown) that controls the steering of the vehicle 10. At this time, if the current position of the vehicle 10 is on the planned route, the vehicle control unit 33 determines the steering angle to follow the planned route. If the current position of the vehicle 10 is far from the planned route, the vehicle control unit 33 determines the steering angle to move closer to the planned route. The vehicle control unit 33 also determines the position and direction of travel of the vehicle 10 at the time of the latest image generation using the same method as described in the offset setting unit 32. The vehicle control unit 33 can then estimate the current position of the vehicle 10 by correcting the position and direction of travel of the vehicle 10 at the time of image generation using the acceleration and yaw rate of the vehicle 10 from the time of image generation to the present time.
[0054] Furthermore, if the level of autonomous driving applied to the vehicle 10 is equivalent to a level of driving assistance in which the driver, in principle, operates the steering wheel, the vehicle control unit 33 may assist the driver by controlling the steering so that the vehicle 10 moves away from the bicycle by an offset distance or more when the vehicle 10's position in the lateral direction approaches the bicycle beyond the approach limit position.
[0055] Furthermore, if the autonomous driving level applied to the vehicle 10 is a level that also controls the speed of the vehicle 10, the vehicle control unit 33 may control the speed of the vehicle 10 depending on whether or not the bicycle is likely to exhibit wobbly behavior. For example, if it is determined that the bicycle is likely to exhibit wobbly behavior, the vehicle control unit 33 may decelerate the vehicle 10 so that its speed is below a predetermined speed (e.g., 20 km / h to 30 km / h). Furthermore, if the vehicle 10 is ahead of the bicycle, the vehicle control unit 33 may accelerate the vehicle 10.
[0056] Figures 4(a) and 4(b) show examples of offset distance settings, respectively. In the example shown in Figure 4(a), an adult is riding the bicycle 401, and it is determined that there is no possibility of the bicycle 401 exhibiting wobbly behavior. Therefore, the lateral offset distance is set to a relatively small value OD1, and the approach limit position AL is set at a position that is offset by OD1 from the bicycle 401. The vehicle 10 is then controlled to travel at a position further away from the bicycle 401 than the approach limit position AL.
[0057] In contrast, in the example shown in Figure 4(b), a child is riding on bicycle 402, so it is determined that bicycle 402 may exhibit wobbly behavior. Therefore, the lateral offset distance is set to a value OD2, which is larger than the offset distance OD1 shown in Figure 4(a), and the approach limit position AL is set at a distance of offset distance OD2 from bicycle 402. Consequently, vehicle 10 is controlled to travel at a relatively large distance from bicycle 402.
[0058] Figure 5 is an operation flowchart of the vehicle control process executed by the processor 23. The processor 23 should execute the vehicle control process according to the following operation flowchart at predetermined intervals.
[0059] The determination unit 31 of the processor 23 determines whether or not it was able to detect a bicycle traveling around the vehicle 10 (step S101). If a bicycle is detected (step S101-Yes), the determination unit 31 determines whether or not the bicycle is likely to exhibit wobbly behavior (step S102).
[0060] If there is a possibility that the bicycle will wobble (step S102-Yes), the offset setting unit 32 of the processor 23 sets the lateral offset distance relative to the bicycle to be relatively large (step S103). On the other hand, if there is no possibility that the bicycle will wobble (step S102-No), the offset setting unit 32 sets the lateral offset distance relative to the bicycle to be relatively small (step S104).
[0061] After step S103 or S104, the vehicle control unit 33 of the processor 23 controls the vehicle 10 to move away from the bicycle by an offset distance or more (step S105). Also, if no bicycle is detected in step S101 (step S101-No), the vehicle control unit 33 controls the vehicle 10 to travel in its own lane (step S106). After step S105 or S106, the processor 23 terminates the vehicle control process.
[0062] As explained above, this vehicle control system determines from the image whether or not the bicycle is likely to exhibit swaying behavior, so it can appropriately set the lateral offset distance even when the bicycle is not actually exhibiting swaying behavior.
[0063] In a modified version, the offset setting unit 32 may correct the offset distance according to the terrain of the road on which the vehicle 10 is traveling. For example, if the road on which the vehicle 10 is traveling is sloped, especially uphill, bicycles traveling around the vehicle 10 are more likely to wobble. Therefore, if the road on which the vehicle 10 is traveling is sloped, the offset setting unit 32 corrects the set offset distance by an additional predetermined distance (for example, 0.3m to 0.6m) according to the likelihood of wobbling behavior. The offset setting unit 32 can determine whether the road on which the vehicle 10 is traveling is sloped or not by referring to the latest position of the vehicle 10 measured by the GPS receiver 3 and information on the road gradient included in the map information.
[0064] Similarly, the offset setting unit 32 may further increase the set offset distance by a predetermined distance if the road the vehicle 10 is traveling on is curved, depending on the likelihood of swaying behavior. In this case as well, the offset setting unit 32 can determine whether the road the vehicle 10 is traveling on is curved by referring to the latest position of the vehicle 10 measured by the GPS receiver 3 and the information on the curvature of the road included in the map information. Alternatively, the offset setting unit 32 may detect lane markings from the image generated by the camera 2 and determine that the road the vehicle 10 is traveling on is curved if the detected lane markings can be approximated by a curve.
[0065] Furthermore, the classifier for determining wobble used in the determination unit 31 may be configured to receive terrain information representing the terrain of the road on which the vehicle 10 is traveling, along with the object region representing the detected bicycle. For example, the terrain information may be represented as a matrix or vector with different values depending on the terrain, such as curves, straight lines, slopes, and flat areas, and is input to the classifier for determining wobble as a channel different from the channel into which the object region is input. The classifier for determining wobble has a layer that performs a fully connected operation between the channel into which the object region is input and the channel into which the terrain information is input, and is pre-trained to output a confidence level that the bicycle will exhibit wobble behavior, taking the terrain information into consideration. As a result, if the terrain of the road on which the vehicle 10 is traveling is such that it is likely to cause the bicycle to wobble (for example, a slope or a curve), it becomes easier to determine that the bicycle is likely to exhibit wobble behavior.
[0066] The computer program that realizes the functions of the processor 23 of the ECU 5 according to the above embodiment or modification may be provided in the form of being recorded on a computer-readable portable recording medium such as semiconductor memory, magnetic recording medium, or optical recording medium.
[0067] As described above, those skilled in the art can make various modifications within the scope of the present invention to suit the implemented form. [Explanation of symbols]
[0068] 1. Vehicle control system 10 vehicles 2 cameras 3 GPS receivers 4. Storage devices 5. Electronic Control Unit (ECU) 21 Communication Interface 22 memory 23 processors 31 Judgment section 32 Offset setting section 33 Vehicle Control Unit
Claims
1. A camera that generates an image representing the surroundings of the vehicle, An ECU that controls the driving of the vehicle, comprising at least one functional module composed of a neural network, It has, The ECU determines whether or not the bicycle is likely to wobble based on the image acquired from the camera. The ECU further sets a predetermined offset distance based on whether or not the bicycle is likely to wobble, in order to output a control signal that causes the vehicle to travel at a distance greater than a predetermined offset distance from the bicycle. Vehicle control device.
2. The vehicle control device according to claim 1, wherein the ECU is configured to determine whether or not the bicycle is likely to exhibit wobbly behavior based on the object region in which the bicycle is detected on the image.
3. The vehicle control device according to claim 2, wherein the ECU is configured to set the offset distance to the bicycle in a direction perpendicular to the direction of extension of the road on which the vehicle is traveling to a value greater than the offset distance when it is determined that the bicycle is not likely to exhibit wobbling behavior.
4. The vehicle control device according to claim 1, wherein the ECU inputs the image to a first classifier pre-trained to detect the bicycle from the image, thereby detecting an object region in the image in which the bicycle is represented, and inputs the object region to a second classifier pre-trained to determine whether or not the bicycle is likely to exhibit a wobbling behavior.
5. The vehicle control device according to claim 4, wherein the ECU determines whether or not the bicycle is likely to exhibit swaying behavior by inputting information representing the terrain of the road on which the vehicle is traveling, along with the object region, to the second identifier.
6. The vehicle control device according to claim 1, wherein the ECU detects the bicycle represented on the image and determines whether or not the bicycle is likely to exhibit a wobbling behavior by inputting the image into a third classifier which has been pre-trained to detect the bicycle represented on the image and determine whether or not the bicycle is likely to exhibit a wobbling behavior.
7. The vehicle control device according to any one of claims 1 to 3, wherein the ECU corrects the offset distance based on the terrain of the road on which the vehicle is traveling.
8. The vehicle control device according to claim 7, wherein the ECU corrects the offset distance to be longer by a predetermined distance when the road on which the vehicle is traveling is an incline or a curve.
9. An ECU that controls the vehicle's operation, comprising at least one functional module composed of a neural network, Based on images obtained from a camera that generates images representing the surroundings of the vehicle, it is determined whether or not the bicycle is likely to wobble. In order to output a control signal that causes the vehicle to travel at a distance greater than a predetermined offset distance from the bicycle, the predetermined offset distance is set based on whether or not the bicycle is likely to wobble. A vehicle control method that includes the following.
10. Based on images acquired from a camera that generates images representing the surroundings of the vehicle, the possibility of the bicycle wobbling is determined. In order to output a control signal that causes the vehicle to travel at a distance greater than a predetermined offset distance from the bicycle, the predetermined offset distance is set based on whether or not the bicycle is likely to wobble. A vehicle control computer program comprising at least one functional module composed of a neural network, which is used to cause an ECU that controls the driving of the vehicle to perform the following actions.
Citation Information
Patent Citations
Vehicle control device, vehicle control method, and program
JP2019185113A
