Vehicle control device, vehicle control method, and computer program product

Abstract

The invention relates to a vehicle control apparatus, a vehicle control method, and a computer program product. A vehicle control device is provided with: a determination unit (31) that detects a bicycle traveling around a vehicle (10) on the basis of an image indicating the surroundings of the vehicle (10), and determines whether or not there is a possibility of a shake behavior of the bicycle on the basis of an object region indicating the bicycle detected on the image; a deviation setting unit (32) that sets the deviation distance with respect to the bicycle in a direction orthogonal to the extending direction of the road on which the vehicle (10) travels, when it is determined that the bicycle is likely to exhibit a shake behavior, to a value greater than the deviation distance when it is determined that the bicycle is not likely to exhibit a shake behavior; and a vehicle control unit (33) that controls the travel of the vehicle (10) so as to be separated from the bicycle by the set deviation distance or more.

Description

Technical Field

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a computer program product for vehicle control. Background Art

[0002] Techniques for driving control of a vehicle based on the recognition result of a two-wheeled vehicle traveling in front of the vehicle are being studied (see Japanese Patent Application Laid-Open No. 2019-185113).

[0003] When the maximum value of the amount of change in the lateral position of a two-wheeled vehicle within a specified distance is greater than or equal to a threshold value, the vehicle control device described in Japanese Patent Application Laid-Open No. 2019-185113 determines that the probability that the two-wheeled vehicle is wobbling is high. Then, the vehicle control device sets a large deviation distance from the dedicated two-wheeled vehicle lane in which the two-wheeled vehicle is traveling, so that the vehicle is kept away from the two-wheeled vehicle.

[0004] In the above technique, the deviation distance is adjusted based on the amount of change in the lateral position of the two-wheeled vehicle. Therefore, for a two-wheeled vehicle that has not wobbled particularly so far, the deviation distance is not set to a large value. However, depending on the rider of a two-wheeled vehicle such as a bicycle, the two-wheeled vehicle sometimes suddenly wobbles toward the lane in which the vehicle is traveling. Summary of the Invention

[0005] Therefore, an object of the present invention is to provide a vehicle control device that can appropriately set a lateral deviation distance with respect to a bicycle traveling around a vehicle.

[0006] According to one embodiment, there is provided a vehicle control device. The vehicle control device includes: a determination unit that detects a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determines whether the bicycle has a possibility of exhibiting a wobbling behavior based on an object region representing the bicycle detected in the image; a deviation setting unit that sets a deviation distance of the vehicle in a direction orthogonal to the extending direction of the road on which the vehicle is traveling with respect to the bicycle to a value larger than the deviation distance when it is determined that the bicycle has no possibility of exhibiting a wobbling behavior, in the case where it is determined that the bicycle has a possibility of exhibiting a wobbling behavior; and a vehicle control unit that controls the traveling of the vehicle so as to be separated from the bicycle by the set deviation distance or more.

[0007] In one embodiment, the determination unit detects an object region representing a bicycle in the image by inputting the image to a first classifier, and determines whether the bicycle has a possibility of exhibiting a wobbling behavior by inputting the object region to a second classifier, where the first classifier has been previously learned to detect a bicycle from the image, and the second classifier has been previously learned to determine the possibility that the bicycle exhibits a wobbling behavior.

[0008] In this case, it may also be set that the determination unit determines the possibility that the bicycle exhibits a wobbling behavior by inputting information indicating the terrain of the road during vehicle travel together with the object area to the second classifier.

[0009] In one embodiment, the determination unit detects an object area representing a bicycle in the image by inputting the image to a third classifier, and determines the possibility that the bicycle exhibits a wobbling behavior, where the third classifier has been pre-trained to detect the bicycle represented in the image and determine the possibility that the bicycle exhibits a wobbling behavior.

[0010] In one embodiment, the deviation setting unit corrects the deviation distance based on the terrain of the road during vehicle travel.

[0011] In this case, it may also be that when the road during vehicle travel is a slope or a curve, the deviation setting unit corrects the deviation distance in such a way that the deviation distance becomes longer by a specified distance.

[0012] According to another embodiment, there is provided a vehicle control method. The vehicle control method includes: detecting a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determining the possibility that the bicycle exhibits a wobbling behavior based on an object area representing the bicycle detected in the image; setting a deviation distance of the vehicle in a direction orthogonal to the extending direction of the road on which the vehicle is traveling with respect to the bicycle to a value larger than the deviation distance when it is determined that the bicycle does not have the possibility of exhibiting a wobbling behavior, in the case where it is determined that the bicycle has the possibility of exhibiting a wobbling behavior; and controlling the travel of the vehicle so as to be separated from the bicycle by more than the set deviation distance.

[0013] According to still another embodiment, there is provided a computer program product for vehicle control. The computer program product for vehicle control includes instructions for causing a processor mounted on a vehicle to execute the following processes: detecting a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determining the possibility that the bicycle exhibits a wobbling behavior based on an object area representing the bicycle detected in the image; setting a deviation distance of the vehicle in a direction orthogonal to the extending direction of the road on which the vehicle is traveling with respect to the bicycle to a value larger than the deviation distance when it is determined that the bicycle does not have the possibility of exhibiting a wobbling behavior, in the case where it is determined that the bicycle has the possibility of exhibiting a wobbling behavior; and controlling the travel of the vehicle so as to be separated from the bicycle by more than the set deviation distance.

[0014] The vehicle control device of the present disclosure has the following effect: it can appropriately set the deviation distance in the lateral direction with respect to a bicycle traveling around the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic configuration diagram of a vehicle control system equipped with a vehicle control device.

[0016] Figure 2 It is a hardware configuration diagram of an electronic control device as an example of the vehicle control device.

[0017] Figure 3 It is a functional block diagram of the processor of the electronic control device related to the vehicle control process.

[0018] Figure 4A It is a diagram showing an example of the setting of the deviation distance.

[0019] Figure 4B It is a diagram showing another example of the setting of the deviation distance.

[0020] Figure 5 It is a flowchart of the operation of the vehicle control process. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Hereinafter, a vehicle control device, a vehicle control method implemented in the vehicle control device, and a vehicle control computer program will be described with reference to the drawings. The vehicle control device detects a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determines whether the bicycle represented in the image has a possibility of exhibiting a wobbling behavior. Further, when it is determined that the bicycle has a possibility of exhibiting a wobbling behavior, the vehicle control device sets the deviation distance of the vehicle in the direction orthogonal to the extending direction of the road on which the vehicle is traveling (hereinafter, referred to as the lateral direction) with respect to the bicycle to a value larger than the deviation distance when it is determined that the bicycle has no possibility of exhibiting a wobbling behavior. Thus, the vehicle control device appropriately sets the deviation distance in the lateral direction even for a bicycle that does not actually exhibit a wobbling behavior.

[0022] In the present embodiment, the bicycle is not limited to a two-wheeler, and may be a bicycle with training wheels for a child to ride, or a tricycle for an elderly person or a child. Further, the bicycle may be a bicycle with auxiliary power, where the auxiliary power is obtained by a motor or the like.

[0023] Figure 1 It is a schematic configuration diagram of a vehicle control system equipped with a vehicle control device. Further, Figure 2This is a hardware configuration diagram of an electronic control unit, which is an example of a vehicle control device. In this embodiment, a vehicle control system 1 mounted on a vehicle 10 and controlling the vehicle 10 includes a camera 2, a GPS (Global Positioning System) receiver 3, a storage device 4, and an electronic control unit (ECU: Electronic Control Unit) 5 as an example of a vehicle control device. The camera 2, the GPS receiver 3, and the storage device 4 are communicably connected to the ECU 5 via a standard-compliant in-vehicle network such as a controller area network. Note that the vehicle 10 is an example of the host vehicle. In addition, the vehicle control system 1 may further include a ranging sensor (not shown), such as a LiDAR (Light Detection and Ranging) or a radar, for measuring the distance from the vehicle 10 to an object existing around the vehicle 10.

[0024] The camera 2 generates an image representing the surroundings of the vehicle 10. The camera 2 is mounted inside the vehicle compartment of the vehicle 10 so as to face a predetermined direction, for example, the front of the vehicle 10. Then, the camera 2 captures an area around the vehicle 10, for example, the front area of the vehicle 10, at regular shooting intervals (for example, 1 / 30 second to 1 / 10 second), and generates an image representing the front area. The image obtained by the camera 2 may be a color image or a grayscale image. Note that a plurality of cameras with different shooting directions or focal lengths may be provided on the vehicle 10.

[0025] Whenever an image is generated, the camera 2 outputs the generated image to the ECU 5 via the in-vehicle network.

[0026] The GPS receiver 3 receives GPS signals from GPS satellites at regular intervals and measures the own position of the vehicle 10 based on the received GPS signals. Then, the GPS receiver 3 outputs positioning information representing the positioning result of the own position of the vehicle 10 based on the GPS signals to the ECU 5 at regular intervals via the in-vehicle network. Note that the vehicle 10 may have a receiver compliant with a satellite positioning system other than the GPS receiver 3. In this case, the receiver may measure the own position of the vehicle 10.

[0027] The storage device 4 is an example of a storage unit, and has, for example, a hard disk device, a non-volatile semiconductor memory, an optical recording medium, and an access device therefor. Further, the storage device 4 stores map information. The map information includes, for example, information on road markings such as lane dividing lines or stop lines for each road section included in a predetermined area represented by the map information, information on road signs, and information on ground objects around the road. Further, the map information may include information on the curvature and gradient of each road section.

[0028] Further, the storage device 4 may have a processor for performing update processing of the map information and processing related to a read request for the map information from the ECU 5. For example, every time the vehicle 10 moves a predetermined distance, the storage device 4 transmits a map information acquisition request together with the current position of the vehicle 10 to a map server via a wireless communication terminal (not shown) mounted on the vehicle 10. Then, the storage device 4 receives map information on a predetermined area around the current position of the vehicle 10 from the map server via the wireless communication terminal. Further, when a read request for the map information is received from the ECU 5, the storage device 4 extracts a relatively narrow range including the current position of the vehicle 10 from the stored map information and outputs it to the ECU 5 via an in-vehicle network.

[0029] The ECU 5 controls the travel of the vehicle 10 according to a predetermined level of autonomous driving. Note that the predetermined level of autonomous driving may be any level of autonomous driving of level 1 or higher defined by the Society of Automotive Engineers (SAE).

[0030] As Figure 2 shown, the ECU 5 includes a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may each be configured as a separate circuit, or may be integrally configured as one integrated circuit.

[0031] The communication interface 21 has an interface circuit for connecting the ECU 5 to the in-vehicle network. Further, every time an image is received from the camera 2, the communication interface 21 transfers the received image to the processor 23. Further, every time positioning information is received from the GPS receiver 3, the communication interface 21 transfers the positioning information to the processor 23. Further, the communication interface 21 transfers the map information read from the storage device 4 to the processor 23.

[0032] The memory 22 is another example of a storage unit, such as a volatile semiconductor memory and a non-volatile semiconductor memory. And, the memory 22 stores various data used in the vehicle control processing executed by the processor 23 of the ECU 5. For example, the memory 22 stores parameters of the camera 2 such as the focal length, the field of view angle, the shooting direction, and the mounting position, map information, and a parameter set for determining various classifiers, where the classifiers are used for the detection of bicycles traveling around the vehicle 10 and the determination of shaking. Moreover, the memory 22 temporarily stores the images generated by the camera 2 and the positioning results of the own position obtained by the GPS receiver 3. Moreover, in addition, the memory 22 temporarily stores various data generated in the middle of the vehicle control processing.

[0033] The processor 23 has one or more CPUs (Central Processing Unit) and their peripheral circuits. The processor 23 may further have other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit, or a graphics processing unit. And, the processor 23 executes the vehicle control processing for the vehicle 10.

[0034] Figure 3 is a functional block diagram of the processor 23 related to the vehicle control processing. The processor 23 has a determination unit 31, a deviation setting unit 32, and a vehicle control unit 33. Each of these units of the processor 23 is, for example, a functional module implemented by a computer program operating on the processor 23. Or, each of these units of the processor 23 may also be a dedicated arithmetic circuit provided in the processor 23.

[0035] The determination unit 31 detects a bicycle traveling around the vehicle 10 based on the image representing the surroundings of the vehicle 10 generated by the camera 2. Then, the determination unit 31 determines whether the detected bicycle shown in the image has the possibility of showing a shaking behavior.

[0036] In the present embodiment, every time the ECU 5 acquires an image from the camera 2, the determination unit 31 inputs the image to the classifier, thereby detecting a bicycle traveling around the vehicle 10. The classifier for bicycle detection is an example of the first classifier. As the classifier for bicycle detection, for example, a deep neural network (DNN) having a convolutional neural network (CNN) - type architecture such as a single - shot multibox detector (SSD) or a faster region - convolutional neural network (Faster R - CNN) can be adopted. Alternatively, the classifier for bicycle detection can also be a DNN having an attention mechanism such as a vision transformer. Or, in addition, the classifier for bicycle detection can also be a classifier based on other machine learning algorithms other than DNN such as a support vector machine or an adaBoost classifier. Such a classifier is pre - learned in advance in a manner of detecting a bicycle from an image using many training images including images representing bicycles according to a prescribed learning algorithm such as the error backpropagation method. The classifier outputs information for determining an object region representing the bicycle detected in the input image. The object region is, for example, an outer circumscribed rectangle of the bicycle represented in the image.

[0037] When a bicycle is detected from the image, the determination unit 31 inputs the object region representing the bicycle to a classifier for determining the possibility that the bicycle exhibits a wobbling behavior (hereinafter, sometimes simply referred to as wobbling). The determination unit 31 may also input the object region to the classifier for wobbling determination after upsampling or downsampling the object region so that the object region representing the bicycle becomes a prescribed size. Thereby, the configuration of the classifier for wobbling determination is simplified. When the object region representing the bicycle is input, the classifier for wobbling determination outputs a confidence score indicating that the bicycle represented by the object region exhibits a wobbling behavior. And, when the confidence score is higher than a prescribed threshold (for example, 0.7 to 0.9), the determination unit 31 determines that there is a possibility that the bicycle exhibits a wobbling behavior, and when the confidence score is less than or equal to the threshold, the determination unit 31 determines that there is no possibility that the bicycle exhibits a wobbling behavior.

[0038] The classifier for shake determination is an example of the second classifier. For example, a CNN-based DNN having, in order from the input side, one or more convolutional layers, one or more fully-connected layers, and an output layer that performs a softmax operation or a sigmoid operation can be used. Alternatively, the classifier for shake determination can also be a DNN having an attention mechanism or a classifier based on other machine learning algorithms.

[0039] The classifier for shake determination uses many training images representing various bicycles and is pre-learned according to a prescribed learning algorithm in such a way that it outputs the confidence level that a bicycle exhibits a shaking behavior based on the characteristics of the bicycle represented by the object region or the characteristics of the rider of the bicycle. For example, if the rider of the bicycle is an elderly person or a child, the rider may sometimes not be able to operate the bicycle without shaking. Therefore, the classifier for shake determination is pre-learned in such a way that the confidence level that a bicycle ridden by an elderly person or a child exhibits a shaking behavior becomes higher. In addition, in the case where the bicycle is a children's bicycle, since the rider is a child, there is a possibility that the bicycle exhibits a shaking behavior. Also, for a bicycle equipped with a baby seat, due to the difficulty of handlebar operation, there is also a possibility that the bicycle exhibits a shaking behavior. Therefore, the classifier for shake determination is pre-learned in such a way that the confidence level that a children's bicycle or a bicycle equipped with a baby seat exhibits a shaking behavior also becomes higher. On the other hand, for a bicycle whose rider is an adult who is not elderly and does not have a baby seat, the possibility of exhibiting a shaking behavior is low. Therefore, the classifier for shake determination is pre-learned in such a way that the confidence level that such a bicycle exhibits a shaking behavior becomes lower.

[0040] Note that in the case where multiple bicycles are detected from one image, the determination unit 31 determines the possibility of each detected bicycle exhibiting a shaking behavior by inputting the object region representing the bicycle to the classifier for shake determination for each detected bicycle. Alternatively, the classifier for shake determination can also be configured to be able to be input with multiple object regions at the same time. For example, the classifier for shake determination can also be configured to be input with different object regions for each channel. In this case, multiple object regions respectively representing the detected bicycles are simultaneously input to the classifier for shake determination, and thus the classifier for shake determination outputs the confidence level that a certain bicycle exhibits a shaking behavior. In this case, the classifier for shake determination can also be pre-learned in such a way that it is easy to determine that there is a possibility that one of two or more bicycles ridden by a parent and a child exhibits a shaking behavior.

[0041] According to the modified example, a classifier can also be configured to detect a bicycle from an image and calculate the confidence that the detected bicycle exhibits a wobbling behavior. The classifier of this modified example is an example of the third classifier, and can be a CNN-based DNN, a DNN with an attention mechanism, or a classifier based on other machine learning algorithms. In this case, the classifier outputs the object region representing the detected bicycle and the confidence in the wobbling behavior obtained for this object region. Then, in this case, the determination unit 31 can also determine whether the bicycle represented by the object region has the possibility of exhibiting a wobbling behavior by comparing the confidence for the object region with a threshold value for each object region.

[0042] For the detected bicycle, the determination unit 31 notifies the deviation setting unit 32 of the determination result as to whether the bicycle has the possibility of exhibiting a wobbling behavior and the position of the object region representing the bicycle on the image.

[0043] The deviation setting unit 32 sets the deviation distance in the lateral direction with respect to the detected bicycle. In the present embodiment, the deviation setting unit 32 sets the deviation distance in the case where it is determined that the detected bicycle has the possibility of exhibiting a wobbling behavior to a value larger than the deviation distance in the case where it is determined that the detected bicycle does not have the possibility of exhibiting a wobbling behavior.

[0044] For example, when there is a bicycle lane on the road where the vehicle 10 is traveling, it is predicted that the bicycle travels within the bicycle lane in principle. Therefore, in this case, the deviation setting unit 32 sets the approaching limit position to the bicycle at a position deviated from the position of the bicycle by the deviation distance toward the own lane side. The deviation setting unit 32 determines whether there is a bicycle lane on the road where the vehicle 10 is traveling with reference to the latest position of the vehicle 10 measured by the GPS receiver 3 and the map information. Alternatively, the deviation setting unit 32 can also determine whether there is a bicycle lane on the road where the vehicle 10 is traveling by inputting the image generated by the camera 2 into a classifier that has been previously learned to detect a bicycle lane. In this case, the deviation setting unit 32 can use, for example, a deep neural network for semantic segmentation such as a fully convolutional network (FCN) or a U-Net as the classifier.

[0045] Furthermore, the deviation setting unit 32 estimates the detected position of the bicycle. Here, each pixel on the image generated by the camera 2 corresponds one-to-one with the azimuth from the camera 2. In addition, it is estimated that the lower end of the object area representing the bicycle indicates the position where the bicycle contacts the road surface. Therefore, the deviation setting unit 32 can estimate the distance and direction to the bicycle based on the position of the lower end of the object area representing the bicycle on the image and the parameters of the camera 2 such as the installation height, focal length, and shooting direction of the camera 2 at the time of image generation, with the position of the camera 2 as the reference. It should be noted that when the vehicle 10 is provided with a distance measuring sensor, the deviation setting unit 32 can also estimate the distance measured by the distance measuring sensor in the azimuth corresponding to the object area representing the bicycle as the distance to the bicycle. Moreover, the deviation setting unit 32 can estimate the position of the bicycle in the world coordinate system based on the distance and direction to the bicycle with the position of the camera 2 as the reference and the position and traveling direction of the vehicle 10 at the time of image generation. Therefore, the deviation setting unit 32 compares the image generated by the camera 2 with the map information to detect the accurate position and traveling direction of the vehicle 10. For example, the deviation setting unit 32 assumes the position and orientation of the vehicle 10, and projects the ground objects on the road or around the road detected from the image onto the map information, or projects the ground objects on the road or around the road around the vehicle 10 represented by the map information onto the image. It should be noted that the ground objects on the road or around the road can be, for example, road markings such as lane dividing lines or stop lines, curbs, or various road signs. Then, the deviation setting unit 32 detects the position and orientation of the vehicle 10 when the ground objects detected from the image match the ground objects represented by the map information the most as the accurate position and traveling direction of the vehicle 10. Moreover, the deviation setting unit 32 detects the lane including the position of the vehicle 10 as the lane on which the vehicle 10 is traveling.

[0046] The deviation setting unit 32 only needs to use the assumed position and orientation of the vehicle 10 and the parameters of the camera 2 such as the focal length, installation height, and shooting direction to determine the position where the ground object is projected on the map or the image. Then, the deviation setting unit 32 calculates the degree of coincidence (for example, the reciprocal of the sum of the squares of the distances between the corresponding ground objects) between the ground objects on the road or around the road detected from the image and the corresponding ground objects represented on the map.

[0047] The deviation setting unit 32 repeatedly executes the above processing while changing the assumed position and orientation of the vehicle 10. Then, the deviation setting unit 32 only needs to detect the assumed position and orientation when the degree of coincidence becomes the maximum as the accurate position and traveling direction of the vehicle 10.

[0048] Note that the deviation setting unit 32 can detect the ground object by inputting the image into a classifier that has been pre - learned to detect the ground object to be detected from the image. The deviation setting unit 32 can use the same classifier as the classifier for detecting a bicycle or the classifier for detecting a dedicated bicycle lane as such a classifier.

[0049] When the position of the bicycle is estimated, the deviation setting unit 32 sets the position that is deviated from the estimated position by a deviation distance toward the center side of the own lane in the lateral direction as the approaching limit position for the bicycle. As described above, the deviation distance in the case where it is determined that the bicycle has a possibility of exhibiting a wobbling behavior is set to a value (e.g., 1.5 m to 2 m) larger than the deviation distance (e.g., 1 m to 1.5 m) in the case where it is determined that the bicycle has no possibility of exhibiting a wobbling behavior.

[0050] Note that, when there is no dedicated bicycle lane on the road where the vehicle 10 is traveling, the deviation setting unit 32 can also set the approaching limit position at a position that is deviated from the position of the bicycle by a deviation distance toward the own lane side. However, in the case where a dedicated bicycle lane is provided within a specified distance from the current position of the bicycle along the road or in the case where the dedicated bicycle lane disappears within a specified distance from the current position of the bicycle along the road, the position of the bicycle in the lateral direction may change. Therefore, when the change point of the presence or absence of the dedicated bicycle lane exists within a specified distance from the current position of the bicycle, the deviation setting unit 32 can also set the approaching limit position at a position that is deviated from the lane dividing line that divides the own lane and is on the bicycle side by a distance obtained by adding a specified correction distance (e.g., 0.5 m) to the deviation distance toward the own lane side. Note that the position of the lane dividing line of the road where the vehicle 10 is traveling can be determined by referring to the map information and the position of the vehicle 10 shown in the latest positioning information.

[0051] In addition, when no bicycle is detected around the vehicle 10, the deviation setting unit 32 sets the position of the lane dividing line that divides the own lane itself or a position that is deviated from the lane dividing line by a specified distance (e.g., 0.1 to 0.5 m) toward the center side of the own lane so that the vehicle 10 can travel along the center of the own lane.

[0052] The deviation setting unit 32 notifies the vehicle control unit 33 of the set approaching limit position.

[0053] The vehicle control unit 33 controls the travel of the vehicle 10 so as to be separated from the bicycle by a distance greater than or equal to a set deviation distance. To this end, the vehicle control unit 33 sets a predetermined travel trajectory so as to be farther from the bicycle than the proximity limit position notified from the deviation setting unit 32 in a predetermined section before and after the position of the bicycle in the extending direction of the road during the travel of the vehicle 10, and sets the predetermined travel trajectory along the own lane. It should be noted that, based on the relative speed and distance between the bicycle and the vehicle 10, the predetermined section is set to be greater than or equal to the distance required until the vehicle 10 overtakes the bicycle. At this time, the vehicle control unit 33 can obtain the relative speed between the bicycle and the vehicle 10 based on the change in the distance between the camera 2 and the bicycle at the time of generation of each of the plurality of images obtained by the camera 2. It should be noted that the distance between the camera 2 and the bicycle is obtained by the same method as the method described in the deviation setting unit 32. Then, the vehicle control unit 33 controls each part of the vehicle 10 so that the vehicle 10 travels along the predetermined travel trajectory.

[0054] When a bicycle is detected, the vehicle control unit 33 sets a predetermined travel trajectory according to a trajectory generation model, where the trajectory generation model is obtained by learning the trajectory of the vehicle 10 when overtaking the bicycle during manual driving by the driver in the past. The trajectory generation model is generated, for example, as the average value of a plurality of trajectories from the time point when the distance from the bicycle becomes a predetermined distance until the vehicle 10 overtakes the bicycle and advances a predetermined distance. Alternatively, the trajectory generation model may be constituted by a DNN that has been pre-learned to output a predetermined travel trajectory. In this case, the vehicle control unit 33 generates a predetermined travel trajectory by inputting the distance from the vehicle 10 to the bicycle and the positions of the vehicle 10 and the bicycle into the trajectory generation model. Then, in the generated predetermined travel trajectory, the vehicle control unit 33 corrects the predetermined travel trajectory so that the vehicle 10 is farther from the bicycle than the proximity limit position in the above-mentioned predetermined section.

[0055] When a predetermined travel trajectory is set, the vehicle control unit 33 controls each part of the vehicle 10 so that the vehicle 10 travels along the predetermined travel trajectory. For this purpose, the vehicle control unit 33 calculates the steering angle of the vehicle 10 for causing the vehicle 10 to travel along the predetermined travel trajectory based on the predetermined travel trajectory and the current position of the vehicle 10, and outputs a control signal corresponding to the steering angle to an actuator (not shown) that controls the steering wheel of the vehicle 10. At this time, if the current position of the vehicle 10 is on the predetermined travel trajectory, the vehicle control unit 33 determines the steering angle so as to follow the predetermined travel trajectory. In addition, if the current position of the vehicle 10 deviates from the predetermined travel trajectory, the vehicle control unit 33 determines the steering angle so as to approach the predetermined travel trajectory. It should be noted that the vehicle control unit 33 obtains the position and traveling direction of the vehicle 10 at the time of generating the latest image by the same method as the method described in the deviation setting unit 32. Then, the vehicle control unit 33 can correct the position and traveling direction of the vehicle 10 at the time of generating the image using the acceleration and yaw rate of the vehicle 10 from the time of generating the image to the current time, thereby estimating the current position of the vehicle 10.

[0056] In addition, when the autonomous driving level applied to the vehicle 10 is a level corresponding to driving assistance in which the steering wheel is basically operated by the driver, when the position of the vehicle 10 in the lateral direction is closer to the bicycle than the approaching limit position, the vehicle control unit 33 can also assist the driver's driving by controlling the steering wheel so as to be separated from the bicycle by a deviation distance or more.

[0057] Moreover, when the autonomous driving level applied to the vehicle 10 is a level that also controls the speed of the vehicle 10, the vehicle control unit 33 can also control the speed of the vehicle 10 according to whether the bicycle is likely to exhibit a wobbling behavior. For example, when it is determined that the bicycle is likely to exhibit a wobbling behavior, the vehicle control unit 33 can decelerate the vehicle 10 so that the speed of the vehicle 10 becomes equal to or lower than a specified speed (for example, 20 km / h to 30 km / h). Moreover, when the vehicle 10 is in front of the bicycle, the vehicle control unit 33 can accelerate the vehicle 10.

[0058] Figure 4A and Figure 4B are diagrams showing an example of setting the deviation distance. In Figure 4A the example shown, the bicycle 401 is ridden by an adult, and it is determined that the bicycle 401 is not likely to exhibit a wobbling behavior. Therefore, the lateral deviation distance is set to a relatively small value OD1, and the approaching limit position AL is set at a position separated from the bicycle 401 by the deviation distance OD1. And the vehicle 10 is controlled to travel at a position farther from the bicycle 401 than the approaching limit position AL.

[0059] In contrast, in the example shown in Figure 4B , the bicycle 402 is ridden by a child, so it is determined that there is a possibility that the bicycle 402 exhibits a wobbling behavior. Therefore, the lateral deviation distance is set to a value OD2 larger than the deviation distance OD1 shown in Figure 4A . The approach limit position AL is set at a position where the deviation distance OD2 has been left from the bicycle 402. Therefore, the vehicle 10 is controlled to travel at a position relatively far from the bicycle 402.

[0060] Figure 5 is a flowchart of the operation of the vehicle control process executed by the processor 23. The processor 23 may execute the vehicle control process according to the following flowchart at every prescribed cycle.

[0061] The determination unit 31 of the processor 23 determines whether a bicycle traveling around the vehicle 10 is detected (step S101). When a bicycle is detected (step S101 - YES), the determination unit 31 determines whether there is a possibility that the bicycle exhibits a wobbling behavior (step S102).

[0062] When there is a possibility that the bicycle exhibits a wobbling behavior (step S102 - YES), the deviation setting unit 32 of the processor 23 sets the lateral deviation distance relative to the bicycle relatively large (step S103). On the other hand, when there is no possibility that the bicycle exhibits a wobbling behavior (step S102 - NO), the deviation setting unit 32 sets the lateral deviation distance relative to the bicycle relatively small (step S104).

[0063] After step S103 or S104, the vehicle control unit 33 of the processor 23 controls the travel of the vehicle 10 so as to leave the deviation distance from the bicycle (step S105). In addition, when a bicycle is not detected in step S101 (step S101 - NO), the vehicle control unit 33 controls the vehicle 10 so that the vehicle 10 travels in the own lane (step S106). After step S105 or S106, the processor 23 ends the vehicle control process.

[0064] As described above, since this vehicle control device determines whether there is a possibility that a bicycle exhibits a wobbling behavior based on an image, it is possible to appropriately set the lateral deviation distance even at a timing when the bicycle does not actually exhibit a wobbling behavior.

[0065] According to the modification example, the deviation setting unit 32 may also correct the deviation distance according to the terrain of the road on which the vehicle 10 is traveling. For example, when the road on which the vehicle 10 is traveling is a slope, especially an uphill slope, the possibility that a bicycle traveling around the vehicle 10 shakes increases. Therefore, when the road on which the vehicle 10 is traveling is a slope, the deviation setting unit 32 corrects the deviation distance set according to the possibility of expressing the shaking behavior so that the deviation distance becomes longer by a specified distance (for example, 0.3 m to 0.6 m). It should be noted that the deviation setting unit 32 can determine whether the road on which the vehicle 10 is traveling is a slope by referring to the latest position of the vehicle 10 measured by the GPS receiver 3 and the information related to the road gradient included in the map information.

[0066] Similarly, when the road on which the vehicle 10 is traveling is curved, the deviation setting unit 32 may also correct the deviation distance set according to the possibility of expressing the shaking behavior so that the deviation distance becomes longer by a specified distance. In this case, the deviation setting unit 32 can also determine whether the road on which the vehicle 10 is traveling is curved by referring to the latest position of the vehicle 10 measured by the GPS receiver 3 and the information related to the road curvature included in the map information. Alternatively, the deviation setting unit 32 may detect the lane dividing line from the image generated by the camera 2, and when the detected lane dividing line can be approximated by a curve, the deviation setting unit 32 determines that the road on which the vehicle 10 is traveling is curved.

[0067] In addition, the classifier for shake determination used in the determination unit 31 may also be configured to input the terrain information representing the terrain of the road on which the vehicle 10 is traveling together with the object area of the detected bicycle. For example, the terrain information is represented as a matrix or vector having different values according to terrains such as curves, straight lines, slopes, and flats, and is input to the classifier for shake determination using a channel different from the channel of the input object area. And the classifier for shake determination has a layer that performs a fully connected operation between the channel of the input object area and the channel of the input terrain information, and thus is pre-learned in such a way that the confidence level of the bicycle expressing the shaking behavior is output while also taking into account the terrain information. Thus, when the terrain of the road on which the vehicle 10 is traveling is a terrain that easily causes the bicycle to shake (for example, a slope or a curve), it is easy to determine that the bicycle has the possibility of expressing the shaking behavior.

[0068] A computer program that implements the functions of the processor 23 of the ECU 5 based on the above-described embodiment or modification example may also be provided as a computer program product, for example, in the form of a computer-readable mobile recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium.

[0069] As described above, those skilled in the art can make various changes within the scope of the present invention in conjunction with the implementation manners.

Claims

1. A vehicle control device, comprising: a determination unit that detects a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determines whether the bicycle is likely to exhibit a shaking behavior based on an object region representing the bicycle detected on the image; a deviation setting unit that sets, when it is determined that the bicycle is likely to exhibit a swaying behavior, a deviation distance of the vehicle from the bicycle in a direction perpendicular to an extending direction of a road on which the vehicle is traveling, to a value greater than the deviation distance when it is determined that the bicycle is not likely to exhibit a swaying behavior; as well as The vehicle control unit controls the travel of the vehicle so that the vehicle is separated from the bicycle by a set departure distance or more.

2. The vehicle control device according to claim 1, wherein: The determination unit detects the object area representing the bicycle on the image by inputting the image into a first classifier, and determines whether the bicycle is likely to exhibit shaking behavior by inputting the object area into a second classifier, wherein the first classifier is pre-learned in a manner to detect the bicycle from the image, and the second classifier is pre-learned in a manner to determine the possibility of the bicycle exhibiting shaking behavior.

3. The vehicle control device according to claim 2, wherein: The determination unit determines whether the bicycle is likely to exhibit a shaking behavior by inputting information indicating the topography of the road on which the vehicle is traveling together with the object region into the second classifier.

4. The vehicle control device according to claim 1, wherein: The determination unit detects the object area representing the bicycle on the image by inputting the image into a third classifier, and determines whether the bicycle is likely to exhibit shaking behavior, wherein the third classifier is pre-learned in a manner to detect the bicycle represented on the image and determine the possibility of the bicycle exhibiting shaking behavior.

5. The vehicle control device according to claim 1 or 2, wherein: The deviation setting unit corrects the deviation distance based on the topography of the road on which the vehicle is traveling.

6. The vehicle control device according to claim 5, wherein: When the road on which the vehicle is traveling is a slope or a curve, the deviation setting unit corrects the deviation distance so that the deviation distance becomes longer by a predetermined distance.

7. A vehicle control method, comprising: detecting a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determining whether the bicycle is likely to exhibit a shaking behavior based on an object region representing the bicycle detected on the image; setting the deviation distance of the vehicle from the bicycle in a direction perpendicular to the extending direction of the road on which the vehicle is traveling when it is determined that the bicycle is likely to exhibit a swaying behavior to a value greater than the deviation distance when it is determined that the bicycle is not likely to exhibit a swaying behavior; as well as The travel of the vehicle is controlled so as to be separated from the bicycle by a set departure distance or more.

8. A computer program product for vehicle control, comprising instructions for causing a processor mounted on a vehicle to execute the following processing: detecting a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determining whether the bicycle is likely to exhibit a wobbling behavior based on an object region representing the bicycle detected on the image; setting the deviation distance of the vehicle from the bicycle in a direction perpendicular to the extending direction of the road on which the vehicle is traveling when it is determined that the bicycle is likely to exhibit a swaying behavior to a value greater than the deviation distance when it is determined that the bicycle is not likely to exhibit a swaying behavior; as well as The travel of the vehicle is controlled so as to be separated from the bicycle by a set departure distance or more.

Images