Vehicle control device, vehicle control method, and computer program for controlling vehicle

Abstract

To provide a vehicle control device capable of setting an offset distance in a lateral direction for bicycles traveling around a vehicle.SOLUTION: A vehicle control device includes: a determination unit 31 that detects a bicycle traveling around a vehicle 10 based on an image showing the surroundings of the vehicle 10 and determines whether or not the bicycle is likely to exhibit swaying behavior based on an object area indicating the bicycle detected in the image; an offset setting unit 32 that sets an offset distance for the bicycle in a direction perpendicular to an extension direction on a road on which the vehicle 10 is traveling when it is determined that there is a possibility the bicycle is likely to exhibit swaying behavior to a value larger than the offset distance when it is determined that there is no possibility the bicycle is likely to exhibit swaying behavior; and a control unit 33 that controls the traveling of the vehicle 10 to be separated from the bicycle by the set offset distance or greater.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a vehicle control computer program.

Background Art

[0002] Techniques for controlling the operation of a host vehicle based on the recognition result of a two-wheeled vehicle traveling in front of the host vehicle have been studied (see Patent Document 1).

[0003] The vehicle control device described in Patent Document 1 determines that there is a high probability that the two-wheeled vehicle is wobbling when the maximum value of the amount of variation in the lateral position of the two-wheeled vehicle at a predetermined distance is equal to or greater than a threshold value. Then, the vehicle control device increases the offset distance from the dedicated two-wheeled vehicle lane in which the two-wheeled vehicle is traveling so as to move the host vehicle away from the two-wheeled vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the above technology, since the offset distance is adjusted based on the amount of variation in the lateral position of the two-wheeled vehicle, the offset distance is not set to a large value for a two-wheeled vehicle that has not been wobbling particularly until then. However, depending on the rider of a two-wheeled vehicle such as a bicycle, the two-wheeled vehicle may suddenly wobble toward the lane side on which the vehicle is traveling.

[0006] Therefore, an object of the present invention is to provide a vehicle control device capable of appropriately setting a lateral offset distance with respect to a bicycle traveling around the vehicle.

Means for Solving the Problems

[0007] According to one embodiment, a vehicle control device is provided. This vehicle control device detects a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determines whether there is a possibility that the bicycle exhibits a wobbling behavior based on an object area representing the bicycle detected in the image. When it is determined that there is a possibility that the bicycle exhibits a wobbling behavior, an offset distance for the bicycle in a direction orthogonal to the extending direction of the road on which the vehicle is traveling is set to a value larger than the offset distance when it is determined that there is no possibility that the bicycle exhibits a wobbling behavior. The vehicle control device includes a determination unit, an offset setting unit, and a control unit. The determination unit determines whether there is a possibility that the bicycle exhibits a wobbling behavior. The offset setting unit sets the offset distance. The control unit controls the traveling of the vehicle so as to be separated from the bicycle by a distance equal to or greater than the set offset distance.

[0008] In one embodiment, the determination unit detects an object area representing the bicycle in the image by inputting the image to a first discriminator pre-trained to detect the bicycle from the image, and inputs the object area to a second discriminator pre-trained to determine the possibility that the bicycle exhibits a wobbling behavior, thereby determining whether there is a possibility that the bicycle exhibits a wobbling behavior.

[0009] In this case, the determination unit may determine whether there is a possibility that the bicycle exhibits a wobbling behavior by inputting information representing the terrain of the road on which the vehicle is traveling to the second discriminator together with the object area.

[0010] In one embodiment, the determination unit inputs the image to a third discriminator pre-trained to detect the bicycle represented in the image and to determine the possibility that the bicycle exhibits a wobbling behavior, thereby detecting the object area representing the bicycle in the image and determining whether there is a possibility that the bicycle exhibits a wobbling behavior.

[0011] In one embodiment, the offset setting unit adjusts the offset distance based on the terrain of the road on which the vehicle is traveling.

[0012] In this case, when the road on which the vehicle is traveling is a slope or a curve, the offset setting unit may correct the offset distance to be longer by a predetermined distance.

[0013] According to another embodiment, a vehicle control method is provided. This vehicle control method detects a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, determines whether there is a possibility that the detected bicycle on the image exhibits a wobbling behavior based on the object area representing the bicycle, and when it is determined that there is a possibility that the bicycle exhibits a wobbling behavior, sets the offset distance for the bicycle in the direction orthogonal to the extending direction of the road on which the vehicle is traveling to a value larger than the offset distance in the case where it is determined that there is no possibility that the bicycle exhibits a wobbling behavior, and controls the traveling of the vehicle to be separated from the bicycle by the set offset distance or more.

[0014] According to still another embodiment, a vehicle control computer program is provided. This vehicle control computer program detects a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, determines whether there is a possibility that the detected bicycle on the image exhibits a wobbling behavior based on the object area representing the bicycle, and when it is determined that there is a possibility that the bicycle exhibits a wobbling behavior, sets the offset distance for the bicycle in the direction orthogonal to the extending direction of the road on which the vehicle is traveling to a value larger than the offset distance in the case where it is determined that there is no possibility that the bicycle exhibits a wobbling behavior, and includes instructions for causing a processor mounted on the vehicle to control the traveling of the vehicle to be separated from the bicycle by the set offset distance or more.

Effect of the Invention

[0015] The vehicle control device according to the present disclosure has an effect that an appropriate lateral offset distance can be set for a bicycle traveling around the vehicle.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0017] Hereinafter, with reference to the drawings, a vehicle control device, a vehicle control method and a vehicle control computer program implemented in the vehicle control device will be described. This vehicle control device detects a bicycle traveling around the vehicle based on an image representing the surroundings of the vehicle, and determines whether there is a possibility that the bicycle represented in the image shows a wobbling behavior. And when it is determined that there is a possibility that the bicycle shows a wobbling behavior, this vehicle control device sets an offset distance with respect to the bicycle in a direction orthogonal to the extending direction of the road on which the vehicle is traveling (hereinafter referred to as the lateral direction) to a value larger than the offset distance in the case where it is determined that there is no possibility that the bicycle shows a wobbling behavior. Thereby, this vehicle control device appropriately sets the offset distance in the lateral direction also for a bicycle that does not actually show a wobbling behavior.

[0018] In the present embodiment, the bicycle is not limited to a two-wheeled vehicle, and may be a bicycle with training wheels for a child to ride, or a tricycle for seniors or children. Also, the bicycle may be a bicycle with auxiliary power obtained by a motor or the like.

[0019] FIG. 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. FIG. 2 is a hardware configuration diagram of an electronic control unit which is an embodiment of the vehicle control device. In the present embodiment, a vehicle control system 1 mounted on a vehicle 10 and controlling the vehicle 10 includes a camera 2, a GPS receiver 3, a storage device 4, and an electronic control unit (ECU) 5 which is an example of the vehicle control device. The camera 2, the GPS receiver 3, the storage device 4, and the ECU 5 are communicably connected via an in-vehicle network conforming to a standard such as a controller area network. Note that the vehicle 10 is an example of the host vehicle. Further, the vehicle control system 1 may further include a distance measuring sensor (not shown) such as LiDAR or radar that measures the distance from the vehicle 10 to an object existing around the vehicle 10.

[0020] The camera 2 generates an image representing the surroundings of the vehicle 10. The camera 2 is attached inside the vehicle 10 so as to face a predetermined direction, for example, the front of the vehicle 10. Then, the camera 2 captures an area around the vehicle 10, for example, the front area of the vehicle 10, at a predetermined imaging period (for example, 1 / 30 second to 1 / 10 second), and generates an image representing the front area. The image obtained by the camera 2 may be a color image or a gray image. Note that the vehicle 10 may be provided with a plurality of cameras having different imaging directions or focal lengths.

[0021] Each time the camera 2 generates an image, the generated image is output to the ECU 5 via the in-vehicle network.

[0022] The GPS receiver 3 receives GPS signals from GPS satellites at a predetermined period, and measures the self-position of the vehicle 10 based on the received GPS signals. Then, the GPS receiver 3 outputs positioning information representing the measurement result of the self-position of the vehicle 10 based on the GPS signals to the ECU 5 via the in-vehicle network at a predetermined period. Note that the vehicle 10 may have a receiver conforming to a satellite positioning system other than the GPS receiver 3. In this case, the receiver may measure the self-position of the vehicle 10.

[0023] The storage device 4 is an example of a storage unit, and has, for example, a hard disk drive, a non-volatile semiconductor memory, or an optical recording medium and its access device. And the storage device 4 stores map information. The map information includes, for example, information representing road markings such as lane dividing lines or stop lines for individual road sections included in a predetermined area represented by the map information, information representing road signs, and information representing features around the road. Further, the map information may include information representing the curvature and gradient of individual road sections.

[0024] Furthermore, the storage device 4 may have a processor for executing processes such as update processing of map information and processing related to a read request for map information from the ECU 5. The storage device 4, for example, every time the vehicle 10 moves a predetermined distance, transmits a request for acquiring map information to a map server together with the current position of the vehicle 10 via a wireless communication terminal (not shown) mounted on the vehicle 10. And the storage device 4 receives map information for a predetermined area around the current position of the vehicle 10 from the map server via the wireless communication terminal. Also, when the storage device 4 receives a read request for map information from the ECU 5, it cuts out a range relatively narrower than the above-mentioned predetermined area including the current position of the vehicle 10 from the stored map information and outputs it to the ECU 5 via the in-vehicle network.

[0025] The ECU 5 controls the running of the vehicle 10 according to a predetermined level of autonomous driving. Note that the predetermined level of autonomous driving can be any one of the levels 1 or higher defined by the Society of Automotive Engineers (SAE).

[0026] As shown in FIG. 2, the ECU 5 has a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may each be configured as separate circuits, or may be integrally configured as one integrated circuit.

[0027] The communication interface 21 has an interface circuit for connecting the ECU 5 to the in-vehicle network. And every time the communication interface 21 receives an image from the camera 2, it transfers the received image to the processor 23. Also, every time the communication interface 21 receives positioning information from the GPS receiver 3, it transfers the positioning information to the processor 23. Furthermore, the communication interface 21 transfers the map information read from the storage device 4 to the processor 23.

[0028] The memory 22 is another example of the storage unit and has, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. And the memory 22 stores various data used in the vehicle control process executed by the processor 23 of the ECU 5. For example, the memory 22 stores parameters of the camera 2 such as focal length, angle of view, shooting direction, and mounting position, map information, and a parameter set for identifying various discriminators used for detecting a bicycle traveling around the vehicle 10 and determining wobbling. Furthermore, the memory 22 temporarily stores the images generated by the camera 2 and the positioning results of the self-position by the GPS receiver 3. Still further, the memory 22 temporarily stores various data generated during the vehicle control process.

[0029] The processor 23 has one or more CPUs (Central Processing Units) and its peripheral circuits. The processor 23 may further have other arithmetic circuits such as a logical operation unit, a numerical operation unit, or a graphic processing unit. And the processor 23 executes a vehicle control process for the vehicle 10.

[0030] Figure 3 is a functional block diagram of the processor 23 regarding the vehicle control process. The processor 23 has a determination unit 31, an offset setting unit 32, and a vehicle control unit 33. Each of these units of the processor 23 is, for example, a functional module realized by a computer program operating on the processor 23. Alternatively, each of these units of the processor 23 may be a dedicated arithmetic circuit provided in the processor 23.

[0031] The determination unit 31 detects a bicycle traveling around the vehicle 10 based on an image representing the surroundings of the vehicle 10 generated by the camera 2. Then, the determination unit 31 determines whether there is a possibility that the detected bicycle shown in the image exhibits a wobbling behavior.

[0032] In the present embodiment, each time the ECU 5 acquires an image from the camera 2, the determination unit 31 inputs the image to the identifier to detect a bicycle traveling around the vehicle 10. The identifier for bicycle detection is an example of the first identifier. The identifier for bicycle detection can be, for example, a deep neural network (DNN) having a convolutional neural network (CNN) type architecture such as Single Shot MultiBox Detector (SSD) or Faster R-CNN. Alternatively, the identifier for bicycle detection may be a DNN having an attention mechanism such as Vision Transformer. Alternatively, the identifier for bicycle detection may be an identifier based on a machine learning algorithm other than DNN, such as a support vector machine or an adaBoost identifier. Such an identifier is pre-trained according to a predetermined learning algorithm such as the error backpropagation method using a large number of teacher images including images representing bicycles so as to detect bicycles from images. The identifier outputs information specifying an object region representing the detected bicycle on the input image. The object region is, for example, an outer circumscribing rectangle of the bicycle shown on the image.

[0033] When the determination unit 31 detects a bicycle from an image, it inputs the object region representing the bicycle into a discriminator for determining whether the object region has a possibility of showing a behavior in which the bicycle wobbles (hereinafter sometimes simply referred to as wobbling). The determination unit 31 may upsample or downsample the object region so that the object region representing the bicycle has a predetermined size and then input it into the discriminator for wobbling determination. Thereby, the configuration of the discriminator for wobbling determination is simplified. When the object region representing the bicycle is input, the discriminator for wobbling determination outputs the reliability that the bicycle represented by the object region shows a wobbling behavior. Then, when the reliability is higher than a predetermined threshold (for example, 0.7 to 0.9), the determination unit 31 determines that there is a possibility that the bicycle shows a wobbling behavior, and when the reliability is equal to or lower than the threshold, the determination unit 31 determines that there is no possibility that the bicycle shows a wobbling behavior.

[0034] The discriminator for wobbling determination is an example of the second discriminator. For example, it can be a CNN-type DNN having, in order from the input side, one or more convolutional layers, one or more fully connected layers, and an output layer that performs a softmax operation or a sigmoid operation. Alternatively, the discriminator for wobbling determination may be a DNN having an attention mechanism or a discriminator based on another machine learning algorithm.

[0035] The wobbling discriminator is pre-trained according to a predetermined learning algorithm using a large number of teacher images representing various bicycles so as to output a reliability indicating the behavior of the bicycle wobbling based on the characteristics of the bicycle or the characteristics of the bicycle rider represented in the object area. For example, if the bicycle rider is an elderly person or a child, the rider may not be able to operate the bicycle so as not to wobble. Therefore, the wobbling discriminator is pre-trained so that the reliability indicating the wobbling behavior is high for a bicycle with an elderly or child rider. Also, when the bicycle is a children's bicycle, there is a possibility that the bicycle will exhibit a wobbling behavior because the rider is a child. Furthermore, even for a bicycle with an infant seat attached, there is a possibility that the bicycle will exhibit a wobbling behavior due to the difficulty of handling the handlebars. Therefore, the wobbling discriminator is also pre-trained so that the reliability indicating the wobbling behavior is high for a children's bicycle or a bicycle with an infant seat attached. On the other hand, for a bicycle with an adult rider who is not an elderly person and without an infant seat, the possibility of exhibiting a wobbling behavior is low. Therefore, the wobbling discriminator is pre-trained so that the reliability indicating the wobbling behavior is low for such a bicycle.

[0036] In addition, when multiple bicycles are detected from one image, the determination unit 31 inputs the object area representing the detected bicycle into the wobbling discriminator for each detected bicycle, thereby determining whether each detected bicycle may exhibit a wobbling behavior. Alternatively, the wobbling discriminator may be configured to be able to input multiple object areas simultaneously. For example, the wobbling discriminator may be configured such that different object areas are input for each channel. In this case, by simultaneously inputting a plurality of object areas representing the respectively detected bicycles into the wobbling discriminator, the wobbling discriminator outputs the reliability indicating that any of the bicycles exhibits a wobbling behavior. In this case, the wobbling discriminator may be pre-trained so that it is likely to be determined that there is a possibility that one of two or more bicycles accompanied by a parent and child exhibits a wobbling behavior.

[0037] According to a modification example, one discriminator may be configured to detect a bicycle from an image and calculate a reliability indicating a behavior in which the detected bicycle wobbles. The discriminator according to this modification example is an example of a third discriminator, and can be a discriminator based on a CNN type or a DNN having an attention mechanism, or another machine learning algorithm. In this case, the discriminator outputs an object region representing the detected bicycle and a reliability indicating a wobbling behavior obtained for the object region. Also in this case, the determination unit 31 may determine whether there is a possibility that the bicycle represented by the object region shows a wobbling behavior by comparing the reliability for the object region with a threshold value for each object region.

[0038] The determination unit 31 notifies the offset setting unit 32 of a determination result as to whether there is a possibility that the detected bicycle shows a wobbling behavior and a position on the image of the object region representing the bicycle.

[0039] The offset setting unit 32 sets a lateral offset distance with respect to the detected bicycle. In the present embodiment, the offset setting unit 32 sets the offset distance when it is determined that there is a possibility that the detected bicycle shows a wobbling behavior to a value larger than the offset distance when it is determined that there is no possibility that the detected bicycle shows a wobbling behavior.

[0040] For example, when there is a dedicated bicycle lane on the road where the vehicle 10 is traveling, the bicycle is expected to travel within the dedicated bicycle lane in principle. Therefore, in this case, the offset setting unit 32 sets the approaching limit position to the bicycle at a position deviated from the bicycle lane side by the offset distance with reference to the position of the bicycle. The offset setting unit 32 refers to the latest position of the vehicle 10 measured by the GPS receiver 3 and the map information to determine whether there is a dedicated bicycle lane on the road where the vehicle 10 is traveling. Alternatively, the offset setting unit 32 may determine whether there is a dedicated bicycle lane on the road where the vehicle 10 is traveling by inputting the image generated by the camera 2 into a discriminator that has been pre-trained to detect the dedicated bicycle lane. In this case, as the discriminator, for example, a DNN for semantic segmentation such as a Fully Convolutional Network (FCN) or a U-Net can be used by the offset setting unit 32.

[0041] Furthermore, the offset setting unit 32 estimates the detected position of the bicycle. Here, each pixel on the image generated by the camera 2 corresponds one-to-one to the orientation from the camera 2. Also, the lower end of the object area representing the bicycle is presumed to represent the position where the bicycle touches the road surface. Therefore, the offset setting unit 32 can estimate the distance and direction to the bicycle with reference to the position of the camera 2 at the time of image generation based on the position of the lower end of the object area representing the bicycle on the image and the parameters of the camera 2 such as the installation height, focal length, and shooting direction of the camera 2. When the vehicle 10 is provided with a ranging sensor, the offset setting unit 32 may estimate the distance measured by the ranging sensor in the orientation corresponding to the object area representing the bicycle as the distance to the bicycle. Furthermore, the offset setting unit 32 can estimate the position of the bicycle in the world coordinate system based on the distance and direction to the bicycle with reference to the position of the camera 2 and the position and traveling direction of the vehicle 10 at the time of image generation. Therefore, in order to detect the accurate position and traveling direction of the vehicle 10, the offset setting unit 32 collates the image generated by the camera 2 with the map information. For example, the offset setting unit 32 projects the ground features on or around the road detected from the image onto the map information assuming the position and orientation of the vehicle 10, or projects the ground features on or around the road around the vehicle 10 represented in the map information onto the image. The ground features on or around the road can be, for example, road markings such as lane dividers or stop lines, curbs, or various road signs. Then, the offset setting unit 32 detects the position and orientation of the vehicle 10 when the ground features detected from the image and the ground features represented in the map information match the most as the accurate position and traveling direction of the vehicle 10. Furthermore, the offset setting unit 32 detects the lane including the position of the vehicle 10 as the own lane in which the vehicle 10 travels.

[0042] The offset setting unit 32 may determine the position on the map or in the image where the ground object is projected by using the assumed position and orientation of the vehicle 10 and the parameters of the camera 2 such as the focal length, the installation height, and the shooting direction. Then, the offset setting unit 32 calculates the degree of coincidence (for example, the reciprocal of the sum of the squares of the distances between the corresponding ground objects) between the ground objects on the road or around the road detected from the image and the corresponding ground objects represented on the map.

[0043] The offset setting unit 32 repeats the above process while changing the assumed position and orientation of the vehicle 10. Then, the offset setting unit 32 may detect the assumed position and orientation when the degree of coincidence is maximized as the accurate position and traveling direction of the vehicle 10.

[0044] Note that the offset setting unit 32 may detect the ground object by inputting the image to an identifier that has been pre-learned to detect the ground object to be detected from the image. As such an identifier, the offset setting unit 32 can use an identifier similar to the identifier used for detecting a bicycle or the identifier used for detecting a bicycle-only lane.

[0045] When the position of the bicycle is estimated, the offset setting unit 32 sets, as the approach limit position to the bicycle, a position that is offset by the offset distance toward the center side of the own lane in the lateral direction with reference to the estimated position. As described above, the offset distance in the case where it is determined that there is a possibility that the bicycle shows a wobbling behavior is set to a larger value (for example, 1.5 m to 2 m) than the offset distance (for example, 1 m to 1.5 m) in the case where it is determined that there is no possibility that the bicycle shows a wobbling behavior.

[0046] In addition, even when there is no dedicated bicycle lane on the road where the vehicle 10 is traveling, the offset setting unit 32 may set the approach limit position at a position deviated from the bicycle position by the offset distance toward the own lane side with reference to the position of the bicycle. However, when a dedicated bicycle lane is provided at a position within a predetermined distance along the road from the current position of the bicycle, or when the dedicated bicycle lane disappears at a position within a predetermined distance along the road from the current position of the bicycle, the position of the bicycle in the lateral direction may vary. Therefore, when a change point where the presence or absence of the dedicated bicycle lane changes exists within a predetermined distance from the current position of the bicycle, the offset setting unit 32 may set the approach limit position at a position deviated from the lane dividing line that divides the own lane and is located on the bicycle side by a distance obtained by adding a predetermined correction distance (for example, 0.5 m) to the offset distance toward the own lane side. The position of the lane dividing line of the road on which the vehicle 10 is traveling may be specified by referring to the position of the vehicle 10 shown in the map information and the latest positioning information.

[0047] Further, when no bicycle is detected around the vehicle 10, the offset setting unit 32 may set the approach limit position to the position of the lane dividing line that divides the own lane itself or a position deviated from the lane dividing line toward the center side of the own lane by a predetermined distance (for example, 0.1 to 0.5 m) so that the vehicle 10 can travel along the center of the own lane.

[0048] The offset setting unit 32 notifies the set approach limit position to the vehicle control unit 33.

[0049] The vehicle control unit 33 controls the running of the vehicle 10 so as to move away from the bicycle by a distance equal to or greater than the set offset distance. To this end, the vehicle control unit 33 sets a planned travel trajectory along the own lane so as to move away from the bicycle by a distance greater than the approaching limit position notified from the offset setting unit 32 in a predetermined section before and after the position of the bicycle in the extending direction of the road on which the vehicle 10 is running. The predetermined section is set to be equal to or greater than the distance required for the vehicle 10 to overtake the bicycle according to the relative speed and distance between the bicycle and the vehicle 10. At this time, the vehicle control unit 33 can obtain the relative speed between the bicycle and the vehicle 10 based on the change in the distance between the camera 2 and the bicycle at the time of generating each of the plurality of images by the camera 2. The distance between the camera 2 and the bicycle is obtained by the same method as described in the offset setting unit 32. Then, the vehicle control unit 33 controls each part of the vehicle 10 so that the vehicle 10 runs along the planned travel trajectory.

[0050] When a bicycle is detected, the vehicle control unit 33 sets a planned travel trajectory according to a trajectory generation model obtained by learning the trajectory of the vehicle 10 when overtaking the bicycle during past manual driving by the driver. The trajectory generation model is generated, for example, as the average of a plurality of trajectories from the time when the distance to the bicycle reaches a predetermined distance until the vehicle 10 overtakes the bicycle and travels a predetermined distance. Alternatively, the trajectory generation model may be composed of a DNN pre-trained to output a planned travel trajectory. In this case, the vehicle control unit 33 generates a planned travel trajectory by inputting the distance between the vehicle 10 and the bicycle and the positions of the vehicle 10 and the bicycle into the trajectory generation model. Then, in the generated planned travel trajectory, the vehicle control unit 33 corrects the planned travel trajectory so that the vehicle 10 moves away from the bicycle by a distance greater than the approaching limit position in the above-mentioned predetermined section.

[0051] When the vehicle control unit 33 sets a planned travel route, it controls each part of the vehicle 10 so that the vehicle 10 travels along the planned travel route. To this end, based on the planned travel route and the current position of the vehicle 10, the vehicle control unit 33 obtains the steering angle of the vehicle 10 for the vehicle 10 to travel along the planned travel route, and outputs a control signal corresponding to the steering angle to an actuator (not shown) that controls the steering of the vehicle 10. At this time, if the current position of the vehicle 10 is on the planned travel route, the vehicle control unit 33 determines the steering angle so as to follow the planned travel route. Also, if the current position of the vehicle 10 is away from the planned travel route, the vehicle control unit 33 determines the steering angle so as to approach the planned travel route. Note that the vehicle control unit 33 obtains the position and traveling direction of the vehicle 10 at the time of generating the latest image by the same method as described in the offset setting unit 32. Then, the vehicle control unit 33 can estimate the current position of the vehicle 10 by correcting the position and traveling direction of the vehicle 10 at the time of generating the image using the acceleration and yaw rate of the vehicle 10 from the time of generating the image to the current time.

[0052] Also, when the level of autonomous driving applied to the vehicle 10 is a level corresponding to driving support in which the driver generally operates the steering, when the position of the vehicle 10 in the lateral direction approaches the bicycle closer than the approach limit position, the vehicle control unit 33 may assist the driver's driving by controlling the steering so as to be separated from the bicycle by more than the offset distance.

[0053] Furthermore, when the level of autonomous driving applied to the vehicle 10 is a level that also controls the speed of the vehicle 10, the vehicle control unit 33 may control the speed of the vehicle 10 according to whether there is a possibility that the bicycle shows a wobbling behavior. For example, when it is determined that there is a possibility that the bicycle shows a wobbling behavior, the vehicle control unit 33 may decelerate the vehicle 10 so that the speed of the vehicle 10 becomes equal to or lower than a predetermined speed (for example, 20 km / h to 30 km / h). Furthermore, when the vehicle 10 is in front of the bicycle, the vehicle control unit 33 may accelerate the vehicle 10.

[0054] Figures 4(a) and 4(b) are diagrams each showing an example of setting the offset distance. In the example shown in Figure 4(a), an adult is riding the bicycle 401, and it is determined that the bicycle 401 is not likely to exhibit a wobbling behavior. Therefore, the lateral offset distance is set to a relatively small value OD1, and the approach limit position AL is set at a position offset by the offset distance OD1 from the bicycle 401. Then, the vehicle 10 is controlled to travel at a position farther from the bicycle 401 than the approach limit position AL.

[0055] On the other hand, in the example shown in Figure 4(b), since a child is riding the bicycle 402, it is determined that the bicycle 402 may exhibit a wobbling behavior. Therefore, the lateral offset distance is set to a value OD2 larger than the offset distance OD1 shown in Figure 4(a), and the approach limit position AL is set at a position offset by the offset distance OD2 from the bicycle 402. Accordingly, the vehicle 10 is controlled to travel at a position relatively far from the bicycle 402.

[0056] Figure 5 is an operation flowchart of vehicle control processing executed by the processor 23. The processor 23 may execute the vehicle control processing according to the following operation flowchart at a predetermined cycle.

[0057] The determination unit 31 of the processor 23 determines whether a bicycle traveling around the vehicle 10 has been detected (step S101). When a bicycle is detected (step S101 - Yes), the determination unit 31 determines whether the bicycle may exhibit a wobbling behavior (step S102).

[0058] When the bicycle may exhibit a wobbling behavior (step S102 - Yes), the offset setting unit 32 of the processor 23 sets the lateral offset distance with respect to the bicycle relatively large (step S103). On the other hand, when the bicycle is not likely to exhibit a wobbling behavior (step S102 - No), the offset setting unit 32 sets the lateral offset distance with respect to the bicycle relatively small (step S104).

[0059] After step S103 or S104, the vehicle control unit 33 of the processor 23 controls the running of the vehicle 10 so as to be separated from the bicycle by a distance equal to or greater than the offset distance (step S105). Also, when the bicycle is not detected in step S101 (step S101 - No), the vehicle control unit 33 controls the vehicle 10 so that the vehicle 10 travels in its own lane (step S106). After step S105 or S106, the processor 23 ends the vehicle control process.

[0060] As described above, since this vehicle control device determines whether there is a possibility that the bicycle shows a swaying behavior from the image, the lateral offset distance can be appropriately set even at a timing when the bicycle is not actually showing a swaying behavior.

[0061] According to a modification, the offset setting unit 32 may correct the offset distance according to the terrain of the road on which the vehicle 10 is running. For example, when the road on which the vehicle 10 is running is a slope, especially an uphill slope, the possibility that a bicycle running around the vehicle 10 sways increases. Therefore, when the road on which the vehicle 10 is running is a slope, the offset setting unit 32 corrects the offset distance set according to the possibility of swaying behavior to be further lengthened by a predetermined distance (for example, 0.3 m to 0.6 m). The offset setting unit 32 may determine whether the road on which the vehicle 10 is running is a slope by referring to the latest position of the vehicle 10 measured by the GPS receiver 3 and the information on the gradient of the road included in the map information.

[0062] Similarly, when the road on which the vehicle 10 is traveling is curved, the offset setting unit 32 may further correct the offset distance set according to the possibility of showing a wobbling behavior by increasing it by a predetermined distance. Also in this case, the offset setting unit 32 may determine whether the road on which the vehicle 10 is traveling is curved by referring to the latest position of the vehicle 10 measured by the GPS receiver 3 and the information regarding the curvature of the road included in the map information. Alternatively, the offset setting unit 32 may detect the lane dividing lines from the image generated by the camera 2, and when the detected lane dividing lines can be approximated by a curve, it may determine that the road on which the vehicle 10 is traveling is curved.

[0063] In addition, the discriminator for wobbling determination used in the determination unit 31 may be configured such that, together with the object region representing the detected bicycle, terrain information representing the terrain of the road on which the vehicle 10 is traveling is input. For example, the terrain information is represented as a matrix or vector having different values according to the terrain such as a curve, a straight line, an uphill slope, or flat, and is input to the discriminator for wobbling determination as a channel different from the channel to which the object region is input. Then, the discriminator for wobbling determination is pre-trained to output the reliability of showing a wobbling behavior of the bicycle in consideration of the terrain information by having a layer that performs a fully connected operation between the channel to which the object region is input and the channel to which the terrain information is input. Thereby, when the terrain of the road on which the vehicle 10 is traveling is a terrain (for example, an uphill slope or a curve) that is likely to cause wobbling of the bicycle, it becomes easier to determine that there is a possibility that the bicycle shows a wobbling behavior.

[0064] The computer program for realizing the function of the processor 23 of the ECU 5 according to the above-described embodiment or modification example may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium.

[0065] As described above, those skilled in the art can make various changes according to the implemented form within the scope of the present invention.

Explanation of Reference Numerals

[0066] 1 Vehicle control system 10 Vehicle 2 Camera 3 GPS receiver 4 Storage device 5 Electronic control unit (ECU) 21 Communication interface 22 Memory 23 Processor 31 Judgment unit 32 Offset setting unit 33 Vehicle control unit

Claims

1. Based on an image representing the surroundings of a vehicle, a bicycle traveling around the vehicle is detected, and based on an object area representing the bicycle detected on the image, it is determined whether there is a possibility that the bicycle exhibits a wobbling behavior; a determination unit; When it is determined that there is a possibility that the bicycle exhibits a wobbling behavior, an offset distance with respect to the bicycle in a direction orthogonal to the extending direction of the road on which the vehicle is traveling is set to a value larger than the offset distance when it is determined that there is no possibility that the bicycle exhibits a wobbling behavior; an offset setting unit; A control unit that controls the travel of the vehicle so as to be separated from the bicycle by a distance equal to or greater than the set offset distance; A vehicle control device having the above.

2. The determination unit inputs the image to a first discriminator pre-trained to detect the bicycle from the image, thereby detecting the object area representing the bicycle on the image, and inputs the object area to a second discriminator pre-trained to determine the possibility that the bicycle exhibits a wobbling behavior, thereby determining whether there is a possibility that the bicycle exhibits a wobbling behavior. The vehicle control device according to claim 1.

3. The determination unit inputs information representing the terrain of the road on which the vehicle is traveling together with the object area to the second discriminator, thereby determining whether there is a possibility that the bicycle exhibits a wobbling behavior. The vehicle control device according to claim 2.

4. The determination unit inputs the image to a third discriminator pre-trained to detect the bicycle represented on the image and to determine the possibility that the bicycle exhibits a wobbling behavior, thereby detecting the object area representing the bicycle on the image and determining whether there is a possibility that the bicycle exhibits a wobbling behavior. The vehicle control device according to claim 1.

5. The offset setting unit corrects the offset distance based on the terrain of the road on which the vehicle is traveling. The vehicle control device according to claim 1 or 2.

6. When the road on which the vehicle is traveling is a slope or a curve, the offset setting unit corrects the offset distance so as to be longer by a predetermined distance. The vehicle control device according to claim 5.

7. Based on an image representing the surroundings of a vehicle, a bicycle traveling around the vehicle is detected, and based on an object area representing the bicycle detected on the image, it is determined whether the bicycle is likely to exhibit a wobbling behavior. When it is determined that the bicycle is likely to exhibit a wobbling behavior, the offset distance with respect to the bicycle in a direction orthogonal to the extending direction of the road on which the vehicle is traveling is set to a value larger than the offset distance when it is determined that the bicycle is not likely to exhibit a wobbling behavior. The vehicle travel is controlled to be separated from the bicycle by at least the set offset distance. A vehicle control method including this.

8. Based on an image representing the surroundings of a vehicle, a bicycle traveling around the vehicle is detected, and based on an object area representing the bicycle detected on the image, it is determined whether the bicycle is likely to exhibit a wobbling behavior. When it is determined that the bicycle is likely to exhibit a wobbling behavior, the offset distance with respect to the bicycle in a direction orthogonal to the extending direction of the road on which the vehicle is traveling is set to a value larger than the offset distance when it is determined that the bicycle is not likely to exhibit a wobbling behavior. The vehicle travel is controlled to be separated from the bicycle by at least the set offset distance. A vehicle control computer program for causing a processor mounted on the vehicle to execute this.

Images