Driving assistance device and program

Abstract

To provide a driving support device and a program capable of appropriately recognizing a parking frame even when a road surface is irradiated with light.SOLUTION: The driving support device 10 recognizes the parking frame 40 based on the image data captured by the vehicle-mounted camera 20 in a situation where the light pattern 50 representing the movement of the vehicle is irradiated on the road surface around the vehicle 100. The light pattern 50 is irradiated so as to blink at a predetermined blinking cycle. The driving support device 10 includes the period identification unit 14 configured to acquire the image data of the road surface captured over a plurality of frames from the vehicle-mounted camera 20, compare the image data of each frame, and identify the non-irradiation period in which the light pattern 50 is not irradiated from the difference between the frames, and the recognition unit 15 configured to recognize the parking frame 40 based on the image data captured in the identified non-irradiation period.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a driving assistance device and a program. [Background technology]

[0002] There is a known technology that projects animations of light onto the road surface around a vehicle, representing the vehicle's movements. This allows people outside the vehicle to intuitively understand the vehicle's upcoming movements. Such a technology is described, for example, in Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2016 / 027315 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, when automatically parking a vehicle in a parking space, if a light animation is projected onto the road surface, there is a possibility that the light animation may be confused with a parking space during image recognition.

[0005] The present invention has been made in consideration of the above circumstances, and a main object of the present invention is to provide a driving assistance device and a program that can properly recognize a parking space even when light is irradiated onto the road surface. [Means for solving the problem]

[0006] A first driving assistance device for solving the above problem is a driving assistance device that recognizes a parking space based on image data captured by a camera unit in a situation where a light pattern that represents the movement of the vehicle is irradiated onto the road surface around the vehicle, the light pattern is irradiated so as to blink at a predetermined blinking cycle, and the device is equipped with a period identification unit that acquires image data of the road surface captured over multiple frames from the camera unit, compares the image data of each frame, and identifies a non-illumination period when the light pattern is not irradiated from the differences between them, and a recognition unit that recognizes the parking space based on image data captured by the camera unit during the non-illumination period identified by the period identification unit.

[0007] According to the above configuration, the parking space is recognized based on the image data captured during the non-illumination period, so that it is possible to prevent erroneous recognition with a light pattern.

[0008] A second driving assistance device for solving the above problem is a driving assistance device that recognizes a parking space based on image data captured by a camera unit in a situation where a light pattern representing the movement of the vehicle is irradiated onto a road surface around the vehicle, the light pattern is irradiated so as to blink at a predetermined blinking cycle, and the device includes: an illumination control unit that estimates the movement of the vehicle and controls the illumination unit to irradiate the light pattern representing the estimated movement of the vehicle; a period determination unit that inputs a non-illumination timing at which the light pattern is not irradiated from the illumination control unit and determines a non-illumination period during which the light pattern is not irradiated; and a recognition unit that recognizes the parking space based on image data captured by the camera unit during the non-illumination period determined by the period determination unit.

[0009] According to the above configuration, the parking space is recognized based on the image data captured during the non-illumination period, so that it is possible to prevent erroneous recognition with a light pattern.

[0010] A first program for solving the above problem is a program implemented by a driving assistance device that recognizes a parking space based on image data captured by a camera unit in a situation where a light pattern representing the movement of the vehicle is irradiated onto the road surface around the vehicle, wherein the light pattern is irradiated so as to blink at a predetermined blinking cycle, and the driving assistance device is caused to perform a period identification step of acquiring image data of the road surface captured over multiple frames from the camera unit, comparing the image data of each frame, and identifying a non-illumination period in which the light pattern is not irradiated from the differences therebetween, and a recognition step of recognizing the parking space based on image data captured by the camera unit during the non-illumination period identified by the period identification step.

[0011] According to the above configuration, the parking space is recognized based on the image data captured during the non-illumination period, so that it is possible to prevent erroneous recognition with a light pattern.

[0012] A second program for solving the above problem is a program implemented by a driving assistance device that recognizes a parking space based on image data captured by a camera unit in a situation where a light pattern representing the movement of the vehicle is projected onto a road surface around the vehicle, the light pattern being projected so as to blink at a predetermined blinking cycle, and causes the driving assistance device to carry out an illumination control step of estimating the movement of the vehicle and controlling the illumination unit to illuminate the light pattern representing the estimated movement of the vehicle, a period specification step of inputting a non-illumination timing at which the illumination unit does not project the light pattern and specifying a non-illumination period during which the light pattern is not projected, and a recognition step of recognizing the parking space based on image data captured by the camera unit during the non-illumination period specified in the period specification step.

[0013] According to the above configuration, the parking space is recognized based on the image data captured during the non-illumination period, so that it is possible to prevent erroneous recognition with a light pattern. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing the configuration of a driving assistance device. [Figure 2] FIG. 10 is a plan view schematically showing irradiation of a light pattern. [Figure 3] 10 is a timing chart showing a non-irradiation period. [Figure 4] 10 is a flowchart showing the flow of a parking process. [Figure 5] FIG. 10 is a plan view schematically showing the movement of a vehicle and the irradiation of a light pattern during parking. [Figure 6] FIG. 10 is a diagram showing the configuration of a driving assistance device according to a second embodiment. [Figure 7] 10 is a flowchart showing the flow of parking processing in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of a driving assistance device and a program according to the present disclosure will be described in detail with reference to the drawings. Note that, between the embodiments and modifications, the same or equivalent parts in the drawings are designated by the same reference numerals, and their description will not be repeated in principle.

[0016] (First embodiment) 1, a vehicle 100, which is the vehicle itself, is equipped with a driving assistance device 10 that performs driving assistance for the vehicle 100. The driving assistance device 10 includes an image recognition device 11 that performs image recognition on image data input from an on-board camera 20 serving as a camera unit, a vehicle control device 12 serving as a vehicle control unit that controls the driving of the vehicle, and an illumination control device 13 serving as an illumination control unit that controls illumination of an illumination unit 22.

[0017] The vehicle-mounted camera 20 captures images of the surroundings of the vehicle 100 and may be a monocular camera or a compound camera. The vehicle-mounted camera 20 captures video at a predetermined frame rate, generates image data, and inputs it to the image recognition device 11. In other words, the vehicle-mounted camera 20 generates image data related to a video (image) consisting of a plurality of frames (still images). The number, position, and type of the vehicle-mounted cameras 20 may be changed as desired. In this embodiment, the vehicle is equipped with a front camera 21a whose imaging range is in front of the vehicle 100, a right-side camera 21b whose imaging range is the right side of the vehicle 100, a left-side camera 21c whose imaging range is the left side of the vehicle 100, and a rear camera 21d whose imaging range is the rear of the vehicle.

[0018] The image recognition device 11 is primarily composed of a microcomputer equipped with a processing unit such as a CPU and storage devices such as various memories. The functions provided by the microcomputer can be provided by software stored in a physical memory device and a computer that executes the software, software alone, hardware alone, or a combination thereof. For example, when the microcomputer is provided by hardware electronic circuits, the functions can be provided by digital circuits including numerous logic circuits or analog circuits. For example, the microcomputer executes programs stored in its own non-transitory tangible storage medium. The programs include, for example, a program that realizes an image recognition function for recognizing the surrounding conditions of the vehicle 100 based on image data input from the onboard camera 20. Execution of the programs results in the execution of a method corresponding to the programs. The non-transitory storage medium is, for example, a non-volatile memory. The programs stored in the non-transitory storage medium can be downloaded and updated via a communication network such as the Internet, for example, via OTA (Over The Air) or other means.

[0019] Like the image recognition device 11, the vehicle control device 12 is mainly configured with a microcomputer. The vehicle control device 12 has various functions related to drive control of the vehicle 100, and these functions control the actuator 30 based on sensor information input from outside the vehicle control device 12. The sensor information includes image data input from the image recognition device 11, etc. The sensor information also includes information from various sensors for measuring the vehicle's driving conditions, such as a vehicle speed sensor and a yaw rate sensor, and information from sensors for detecting other vehicles and obstacles, such as a millimeter-wave radar and a laser radar (LiDAR). The sensor information may also include various sensors for detecting various driver operation amounts, such as an accelerator sensor for detecting the driver's accelerator operation amount, a brake sensor for detecting the brake operation amount, and a steering angle sensor for detecting the driver's steering amount (steering angle) of the steering wheel.

[0020] A representative function among the various functions related to drive control of the vehicle 100 is, for example, an advanced parking assist function (APA). This advanced parking assist function is a type of driving assistance function. Note that the vehicle may also be provided with other vehicle control functions related to driving assistance. This vehicle control function is realized by the vehicle control device 12 executing a program stored in the storage device of the driving assistance device 10 or the vehicle control device 12. Note that this program may be downloadable and updatable via a communication network.

[0021] The actuators 30 include, for example, actuators for driving the vehicle 100, such as a motor that serves as the main engine of the vehicle 100. The actuators 30 also include actuators for controlling the behavior of the vehicle 100, such as an actuator for operating a steering wheel, an actuator for operating a brake pedal, and an actuator for operating an accelerator pedal. The actuators 30 may also include devices for operating accessories of the vehicle 100, such as a display, a speaker, an indicator, and a headlight. The actuators 30 control the traveling and operation of the vehicle 100. When the advanced parking assist function is implemented, the vehicle control device 12 controls the traveling of the vehicle 100 so as to park the vehicle 100 in a parking stall 40 by controlling the actuators 30.

[0022] The illumination control device 13, like the image recognition device 11, is mainly composed of a microcomputer. The illumination control device 13 has a road lighting function that controls the illumination unit 22 to illuminate (project) a light pattern 50 onto the road surface to display figures and the like. The illumination unit 22 consists of a forward illumination unit 22a provided at the front of the vehicle 100 and a rear illumination unit 22b provided at the rear of the vehicle 100. The forward illumination unit 22a illuminates the light pattern 50 onto the road surface in front of the vehicle 100 to display figures and the like, while the rear illumination unit 22b illuminates the light pattern 50 onto the road surface behind the vehicle 100 to display figures and the like. The forward illumination unit 22a and the rear illumination unit 22b are, for example, projection lamps or lasers. The forward illumination unit 22a and the rear illumination unit 22b are configured to illuminate using a dynamic lighting method. Therefore, the light pattern 50 blinks at a predetermined blinking cycle.

[0023] This road lighting function is realized by the illumination control device 13 executing a program stored in a storage device of the driving assistance device 10 or the illumination control device 13. This road lighting function is one of the functions related to driving assistance. Note that this program may be downloadable and updatable via a communication network.

[0024] The road surface lighting function may be executed together with the advanced parking assist function by the vehicle control device 12. Specifically, as shown in FIG. 2(b), when the vehicle 100 moves to park in the parking space 40, a light pattern 50 indicating the traveling direction of the vehicle 100 (in FIG. 2(b), the rear of the vehicle 100) is projected onto the road surface to represent the operation the vehicle 100 will perform. As described above, the light pattern 50 blinks, so that the state shown in FIG. 2(a) and the state shown in FIG. 2(b) alternate. The light pattern 50 shown in FIG. 2(b) is a light pattern configured by displaying two lines that spread out in a roughly fan-like shape toward the rear of the vehicle 100 and displaying five roughly wave-shaped lines between the two lines. By projecting the light pattern 50 indicating the traveling direction of the vehicle 100, people around the vehicle 100 can be aware of the movement of the vehicle 100 in advance, thereby improving safety when the advanced parking assist function is executed.

[0025] However, when the light pattern 50 is projected onto the road surface, there is a possibility that it may be erroneously recognized as a parking space 40 as shown in FIG. 2(b). Therefore, in this embodiment, in order to prevent erroneous recognition, the parking space 40 is recognized based on image data captured at a timing when there is no irradiation from the projection unit 22. This will be described in detail below.

[0026] The image recognition device 11 has a function as a period determination unit 14 and a function as a recognition unit 15. These functions are realized by the image recognition device 11 executing a program stored in the driving assistance device 10 or a storage device of the image recognition device 11.

[0027] The period determination unit 14 first acquires (inputs) image data of the road surface illuminated by the light pattern 50 from the in-vehicle camera 20. At this time, the image data generated by capturing multiple frames is acquired. For example, as shown in FIG. 3, image data (video data) consisting of multiple frames D0 to D5 is acquired. The frames D0 to D5 are acquired at frame acquisition times T0 to T5, respectively. It is desirable that the interval between the frame acquisition times T0 to T5, i.e., the frame period (the reciprocal of the frame rate), be faster than the blinking period of the light pattern 50.

[0028] The period determination unit 14 then compares the image data (still image data) of each frame D0 to D5 and, based on the differences between them, determines the non-irradiation period during which the light pattern 50 is not irradiated. More specifically, the period determination unit 14 performs pattern matching of the light pattern 50 irradiated from the irradiation unit 22 for each of the image data (still image data) of each frame D0 to D5. That is, for each of the image data (still image data) of each frame D0 to D5, it determines whether a light shape matching the shape of the light pattern 50 exists in the image data (still image data). The light pattern 50 irradiated from the irradiation unit 22 is pre-stored in a storage device of the image recognition device 11. As another example, the irradiated light pattern 50 may be input from the irradiation control device 13. In FIG. 3, it is determined that the light pattern 50 is included (present) in the image data of frames D0 and D3, and that the light pattern 50 is not included (absent) in the image data of frames D1, D2, D4, and D5.

[0029] The period determination unit 14 then compares the presence or absence of the light pattern 50 in the image data (still image data) of each of the frames D0 to D5, and identifies a non-irradiation period in which the light pattern 50 is not irradiated based on the difference in the presence or absence of the light pattern 50. In FIG. 3, since the light pattern 50 is not present in the image data of frames D1 and D2, the period determination unit 14 determines that at least the period from frame acquisition timing T1 to T2 is the non-irradiation period. Furthermore, since the light pattern 50 is not present in the image data of frames D4 and D5, the period determination unit 14 determines that at least the period from frame acquisition timing T4 to T5 is the non-irradiation period. As a result, the period determination unit 14 determines that the length of the non-irradiation period corresponds to the length from frame acquisition timing T1 to frame acquisition timing T2, and that the set interval (set cycle) of the non-irradiation period is the interval from frame acquisition timing T1 to frame acquisition timing T4. This enables the period determination unit 14 to determine the start and end points of the non-irradiation period.

[0030] Then, the period specifying unit 14 notifies the specified non-irradiation period to the recognition unit 15. Specifically, the period specifying unit 14 notifies the recognition unit 15 of the start and end times of the specified non-irradiation period.

[0031] The recognition unit 15 inputs image data captured by the in-vehicle camera 20 during the non-illumination period identified by the period identification unit 14. Then, the recognition unit 15 recognizes the parking space 40 based on the image data in which the light pattern 50 is not irradiated. In the example of FIG. 3, the parking space 40 is recognized based on the image data of frames D1 and D4. Note that the method for recognizing the parking space 40 itself utilizes well-known technology. The recognition unit 15 inputs the recognized parking space 40 to the vehicle control device 12.

[0032] Then, the vehicle control device 12 controls the actuator 30 to move the vehicle 100 into the parking space 40 recognized by the recognition unit 15 and park the vehicle 100 therein.

[0033] Next, a flow of parking the vehicle 100 in the parking space 40 while irradiating the light pattern 50 onto the road surface will be described. FIG. 4 is a flowchart showing the flow of the parking process. The parking process is performed by the driving assistance device 10. FIG. 5 is a diagram schematically showing the movement of the vehicle 100 and the irradiation of the light pattern 50 during parking. The parking process is started, for example, when the implementation of the advanced parking assist function is selected based on the operation of a selection button.

[0034] When the parking process starts, first, the image recognition device 11 recognizes a parking space 40 available for parking (i.e., a parking space 40 in which no other vehicle is parked) from the image data acquired from the vehicle-mounted camera 20 (step S101). As shown in FIG. 5(a), the road surface lighting function has not been started at the start of the parking process. Therefore, the parking space 40 is recognized by the normal image recognition method. In FIG. 5(a), the parking space 40 on the right side of the vehicle 100 is recognized.

[0035] The image recognition device 11 displays the recognized parking space 40 on the monitor of the vehicle 100 in a state where it can be selected by the driver (step S102). When the driver selects the target parking space 40 where the vehicle 100 is to be parked, the vehicle control device 12 causes the vehicle 100 to move forward and stop at a reverse start position corresponding to the selected target parking space 40 in order to reverse the vehicle 100 and park it in the target parking space 40 (step S103). In step S103, the vehicle control device 12 controls the various actuators 30 so as to cause the vehicle 100 to move forward to the reverse start position.

[0036] In Fig. 5(a), the target parking space 40 is on the right side of the vehicle 100, so the vehicle control device 12 controls the actuator 30 to move the vehicle 100 in the direction of the arrow (upper right direction in Fig. 5(a)) and stop the vehicle at the position shown in Fig. 5(b). Note that in step S103, the driver is prompted to select the target parking space 40, but the driving assistance device 10 may select it autonomously.

[0037] At the reverse start position, it is determined whether the shift position has been changed from drive "D" to reverse "R" (step S104). If the result of this determination is negative, step S104 is performed again after a predetermined time has elapsed, and the vehicle waits at the reverse start position.

[0038] If the determination result in step S104 is positive, the illumination control device 13 starts the road surface lighting function (step S105). That is, the illumination control device 13 estimates the traveling direction of the vehicle 100 and starts controlling the illumination unit 22 to illuminate the road surface with a light pattern 50 indicating the estimated traveling direction. The traveling direction of the vehicle 100 can be estimated based on the shift position, and in this case, it is estimated that the vehicle 100 will move backward. Note that the steering angle included in the sensor information may be input, and a more detailed traveling direction of the vehicle 100 may be estimated based on the shift position and the steering angle.

[0039] Then, the illumination control device 13 reads out the light pattern 50 corresponding to the estimated traveling direction from the storage device, and controls the illumination unit 22 to illuminate the light pattern 50. In the case of FIG. 5(b), the illumination control device 13 estimates that the vehicle 100 will move backward, and controls the rear illumination unit 22b to illuminate the road surface with the light pattern 50 indicating that the vehicle 100 is moving backward. Thereafter, the road surface lighting function is continuously performed until the vehicle 100 is parked. This step S105 corresponds to the illumination control step.

[0040] At this time, as described above, the irradiation unit 22 irradiates the light pattern 50 at a predetermined blinking cycle. At this time, the blinking cycle is very fast, and to the human eye, it appears as if the light pattern 50 is being continuously irradiated. Note that the blinking cycle may be slowed down so that the human eye can recognize that the light pattern 50 is blinking.

[0041] Next, the period determination unit 14 of the image recognition device 11 acquires image data (video data) of the road surface (the road surface illuminated with the light pattern 50) captured over multiple frames by the in-vehicle camera 20 (step S106). Then, the period determination unit 14 performs pattern matching of the light pattern 50 illuminated from the illumination unit 22 with the image data (still image data) of each acquired frame (step S107). That is, it determines whether or not the light pattern 50 is included in the image data of each frame. Then, the period determination unit 14 identifies a non-illumination period based on the presence or absence of the light pattern 50 (step S108). Steps S106 to S108 correspond to the period determination step.

[0042] Thereafter, the recognition unit 15 inputs image data captured by the in-vehicle camera 20 during the non-illumination period identified by the period identification unit 14 (step S109). Then, the recognition unit 15 recognizes the parking space 40 from the image data (step S110). Steps S109 to S110 correspond to a recognition step.

[0043] Based on the parking space 40 recognized in step S110, the vehicle control device 12 controls the actuator 30 to guide the vehicle 100 into the parking space 40 (step S111). As a result, the vehicle control device 12 controls the actuator 30 to move the vehicle 100 backward from the reverse start position shown in FIG. 5(b) and guide the vehicle 100 into the parking space 40.

[0044] Then, the vehicle control device 12 determines whether the vehicle 100 has been parked inside the parking space 40 (step S112). If the determination result is negative, the driving assistance device 10 proceeds to the processing of step S109 again. On the other hand, if the determination result is positive (if parking is completed), the driving assistance device 10 terminates the road lighting function and the advanced parking assist function, and terminates the parking processing. Note that the processing of step S110 (image recognition processing), step S111 (vehicle control processing), and step S112 may be performed not only during the non-illumination period identified in step S108, but also in parallel during periods other than the non-illumination period, for example, during the illumination period in which the light pattern 50 is illuminated.

[0045] The effects of the first embodiment will be described.

[0046] The period determination unit 14 acquires image data (video data) of the road surface captured over multiple frames, compares the image data (still image data) of each frame, and determines a non-illumination period during which the light pattern 50 is not irradiated based on the differences between the image data. The recognition unit 15 then recognizes the parking space 40 based on the image data captured during the determined non-illumination period. Therefore, even if the light pattern 50 is irradiated onto the road surface, the parking space 40 can be accurately recognized without being obstructed by the light pattern 50. Furthermore, irradiating the light pattern 50 onto the road surface allows people around the vehicle 100 to be notified in advance of the traveling direction of the vehicle 100, thereby improving safety when parking.

[0047] The light pattern 50 is predetermined and can be detected with high accuracy by pattern matching. Therefore, the period identification unit 14 detects the presence or absence of the light pattern 50 by performing pattern matching of the light pattern 50 for each image data. This makes it possible to identify the non-irradiation period with high accuracy.

[0048] During the irradiation period, the recognition unit 15 performs image recognition processing on the image data captured during the non-irradiation period to recognize the parking space 40. This allows the image recognition processing to be performed in parallel using the irradiation period, thereby making it possible to use time efficiently.

[0049] (Second embodiment) The configuration of the driving support device 10 of the first embodiment may be partially changed. A second embodiment in which the configuration of the driving support device 10 of the first embodiment is partially changed will be described below.

[0050] As shown in Fig. 6, the period determination unit 14 of the second embodiment receives the irradiation timing and non-irradiation timing of the light pattern 50 from the irradiation control device 13 and determines the non-irradiation period. The flow of the parking process of the second embodiment will be described in detail with reference to Fig. 7. The processing from steps S201 to S204 of the parking process of the second embodiment is the same as the processing from steps S101 to S104 of the parking process of the first embodiment. Therefore, the description thereof will be omitted.

[0051] If the determination result in step S204 is positive, the illumination control device 13 starts controlling the illumination unit 22 in the same manner as step S105 in the first embodiment (step S205). Thereafter, the road surface lighting function is continuously performed until the vehicle 100 is parked. Step S205 corresponds to an illumination control step.

[0052] During the execution of the road surface lighting function, the illumination control device 13 inputs the timing of non-illumination, that is, the start timing of the non-illumination period, to the image recognition device 11 (step S206). When the non-illumination timing is input, the period determination unit 14 of the image recognition device 11 determines that the non-illumination period has started (step S207). Step S207 corresponds to the period determination step.

[0053] The image recognition device 11 acquires image data of the road surface captured during the non-illumination period from the in-vehicle camera 20 (step S208). Also, while the road surface lighting function is being performed, when the illumination timing arrives (i.e., the end timing of the non-illumination period), the illumination control device 13 inputs this information to the image recognition device 11 (step S209). When the illumination timing is input, the period determination unit 14 of the image recognition device 11 determines that the non-illumination period has ended (step S210). When the non-illumination period ends, the recognition unit 15 recognizes the target parking space 40 from the image data acquired in step S208 (image data captured during the non-illumination period) (step S211). Step S211 corresponds to the recognition step. Based on the parking space 40 recognized in step S211, the vehicle control device 12 controls the actuator 30 to guide the vehicle 100 into the parking space 40 (step S212).

[0054] Then, the driving assistance device 10 determines whether or not the vehicle 100 is parked inside the parking space 40 (step S213). If the determination result is negative, the driving assistance device 10 proceeds to the process of step S206 again. On the other hand, if the determination result is positive (if the vehicle 100 has been parked), the driving assistance device 10 ends the road lighting function and the advanced parking assist function, and ends the parking process.

[0055] In parallel with the processing of steps S211 to S212, the illumination control device 13 controls the illumination unit 22 to illuminate the road surface with the light pattern 50. In addition, in steps S206 to S213, the vehicle control device 12 continuously controls the actuator 30 to guide the vehicle 100 into the parking stall 40, and in step S211, the positions of the vehicle 100 and the parking stall 40 are confirmed.

[0056] According to the second embodiment, the period specifying unit 14 inputs the non-irradiation timing when the light pattern 50 is not irradiated from the irradiation control device 13, and specifies the non-irradiation period when the light pattern 50 is not irradiated. This eliminates the need to compare image data with each other or to perform pattern matching of the light pattern 50, thereby reducing the processing load.

[0057] (Variation) Although the period identification unit 14 in the first embodiment determines the presence or absence of the light pattern 50 by performing pattern matching of the light pattern 50 for the image data of each frame, other methods may be used to determine the presence or absence of the light pattern 50. For example, the period identification unit 14 may determine the presence or absence of the light pattern 50 by taking the difference between the image data (still image data) of previous and next frames and comparing the difference with the light pattern 50.

[0058] 3, the light pattern 50 is detected by taking the difference between the image data of frame D1 and the image data of frame D2 and determining whether or not the difference matches the light pattern 50. Similarly, the light pattern 50 is detected by taking the difference between frame D2 and frame D3, the difference between frame D3 and frame D4, and the difference between frame D4 and frame D5 and determining whether or not these differences match the light pattern 50.

[0059] In the above embodiment, the driving assistance device 10 includes the image recognition device 11, the vehicle control device 12, and the illumination control device 13, each configured as a separate device (microcomputer). However, two or all of these devices may be implemented by a single control device. For example, the image recognition device 11 and the vehicle control device 12 may be implemented by a single control device. The image recognition device 11 may also include some or all of the functions of the vehicle control device 12, or some or all of the functions of the illumination control device 13. Similarly, the vehicle control device 12 may also include some or all of the functions of the image recognition device 11, or some or all of the functions of the illumination control device 13. Similarly, the illumination control device 13 may also include some or all of the functions of the image recognition device 11, or some or all of the functions of the vehicle control device 12.

[0060] In the above embodiment, the driving assistance device 10 does not necessarily have to include the vehicle control device 12. That is, the vehicle control device 12 may be provided externally to the driving assistance device 10. Similarly, in the first embodiment, the driving assistance device 10 does not necessarily have to include the irradiation control device 13. That is, the irradiation control device 13 may be provided externally to the driving assistance device 10.

[0061] In the above embodiment, when the advanced parking assist function is implemented, the vehicle 100 is moved backward to park in the parking space 40, but the vehicle 100 may be moved forward to park in the parking space 40. In this case, the forward illumination unit 22a emits the light pattern 50 indicating that the vehicle 100 is moving forward.

[0062] The controller and methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the controller and methods described herein may be implemented by a special-purpose computer configured with a processor configured with one or more dedicated hardware logic circuits. Alternatively, the controller and methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0063] Below, the technical ideas that can be derived from the above-described embodiment and modifications will be additionally described.

[0064] [Configuration 1] A driving assistance device (10) that recognizes a parking space (40) based on image data captured by a camera unit (20, 21a to 21d) in a situation where a light pattern (50) representing the movement of a vehicle (100) is projected onto a road surface around the vehicle, the light pattern is irradiated so as to blink at a predetermined blinking cycle, a period determination unit (14) that acquires image data of the road surface photographed over a plurality of frames from the camera unit, compares the image data of each frame, and determines a non-irradiation period during which the light pattern is not irradiated based on differences therebetween; a recognition unit (15) that recognizes the parking space based on image data captured by the camera unit during the non-illumination period specified by the period specification unit.

[0065] [Configuration 2] The driving assistance device according to configuration 1, wherein the period determination unit performs pattern matching of the light pattern for image data of each frame to detect the presence or absence of the light pattern and determine the non-illumination period.

[0066] [Configuration 3] The driving assistance device according to configuration 1, wherein the period determination unit takes a difference between image data in previous and next frames and compares the difference with the light pattern to detect the presence or absence of the light pattern and determine the non-illumination period.

[0067] [Configuration 4] A driving assistance device (10) that recognizes a parking space (40) based on image data captured by a camera unit (20, 21a to 21d) in a situation where a light pattern (50) representing the movement of a vehicle (100) is projected onto a road surface around the vehicle, the light pattern is irradiated so as to blink at a predetermined blinking cycle, an illumination control unit (13) that estimates a movement of the host vehicle and controls the illumination unit to illuminate the light pattern that represents the estimated movement of the host vehicle; a period specifying unit (14) that receives input of a non-irradiation timing at which the light pattern is not irradiated from the irradiation control unit and specifies a non-irradiation period during which the light pattern is not irradiated; a recognition unit (15) that recognizes the parking space based on image data captured by a camera unit during the non-illumination period specified by the period specification unit.

[0068] [Configuration 5] The driving assistance device according to any one of configurations 1 to 4, wherein the recognition unit performs image recognition processing on image data captured by the camera unit during the non-illumination period during an illumination period in which the light pattern is illuminated.

[0069] [Configuration 6] The driving assistance device according to any one of configurations 1 to 5, further comprising a vehicle control unit (12) that controls the movement of the host vehicle so as to park the host vehicle within the parking space recognized by the recognition unit.

[0070] [Configuration 7] A program executed by a driving assistance device (10) that recognizes a parking space (40) based on image data captured by a camera unit (20, 21a to 21d) in a situation where a light pattern (50) representing a movement of a vehicle (100) is projected onto a road surface around the vehicle, the program comprising: the light pattern is irradiated so as to blink at a predetermined blinking cycle, The driving assistance device includes: a period identification step of acquiring image data of the road surface photographed over a plurality of frames from the camera unit, comparing the image data of each frame, and identifying a non-irradiation period during which the light pattern is not irradiated based on differences therebetween; a recognition step of recognizing the parking space based on image data captured by the camera unit during the non-irradiation period specified by the period specification step.

[0071] [Configuration 8] A program executed by a driving assistance device (10) that recognizes a parking space (40) based on image data captured by a camera unit (20, 21a to 21d) in a situation where a light pattern (50) representing a movement of a vehicle (100) is projected onto a road surface around the vehicle, the program comprising: the light pattern is irradiated so as to blink at a predetermined blinking cycle, The driving assistance device includes: an irradiation control step of estimating a movement of the host vehicle and controlling an irradiation unit to irradiate the light pattern representing the estimated movement of the host vehicle; a period specifying step of inputting a non-irradiation timing at which the irradiation unit does not irradiate the light pattern and specifying a non-irradiation period during which the light pattern is not irradiated; a recognition step of recognizing the parking space based on image data captured by a camera unit during the non-irradiation period specified in the period specification step. [Explanation of symbols]

[0072] 10...driving assistance device, 11...image recognition device, 12...vehicle control device, 13...illumination control device, 14...period determination unit, 15...recognition unit, 20...on-board camera, 21a...front camera, 21b...right side camera, 21c...left side camera, 21d...rear camera, 22...illumination unit, 30...actuator, 40...parking space, 50...light pattern, 100...vehicle

Claims

1. A driving assistance device (10) that recognizes a parking space (40) based on image data captured by a camera unit (20, 21a to 21d) in a situation where a light pattern (50) representing the movement of a vehicle (100) is projected onto a road surface around the vehicle, the light pattern is irradiated so as to blink at a predetermined blinking cycle, a period determination unit (14) that acquires image data of the road surface photographed over a plurality of frames from the camera unit, compares the image data of each frame, and determines a non-irradiation period during which the light pattern is not irradiated based on differences therebetween; A driving assistance device comprising: a recognition unit (15) that recognizes the parking space based on image data captured by the camera unit during the non-illumination period specified by the period specification unit.

2. The driving assistance device according to claim 1 , wherein the period determination unit performs pattern matching of the light pattern on image data of each frame to detect the presence or absence of the light pattern and determine the non-illumination period.

3. The driving assistance device according to claim 1 , wherein the period determination unit determines the non-illumination period by taking a difference between image data in previous and next frames and comparing the difference with the light pattern to detect the presence or absence of the light pattern.

4. A driving assistance device (10) that recognizes a parking space (40) based on image data captured by a camera unit (20, 21a to 21d) in a situation where a light pattern (50) representing the movement of a vehicle (100) is projected onto a road surface around the vehicle, the light pattern is irradiated so as to blink at a predetermined blinking cycle, an illumination control unit (13) that estimates a movement of the host vehicle and controls the illumination unit to illuminate the light pattern that represents the estimated movement of the host vehicle; a period specifying unit (14) that receives input of a non-irradiation timing at which the light pattern is not irradiated from the irradiation control unit and specifies a non-irradiation period during which the light pattern is not irradiated; A driving assistance device comprising: a recognition unit (15) that recognizes the parking space based on image data captured by a camera unit during the non-illumination period specified by the period specification unit.

5. The driving assistance device according to any one of claims 1 to 4, wherein the recognition unit performs image recognition processing on image data captured by the camera unit during the non-illumination period during an illumination period in which the light pattern is illuminated.

6. A vehicle control unit (12) that controls the movement of the vehicle so as to park the vehicle in the parking space recognized by the recognition unit. The driving assistance device according to any one of claims 1 to 4.

7. A program executed by a driving assistance device (10) that recognizes a parking space (40) based on image data captured by a camera unit (20, 21a to 21d) in a situation where a light pattern (50) representing the movement of a vehicle (100) is irradiated onto a road surface around the vehicle, the program comprising: the light pattern is irradiated so as to blink at a predetermined blinking cycle, The driving assistance device includes: a period identification step of acquiring image data of the road surface photographed over a plurality of frames from the camera unit, comparing the image data of each frame, and identifying a non-irradiation period during which the light pattern is not irradiated based on differences therebetween; a recognition step of recognizing the parking space based on image data captured by the camera unit during the non-irradiation period specified by the period specification step.

8. A program executed by a driving assistance device (10) that recognizes a parking space (40) based on image data captured by a camera unit (20, 21a to 21d) in a situation where a light pattern (50) representing the movement of a vehicle (100) is irradiated onto a road surface around the vehicle, the program comprising: the light pattern is irradiated so as to blink at a predetermined blinking cycle, The driving assistance device includes: an irradiation control step of estimating a movement of the host vehicle and controlling an irradiation unit to irradiate the light pattern representing the estimated movement of the host vehicle; a period specifying step of inputting a non-irradiation timing at which the irradiation unit does not irradiate the light pattern and specifying a non-irradiation period during which the light pattern is not irradiated; a recognition step of recognizing the parking space based on image data captured by a camera unit during the non-irradiation period specified in the period specification step.

Images