Driving assistance device, driving assistance method, storage medium, and program

Abstract

A driving assistance device according to the present disclosure is mounted to a moving body and comprises: an image capture means that captures an image including subjects in the cabin of the moving body and outside the moving body; a recognition means that uses the captured image to recognize a person in the image; an estimation means that estimates one or more characteristics of the recognized person in the image; a detection means that detects, as a passerby, a person in the image having the estimated one or more characteristics each satisfying a predetermined criterion; and an output control means that causes a warning related to the presence of the detected passerby to be output from an output device, wherein the one or more characteristics include at least one from among the distance of the person from the driving assistance device, the movement speed of the person relative to the moving body, and the height of the person.

Description

Driving assistance device, driving assistance method, storage medium, and program

[0001] The present invention relates to a driving assistance device, a driving assistance method, a storage medium, and a program.

[0002] In recent years, a driving assistance technology has been proposed that analyzes an image of the surroundings of a vehicle (also called a moving object) captured by a camera installed inside the vehicle, and when it is determined that an object in the image is a cautionary object, displays a warning image on a display device at the predicted position of the cautionary object (Patent Document 1).If the driving assistance technology notifies the driver of warnings regarding cautionary objects as needed, the driver can pay attention to highly dangerous objects and appropriately avoid danger.

[0003] International Publication No. 2019 / 159344

[0004] However, there are cases where it is desirable to provide a driver with a warning about pedestrians passing beside or behind the vehicle using a drive recorder installed inside the vehicle. In many cases, it is necessary to detect pedestrians around the vehicle using images from a camera positioned so that the interior of the vehicle is included in its capture range. In such cases, it is necessary to reduce false detections, such as detecting a person inside the vehicle as a pedestrian, and to accurately detect pedestrians outside the vehicle from the captured images.

[0005] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to realize a technology that can accurately detect passersby outside a vehicle when using an image that includes both the interior of the vehicle and the exterior of the vehicle.

[0006] According to the present invention, there is provided a driving assistance device attached to a moving body, comprising: an imaging means for capturing an image including the interior of the moving body and a subject outside the moving body; a recognition means for recognizing a person in the image using the captured image; an estimation means for estimating one or more characteristics of the person in the recognized image; a detection means for detecting a person in the image whose estimated one or more characteristics each satisfy a predetermined standard as a passerby; and an output control means for causing an output device to output a warning regarding the presence of the detected passerby, wherein the one or more characteristics include at least one of the distance of the person from the driving assistance device, the moving speed of the person relative to the moving body, and the height of the person.

[0007] According to the present invention, when a captured image including the interior of the vehicle and the exterior of the vehicle is used, it is possible to accurately detect a passerby outside the vehicle.

[0008] Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are designated by the same reference numerals.

[0009] The accompanying drawings are included in the specification, constitute a part thereof, illustrate embodiments of the present invention, and are used to explain the principles of the present invention together with the description thereof. A diagram schematically showing an example of the configuration of a moving body (vehicle) according to an embodiment. A diagram schematically showing an example of the external configuration of a drive recorder as an example of a driving assistance device according to an embodiment. A diagram schematically showing the external configuration of an output device according to an embodiment. A block diagram showing an example of the hardware configuration of a drive recorder according to an embodiment. A block diagram showing an example of the functional configuration of a drive recorder according to an embodiment. A diagram explaining a person's characteristics (relative moving speed) according to an embodiment. A diagram explaining a person's characteristics (distance from the drive recorder) according to an embodiment. A flowchart showing a series of operations of a driving assistance process according to an embodiment.

[0010] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0011] In this embodiment, the driving assistance device is described as a drive recorder that is installed inside a vehicle, captures images of the front, sides, and rear of the vehicle, and records the state of the vehicle while it is traveling on a recording medium. However, the driving assistance device may be any device other than a drive recorder as long as it is installed in a vehicle and can capture images of the environment outside the vehicle. Furthermore, in the following embodiment, the moving object is described as a vehicle, for example. The vehicle is typically a four-wheeled vehicle, but this embodiment may also be applied to other types of vehicles.

[0012] An example of the mounting position of a drive recorder will be described with reference to Fig. 1. A drive recorder 101 is mounted on a vehicle 100. The drive recorder 101 is mounted near the rearview mirror on the windshield of the vehicle 100 at a position that does not obstruct the driver's field of view, or on the rearview mirror of the vehicle 100. The mounting position of the drive recorder 101 is not limited to this, and it may be any position that can capture an image of the outside world of the vehicle 100.

[0013] For example, the drive recorder 101 can transmit information that needs to be output to the driver to an external output device 102 wirelessly or via a wired connection, and can output the transmitted warning from the output device 102. Note that in the description of this embodiment, a case will be described in which the output device 102 is an external device of the drive recorder 101. However, the drive recorder 101 may also be provided with an output device (i.e., a display for display or a speaker for audio output).

[0014] <External Configuration Example of Drive Recorder> An external configuration example of the drive recorder 101 will be described with reference to Fig. 2A . The drive recorder 101 has an imaging device 202 on a front surface 201, which captures images of the forward direction (movement direction) of the vehicle 100. The imaging device 202 is, for example, an imaging device with a wide-angle lens. The drive recorder 101 also has an imaging device 211 on a rear surface 210, which captures images of the sides and rear of the vehicle 100. The imaging device 211 is, for example, an imaging device with a fisheye lens. The image captured by the imaging device 211 includes the driver of the vehicle 100. 203 constitutes the top surface of the drive recorder 101.

[0015] <External Configuration Example of Output Device> An external configuration example of the output device 102 will be described with reference to Fig. 2B . The output device 102 is a device for outputting information to occupants of the vehicle 100. The output device 102 includes a display device (e.g., a display or a light-emitting element) that provides visual information and an audio device (e.g., a speaker) that provides auditory information. The output device 102 displays information on the display device by communicating with the drive recorder 101 wirelessly or via a wire.

[0016] The output device 102 has a display device on its upper surface 220. The output device 102 can color-display a partial area (e.g., 222) within a circle 221 representing the periphery of the vehicle 100 in accordance with an instruction from the drive recorder 101. The color-displayed area 222 serves as a warning to be output to the driver or the like that a risk exists in the corresponding direction. The output device 102 is installed, for example, on a dashboard near the driver's seat of the vehicle 100. The output device 102 may further have a half mirror so as to reflect light output from the upper surface 220 toward the driver.

[0017] 3, an example of the hardware configuration of the drive recorder 101 will be described. The drive recorder 101 includes, for example, a control unit 301, a storage device 302, an imaging device 303, an input device 304, and a communication device 305.

[0018] The control unit 301 includes a processor such as a CPU (Central Processing Unit) and controls the overall operation of the drive recorder 101. The control unit 301 may be realized, for example, by the CPU expanding and executing a program stored in the storage device 302. In addition to the CPU, the control unit 301 may further include a GPU (Graphics Processing Unit) and dedicated circuits for image processing, machine learning, and the like.

[0019] The storage device 302 is a device that stores data and programs related to the operation of the drive recorder 101. For example, the storage device 302 may store a program that defines the operation of the drive recorder 101, temporary data when the program is executed, images captured by the imaging device 303, environmental sounds collected by a microphone, etc. The storage device 302 may be realized by a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). Furthermore, the storage device 302 may include an auxiliary storage device such as an SSD (Solid State Drive).

[0020] The imaging device 303 includes a camera for capturing images of the external environment in front of the vehicle 100. The imaging device 303 may be, for example, a wide-angle camera. This camera is the imaging device 202 described above, located in front of the drive recorder 101 (on the front side of the vehicle 100). The imaging device 303 also includes a camera capable of capturing images of the rear and left and right sides of the vehicle 100, i.e., the imaging device 211 described above. This camera may be a camera with a fisheye lens. The camera capturing images of the rear is located behind the drive recorder (on the rear side of the vehicle). In this embodiment, an example will be described in which the same camera captures images of the rear and sides of the vehicle. However, the camera capturing images of the rear and sides may be separate cameras. The camera capturing images of the sides may be located on the side of the drive recorder. Some of the cameras capturing images in multiple directions may be separate from the drive recorder.

[0021] The input device 304 is a device for inputting instructions from an occupant of the vehicle 100. The input device 304 includes, for example, a button or a touch panel. The input device 304 may also include a microphone for inputting the voice of the occupant and external sounds.

[0022] The communication device 305 is a device that allows the drive recorder 101 to communicate with the output device 102 via wired or wireless communication. The communication device 305 transmits warning information generated by the driving assistance process in the control unit 301 to the output device 102. The communication device 305 may communicate with other devices via wired communication or wirelessly. The wireless communication may be communication that complies with a protocol such as short-range communication (e.g., Bluetooth (registered trademark)). The communication device 305 may be configured to be able to communicate with the control ECU of the vehicle 100 via wired or wireless communication. The communication device 305 may be omitted.

[0023] <Example of Functional Configuration of Drive Recorder> An example of the functional configuration realized by the control unit 301 of the drive recorder 101 will be described with reference to Fig. 4. Each unit shown in Fig. 4 is realized, for example, by the CPU of the control unit 301 executing a computer program.

[0024] The image acquisition unit 401 acquires a front captured image from the imaging device 202 that captures an image of the area in front of the drive recorder 101. The image acquisition unit 401 also acquires a rear captured image from the imaging device 211 that captures an image of the area behind the drive recorder 101. The rear captured image includes an object outside the vehicle and the interior of the vehicle. For example, the rear captured image includes a passerby walking outside the vehicle 100 and an occupant present inside the vehicle.

[0025] The person recognition unit 402 recognizes people included in the image (rear captured image) acquired by the image acquisition unit 401. The person recognition unit 402 may use a neural network that inputs the rear captured image and outputs a person recognition result. When at least a part of a person's body is included in the image, the person recognition unit 402 may be able to recognize the part as a person.

[0026] The characteristic estimation unit 403 estimates one or more characteristics of the person in the image recognized by the person recognition unit 402. The one or more characteristics include, for example, at least one of the distance of the person from the drive recorder 101, the moving speed of the person relative to the vehicle 100, and the height of the person. The characteristics of the person will be described later.

[0027] The passerby detection unit 404 determines whether one or more characteristics of the person estimated by the characteristic estimation unit 403 satisfy predetermined criteria, and detects a person who satisfies all of the criteria as a passerby. The criteria that the characteristics must satisfy will be described later.

[0028] The output control unit 405 causes the output device 102 to output a warning when the detected pedestrian approaches within a predetermined range from the traveling trajectory of the vehicle 100. For example, the control unit 301 estimates a predicted trajectory of the detected pedestrian, and when a warning is necessary, causes the output device 102 to output a warning indicating the direction in which the pedestrian is present.

[0029] Next, the characteristics of a person according to this embodiment and the criteria that the characteristics must satisfy will be described with reference to FIGS.

[0030] FIG. 5 shows a specific example in which the person characteristic is the person's relative movement speed with respect to the vehicle 100. For example, suppose that persons 501 and 502 are riding in the vehicle 100, and person 503 is passing outside the vehicle. The characteristic estimation unit 403 can estimate the vehicle's predicted trajectory and movement speed based on changes in (time-series) front-view captured images or (time-series) rear-view captured images. Furthermore, the characteristic estimation unit 403 can estimate the movement speeds of persons 501, 502, and 503 using (time-series) rear-view captured images. For example, the characteristic estimation unit 403 estimates the absolute values ​​of the movement speeds of persons 501 and 502 relative to the vehicle 100 based on changes in the areas of these persons 501 and 502 or the predicted trajectories of the persons. Furthermore, the characteristic estimation unit 403 estimates the absolute value of the movement speed of person 503 relative to the vehicle 100 based on changes in the area of ​​person 503 or the predicted trajectory of person 503. The moving speed of the person may be estimated by any known method other than the above-described method, for example, by inputting a rearward captured image into a neural network.

[0031] If the absolute value of the person's relative movement speed with respect to the vehicle 100 is greater than a predetermined threshold, the passerby detection unit 404 determines that the person in the image satisfies the criterion for the person's relative movement speed with respect to the vehicle 100. For example, the absolute values ​​of the relative movement speeds of the person 501 and the person 502 with respect to the vehicle 100 are generally small (i.e., smaller than the threshold). In this case, for example, the passerby detection unit 404 determines that the person 501 and the person 502 do not satisfy the relative movement speed criterion and does not detect the person 501 and the person 502 as passersby. On the other hand, if the absolute value of the person 503's relative movement speed with respect to the vehicle 100 exceeds the threshold, the passerby detection unit 404 determines that the person 503 satisfies the relative movement speed criterion and detects the person 503 as a passerby. In this way, by taking the person's relative movement speed into consideration, it is possible to exclude from processing persons having a movement speed that is inappropriate for a passerby, thereby enabling accurate detection of passersby outside the vehicle.

[0032] 6 shows a specific example in which the characteristic of a person is the distance of the person from the drive recorder 101. As in the case of FIG. 5, for example, it is assumed that persons 501 and 502 are riding in the vehicle 100, and a person 503 is passing outside the vehicle.

[0033] The characteristic estimation unit 403 uses the (time-series) rearward captured images to estimate the distances from the drive recorder 101 to the person 501, the person 502, and the person 503. The distances from the drive recorder 101 to the people may be estimated by a known method, for example, by inputting the rearward captured images into a neural network.

[0034] If the distances of the persons 501 and 502 from the drive recorder 101 are equal to or less than a threshold value corresponding to the interior of the vehicle 100 (for example, a threshold value indicating the range 601), the passerby detection unit 404 determines that the persons 501 and 502 do not satisfy the distance criterion. Therefore, the persons 501 and 502 are not detected as passersby.

[0035] On the other hand, if the distance of person 503 from drive recorder 101 is greater than the threshold corresponding to the interior of vehicle 100 and is equal to or less than a threshold that defines the distance (for example, the threshold indicating range 602), passerby detection unit 404 determines that person 503 meets the criterion for the distance from the drive recorder. Therefore, person 503 is detected as a passerby. In this way, by taking the distance from drive recorder 100 into consideration, it is possible to exclude from processing persons who are located at a distance that is inappropriate for a passerby, and it becomes possible to accurately detect passersby outside the vehicle.

[0036] The characteristic of a person may be the height of the person. Generally, the size of a person's face varies little from person to person, and is therefore a useful parameter when detecting a person from an image. However, if a person is far away from the imaging device, the size of the face in the image becomes very small. For this reason, in this embodiment, the height of a person is used as a characteristic of a person in order to perform an effective evaluation using image information even when the person is far away from the imaging device. However, the size of a person's face may also be used as a characteristic of a person.

[0037] The characteristic estimation unit 403 estimates the height of the person based on the size of the person area in the image. The height of the person may be estimated by a known method, for example, by inputting the rear captured image into a neural network.

[0038] The passerby detection unit 404 determines that a person meets a height standard if the person's height falls within a range determined according to the person's distance from the drive recorder 101. When estimating a person's characteristics, such as height, from a captured image, the greater the distance of the person, the greater the likelihood of an estimation error. Therefore, in this embodiment, a permissible error range for the estimated height is determined in advance for each distance from the drive recorder 101. In this manner, in this embodiment, if the estimated height exceeds the permissible error range, the person is not detected as a passerby. In this way, by taking the person's height into consideration, people whose heights fall within an inappropriate range for a passerby can be excluded from the processing target, thereby enabling accurate detection of passersby outside the vehicle.

[0039] <Series of Operations of Driving Assistance Processing in Drive Recorder> Next, a series of operations of driving assistance processing executed in the drive recorder 101 will be described with reference to Fig. 7. Note that this processing is realized by the CPU of the control unit 301 executing a computer program stored in the storage device 302.

[0040] In the following description, a case where a person in an image that satisfies predetermined criteria is detected as a passerby is described as an example of a process for detecting passersby. However, the process for detecting passersby may include any other detection process. In this embodiment, by not detecting a person in an image whose characteristics do not satisfy predetermined criteria as a passerby, it is possible to reduce false detection of passersby, and as a result, it is possible to accurately detect passersby outside the vehicle.

[0041] In S701, the image acquisition unit 401 acquires a rear captured image capturing an image of the area behind the drive recorder 101. The rear captured image includes an object outside the vehicle and the interior of the vehicle. In S702, the person recognition unit 402 recognizes a person in the image. In S703, the characteristic estimation unit 403 estimates one or more characteristics of the person in the image. As described above, the person's characteristics include, for example, at least one of the person's distance from the drive recorder 101, the person's moving speed relative to the vehicle 100, and the person's height.

[0042] In S704, the passerby detection unit 404 determines whether the estimated characteristics satisfy predetermined criteria. Specifically, as described with reference to FIGS. 4, 5, and 6, the passerby detection unit 404 determines whether the estimated characteristics, such as the person's height, satisfy each predetermined criterion. If the passerby detection unit 404 determines that the characteristics of the passerby satisfy the predetermined criterion (if multiple characteristics are handled, all criteria are satisfied), the passerby detection unit 404 proceeds to S705; otherwise, the passerby detection unit 404 terminates this series of processes. In S705, the passerby detection unit 404 detects the person being processed as a passerby. For example, in the examples of FIGS. 5 and 6 described above, person 503 is detected as a passerby.

[0043] In S706, the output control unit 405 determines whether it is necessary to output a warning to the driver regarding the detected pedestrian. For example, the output control unit 405 determines that it is necessary to output a warning when the detected pedestrian approaches a position within a predetermined range from the traveling trajectory of the vehicle 100. If the output control unit 405 determines that it is necessary to output a warning, the process proceeds to S707; otherwise, the process ends.

[0044] In S707, the output control unit 405 causes the output device 102 to output a warning indicating the direction in which a passerby is present (for example, the area 222 in FIG. 2B ). For example, the output control unit 405 causes the output device 102 to output a warning indicating the direction in which a passerby is present, based on a trajectory that has already been obtained for the detected passerby.

[0045] In the above-described embodiment, in order to detect passersby around the vehicle, it is determined whether the characteristics of the person satisfy a criterion. The control unit 302 may perform a process of excluding registered occupants from the processing target before the characteristic estimation unit 403 performs a process of estimating the characteristics of the person. For example, the control unit 302 may register the faces of the occupants in the drive recorder 101 in advance (through a user operation, etc.). Then, the control unit 302 may identify, for example, people in the recognized image who have facial characteristics that match the pre-registered facial characteristics. The characteristic estimation unit 403 may estimate one or more characteristics of people other than the identified person. In this way, unnecessary estimation processes for the occupants can be omitted, thereby achieving faster processing and lower power consumption.

[0046] In the above embodiment, the warning is output from the output device 102. However, the warning may be output from at least one of a smartphone, a tablet device, a head-mounted display, and a wearable device. These devices communicate with the drive recorder 101 wirelessly or via a cable to display the warning on a display device.

[0047] Summary of the embodiment 1. A driving assistance device (e.g., 101) of the above embodiment is a driving assistance device attached to a moving body (e.g., 100), and includes: an imaging unit (e.g., 401) that captures an image including an interior of the moving body and a subject outside the moving body; a recognition unit (e.g., 402) that recognizes a person in the image using the captured image; an estimation unit (e.g., 403) that estimates one or more characteristics of the person in the recognized image; a detection unit (e.g., 404) that detects, as a passerby, a person in the image whose estimated one or more characteristics each satisfy a predetermined standard; and an output control unit (e.g., 405) that causes an output device to output a warning regarding the presence of the detected passerby, wherein the one or more characteristics include at least one of a distance of the person from the driving assistance device, a moving speed of the person relative to the moving body, and a height of the person.

[0048] According to this embodiment, when a captured image including the interior of the vehicle and the exterior of the vehicle is used, it is possible to accurately detect passersby outside the vehicle.

[0049] 2. In the driving assistance device according to the above embodiment, the detection means determines that the person in the image satisfies the criterion for the distance from the driving assistance device when the distance from the person in the image to the driving assistance device is greater than a threshold corresponding to the interior of the moving object and is equal to or less than a threshold that defines a far distance.

[0050] According to this embodiment, by taking into account the distance from the drive recorder 100, people located at a distance that is inappropriate for a pedestrian can be excluded from the processing target, thereby enabling accurate detection of pedestrians outside the vehicle.

[0051] 3. In the driving assistance device according to the above embodiment, when an absolute value of the relative movement speed of the person with respect to the moving object is greater than a predetermined threshold, the detection means determines that the person in the image satisfies a criterion for the relative movement speed of the person with respect to the moving object.

[0052] According to this embodiment, by taking into account the relative movement speed of a person, people whose movement speed is inappropriate for a pedestrian can be excluded from the processing target, making it possible to accurately detect pedestrians outside the vehicle.

[0053] 4. In the driving assistance device according to the above embodiment, the detection means determines that the person in the image satisfies a standard for person height if the height of the person falls within a range determined according to a distance of the person from the driving assistance device.

[0054] According to this embodiment, by taking into consideration the height of a person, people whose height falls within a range that is inappropriate for a pedestrian can be excluded from the processing target, thereby enabling accurate detection of pedestrians outside the vehicle.

[0055] 5. In the driving assistance device according to the above embodiment, the estimation means estimates one or more characteristics of people in the recognized image other than people having facial features that match pre-registered facial features.

[0056] According to this embodiment, unnecessary estimation processing for the occupant can be omitted, and processing speed can be increased and power consumption can be reduced.

[0057] 6. In the driving assistance device according to the above embodiment, the output control means causes the output device to output a warning indicating the direction in which the detected pedestrian is present.

[0058] According to this embodiment, the driver can quickly and easily grasp where the target of the warning is located in the surrounding area.

[0059] 7. In the driving assistance device according to the above embodiment, the output device is included in the driving assistance device.

[0060] According to this embodiment, driving assistance can be performed using a single driving assistance device with a configuration that is easy to install.

[0061] 8. In the driving assistance device according to the above embodiment, the output device communicates with the driving assistance device wirelessly or via a wire, thereby displaying the warning on a display device of the output device.

[0062] According to this embodiment, it is possible to arrange the output device at a position where it can be easily perceived by the driver.

[0063] 9. In the driving assistance device according to the above embodiment, the output device is at least one of a smartphone, a tablet terminal, a head-mounted display, and a wearable terminal.

[0064] According to this embodiment, a separately used device can be used as the output device, which makes it possible to reduce the manufacturing cost of the driving assistance device.

[0065] 10. A driving assistance method in the above embodiment is a driving assistance method executed by a driving assistance device (e.g., 101) attached to a moving body (e.g., 100), comprising: capturing an image including an interior of the moving body and a subject outside the moving body (e.g., S701); recognizing a person in the image using the captured image (e.g., S702); estimating one or more characteristics of the person in the recognized image (e.g., S703); detecting, as a passerby, a person in the image whose estimated one or more characteristics each satisfy a predetermined criterion (e.g., S704); and outputting, from an output device, a warning regarding the presence of the detected passerby (e.g., S705), wherein the one or more characteristics include at least one of a distance of the person from the driving assistance device, a moving speed of the person relative to the moving body, and a height of the person.

[0066] According to this embodiment, when a captured image including the interior of the vehicle and the exterior of the vehicle is used, it is possible to accurately detect passersby outside the vehicle.

[0067] 11. A program in the above embodiment is a program for causing a computer to function as each means of a driving assistance device (e.g., 101) attached to a moving body (e.g., 100), wherein the driving assistance device comprises: an imaging means (e.g., 401) that captures an image including an interior of the moving body and a subject outside the moving body; a recognition means (e.g., 402) that recognizes a person in the image using the captured image; an estimation means (e.g., 403) that estimates one or more characteristics of the person in the recognized image; a detection means (e.g., 404) that detects, as a passerby, a person in the image whose estimated one or more characteristics each satisfy a predetermined standard; and an output control means (e.g., 405) that causes an output device to output a warning regarding the presence of the detected passerby, wherein the one or more characteristics include at least one of a distance of the person from the driving assistance device, a moving speed of the person relative to the moving body, and a height of the person.

[0068] According to this embodiment, when a captured image including the interior of the vehicle and the exterior of the vehicle is used, it is possible to accurately detect passersby outside the vehicle.

[0069] 12. The storage medium in the above embodiment is a storage medium (e.g., 302) that stores a program for causing a computer to function as each means of a driving assistance device (e.g., 101) attached to a moving body (e.g., 100), wherein the driving assistance device comprises: an imaging means (e.g., 401) that captures an image including an interior of the moving body and a subject outside the moving body; a recognition means (e.g., 402) that recognizes a person in the image using the captured image; an estimation means (e.g., 403) that estimates one or more characteristics of the person in the recognized image; a detection means (e.g., 404) that detects, as a passerby, a person in the image whose estimated one or more characteristics each satisfy a predetermined standard; and an output control means (e.g., 405) that causes an output device to output a warning regarding the presence of the detected passerby, wherein the one or more characteristics include at least one of a distance of the person from the driving assistance device, a moving speed of the person relative to the moving body, and a height of the person.

[0070] According to this embodiment, when a captured image including the interior of the vehicle and the exterior of the vehicle is used, it is possible to accurately detect passersby outside the vehicle.

[0071] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention.

[0072] 100... vehicle, 101... drive recorder, 201... control unit, 202... storage device, 203... imaging device, 205... output device

Claims

1. A driving assistance device attached to a moving body, comprising: an imaging means for capturing an image including the interior of the moving body and a subject outside the moving body; a recognition means for recognizing a person in the captured image using the captured image; an estimation means for estimating one or more characteristics of the person in the recognized image; a detection means for detecting a person in the image whose one or more estimated characteristics each satisfy a predetermined standard as a passerby; and an output control means for causing an output device to output a warning regarding the presence of the detected passerby, wherein the one or more characteristics include at least one of the distance of the person from the driving assistance device, the moving speed of the person relative to the moving body, and the height of the person.

2. The driving assistance device according to claim 1, characterized in that the detection means determines that the person in the image meets the criterion for the distance from the driving assistance device to the person when the distance from the person in the image to the driving assistance device is greater than a threshold corresponding to the interior of the moving body and is equal to or less than a threshold defining the distance.

3. The driving assistance device according to claim 1, characterized in that the detection means determines that the person in the image satisfies a criterion for the person's relative moving speed with respect to the moving body when the absolute value of the person's relative moving speed with respect to the moving body is greater than a predetermined threshold value.

4. The driving assistance device of claim 1, characterized in that the detection means determines that a person in the image meets a standard for person height if the person's height falls within a range determined according to the person's distance from the driving assistance device.

5. The driving assistance device according to claim 1, characterized in that the estimation means estimates one or more characteristics of persons in the recognized image other than persons having facial features matching pre-registered facial features.

6. A driving support device according to claim 1, characterized in that the output control means causes the output device to output a warning indicating the direction in which the detected pedestrian is present.

7. The driving assistance device according to claim 1, wherein the output device is included in the driving assistance device.

8. A driving assistance device as described in claim 1, characterized in that the output device communicates with the driving assistance device wirelessly or via a wired connection to cause the warning to be displayed on a display device of the output device.

9. The driving assistance device according to claim 8, wherein the output device is at least one of a smartphone, a tablet terminal, a head-mounted display, and a wearable terminal.

10. A driving assistance method executed by a driving assistance device attached to a moving body, comprising: capturing an image including the interior of the moving body and a subject outside the moving body; recognizing a person in the image using the captured image; estimating one or more characteristics of the person in the recognized image; detecting a person in the image whose one or more estimated characteristics each satisfy a predetermined criterion as a passerby; and outputting a warning regarding the presence of the detected passerby from an output device, wherein the one or more characteristics include at least one of the distance of the person from the driving assistance device, the moving speed of the person relative to the moving body, and the height of the person.

11. A program for causing a computer to function as each means of a driving assistance device attached to a moving body, the driving assistance device comprising: an imaging means for capturing an image including the interior of the moving body and a subject outside the moving body; a recognition means for recognizing a person in the captured image using the captured image; an estimation means for estimating one or more characteristics of the person in the recognized image; a detection means for detecting a person in the image whose one or more estimated characteristics each satisfy a predetermined standard as a passerby; and an output control means for causing an output device to output a warning regarding the presence of the detected passerby, wherein the one or more characteristics include at least one of the distance of the person from the driving assistance device, the moving speed of the person relative to the moving body, and the height of the person.

12. A storage medium storing a program for causing a computer to function as each means of a driving assistance device attached to a moving body, the driving assistance device comprising: an imaging means for capturing an image including the interior of the moving body and a subject outside the moving body; a recognition means for recognizing a person in the captured image using the captured image; an estimation means for estimating one or more characteristics of the person in the recognized image; a detection means for detecting a person in the image whose one or more estimated characteristics each satisfy a predetermined standard as a passerby; and an output control means for causing an output device to output a warning regarding the presence of the detected passerby, wherein the one or more characteristics include at least one of the distance of the person from the driving assistance device, the relative moving speed of the person with respect to the moving body, and the height of the person.