Driving assistance device, driving assistance method, storage medium and program

Abstract

A driving assistance device according to the present disclosure is mounted to a moving body and comprises: an image capture means that captures an image including subjects in the cabin of the moving body and outside the moving body; a recognition means that uses the captured image to recognize a person in the image; an estimation means that estimates one or more characteristics of the recognized person in the image; a detection means that detects, as a passerby, a person in the image having the estimated one or more characteristics each satisfying a predetermined criterion; and an output control means that causes a warning related to the presence of the detected passerby to be output from an output device, wherein the one or more characteristics include at least one from among the distance of the person from the driving assistance device, the movement speed of the person relative to the moving body, and the height of the person.

Description

Technical Field

[0001] The present invention relates to a driving support device, a driving support method, a storage medium, and a program.

Background Art

[0002] In recent years, there has been proposed a driving support technique in which an image of the surroundings of the host vehicle captured by a camera installed inside the vehicle (also referred to as a moving object) is analyzed, and when an object in the image is determined to be an object requiring attention, a warning image is displayed at the predicted position of the object requiring attention on a display device (Patent Document 1). If the driver is notified of warnings regarding attention targets at any time by the driving support technique, the driver can pay attention to highly dangerous targets and appropriately avoid risks.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, there are cases where it is desired to provide a driver with a warning for pedestrians passing by the side or rear of the host vehicle by a drive recorder installed inside the vehicle. In many cases, it is necessary to detect pedestrians around the vehicle using an image of a camera arranged so as to include the interior of the vehicle in the imaging range. In such a case, it is necessary to reduce false detections such as detecting a person inside the vehicle as a pedestrian, and accurately detect pedestrians outside the vehicle from the captured image.

[0005] The present invention has been made in view of the above problems, and an object thereof is to realize a technique capable of accurately detecting pedestrians outside the vehicle when using an imaging image including the interior of the vehicle and the outside of the vehicle.

Means for Solving the Problems

[0006] According to the present invention, A driving assistance device that is attached to a mobile vehicle, An imaging means for capturing an image including the interior of the moving body and a subject outside the moving body, A recognition means that recognizes a person in an image using an image that has been captured, An estimation means for estimating one or more characteristics of a person in a recognized image, A detection means for detecting a person in the image whose estimated characteristics each satisfy predetermined criteria as a passerby, The system includes an output control means that causes an output device to output a warning regarding the presence of the detected passerby, The provided driving assistance device is characterized in that one or more of the above characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person moving with respect to the moving object, and the height of the person. [Effects of the Invention]

[0007] According to the present invention, when using an image captured that includes both the interior and exterior of the vehicle, it becomes possible to accurately detect pedestrians outside the vehicle.

[0008] Other features and advantages of the present invention will become apparent from the following description with reference to the accompanying drawings. In the accompanying drawings, the same or similar components are given the same reference numeral. [Brief explanation of the drawing]

[0009] The attached drawings are included in the specification and constitute part thereof, illustrating embodiments of the present invention and are used together with the description to explain the principles of the present invention. [Figure 1] A schematic diagram showing an example of the configuration of a mobile body (vehicle) according to the embodiment. [Figure 2A] This figure schematically shows an example of the external configuration of a drive recorder as an example of a driver assistance device according to the embodiment. [Figure 2B] A schematic diagram showing the external configuration of the output device according to the embodiment. [Figure 3] Block diagram showing an example of the hardware configuration of a drive recorder according to this embodiment. [Figure 4] Block diagram showing an example of the functional configuration of a drive recorder according to the embodiment. [Figure 5] A diagram illustrating the characteristics (relative movement speed) of a person according to the embodiment. [Figure 6] A diagram illustrating the characteristics of a person according to the embodiment (distance from the dashcam). [Figure 7] A flowchart illustrating a series of operations of the driver assistance process according to the embodiment. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more features from the multiple features described in the embodiments may be combined arbitrarily. Furthermore, identical or similar configurations will be given the same reference numeral, and redundant descriptions will be omitted.

[0011] In this embodiment, the driving assistance device is described as a drive recorder that is installed inside the vehicle, captures images of the front, sides, and rear of the vehicle, and records the vehicle's movements on a recording medium. However, the driving assistance device may be a device other than a drive recorder, as long as it is installed in the vehicle and capable of capturing images of the outside world. Furthermore, in the following embodiment, the moving object is described as a vehicle, for example. The vehicle is typically a four-wheeled vehicle, but this embodiment may be applied to other types of vehicles.

[0012] Referring to FIG. 1, an example of the mounting position of the drive recorder will be described. In the vehicle 100, a drive recorder 101 is mounted. The drive recorder 101 is mounted at a position that does not obstruct the driver's view of the front glass of the vehicle 100 and near the rearview mirror, or on the rearview mirror of the vehicle 100. The mounting position of the drive recorder 101 is not limited to this, and it may be any position where the outside of the vehicle 100 can be imaged.

[0013] The drive recorder 101 can, for example, wirelessly or wiredly transmit information that needs to be output to the driver to an external output device 102, and cause the output device 102 to output the transmitted warning. In the description of this embodiment, the case where the output device 102 is an external device of the drive recorder 101 will be described as an example. However, the drive recorder 101 may be provided with an output device (that is, a display for display or a speaker for audio output).

[0014] <Example of the external configuration of the drive recorder> Referring to FIG. 2A, an example of the external configuration of the drive recorder 101 will be described. The drive recorder 101 includes an imaging device 202 on the front surface 201, and images the front direction (moving direction) of the vehicle 100. The imaging device 202 is, for example, an imaging device provided with a wide-angle lens. Further, the drive recorder 101 includes an imaging device 211 on the back surface 210, and images the sides and rear of the vehicle 100. The imaging device 211 is, for example, an imaging device provided with a fish-eye lens. The image captured by the imaging device 211 includes the driver of the vehicle 100. 203 constitutes the upper surface of the drive recorder 101.

[0015] <Example of the external configuration of the output device> Referring to FIG. 2B, an example of the external configuration of the output device 102 will be described. The output device 102 is a device for outputting information to the passengers of the vehicle 100. The output device 102 includes a display device (such as a display or a light-emitting element) for providing visual information and an acoustic device (such as a speaker) for providing auditory information. The output device 102 communicates with the drive recorder 101 wirelessly or via a wire to display information on the display device.

[0016] The output device 102 has a display device on its upper surface 220. The output device 102 can color and display a partial area (such as 222) within a circle 221 representing the surroundings of the vehicle 100 in response to an instruction from the drive recorder 101. The colored area 222 serves as a warning to output to the driver or the like that there is a risk in the corresponding direction. The output device 102 is installed, for example, on the dashboard near the driver's seat of the vehicle 100. The output device 102 may further include a half mirror to reflect the light output from the upper surface 220 in the direction of the driver.

[0017] <Example of the Hardware Configuration of the Drive Recorder> Referring to FIG. 3, an example of the hardware configuration of the drive recorder 101 will be described. The drive recorder 101 includes, for example, a control unit 301, a storage device 302, an imaging device 303, an input device 304, and a communication device 305.

[0018] The control unit 301 includes a processor such as a CPU (Central Processing Unit) and controls the overall operation of the drive recorder 101. The control unit 301 may be realized, for example, by the CPU expanding and executing a program stored in the storage device 302. In addition to the CPU, the control unit 301 may further include a GPU (Graphics Processing Unit) and dedicated circuits for image processing and machine learning.

[0019] The storage device 302 is a device that stores data and programs related to the operation of the drive recorder 101. For example, the storage device 302 may store a program that defines the operation of the drive recorder 101, temporary data during the execution of this program, images captured by the imaging device 303, and ambient sounds picked up by the microphone. The storage device 302 may be implemented using memory such as RAM (Random Access Memory) or ROM (Read Only Memory). Furthermore, the storage device 302 may include an auxiliary storage device such as an SSD (Solid State Drive).

[0020] The imaging device 303 includes a camera for imaging the external environment in front of the vehicle 100. The imaging device 303 may be, for example, a wide-angle camera. This camera is the imaging device 202 described above, located in front of the drive recorder 101 (front side of the vehicle 100). The imaging device 303 also includes a camera capable of imaging the rear and left and right sides of the vehicle 100, i.e., the imaging device 211 described above. This camera may be a camera equipped with a fisheye lens. The camera for imaging the rear is located behind the drive recorder (rear side of the vehicle). In this embodiment, the case where the rear and sides of the vehicle are imaged with the same camera is described as an example. However, the camera for imaging the rear and the camera for imaging the sides may be separate cameras. The camera for imaging the sides may be located on the side of the drive recorder. Some of the cameras that image in multiple directions may be separate from the drive recorder.

[0021] The input device 304 is a device for receiving instructions from the occupants of the vehicle 100. The input device 304 may include, for example, buttons or a touch panel. The input device 304 may also include a microphone for receiving the occupants' voices and sounds from the outside world.

[0022] The communication device 305 is a device for the drive recorder 101 to communicate with the output device 102 by wire or wireless connection. The communication device 305 transmits warning information generated by the driving assistance processing in the control unit 301 to the output device 102. The communication device 305 may communicate with other devices by wire or wireless connection. Wireless communication may be communication compliant with a protocol such as short-range communication (e.g., Bluetooth®). The communication device 305 may be configured to communicate with the control ECU of the vehicle 100 by wire or wireless connection. The communication device 305 may be omitted.

[0023] <Example of a dashcam's functional configuration> Referring to Figure 4, an example of the functional configuration realized by the control unit 301 of the drive recorder 101 will be described. Each part shown in Figure 4 is realized, for example, by the CPU of the control unit 301 executing a computer program.

[0024] The image acquisition unit 401 acquires a forward-facing image from the imaging device 202, which images the area in front of the drive recorder 101. The image acquisition unit 401 also acquires a rear-facing image from the imaging device 211, which images the area behind the drive recorder 101. The rear-facing image includes both subjects outside the vehicle and the interior of the vehicle. For example, the rear-facing image may include pedestrians walking outside the vehicle 100 and occupants inside the vehicle.

[0025] The person recognition unit 402 recognizes a person included in the image (rear-view image) acquired by the image acquisition unit 401. The person recognition unit 402 may use a neural network that takes the rear-view image as input and outputs a person recognition result. The person recognition unit 402 may also be able to recognize a part of a person as a person if at least a part of the person's body is included in the image.

[0026] The characteristic estimation unit 403 estimates one or more characteristics of a person in the image recognized by the person recognition unit 402. One or more characteristics include, for example, the distance of the person from the drive recorder 101, the relative speed of the person relative to the vehicle 100, and the person's height. The characteristics of the person will be described later.

[0027] The pedestrian detection unit 404 determines whether one or more characteristics of a person estimated by the characteristic estimation unit 403 meet predetermined criteria, and detects a person who meets all of those criteria as a pedestrian. The criteria that the characteristics must meet will be described later.

[0028] The output control unit 405 causes the output device 102 to output a warning when the detected pedestrian approaches a predetermined position within a range from the vehicle 100's travel path. For example, the control unit 301 estimates the predicted trajectory for the detected pedestrian and, if a warning is necessary, causes the output device 102 to output a warning indicating the direction in which the pedestrian is located.

[0029] Next, with reference to Figures 5 and 6, the characteristics of the person according to this embodiment and the criteria that those characteristics must satisfy will be described.

[0030] Figure 5 shows a specific example where the characteristic of a person is their relative speed to the vehicle 100. For example, assume that people 501 and 502 are riding in the vehicle 100, and person 503 is walking outside the vehicle. The characteristic estimation unit 403 can estimate the predicted trajectory and speed of the vehicle according to changes in the (time-series) forward-captured image or the (time-series) rear-captured image. The characteristic estimation unit 403 can also estimate the speeds of people 501, 502, and 503 using the (time-series) rear-captured image. For example, the characteristic estimation unit 403 estimates the absolute value of the relative speed of people 501 and 502 to the vehicle 100 based on changes in their areas or their predicted trajectories. The characteristic estimation unit 403 also estimates the absolute value of the relative speed of person 503 to the vehicle 100 based on changes in person 503's area or person 503's predicted trajectory. Furthermore, the speed of a person's movement may be estimated using any known method, not limited to the method described above. For example, it may be estimated by inputting a rearward-facing image into a neural network.

[0031] The pedestrian detection unit 404 determines that a person in the image meets the criteria for relative movement speed with respect to the vehicle 100 if the absolute value of the person's relative movement speed with respect to the vehicle 100 is greater than a predetermined threshold. For example, the absolute values ​​of the relative movement speeds of person 501 and person 502 with respect to the vehicle 100 are generally small (i.e., less than the threshold). In this case, for example, the pedestrian detection unit 404 determines that person 501 and person 502 do not meet the criteria for relative movement speed and does not detect person 501 and person 502 as pedestrians. On the other hand, if the absolute value of person 503's relative movement speed with respect to the vehicle 100 exceeds the threshold, the pedestrian detection unit 404 determines that person 503 meets the criteria for relative movement speed and detects them as a pedestrian. In this way, by considering the relative movement speed of people, it is possible to exclude people with movement speeds unsuitable for pedestrians from processing, and pedestrians outside the vehicle can be detected with high accuracy.

[0032] Figure 6 shows a specific example where the person's characteristic is their distance from the drive recorder 101. Similar to Figure 5, for example, assume that people 501 and 502 are inside vehicle 100, and person 503 is walking outside the vehicle.

[0033] The characteristic estimation unit 403 estimates the distance from the drive recorder 101 to persons 501, 502, and 503 using the (time-series) rear-view images. The distance from the drive recorder 101 to the persons may be estimated by a known method, for example, by inputting the rear-view images into a neural network.

[0034] The pedestrian detection unit 404 determines that if the distance of persons 501 and 502 from the drive recorder 101 is below a threshold corresponding to the interior of the vehicle 100 (for example, a threshold indicating the range of 601), persons 501 and 502 do not meet the distance criterion. Therefore, persons 501 and 502 are not detected as pedestrians.

[0035] On the other hand, the pedestrian detection unit 404 determines that if the distance of person 503 from the drive recorder 101 is greater than the threshold corresponding to the interior of the vehicle 100 and less than or equal to a threshold defining far distance (for example, a threshold indicating the range of 602), then person 503 meets the criteria for distance from the drive recorder. Therefore, person 503 is detected as a pedestrian. In this way, the drive recorder 101 By considering the distance from the vehicle, individuals located at an unsuitable distance for being pedestrians can be excluded from processing, enabling accurate detection of pedestrians outside the vehicle.

[0036] A person's characteristic may also be their height. Generally, the size of a person's face varies little from person to person, making it a useful parameter when detecting a person from an image. However, when a person is far away from the imaging device, the size of their face in the image becomes very small. Therefore, in this embodiment, a person's height is used as a person's characteristic in order to perform an effective evaluation using image information even when a person is far away from the imaging device. However, the size of a person's face may also be used as a person's characteristic.

[0037] The characteristic estimation unit 403 estimates the height of a person based on the size of the person's region in the image. The person's height may be estimated using a known method, for example, by inputting the back-view image into a neural network.

[0038] The pedestrian detection unit 404 determines that a person meets the height criteria if their height falls within a range determined according to the distance of the person from the drive recorder 101. When estimating a person's characteristics, such as height, from captured images, the estimation error tends to increase as the distance to the person increases. Therefore, in this embodiment, an acceptable range of error in estimated height is predetermined for each distance from the drive recorder 101. In this way, in this embodiment, if the estimated height exceeds the acceptable range of error, the person is not detected as a pedestrian. In this way, by considering the person's height, it is possible to exclude people with heights outside the range that are unsuitable for being pedestrians from processing, and pedestrians outside the vehicle can be detected with high accuracy.

[0039] Next, with reference to Figure 7, a series of operations of the driving assistance processing performed in the drive recorder 101 will be described. This processing is realized by the CPU of the control unit 301 executing a computer program stored in the storage device 302.

[0040] In the following explanation, the process for detecting pedestrians is described using the example of detecting people in an image who meet predetermined criteria as pedestrians. However, the process for detecting pedestrians may include other arbitrary detection processes. In this embodiment, by not detecting people in an image whose characteristics do not meet predetermined criteria as pedestrians, false detections of pedestrians can be reduced, and as a result, pedestrians outside the vehicle can be detected with high accuracy.

[0041] In S701, the image acquisition unit 401 acquires a rear-view image captured from behind the drive recorder 101. The rear-view image includes subjects outside the vehicle and the interior of the vehicle. In S702, the person recognition unit 402 recognizes a person in the image. In S703, the characteristic estimation unit 403 estimates one or more characteristics of the person in the image. As described above, the characteristics of the person include, for example, the distance of the person from the drive recorder 101, the relative speed of the person relative to the vehicle 100, and the height of the person.

[0042] In S704, the pedestrian detection unit 404 determines whether the estimated characteristics meet predetermined criteria. Specifically, as explained with reference to Figures 4, 5, and 6, it determines whether the estimated characteristics, such as the person's height, meet the respective predetermined criteria. If the pedestrian detection unit 404 determines that the characteristics of the pedestrian meet the predetermined criteria (all criteria if multiple characteristics are handled), it proceeds to S705; otherwise, it terminates this series of processes. In S705, the pedestrian detection unit 404 detects the person being processed as a pedestrian. For example, in the examples of Figures 5 and 6 above, person 503 is detected as a pedestrian.

[0043] In S706, the output control unit 405 determines whether it is necessary to output a warning to the driver regarding the detected pedestrian. For example, the output control unit 405 determines that it is necessary to output a warning if the detected pedestrian approaches a predetermined range from the vehicle 100's travel trajectory. If the output control unit 405 determines that it is necessary to output a warning, it proceeds to S707; otherwise, it terminates this series of processes.

[0044] In S707, the output control unit 405 causes the output device 102 to output a warning indicating the direction in which a pedestrian is located (for example, area 222 in Figure 2B). The output control unit 405 causes the output device 102 to output a warning indicating the direction in which a pedestrian is located, for example, based on the trajectory already obtained for the detected pedestrian.

[0045] In the above-described embodiment, in order to detect pedestrians present around the vehicle, the system determines whether the characteristics of the person meet the criteria. 301 Before the characteristic estimation unit 403 performs the process of estimating the characteristics of individuals, the unit may perform a process to exclude registered crew members from the processing target. For example, the control unit 301 The driver registers the occupant's face in the drive recorder 101 in advance (through user operation, etc.). Then, the control unit... 301 For example, the system may identify individuals in the recognized image whose facial features match those of pre-registered individuals. The characteristic estimation unit 403 only needs to estimate one or more characteristics of individuals other than the identified person. In this way, unnecessary estimation processing for occupants can be omitted, resulting in faster processing and reduced power consumption.

[0046] Furthermore, although the above-described embodiment explained the case in which a warning is output by the output device 102, the warning may also be output by at least one of a smartphone, tablet terminal, head-mounted display, and wearable terminal. These devices communicate with the drive recorder 101 wirelessly or via a wired connection to display the warning on the display device.

[0047] <Summary of Embodiments> 1. The driving support device (for example, 101) of the above embodiment is: A driving assistance device that is attached to a mobile body (for example, 100), An imaging means (e.g., 401) for capturing an image including the interior of the moving body and a subject outside the moving body, A recognition means (e.g., 402) that recognizes a person in an image using the captured image, Estimation means (e.g., 403) for estimating one or more characteristics of a person in a recognized image, A detection means (e.g., 404) for detecting a person in the image as a passerby if one or more of the estimated characteristics each satisfy predetermined criteria, The system includes an output control means (e.g., 405) that causes an output device to output a warning regarding the presence of the detected passerby, The one or more characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person relative to the moving object, and the height of the person.

[0048] According to this embodiment, when using an image captured that includes both the interior and exterior of the vehicle, it becomes possible to accurately detect pedestrians outside the vehicle.

[0049] 2. In the above embodiment of the driving assistance device, The detection means determines that the person in the image meets the criteria for the person's distance from the driving assistance device if the distance of the person in the image from the driving assistance device is greater than a threshold corresponding to the interior of the moving object and less than or equal to a threshold defining far away.

[0050] According to this embodiment, the drive recorder 101 By considering the distance from the vehicle, individuals located at an unsuitable distance for being pedestrians can be excluded from processing, enabling accurate detection of pedestrians outside the vehicle.

[0051] 3. In the above embodiment of the driving assistance device, The detection means determines that the person in the image meets the criteria for the person's relative movement speed to the moving object if the absolute value of the person's relative movement speed to the moving object is greater than a predetermined threshold.

[0052] According to this embodiment, by taking into account the relative movement speed of the person, it is possible to exclude from processing people who are moving at a speed unsuitable for being pedestrians, and to accurately detect pedestrians outside the vehicle.

[0053] 4. In the driving assistance device of the above embodiment, The detection means determines that the person in the image meets the height criteria if the person's height falls within a range determined according to the distance of the person from the driving assistance device.

[0054] According to this embodiment, by taking into account the height of the person, it is possible to exclude from processing individuals whose height falls outside the range of what is considered a suitable pedestrian, and thus enable accurate detection of pedestrians outside the vehicle.

[0055] 5. In the driving assistance device of the above embodiment, The estimation means estimates one or more characteristics of people in the recognized image, excluding those whose facial features match those of pre-registered facial features.

[0056] According to this embodiment, unnecessary estimation processing for the occupants can be omitted, resulting in faster processing and reduced power consumption.

[0057] 6. In the above embodiment of the driving assistance device, The output control means causes the output device to output a warning indicating the direction in which the detected pedestrian is located.

[0058] According to this embodiment, the driver can quickly and easily determine where the warning object is located in their surroundings.

[0059] 7. In the driving assistance device of the above embodiment, The output device is included in the driving support device.

[0060] According to this embodiment, it becomes possible to provide driving assistance with a configuration that is easy to install using a standalone driving assistance device.

[0061] 8. In the driving assistance device of the above embodiment, The output device communicates with the driving assistance device wirelessly or via a wired connection to display the warning on the output device's display device.

[0062] According to this embodiment, it becomes possible to place the output device in a position that is easily perceptible to the driver.

[0063] 9. In the driving assistance device of the above embodiment, The output device is at least one of a smartphone, a tablet device, a head-mounted display, and a wearable device.

[0064] According to this embodiment, a separately used device can be used as an output device, making it possible to reduce the manufacturing cost of the driver assistance system.

[0065] 10. The driving assistance method in the above embodiment is: A driving assistance method performed by a driving assistance device (e.g., 101) attached to a mobile body (e.g., 100), To capture an image including the interior of the moving object and a subject outside the moving object (for example, S701), Recognizing a person in an image using the captured image (for example, S702), Estimating one or more characteristics of a person in a recognized image (e.g., S703), Detecting a person in the image whose estimated characteristics each satisfy predetermined criteria as a passerby (for example, S704), The system includes outputting a warning from an output device regarding the presence of the detected passerby (for example, S705), The one or more characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person relative to the moving object, and the height of the person.

[0066] According to this embodiment, when using an image captured that includes both the interior and exterior of the vehicle, it becomes possible to accurately detect pedestrians outside the vehicle.

[0067] 11. The program in the above embodiment is A program for causing a computer to function as a means of a driving assistance device (e.g., 101) attached to a mobile body (e.g., 100), wherein the driving assistance device is An imaging means (e.g., 401) for capturing an image including the interior of the moving body and a subject outside the moving body, A recognition means (e.g., 402) that recognizes a person in an image using the captured image, Estimation means (e.g., 403) for estimating one or more characteristics of a person in a recognized image, A detection means (e.g., 404) for detecting a person in the image as a passerby if one or more of the estimated characteristics each satisfy predetermined criteria, The system includes an output control means (e.g., 405) that causes an output device to output a warning regarding the presence of the detected passerby, The one or more characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person relative to the moving object, and the height of the person.

[0068] According to this embodiment, when using an image captured that includes both the interior and exterior of the vehicle, it becomes possible to accurately detect pedestrians outside the vehicle.

[0069] 12. The storage medium in the above embodiment is A storage medium (e.g., 302) for storing a computer that stores programs for causing a driving assistance device (e.g., 101) to function as a means of a driving assistance device (e.g., 101) attached to a mobile body (e.g., 100), wherein the driving assistance device is An imaging means (e.g., 401) for capturing an image including the interior of the moving body and a subject outside the moving body, A recognition means (e.g., 402) that recognizes a person in an image using the captured image, Estimation means (e.g., 403) for estimating one or more characteristics of a person in a recognized image, A detection means (e.g., 404) for detecting a person in the image as a passerby if one or more of the estimated characteristics each satisfy predetermined criteria, The system includes an output control means (e.g., 405) that causes an output device to output a warning regarding the presence of the detected passerby, The one or more characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person relative to the moving object, and the height of the person.

[0070] According to this embodiment, when using an image captured that includes both the interior and exterior of the vehicle, it becomes possible to accurately detect pedestrians outside the vehicle.

[0071] The invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of the gist of the invention. [Explanation of Symbols]

[0072] 100...Vehicle, 101...Drive recorder, 201...Control unit, 202...Storage device, 203...Imaging device, 205...Output device

Claims

1. A driving assistance device that is attached to a mobile vehicle, An imaging means for capturing an image including the interior of the moving body and a subject outside the moving body, A recognition means that recognizes a person in an image using an image that has been captured, An estimation means for estimating one or more characteristics of a person in a recognized image, A detection means for detecting a person in the image whose estimated characteristics each satisfy predetermined criteria as a passerby, The system includes an output control means that causes an output device to output a warning regarding the presence of the detected passerby, A driving assistance device characterized in that one or more of the above characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person moving with respect to the moving object, and the height of the person.

2. The driving assistance device according to claim 1, wherein the detection means determines that the person in the image meets the criteria for the person's distance from the driving assistance device when the distance of the person in the image from the driving assistance device is greater than a threshold corresponding to the interior of the moving object and less than or equal to a threshold defining far away.

3. The driving assistance device according to claim 1, characterized in that the detection means determines that the person in the image meets the criteria for the person's relative movement speed with respect to the moving object if the absolute value of the person's relative movement speed with respect to the moving object is greater than a predetermined threshold.

4. The driving assistance device according to claim 1, characterized in that the detection means determines that the person in the image meets the height criteria for the person if the person's height falls within a range determined according to the distance of the person from the driving assistance device.

5. The driving assistance device according to claim 1, characterized in that the estimation means estimates one or more characteristics of persons in the recognized image other than those whose facial features match those of persons

6. The driving assistance device according to claim 1, characterized in that the output control means causes the output device to output a warning indicating the direction in which the detected pedestrian is located.

7. The driver assistance device according to claim 1, characterized in that the output device is included in the driver assistance device.

8. The driver assistance device according to claim 1, characterized in that the output device communicates with the driver assistance device wirelessly or via a wired connection to cause the warning to be displayed on the display device of the output device.

9. The driving assistance device according to claim 8, characterized in that the output device is at least one of a smartphone, a tablet device, a head-mounted display, and a wearable device.

10. A driving assistance method performed by a driving assistance device attached to a mobile vehicle, To capture an image that includes the interior of the moving body and a subject outside the moving body, Recognizing people in an image using the captured image, To estimate one or more characteristics of a person in a recognized image, Detecting a person in the image whose estimated characteristics each satisfy predetermined criteria as a passerby, The system includes, , outputting a warning from an output device regarding the presence of the detected passerby, A driving assistance method characterized in that the one or more characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person moving with respect to the moving object, and the height of the person.

11. A program for causing a computer to function as one of the means of a driving assistance device attached to a mobile body, wherein the driving assistance device is An imaging means for capturing an image including the interior of the moving body and a subject outside the moving body, A recognition means that recognizes a person in an image using an image that has been captured, An estimation means for estimating one or more characteristics of a person in a recognized image, A detection means for detecting a person in the image whose estimated characteristics each satisfy predetermined criteria as a passerby, The system includes an output control means that causes an output device to output a warning regarding the presence of the detected passerby, A program characterized in that one or more of the above characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person relative to the moving object, and the height of the person.

12. A storage medium for storing a computer that stores a program for causing a driving assistance device to function as one of the means of a driving assistance device attached to a mobile body, wherein the driving assistance device is An imaging means for capturing an image including the interior of the moving body and a subject outside the moving body, A recognition means that recognizes a person in an image using an image that has been captured, An estimation means for estimating one or more characteristics of a person in a recognized image, A detection means for detecting a person in the image whose estimated characteristics each satisfy predetermined criteria as a passerby, The system includes an output control means that causes an output device to output a warning regarding the presence of the detected passerby, A storage medium characterized in that one or more of the above characteristics include at least one of the distance of the person from the driving assistance device, the relative speed of the person moving with respect to the moving object, and the height of the person.

Images