Travel assistance device, travel assistance method, and recording medium

Abstract

Provided are a travel assistance device, a travel assistance method, and a recording medium that make it possible to more quickly and accurately notify a vehicle or the like that is traveling a road of a person or the like that is in a blind spot.　A travel assistance device according to the present invention comprises a first acquisition means that acquires first image data for ascertaining the situation around a road from a first camera that can capture images of the road, a second acquisition means that acquires second image data obtained by capturing images in the advancement direction of a vehicle that is traveling the road from a second camera that is installed on the vehicle, a detection means that detects subjects in a prescribed area and the orientations thereof in each of the first image data and the second image data, a comparison means that compares the numbers of subjects in each orientation, and a notification means that gives a prescribed warning to the vehicle when the numbers of subjects in each orientation do not match in the results of the comparison.

Description

Driving support device, driving support method, and recording medium

[0001] The present invention relates to a driving assistance device, a driving assistance method, and a recording medium.

[0002] Patent Document 1 discloses a driving assistance control device that can improve driving safety by acquiring information on areas in the driver's blind spot and assisting the driver. According to the document, the driving assistance control device includes a first image data acquisition unit that acquires image data of a pedestrian crossing area captured by a vehicle camera, and a second image data acquisition unit that acquires image data captured by an external camera. Furthermore, the driving assistance control device compares a first shape of a pedestrian in the crossing area estimated based on the image data captured by the vehicle camera with a second shape of the pedestrian in the crossing area captured by the external camera. The driving assistance control device then issues an alert when the first shape and the second shape differ.

[0003] Japanese Patent Application Laid-Open No. 2022-148428

[0004] The driving assistance control device in Patent Document 1 focuses on pedestrian crossing areas, such as crosswalks, and compares the number of pedestrians (referred to as a first mode and a second mode). However, in order to warn vehicles earlier, it is necessary to also identify pedestrians on the sidewalk before they enter the crosswalk. On the other hand, if the area for identifying pedestrians is expanded, people who are not about to cross the road will also be detected. Therefore, the invention in Patent Document 1 has the problem of not being able to accurately warn vehicles that may cross these pedestrians.

[0005] The present disclosure aims to provide a driving assistance device, a driving assistance method, and a recording medium that can notify vehicles traveling on a road of a person or the like located in a blind spot earlier and more accurately.

[0006] According to a first aspect, there is provided a driving assistance device comprising: a first acquisition means for acquiring first image data for grasping the situation around the road from a first camera capable of photographing the road; a second acquisition means for acquiring second image data photographing the direction of travel of the vehicle from a second camera mounted on a vehicle traveling on the road; a detection means for detecting subjects and their orientations in a predetermined area captured in the first image data and the second image data; a comparison means for comparing the number of subjects for each orientation; and a notification means for issuing a predetermined warning to the vehicle if the number of subjects for each orientation does not match as a result of the comparison.

[0007] According to a second aspect, there is provided a driving assistance method that acquires first image data for grasping the situation around the road from a first camera that can photograph the road, acquires second image data that captures the direction in which the vehicle is traveling from a second camera mounted on a vehicle traveling on the road, detects objects and their orientations in a specified area that are captured in the first image data and the second image data, compares the number of objects for each orientation for the objects, and if the number of objects for each orientation does not match as a result of the comparison, notifies the vehicle of a specified warning.

[0008] According to a third aspect, there is provided a recording medium having recorded thereon a program that causes a computer to execute the following processes: a process of acquiring first image data for grasping the situation around the road from a first camera capable of photographing the road; a process of acquiring second image data photographing the direction in which the vehicle is traveling from a second camera mounted on a vehicle traveling on the road; a process of detecting objects and their orientations in a specified area that are captured in the first image data and the second image data; a process of comparing the number of objects for each orientation for the objects; and a process of issuing a specified warning to the vehicle if the number of objects for each orientation does not match as a result of the comparison.

[0009] According to the present disclosure, it is possible to provide a driving assistance device, a driving assistance method, and a recording medium that can more quickly and accurately notify vehicles traveling on a road of people or the like located in blind spots.

[0010] FIG. 1 is a diagram illustrating one configuration of the present disclosure. FIG. 2 is a flowchart illustrating the operation of the present disclosure. FIG. 3 is a diagram for explaining the operation of the present disclosure. FIG. 4 is a diagram for explaining the operation of the present disclosure. FIG. 5 is a diagram illustrating one configuration of the present disclosure. FIG. 6 is a diagram illustrating an example of a marker indicating the position and movement of a subject. FIG. 7 is a flowchart illustrating the operation of the present disclosure. FIG. 8 is a diagram for explaining the operation of the present disclosure. FIG. 9 is another diagram for explaining the operation of the present disclosure. FIG. 10 is a diagram illustrating another configuration of the present disclosure. FIG. 11 is another flowchart illustrating the operation of the present disclosure. FIG. 12 is another diagram for explaining the operation of the present disclosure. FIG. 13 is a diagram illustrating the configuration of a computer constituting a driving assistance device of the present disclosure.

[0011] First, an overview of one embodiment of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. The reference numerals in the drawings attached to this overview are attached to each element for convenience as an example to facilitate understanding, and are not intended to limit the present disclosure to the illustrated form. Furthermore, connecting lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional lines. Unidirectional arrows are used to schematically indicate the flow of main signals (data) and do not exclude bidirectionality. A program is executed via a computer device, which includes, for example, a processor, a storage device, an input device, a communication interface, and, if necessary, a display device. Furthermore, this computer device is configured to be able to communicate with internal or external devices (including computers) via the communication interface, whether wired or wireless. Although ports or interfaces are present at the input / output connection points of each block in the drawings, they are not shown.

[0012] In one embodiment, the present disclosure can be realized by a driving assistance device 10 including, as shown in FIG. 1 , a first acquisition unit 11, a second acquisition unit 12, a detection unit 13, a comparison unit 14, and a notification unit 15. More specifically, the first acquisition unit 11 acquires first image data for grasping the conditions around a road from a first camera C1 capable of capturing images of the road. The second acquisition unit acquires second image data capturing an image of the vehicle's traveling direction from a second camera C2 mounted on a vehicle traveling on the road. The detection unit 13 detects objects and their orientations in a predetermined area captured in the first image data and the second image data. The comparison unit 14 compares the number of objects for each orientation. If the comparison results in a discrepancy in the number of objects for each orientation, the notification unit 15 issues a predetermined warning to the vehicle.

[0013] The driving assistance device 10 configured as described above operates as follows: First, the driving assistance device 10 acquires first image data for grasping the conditions around the road from the first camera C1 capable of photographing the road (step S01 in FIG. 2).

[0014] Next, the driving assistance device 10 acquires second image data from a second camera mounted on the vehicle traveling on the road, the second image data capturing the image in the traveling direction of the vehicle (step S02 in FIG. 2). Note that the order of steps S01 and S02 in FIG. 2 may be reversed, or the first and second image data may be acquired in parallel.

[0015] Next, the driving assistance device 10 detects the subject and its orientation in a predetermined area captured in the first image data and the second image data (step S03 in FIG. 2 ). For example, if the subject is a pedestrian, the orientation of the subject can be determined from the direction of the face, toes, etc. Here, the description will be given assuming that a parked vehicle V2 and pedestrians P1 to P5 are present around a crosswalk as shown in FIG. 3 . In FIG. 3 , the top of the figure is north, and the bottom of the figure is south. Pedestrians facing upward (north) in the figure are represented as "PN," and pedestrians facing downward (south) in the figure are represented as "PS." In the example of FIG. 3 , only P3 is the pedestrian PN facing upward (north), and pedestrians PS facing downward (south) in the figure are P1 and P2.

[0016] Next, the driving assistance device 10 compares the number of subjects for each orientation (step S04 in FIG. 2). If the comparison result shows that the number of subjects for each orientation does not match (No in step S05 in FIG. 2), the driving assistance device 10 issues a predetermined warning to the vehicle (step S06 in FIG. 2). This predetermined warning may include a notice that the number of subjects does not match. On the other hand, if the number of subjects for each orientation matches (Yes in step S05 in FIG. 2), the driving assistance device 10 omits the subsequent processing.

[0017] For example, as shown in FIG. 4, image data captured by the first camera C1 captures pedestrians P1 to P5, and the counts are PN=1 and PS=2. On the other hand, image data captured by the second camera C2 of vehicle V1 may capture pedestrian P1 in the shadow of parked vehicle V2, making it impossible to detect pedestrian P1. In this case, the counts are PN=1 and PS=1. In this case, the driving assistance device 10 can issue a predetermined warning to vehicle V1, informing it that the number of detected subjects (pedestrians) does not match. This allows the driver of vehicle V1 to be notified that a pedestrian is in the blind spot of the second camera.

[0018] In a more preferred embodiment, the driving assistance device 10 can issue a predetermined warning to the vehicle V1, informing the vehicle V1 of the orientation of the subject inconsistent with the direction of the inconsistency. For example, in the example described above, the number of pedestrians PN facing upward (north) in the figure matches, but the number of pedestrians PS facing downward (south) in the figure does not match. In this case, the driving assistance device 10 can notify the vehicle V1 that a pedestrian may be coming out of a blind spot from the left side (north direction in FIG. 4 ) to the right (south) as viewed from the vehicle V1.

[0019] As described above, according to the present disclosure, it is possible to more quickly and accurately notify a vehicle V1 or the like traveling on a road of a person or the like located in a blind spot. In the above example, pedestrians P4 and P5 are not facing either direction, and therefore are not included in either PN or PS. This makes it possible to exclude from the determination targets subjects (pedestrians) who do not cross the crosswalk, i.e., who are unlikely to cross the vehicle V1. Of course, it is also possible to count these pedestrians who are not facing either direction, compare the counts, and notify the vehicle V1 if there is a discrepancy.

[0020] In the above explanation, an example of detecting pedestrians from images captured by the first and second cameras C1 and C2 was given, but it is also possible to detect objects other than pedestrians. In this case, too, the number of pedestrians can be counted for each direction, compared, and if there is a mismatch, the vehicle can be notified.

[0021] [First Embodiment] Next, a first embodiment will be described, in which not only the orientation of a subject but also the movement of the subject is measured. Fig. 5 is a diagram showing one configuration of the present disclosure. Referring to Fig. 5, a driving assistance device 100 including a first acquisition unit 101, a second acquisition unit 102, a detection unit 103, a matching unit 104, a notification unit 105, and a movement measurement unit 106 is shown.

[0022] The first acquisition unit 101 is connected via a network to a camera C1 that is installed to grasp the situation around a crosswalk on a road. The first acquisition unit 101 acquires first image data of an image of the area around the crosswalk on a road from the camera C1.

[0023] The second acquisition unit 102 acquires image data from a camera C2 mounted on a vehicle heading toward the location where the crosswalk is installed. This camera C2 is a camera that captures images in the direction of travel of the vehicle, and the image data from the camera C2 captures the location where the crosswalk is installed. Therefore, the first acquisition unit 101 and the second acquisition unit 102 correspond to the first acquisition means 11 and the second acquisition means 12, respectively. Note that the vehicle and the driving assistance device 100 can exchange data and the like using 5G (fifth generation mobile communication system, including local 5G), LTE (Long Term Evolution), or other short-range wireless communication methods.

[0024] The detection unit 103 detects the position of an object around the crosswalk that is captured in the image data of the camera C1, and sends the image data together with the movement measurement unit 106. The detection unit 103 also detects an object near the crosswalk that is captured in the image data of the camera C2, and sends the image data together with the movement measurement unit 106. Therefore, the detection unit 103 corresponds to the detection means 13 described above. In the following explanation, it is assumed that the object to be detected by the detection unit 103 is a pedestrian.

[0025] The movement measurement unit 106 tracks the pedestrians detected by the detection unit 103, measures the movement direction and speed for each of the cameras C1 and C2, and sends the measurement results to the matching unit 104. In this embodiment, this movement direction is used as the orientation of the subject. Furthermore, since the movement measurement unit 106 measures the speed of the subject, it corresponds to a speed measurement means and a movement direction measurement means.

[0026] The matching unit 104 compares the number of pedestrians for each direction detected from the image data of camera C1 with the number of pedestrians for each direction detected from the image data of camera C2. If the comparison result shows that the number of pedestrians for a certain direction detected from the image data of camera C1 is greater than the number of pedestrians for the same direction detected from the image data of camera C2, the matching unit 104 requests the notification unit 105 to notify the vehicle. Therefore, the matching unit 104 corresponds to the above-mentioned matching means 14.

[0027] When requested by the comparison unit 104 to notify a vehicle, the notification unit 105 transmits a notification to the vehicle informing the vehicle that there may be a pedestrian in its blind spot. This notification is transmitted in a form that can be output from, for example, a display device or speaker of the vehicle's car navigation system. It is also desirable that this notification include the direction and speed of movement of the subject measured by the movement measurement unit 106.

[0028] The first acquisition unit 101, the detection unit 103, and the movement measurement unit 106 may be provided in a device that generates target object information indicating the type of object on the road and its moving direction and speed from a camera image. Therefore, the driving assistance device 100 can also be configured by adding the second acquisition unit 102, the matching unit 104, and the notification unit 105 to a device that can generate target object information.

[0029] Next, the operation of this embodiment will be described in detail with reference to the drawings. Fig. 6 is a flowchart showing the operation of this disclosure. Referring to Fig. 6, first, the driving assistance device 100 acquires image data from the camera C1 and image data from the camera C2 (steps S001 and S002).

[0030] Next, the driving assistance device 100 detects a subject (pedestrian) in the acquired image data (step S003). This subject detection mechanism can use various technologies known as object recognition technologies. For example, the detection unit 103 can employ a method of detecting an object in an image using an inference model created in advance by machine learning or the like.

[0031] Next, the driving assistance device 100 tracks the position of the subject (pedestrian) and analyzes the movement of the subject (pedestrian) (step S004). FIG. 7 shows an example of a marker representing the movement direction and speed of the pedestrian measured by the movement measurement unit 106. In the example of FIG. 7, the center of the marker circle represents the position (x, y) of the pedestrian. The direction of the line segment extending from the marker circle represents the movement direction of the pedestrian, and the length of the line segment represents the speed. This movement direction may be an azimuth angle, or it may be the orientation of the subject (pedestrian) relative to the road.

[0032] Next, the driving assistance device 100 compares the number of subjects (pedestrians) detected in each direction of movement from the image data of the camera C1 with the number of subjects (pedestrians) detected in each direction of movement from the image data of the camera C2 (step S005). During this comparison, strict matching of the movement direction is not performed, and it may be determined, for example, whether the subject is moving in the forward direction or the reverse direction relative to the direction perpendicular to the road.

[0033] If, as a result of the comparison, the number of subjects (pedestrians) detected by camera C1 in either direction of movement is greater than the number of subjects (pedestrians) detected by camera C2 (Yes in step S006), the driving assistance device 100 issues a warning to the vehicle (step S007).

[0034] On the other hand, if the number of subjects (pedestrians) detected by camera C2 matches the number of subjects (pedestrians) detected by camera C1 in all directions, the driving assistance device 100 does nothing (No in step S006). Alternatively, if the number of subjects (pedestrians) detected by camera C2 is greater than the number of subjects (pedestrians) detected by camera C1, the driving assistance device 100 also does nothing (No in step S006). This is because if the number of subjects (pedestrians) detected by camera C2 is greater than the number of subjects (pedestrians) detected by camera C1, it is highly likely that the subjects (pedestrians) are captured by the vehicle's camera C2. In this embodiment, notifications to the vehicle are suppressed in such cases. By reducing the number of notifications sent to the vehicle in this way, the effectiveness of the alert notification can be improved.

[0035] Figure 8 is a diagram plotting pedestrians and vehicles detected from images captured by camera C1 on a map. Camera C1 is able to capture pedestrians P1 to P5 because it has few blind spots caused by parked vehicle V2. Figure 9 is a diagram plotting pedestrians and vehicles detected from images captured by camera C2 on a map. Camera C2 is unable to detect pedestrians P1, P4, and P6 because it has a blind spot caused by parked vehicle V2.

[0036] In such a case, the driving assistance device 100 compares the number of pedestrians according to the direction of movement of the pedestrians. As a result, it is determined that the number of pedestrians crossing the road from top to bottom differs, and therefore the driving assistance device 100 can notify the vehicle V1 in FIG. 9 that pedestrians P1 and P6 not captured by the camera C2 are present in the vehicle's direction of travel. Note that pedestrian P4 has no direction of movement, i.e., is not moving, and is therefore excluded from the above-mentioned comparison. This allows moving pedestrians and the like to be notified, thereby enhancing the effectiveness of alerting them.

[0037] Furthermore, in this embodiment, since the direction of movement is used as the orientation of the pedestrian, the pedestrians captured by the two cameras can be matched more accurately.

[0038] [Second Embodiment] A second embodiment, in which a function for comparing the orientations of multiple types of subjects is added to the above-described driving assistance device, will be described in detail with reference to the drawings. FIG. 10 is a diagram showing the configuration of a driving assistance device 100a of the present disclosure. The difference from the first embodiment shown in FIG. 5 is that the detection unit 103a, the movement measurement unit 106a, the comparison unit 104a, and the notification unit 106a are configured to issue a note by detecting and comparing multiple types of subjects. Since the other configurations are the same as those of the first embodiment, the following description will focus on the differences.

[0039] The detection unit 103a detects pedestrians and bicycles as subjects around the crosswalk that appear in the image data from camera C1. The detection unit 103a sends the positions of the pedestrians and bicycles along with the image data to the movement measurement unit 106. Similarly, the detection unit 103a detects the positions of pedestrians and bicycles near the crosswalk that appear in the image data from camera C2, and sends these positions along with the image data to the movement measurement unit 106a.

[0040] The movement measurement unit 106a tracks the pedestrians and cyclists detected by the detection unit 103a, measures their movement direction and speed, and sends the measurement results to the matching unit 104a.

[0041] The matching unit 104a compares the number of pedestrians for each direction detected from the image data of camera C1 with the number of pedestrians for each direction detected from the image data of camera C2. If the comparison shows that the number of pedestrians for a certain direction detected from the image data of camera C1 is greater than the number of pedestrians for the same direction detected from the image data of camera C2, the matching unit 104a requests the notification unit 105 to notify the vehicle. The matching unit 104a also compares the number of bicycles for each direction detected from the image data of camera C1 with the number of bicycles for each direction detected from the image data of camera C2. If the comparison shows that the number of bicycles for a certain direction detected from the image data of camera C1 is greater than the number of bicycles for the same direction detected from the image data of camera C2, the matching unit 104a requests the notification unit 105a to notify the vehicle.

[0042] When requested by the collating unit 104a to notify a vehicle, the notifying unit 105a transmits a notification to the vehicle in question informing the vehicle that there is a possibility that a pedestrian or a bicycle is in the blind spot.

[0043] Next, the operation of this embodiment will be described in detail with reference to the drawings. FIG. 11 is a flowchart showing the operation of this disclosure. The operation differs from the first embodiment shown in FIG. 6 in steps S103 and after. The driving assistance device 100a detects bicycles in addition to pedestrians (step S103) and analyzes their movements (step S104). The driving assistance device 100a then compares the number of bicycles for each direction of movement (step S105). If, as a result of the comparison, the number of bicycles detected by camera C1 is greater than the number of bicycles detected by camera C2 in any direction of movement (Yes in step S106), the driving assistance device 100a issues a warning to the vehicle (step S107).

[0044] FIG. 12 is a diagram plotting pedestrians and bicycles detected from images captured by camera C1 on a map. The symbols B1 and B2 in the figure are markers indicating the location, movement direction, and speed of the bicycle. As in FIG. 9 , the presence of a parked vehicle V2 may prevent pedestrian P1 and bicycle B2 from being detected in the image captured by camera C2. In this case, the driving assistance device 100a can notify vehicle V1 that pedestrian P1 and bicycle B2 are present in its direction of travel but are not captured by camera C2. Without a notification from the driving assistance device 100a, vehicle V1 may overtake parked vehicle V2 in anticipation of pedestrians P2 and P3 crossing the crosswalk, potentially resulting in an intersection with pedestrian P1 and bicycle B2. However, by providing a notification from the driving assistance device 100a, vehicle V1 can pay attention to pedestrian P1 and bicycle B2 in its blind spot.

[0045] In the above embodiment, bicycles are added to pedestrians as subjects to be matched, but the combination of subjects to be matched is not limited to this. If necessary, it is also possible to add vulnerable road users, such as those using wheelchairs and strollers, to the subjects to be matched.

[0046] [Third Embodiment] A third embodiment in which a collision prediction function with a pedestrian or the like is added to the driving assistance device described above will be described in detail with reference to the drawings. Fig. 13 is a diagram showing the configuration of a driving assistance device 100b according to the present disclosure. The differences from the first embodiment shown in Fig. 5 are that a collision prediction unit 107 is added to the driving assistance device 100b and that a notification unit 105 notifies the result of the collision prediction. Since the other configurations are the same as those of the first embodiment, the following description will focus on the differences.

[0047] The collision prediction unit 107 predicts whether or not there is a possibility that the vehicle equipped with the camera C2 will collide (intersect) with the pedestrian, based on the moving direction and speed of the pedestrian determined by the comparison unit 104 not to have been captured by the camera C2, and the vehicle movement determined from the image of the camera C2. The collision prediction unit 107 sends the prediction result to the notification unit 105b. This collision prediction unit 107 corresponds to a collision prediction means that predicts a collision between the subject and the vehicle.

[0048] The notification unit 105b notifies the vehicle of the information on the pedestrian not captured by the camera C2, as in the first embodiment, as well as the result of the determination of the possibility of a collision with the pedestrian.

[0049] Next, the operation of this embodiment will be described in detail with reference to the drawings. Fig. 14 is a flowchart showing the operation of the present disclosure. The operation differs from that of the first embodiment shown in Fig. 6 in that, after step S006, the driving assistance device 100b predicts the possibility of a collision with a pedestrian in the blind spot of the camera C2 (step S207) and notifies the vehicle of this (step S208).

[0050] FIG. 15 is a diagram plotting pedestrians and bicycles detected from images captured by camera C1 on a map. Similar to FIG. 9 , pedestrians P1 and P6 may not be detected in images captured by camera C2 due to the presence of parked vehicle V2. Furthermore, as a result of predicting the possibility of a collision between vehicle V1 and pedestrians P1 and P6 and vehicle V1, it is determined that pedestrian P6 may collide with vehicle V1. In this case, driving assistance device 100b can notify vehicle V1 that pedestrians P1 and P6 not captured by camera C2 are present in its direction of travel and that there is a possibility of a collision with pedestrian P6. Upon receiving the notification from driving assistance device 100b, vehicle V1 can pay attention to pedestrians P1 and P6 in its blind spot and take measures such as slowing down or changing course to avoid a collision with pedestrian P6.

[0051] In the above example, the subject was a pedestrian, but a bicycle or other object could also be set as the subject. In this case, the possibility of a collision with a bicycle located in a blind spot can be predicted and the result can be provided to the vehicle. In particular, since bicycles travel faster than pedestrians and often cross the road diagonally, providing a collision prediction can effectively prevent accidents between vehicles and bicycles.

[0052] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and further modifications, substitutions, and adjustments can be made without departing from the basic technical concept of the present disclosure. For example, the network configurations, element configurations, and data representation formats shown in the drawings are examples intended to aid in understanding the present disclosure, and are not limited to the configurations shown in these drawings.

[0053] For example, in the first to third embodiments described above, an example was given in which the area around a crosswalk was set as the predetermined area, but the locations to which the present disclosure can be applied are not limited to this. For example, the present disclosure can also be applied to an intersection with a crosswalk.

[0054] (Hardware Configuration) In each embodiment of the present disclosure, each component of each device represents a functional unit block. Some or all of the components of each device are realized by an arbitrary combination of an information processing device 900 and a program, for example, as shown in FIG. 16 . FIG. 16 is a block diagram showing an example of the hardware configuration of the information processing device 900 that realizes each component of each device. The information processing device 900 includes, as an example, the following configuration: - CPU (Central Processing Unit) 901 - ROM (Read Only Memory) 902 - RAM (Random Access Memory) 903 - Program 904 loaded into RAM 903 - Storage device 905 that stores the program 904 - Drive device 907 that reads and writes to a recording medium 906 - Communication interface 908 that connects to a communication network 909 - Input / output interface 910 that inputs and outputs data - Bus 911 that connects each component

[0055] Each component of each device in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. That is, the CPU 901 in FIG. 16 executes an object detection program and a matching program, and performs an update process for each calculation parameter stored in the RAM 903, the storage device 905, etc. The program 904 that realizes the function of each component of each device is stored in the storage device 905 or the ROM 902 in advance, for example, and is read by the CPU 901 as needed. Note that the program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in advance on the recording medium 906, and the drive device 907 may read the program and supply it to the CPU 901.

[0056] There are various variations in the method of realizing each device. For example, each device may be realized by any combination of a separate information processing device 900 and a program for each component. Furthermore, multiple components of each device may be realized by any combination of a single information processing device 900 and a program. That is, each unit (processing means, function) of the driving assistance device shown in the first to third embodiments can be realized by a computer program that causes a processor installed in the device to execute each of the above-mentioned processes using its hardware.

[0057] In addition, some or all of the components of each device may be realized by other general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus.

[0058] Some or all of the components of each device may be realized by a combination of the above-mentioned circuits and programs.

[0059] When some or all of the components of each device are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in a form in which each device is connected via a communication network.

[0060] It should be noted that the above-described embodiments are preferred embodiments of the present disclosure, and the scope of the present disclosure is not limited to only the above-described embodiments. In other words, those skilled in the art can modify or substitute the above-described embodiments to construct various modified forms without departing from the gist of the present disclosure.

[0061] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0062] [Supplementary Note 1] A driving assistance device comprising: a first acquisition means for acquiring first image data for grasping the situation around a road from a first camera capable of capturing an image of a road; a second acquisition means for acquiring second image data capturing an image of the vehicle's traveling direction from a second camera mounted on a vehicle traveling on the road; a detection means for detecting objects and their orientations in a predetermined area captured in the first image data and the second image data; a comparison means for comparing the number of objects for each orientation; and a notification means for issuing a predetermined warning to the vehicle if the comparison results in a discrepancy in the number of objects for each orientation. [Supplementary Note 2] The detection means of the driving assistance device may be configured to detect the orientation of the object relative to the road as the orientation of the object. [Supplementary Note 3] The comparison means of the driving assistance device may be configured to exclude stationary objects from the objects to be compared. [Supplementary Note 4] The notification means of the above-mentioned driving support device may be configured to notify the vehicle when the number of different orientations of the subject captured by the first camera is greater than the number of different orientations of the subject captured by the second camera. [Supplementary Note 5] The above-mentioned driving support device may further include speed measurement means for measuring the speed of the subject, and the notification means may further notify the vehicle of the speed of the subject. [Supplementary Note 6] The above-mentioned driving support device may further include movement direction measurement means for measuring the movement direction of the subject, and the notification means may further notify the vehicle of the movement direction of the subject. [Supplementary Note 7] The above-mentioned driving support device may further include collision prediction means for predicting a collision between the subject and the vehicle, and the notification means may be configured to notify the vehicle of the result of the collision prediction as a message. [Supplementary Note 8] In the above-described driving assistance device, the subjects may include pedestrians and bicycles, and the verification means may verify the number of pedestrians and bicycles for each direction.[Supplementary Note 9] A driving assistance method comprising: acquiring first image data for grasping a situation around a road from a first camera capable of photographing a road; acquiring second image data photographing a direction in which the vehicle is traveling from a second camera mounted on a vehicle traveling on the road; detecting objects and their orientations in a predetermined area captured in the first image data and the second image data; comparing the number of objects for each orientation; and if, as a result of the comparison, the numbers of objects for each orientation do not match, issuing a predetermined warning to the vehicle. [Supplementary Note 10] A recording medium storing a program causing a computer to execute the following steps: acquiring first image data from a first camera capable of capturing images of a road to grasp the conditions around the road; acquiring second image data captured in the direction of travel of the vehicle from a second camera mounted on a vehicle traveling on the road; detecting objects and their orientations in a predetermined area captured in the first image data and the second image data; comparing the number of objects for each orientation; and issuing a predetermined warning to the vehicle if the number of objects for each orientation does not match as a result of the comparison. The embodiments described in each of the above supplementary notes can be combined with each other after making necessary modifications. For example, a configuration that combines the contents of Supplementary Note 2 and Supplementary Note 3, detects the orientation of the object relative to the road, and excludes stationary objects from the detection target, is also within the scope of the present specification. The embodiments of Supplementary Note 9 to Supplementary Note 10 can be expanded into the embodiments of Supplementary Note 2 to Supplementary Note 8, as with Supplementary Note 1.

[0063] The disclosures of the above-cited patent documents are incorporated herein by reference and may be used as the basis or part of this disclosure, as necessary. Modifications and adjustments of the embodiments and examples are possible within the scope of this disclosure (including the claims), and further based on its basic technical concept. Furthermore, various combinations and selections (including partial deletions) of various disclosed elements (including elements of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible within the scope of this disclosure. In other words, this disclosure naturally includes various modifications and alterations that would be possible by a person skilled in the art in accordance with the entire disclosure, including the claims, and the technical concept. In particular, with regard to the numerical ranges described herein, any numerical value or subrange within that range should be construed as specifically described, even if not otherwise specified. Furthermore, the disclosures of the above-cited documents, when used in part or in whole in combination with the disclosures herein as part of this disclosure, in accordance with the spirit of this disclosure, are also deemed to be included in the disclosures of this application.

[0064] REFERENCE SIGNS LIST 10 Driving support device 11 First acquisition means 12 Second acquisition means 13 Detection means 14 Collation means 15 Notification means 100, 100a, 100b Driving support device 101 First acquisition unit 102 Second acquisition unit 103, 103a Detection unit 104, 104a Collation unit 105, 105a, 105b Notification unit 106 Movement measurement unit 107 Collision prediction unit 900 Information processing device 901 CPU (Central Processing Unit) 902 ROM (Read Only Memory) 903 RAM (Random Access Memory) 904 Program 905 Storage device 906 Recording medium 907 Drive device 908 Communication interface 909 Communication network 910 Input / output interface 911 Bus B1, B2 Marker (bicycle) C1, C2 Camera P1 to P6 Marker (pedestrian) V1 Vehicle V2 Parked vehicle

Claims

1. A driving assistance device comprising: a first acquisition means for acquiring first image data for grasping the situation around a road from a first camera capable of photographing a road; a second acquisition means for acquiring second image data photographing the direction of travel of a vehicle from a second camera mounted on a vehicle traveling on said road; a detection means for detecting objects and their orientations in a predetermined area captured in said first image data and said second image data; a comparison means for comparing the number of objects for each orientation with respect to said objects; and a notification means for issuing a predetermined warning to said vehicle if the number of objects for each orientation does not match as a result of said comparison.

2. A driving assistance device according to claim 1, wherein said detection means detects the orientation of said subject relative to said road as the orientation of said subject.

3. A driving assistance device according to claim 1 or 2, wherein the verification means excludes stationary subjects from the subjects to be verified.

4. A driving assistance device according to any one of claims 1 to 3, wherein the notification means notifies the vehicle when the number of different orientations of the subject captured by the first camera is greater than the number of different orientations of the subject captured by the second camera.

5. A driving assistance device according to any one of claims 1 to 4, further comprising speed measurement means for measuring the speed of the subject, and wherein the notification means further notifies the vehicle of the speed of the subject.

6. A driving assistance device according to any one of claims 1 to 5, further comprising a movement direction measuring means for measuring the movement direction of the subject, and wherein the notification means further notifies the vehicle of the movement direction of the subject.

7. A driving assistance device according to any one of claims 1 to 6, further comprising a collision prediction means for predicting a collision between the subject and the vehicle, wherein the notification means notifies the result of the collision prediction as a message to the vehicle.

8. A driving assistance device according to any one of claims 1 to 7, wherein the subjects include pedestrians and bicycles, and the verification means verifies the number of pedestrians and bicycles for each direction.

9. A driving assistance method comprising: acquiring first image data for grasping the conditions around a road from a first camera capable of photographing a road; acquiring second image data photographing the direction of travel of a vehicle from a second camera mounted on a vehicle traveling on the road; detecting objects and their orientations in a predetermined area captured in the first image data and the second image data; comparing the number of objects for each orientation; and, if the number of objects for each orientation does not match as a result of the comparison, issuing a predetermined warning to the vehicle.

10. A recording medium having recorded thereon a program that causes a computer to execute the following processes: a process of acquiring first image data for grasping the situation around the road from a first camera capable of photographing the road; a process of acquiring second image data photographing the direction of travel of the vehicle from a second camera mounted on a vehicle traveling on the road; a process of detecting subjects and their orientations in a specified area that are captured in the first image data and the second image data; a process of comparing the number of subjects for each orientation for the subjects; and a process of issuing a specified warning to the vehicle if the number of subjects for each orientation does not match as a result of the comparison.

Images