Information notification device and information notification method

Abstract

To provide an information notification device and an information notification method capable of providing correct position information of a user vehicle to a user.SOLUTION: In an information notification device 10, a communication control section 121 communicates with a terminal used by a user. When a notification start instruction is received from the terminal, an image acquisition section 122 acquires photographed images of one or a plurality of on-vehicle cameras obtained by photographing a peripheral environment of a user vehicle from the one or the plurality of on-vehicle cameras mounted on the user vehicle of the user. A detection section 123 detects the user from one of the photographed images of the on-vehicle cameras. When the user is detected from one of the photographed images of the on-vehicle cameras, a calculation section 124 calculates position relation between the user vehicle and the user based on the photographed images of the on-vehicle cameras. A notification section 125 notifies the terminal of information including the position relation between the user vehicle and the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information notification device and an information notification method. [Background technology]

[0002] It may be difficult for a user to find a vehicle in a parking lot. Patent Document 1 discloses a technology for providing a user terminal with vehicle position information detected by GPS (Global Positioning System) information. Patent Document 1 also discloses a technology for providing a user terminal with images of the surroundings of a vehicle in response to a shooting instruction from the user terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-210617 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the background art of Patent Document 1, since GPS information may not be available in environments such as underground parking lots, accurate location information of the user's vehicle may not be provided to the user. Also, in the background art of Patent Document 1, if the scenery around the vehicle shown in the captured image provided to the user does not match the scenery around the user, it is difficult for the user to identify the accurate location of the user's vehicle.

[0005] In consideration of such problems, the present disclosure has an object to provide an information notification device and an information notification method that can provide accurate location information of a user's vehicle to a user. [Means for solving the problem]

[0006] The information notification device of the present disclosure includes: A communication control unit that communicates with a terminal used by a user; an image acquisition unit that acquires, when a notification start instruction is received from the terminal, images captured by one or more vehicle-mounted cameras mounted on the user vehicle of the user, the images capturing the surrounding environment of the user vehicle from the one or more vehicle-mounted cameras; A detection unit that detects the user from any of the images captured by the one or more vehicle-mounted cameras; a calculation unit that calculates a positional relationship between the user vehicle and the user based on the images captured by the one or more vehicle-mounted cameras when the user is detected from any of the images captured by the one or more vehicle-mounted cameras; A notification unit that notifies the terminal of information including a positional relationship between the user vehicle and the user.

[0007] The information notification method of the present disclosure includes: The computer Communicating with a terminal used by a user, When a notification start instruction is received from the terminal, images captured by one or more vehicle-mounted cameras mounted on the user vehicle of the user are acquired, the images capturing the surrounding environment of the user vehicle being captured by the one or more vehicle-mounted cameras; Detecting the user from any of the images captured by the one or more vehicle-mounted cameras; When the user is detected from any of the images captured by the one or more vehicle-mounted cameras, a positional relationship between the user vehicle and the user is calculated based on the images captured by the one or more vehicle-mounted cameras; Information including a positional relationship between the user vehicle and the user is notified to the terminal. Effect of the Invention

[0008] The present disclosure makes it possible to provide an information notification device and an information notification method that can provide accurate location information of a user's vehicle to a user. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of a configuration of an information notification system according to a first embodiment. [Diagram 2] 4 shows an example of a graphic generated by the information notification device of the information notification system according to the first embodiment. [Diagram 3] 5 is a flowchart showing an example of the operation of the information notification device in the information notification system according to the first embodiment. [Figure 4] FIG. 11 is a block diagram showing an example of a configuration of an information notification system according to a second embodiment. [Diagram 5] 13 shows an example of a surrounding environment image created by an information notification device of an information notification system according to a second embodiment. [Figure 6] 10 is a flowchart showing an example of the operation of an information notification device in the information notification system according to the second embodiment. [Figure 7] 13 shows an example of a surrounding environment image created by an information notification device of an information notification system according to a third embodiment. [Figure 8] 13 is a flowchart showing an example of the operation of an information notification device in the information notification system according to the third embodiment. [Figure 9] 13 is a flowchart showing an example of the operation of an information notification device in an information notification system according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, specific embodiments to which the present disclosure is applied will be described in detail with reference to the drawings. In each drawing, the same elements are given the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0011] (First embodiment) First, the configuration of an information notification system 1 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the information notification system 1 according to the first embodiment.

[0012] As shown in Fig. 1, the information notification system 1 includes an information notification device 10 and a terminal 20 used by a user. The information notification device 10 is a device that communicates with the user's terminal 20 and provides the terminal 20 with a notification including accurate position information of the user's vehicle. The information notification device 10 is mounted on, for example, the user's vehicle. The terminal 20 is a device such as a smartphone or tablet used by the user.

[0013] The information notification device 10 includes one or more vehicle-mounted cameras 11, a control unit 12, a first communication unit 13, and a storage unit . The vehicle-mounted camera 11 is a camera mounted on a user's vehicle. The vehicle-mounted camera 11 is a camera for capturing overhead images, and captures images such as still images and videos of the surrounding environment of the user's vehicle.

[0014] The vehicle-mounted camera 11 is a front camera 111, a rear camera 112, a left side camera 113, or a right side camera 114. The front camera 111 is mounted in front of the user vehicle and captures the surrounding environment in front of the user vehicle. The rear camera 112 is mounted in the rear of the user vehicle and captures the surrounding environment behind the user vehicle. The left side camera 113 is mounted on the left side of the user vehicle and captures the surrounding environment on the left side of the user vehicle. The right side camera 114 is mounted on the right side of the user vehicle and captures the surrounding environment on the right side of the user vehicle. The information notification device 10 of this embodiment is equipped with the front camera 111, the rear camera 112, the left side camera 113, and the right side camera 114. Therefore, the entire surrounding environment of the user vehicle is captured.

[0015] Each of the front camera 111, the rear camera 112, the left side camera 113, and the right side camera 114 may be a camera installed in an upper part of the cabin of the user's vehicle facing in each direction. Also, each of the front camera 111, the rear camera 112, the left side camera 113, and the right side camera 114 may be a camera for a drive recorder.

[0016] The control unit 12 has a function of controlling one or more vehicle-mounted cameras 11, the first communication unit 13, and the storage unit . The first communication unit 13 has a function of communicating with other devices by a wireless communication method such as 4G, 5G, WiFi, etc. The first communication unit 13 may have a function of communicating with other devices by a direct wireless communication method such as Bluetooth (registered trademark). The storage unit 14 has a function of storing various information. For example, the storage unit 14 stores user recognition information, which will be described later.

[0017] The control unit 12 includes a first communication control unit 121, an image acquisition unit 122, a detection unit 123, a calculation unit 124, and a notification unit 125. The first communication control unit 121 controls the first communication unit 13. Specifically, the first communication control unit 121 communicates with the terminal 20 via the first communication unit 13. For example, the first communication control unit 121 receives from the terminal 20 a notification start instruction which is an instruction for starting a notification process to the terminal 20 of the user. In addition, the first communication control unit 121 receives from the terminal 20 a notification end instruction which is an instruction for ending the notification process to the terminal 20 of the user.

[0018] When a notification start instruction is received from the terminal 20, the image acquisition unit 122 acquires images captured by each of the one or more vehicle-mounted cameras 11. When a notification start instruction is received, the image acquisition unit 122 may start each vehicle-mounted camera 11 and cause each vehicle-mounted camera 11 to start capturing images of the environment around the user's vehicle.

[0019] The detection unit 123 detects a user from any of the images captured by each vehicle-mounted camera 11 using user recognition information registered in advance in the storage unit 14. For example, the detection unit 123 detects a user from the images captured by each vehicle-mounted camera 11 based on user recognition information learned by a deep learning algorithm such as CNN (Convolutional Neural Network). The user recognition information includes physical characteristics of the user, such as the shape and position of the user's eyes. The user recognition information also includes characteristics of the user's whole-body movements, such as the way the user walks. The user recognition information also includes characteristics of the user's overall appearance on that day, such as the clothes the user is wearing.

[0020] When a user is detected from any of the images captured by the vehicle-mounted cameras 11, the calculation unit 124 calculates the positional relationship between the user vehicle and the user based on the images captured by each vehicle-mounted camera 11. When a user is detected from any of the images captured by the vehicle-mounted cameras 11, the calculation unit 124 calculates the traveling direction of the user based on the images captured by each vehicle-mounted camera 11. Specifically, when a user is detected from an image captured by the vehicle-mounted camera 11, the calculation unit 124 calculates the positional relationship between the vehicle-mounted camera 11 from which the user was detected and the user from the image captured by the vehicle-mounted camera 11 from which the user was detected. The positional relationship between the vehicle-mounted camera 11 from which the user was detected and the user, that is, represents the positional relationship between the user vehicle and the user. Here, the positional relationship between the user vehicle and the user is represented by the relative position of the user with respect to the user vehicle, that is, the distance and direction. The calculation unit 124 calculates the distance of the user from the ground position of the user shown in the image captured by the vehicle-mounted camera 11 from which the user was detected to the user vehicle. The calculation unit 124 also calculates the direction of the user relative to the user vehicle from the lateral position of the user shown in the image captured by the in-vehicle camera 11 where the user is detected and the calculated distance. The calculation unit 124 also calculates the traveling direction of the user from the orientation of the user's body shown in the image captured by the in-vehicle camera 11 where the user is detected. Note that the calculation unit 124 is not limited to the orientation of the user shown in the image captured by the in-vehicle camera 11, and may calculate the traveling direction of the user from the movement in the time direction of an object (i.e., the user) shown in the image captured by the in-vehicle camera 11.

[0021] The notification unit 125 transmits a notification including the positional relationship between the user vehicle and the user calculated by the calculation unit 124 to the terminal 20 via the first communication unit 13. Furthermore, the notification unit 125 transmits a notification including the traveling direction of the user calculated by the calculation unit 124 to the terminal 20 via the first communication unit 13. Specifically, the notification unit 125 creates a graphic as shown in FIG. 2, which represents the positional relationship between the user vehicle and the user and the traveling direction of the user. FIG. 2 is a diagram showing an example of a graphic generated by the information notification device 10 of the information notification system 1 according to the first embodiment. As shown in FIG. 2, an icon X1 indicating the user vehicle and an icon X2 indicating the user are arranged in the graphic. In the graphic, the positional relationship between the user vehicle and the user is represented by a dotted circle indicating a distance from the user vehicle, such as 10 m or 20 m, and the arrangement positions of the icon X1 indicating the user vehicle and the icon X2 indicating the user. In addition, in the graphic, the traveling direction of the user is represented by the protruding direction of the icon X2 indicating the user. The icon X2 representing the user may be placed in the center, and the icon X1 representing the user's vehicle may be displayed in the periphery. In this case, the user's traveling direction may be fixed, for example, in the upward direction of the graphic. Here, when the user's traveling direction changes, the position of the icon X1 representing the vehicle shifts accordingly so as to make a circular motion. Also, when the direction on the graphic is fixed and the user changes the traveling direction, the protrusion direction of the icon X2 representing the user may change. Also, the shapes of the icon X1 and the icon X2 are not limited to this. For example, the icon X1 representing the vehicle may also have a protrusion so that the direction of the vehicle can be known. Then, the notification unit 125 transmits an instruction to display the notification by the graphic to the terminal 20.

[0022] The notification unit 125 may transmit, via the first communication unit 13, to the terminal 20 an instruction to display images captured by each vehicle-mounted camera 11, including an image captured by the vehicle-mounted camera 11 detected by the user. The notification unit 125 also creates a surrounding environment image such as an around view monitor image from the images captured by each vehicle-mounted camera 11. The notification unit 125 then creates a surrounding environment image that reflects the positional relationship between the user's vehicle and the user and the user's traveling direction. Furthermore, the notification unit 125 may transmit an instruction to display the surrounding environment image to the terminal 20 via the first communication unit 13.

[0023] Furthermore, when a user is detected by the detection unit 123, the notification unit 125 may transmit, to the terminal 20 via the first communication unit 13, a voice output instruction for the terminal 20 to output a voice notification. Furthermore, when a user is detected by the detection unit 123, the notification unit 125 may transmit, to the terminal 20 via the first communication unit 13, a vibration output instruction for the terminal 20 to output a vibration notification.

[0024] The terminal 20 includes an operation input unit 21, a control unit 22, a second communication unit 23, a display unit 24, a sound output unit 25, and a vibration unit 26. The operation input unit 21 is a touch panel or the like, and has a function of accepting input of user instructions by user operations. The control unit 22 has a function of controlling the operation input unit 21, the second communication unit 23, the display unit 24, the audio output unit 25, and the vibration unit .

[0025] The second communication unit 23 has a function of communicating with other devices by a wireless communication method such as 4G, 5G, WiFi, etc. The display unit 24 is a display or the like, and has a function of displaying images. The audio output unit 25 is a speaker or the like, and has a function of outputting audio. The vibration unit 26 is a vibration generator or the like, and has a function of outputting vibration. The control unit 22 includes a second communication control unit 221 , a display control unit 222 , an operation control unit 223 , a sound output control unit 224 , and a vibration control unit 225 .

[0026] The second communication control unit 221 controls the second communication unit 23. Specifically, the second communication control unit 221 communicates with the information notification device 10 via the second communication unit 23. For example, the second communication control unit 221 transmits a notification start instruction to the information notification device 10. In addition, the second communication control unit 221 transmits a notification end instruction to the information notification device 10.

[0027] The display control unit 222 controls the display unit 24. Specifically, when the display control unit 222 receives an instruction to display graphics or the like from the information notification device 10, the display control unit 222 causes the display unit 24 to display the graphics or the like in accordance with the display instruction. The operation control unit 223 controls the operation input unit 21. Specifically, the operation control unit 223 acquires from the operation input unit 21 user instructions such as a notification start instruction and a notification end instruction.

[0028] The audio output control unit 224 controls the audio output unit 25. Specifically, when the audio output control unit 224 receives an audio output instruction for audio notification from the information notification device 10, the audio output control unit 224 causes the audio output unit 25 to output the audio notification in accordance with the audio output instruction. The vibration control unit 225 controls the vibration unit 26. Specifically, when the vibration control unit 225 receives a vibration output instruction for a vibration notification from the information notification device 10, the vibration control unit 225 causes the vibration unit 26 to output a vibration notification in accordance with the vibration output instruction.

[0029] Next, the operation of the information notification device 10 of the information notification system 1 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the operation of the information notification device 10 of the information notification system 1 according to the first embodiment.

[0030] As shown in FIG. 3, first, the first communication control unit 121 of the information notification device 10 of the information notification system 1 receives a notification start instruction from the user terminal 20 via the first communication unit 13 (step S101). Next, the image acquisition unit 122 acquires images captured by one or more vehicle-mounted cameras 11 mounted on the user's vehicle (step S102).

[0031] Next, the detection unit 123 detects a user from any of the images captured by each vehicle-mounted camera 11, using user recognition information registered in advance in the storage unit 14 (step S103). Next, if a user is detected from any of the images captured by each vehicle-mounted camera 11 (YES in step S103), the process proceeds to step S104. On the other hand, if the user is not detected by the detection unit 123 (NO in step S103), the calculation unit 124 proceeds to step S107.

[0032] In the process of step S104, the calculation unit 124 calculates the positional relationship between the user's vehicle and the user and the traveling direction of the user based on the images captured by each of the in-vehicle cameras 11.

[0033] Next, the notification unit 125 creates a graphic representing the positional relationship between the user vehicle and the user and the user's traveling direction (step S105). For example, the notification unit 125 creates a graphic as shown in FIG. 2. In the graphic, an icon X1 representing the user vehicle and an icon X2 representing the user are arranged. In the graphic, the positional relationship between the user vehicle and the user is represented by a dotted circle indicating a distance from the user vehicle, such as 10 m or 20 m, and the arrangement positions of the icon X1 representing the user vehicle and the icon X1 representing the user. In addition, in the graphic, the user's traveling direction is represented by the protruding direction of the icon X2 representing the user.

[0034] Next, the notification unit 125 transmits an instruction to display a graphic notification to the terminal 20 (step S106). When the display control unit 222 of the terminal 20 receives the instruction to display a graphic notification, the display control unit 222 displays the graphic on the display unit 24.

[0035] Next, the first communication control unit 121 determines whether or not a notification end instruction has been received from the terminal 20 via the first communication unit 13 (step S107). If it is determined that the notification end instruction has been received (YES in step S107), the process ends. On the other hand, if it is determined that the notification end instruction has not been received (NO in step S107), the process returns to step S103.

[0036] The information notification device 10 of the information notification system 1 according to the first embodiment uses images captured by the vehicle-mounted camera 11 without using GPS information. Therefore, the information notification device 10 can provide accurate position information of the user's vehicle to the user's terminal 20 even in an environment such as an underground parking lot where position information such as GPS information may not be available. Furthermore, the information notification device 10 can provide the user's terminal 20 with the positional relationship between the user's vehicle and the user and the user's traveling direction. Therefore, even if the information notification device 10 provides the user with captured images of the surroundings of the user's vehicle, and the scenery around the user does not match the scenery around the user, the user can identify the exact position of the vehicle.

[0037] Second embodiment Next, the configuration of the information notification system 2 according to the second embodiment will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a block diagram showing an example of the configuration of the information notification system 2 according to the second embodiment.

[0038] The information notification system 2 has the following different configuration compared to the information notification system 1 according to the first embodiment. Instead of the information notification device 10 and the terminal 20, the information notification system 2 includes an information notification device 30, a terminal 40, and one or more outside-vehicle cameras 50.

[0039] The information notification device 30 further includes the following configuration compared to the information notification device 10 according to the first embodiment. The terminal 40 further includes the following configuration compared to the terminal 20 according to the first embodiment. The following mainly describes the different configurations between the two devices, and omits a description of the similar configurations.

[0040] The information notification device 30 is a camera capable of communicating within a predetermined range, and communicates with one or more exterior cameras 50 located outside the user's vehicle. The exterior camera 50 is, for example, a camera mounted on another vehicle in the vicinity of the user's vehicle, or a surveillance camera installed in a parking lot, etc. The exterior camera 50 is a camera for capturing overhead images, and captures images such as still images and videos of the environment surrounding the exterior camera 50.

[0041] Moreover, the information notification device 30 includes a first communication unit 13A and a first communication unit 13B as a specific configuration of the first communication unit 13. The first communication unit 13A has a function of communicating by a wireless communication method such as 4G, 5G, or WiFi. The first communication unit 13B has a function of communicating by a wireless communication method such as Bluetooth (registered trademark) or NFC (Near field communication). In the following description, the first communication unit 13A and the first communication unit 13B are collectively referred to as the first communication unit 13.

[0042] Furthermore, the control unit 12 of the information notification device 30 includes a 1A communication control unit 121A and a 1B communication control unit 121B as a specific configuration of the first communication control unit 121. The 1A communication control unit 121A controls the 1A communication unit 13A. The 1B communication control unit 121B controls the 1B communication unit 13B. In the following description, the 1A communication control unit 121A and the 1B communication control unit 121B are collectively referred to as the first communication control unit 121.

[0043] The first communication control unit 121 communicates with the terminal 20 via the first communication unit 13. For example, the first communication control unit 121 receives from the terminal 20 a notification start instruction, which is an instruction to start a notification process to the user's terminal 20. The first communication control unit 121 also receives from the terminal 20 a notification end instruction, which is an instruction to end a notification process to the user's terminal 20. The first communication control unit 121 also identifies one or more outside vehicle cameras 50 with which communication is possible within a predetermined range. The first communication control unit 121 communicates with each of the identified outside vehicle cameras 50. The first communication control unit 121 then acquires camera metadata from each of the identified outside vehicle cameras 50. The camera metadata includes information such as the position, angle of view, and orientation of the outside vehicle cameras 50.

[0044] When a notification start instruction is received from terminal 20, image acquisition unit 122 acquires images captured by each in-vehicle camera 11 and also acquires images captured by each outside vehicle camera 50 from each outside vehicle camera 50. When a notification start instruction is received, image acquisition unit 122 may start each in-vehicle camera 11 and cause each outside vehicle camera 11 to start capturing images. When a notification start instruction is received, image acquisition unit 122 may start each outside vehicle camera 50 and cause each outside vehicle camera 50 to start capturing images.

[0045] The calculation unit 124 calculates the positional relationship between each of the on-board cameras 11 and each of the outside-vehicle cameras 50 based on the images captured by each of the on-board cameras 11 and the images captured by each of the outside-vehicle cameras 50. For example, the calculation unit 124 calculates the positional relationship between the on-board camera 11 and the first and second outside-vehicle cameras 50 by the following method.

[0046] The calculation unit 124 refers to the camera metadata of the on-board camera 11, the first outside vehicle camera 50, and the second outside vehicle camera 50. The calculation unit 124 identifies an object that appears in common in the images captured by the on-board camera 11 and the first outside vehicle camera 50, and calculates the positional relationship between the on-board camera 11 and the first outside vehicle camera 50 based on the position and orientation of the identified object in the captured image. The calculation unit 124 then identifies an object that appears in common in the images captured by the first outside vehicle camera 50 and the second outside vehicle camera 50, and calculates the positional relationship between the first outside vehicle camera 50 and the second outside vehicle camera 50 based on the position and orientation of the identified object in the captured image. Furthermore, the calculation unit 124 calculates the positional relationship between the on-board camera 11 and the second outside vehicle camera 50 based on the positional relationship between the on-board camera 11 and the first outside vehicle camera 50 and the positional relationship between the first outside vehicle camera 50 and the second outside vehicle camera 50.

[0047] Also, assume that the first exterior camera 50 is captured in an image captured by the on-board camera 11, and the second exterior camera 50 is captured in an image captured by the first exterior camera 50. The calculation unit 124 refers to the camera metadata of the on-board camera 11, the first exterior camera 50, and the second exterior camera 50. The calculation unit 124 calculates the positional relationship between the on-board camera 11 and the first exterior camera 50 from the size, position, orientation, etc. of the first exterior camera 50 captured in the image captured by the on-board camera 11. The calculation unit 124 then calculates the positional relationship between the first exterior camera 50 and the second exterior camera 50 from the size, position, orientation, etc. of the second exterior camera 50 captured in the image captured by the first exterior camera 50. Furthermore, the calculation unit 124 calculates the positional relationship between the on-board camera 11 and the first exterior camera 50 and the positional relationship between the first exterior camera 50 and the second exterior camera 50.

[0048] The calculation unit 124 also refers to camera metadata of the on-board camera 11, the first exterior camera 50, and the second exterior camera 50. The calculation unit 124 may calculate the positional relationship between the on-board camera 11, the first exterior camera 50, and the second exterior camera 50 from characteristic objects (e.g., signs, stairs, and entrances and exits) that appear in the images captured by the on-board camera 11, the first exterior camera 50, and the second exterior camera 50.

[0049] The detection unit 123 detects a user from either the image captured by each in-vehicle camera 11 or the image captured by each exterior camera 50, using user recognition information registered in advance in the storage unit 14. For example, the detection unit 123 detects a user from either the image captured by each in-vehicle camera 11 or the image captured by each exterior camera 50, based on user recognition information learned by a deep learning algorithm such as CNN.

[0050] When a user is detected from either the image captured by each on-board camera 11 or the image captured by each outside camera 50, the calculation unit 124 calculates the positional relationship between the user vehicle and the user and the traveling direction of the user based on the image captured by each on-board camera 11 and the image captured by each outside camera 50. For example, when a user is detected from an image captured by the on-board camera 11, the calculation unit 124 calculates the positional relationship between the on-board camera 11 in which the user is detected and the user from the image captured by the on-board camera 11 in which the user is detected. Here, the positional relationship between the on-board camera 11 in which the user is detected and the user represents the positional relationship between the user vehicle and the user. Furthermore, when a user is detected from an image captured by the outside camera 50, the calculation unit 124 calculates the positional relationship between the on-board camera 50 in which the user is detected and the user from the image captured by the outside camera 50 in which the user is detected. Then, the calculation unit 124 calculates the positional relationship between the on-board camera 11 and the user based on the positional relationship between the on-board camera 50 in which the user is detected and the user and the positional relationship between each on-board camera 11 and each outside camera 50. Here, the positional relationship between the vehicle-mounted camera 11 and the user refers to the positional relationship between the user's vehicle and the user.

[0051] The notification unit 125 transmits a notification including the positional relationship between the user vehicle and the user and the traveling direction of the user calculated by the calculation unit 124 to the terminal 20 via the first communication unit 13. Specifically, the notification unit 125 creates a surrounding environment image such as an around view monitor image from the images captured by each in-vehicle camera 11 and the images captured by each exterior camera 50. Furthermore, the notification unit 125 creates a surrounding environment image as shown in FIG. 5, which reflects the positional relationship between the user vehicle and the user and the traveling direction of the user. FIG. 5 is a diagram showing an example of a surrounding environment image created in the information notification device 30 of the information notification system 2 according to the second embodiment. As shown in FIG. 5, an icon Y1 indicating the user vehicle is arranged in the surrounding environment image. The icon Y1 indicating the user vehicle is a rectangular frame surrounding the user vehicle. In addition, an icon Y2 indicating the user is arranged in the surrounding environment image. In the surrounding environment image, the positional relationship between the user vehicle and the user is represented by a dotted circle indicating the distance from the user vehicle and the arrangement positions of the icon Y1 indicating the user vehicle and the icon Y2 indicating the user. In the surrounding environment image, the protrusion direction of the icon Y2 representing the user represents the user's traveling direction. Then, the notification unit 125 transmits an instruction to display a notification by the surrounding environment image to the terminal 20 via the first communication unit 13.

[0052] The notification unit 125 may create a graphic that reflects the positional relationship between the user vehicle and the user and the traveling direction of the user. Specifically, the notification unit 125 generates a graphic that shows only an icon indicating the user vehicle and an icon indicating the user, as shown in FIG. 2, and their positional relationship. Here, in the graphic, the positional relationship between the user vehicle and the user is represented by a dotted circle indicating a distance from the user vehicle, such as 10 m or 20 m, and the arrangement positions of the icon indicating the user vehicle and the icon indicating the user. In addition, in the graphic, the traveling direction of the user is represented by the protrusion direction of the icon indicating the user. Note that the icon indicating the user may be placed in the center, and the icon indicating the user vehicle may be represented in the periphery. In this case, the traveling direction of the user may be fixed, for example, in the upward direction of the graphic. In this case, when the traveling direction of the user changes, the position of the icon indicating the vehicle shifts so as to move in a circular motion. In addition, when the direction on the graphic is fixed and the user changes the traveling direction, the protrusion direction of the icon indicating the user may change. In addition, the icon and the shape of the icon are not limited to this. For example, the icon indicating the vehicle may also be provided with a protrusion so that the direction of the vehicle can be understood. The notification unit 125 may generate a graphic showing only the positional relationship between an icon showing the user's vehicle, an icon showing the user, and an icon showing another vehicle different from the user's vehicle. The notification unit 125 may then transmit an instruction to display the graphic notification to the terminal 20 via the first communication unit 13.

[0053] Furthermore, when a user is detected from either an image captured by each in-vehicle camera 11 or an image captured by each exterior camera 50, the notification unit 125 may transmit, via the first communication unit 13, a voice output instruction for the terminal 20 to output a voice notification to the terminal 20. Furthermore, when a user is detected from either an image captured by each in-vehicle camera 11 or an image captured by each exterior camera 50, the notification unit 125 may transmit, via the first communication unit 13, a vibration output instruction for the terminal 20 to output a vibration notification to the terminal 20.

[0054] The terminal 40 also includes a second A communication unit 23A and a second B communication unit 23B as specific configurations of the second communication unit 23. The second A communication unit 23A has a function of communicating by a wireless communication method such as 4G, 5G, or WiFi. The second B communication unit 23B has a function of communicating by a wireless communication method such as Bluetooth or NFC. In the following description, the second A communication unit 23A and the second B communication unit 23B are collectively referred to as the second communication unit 23.

[0055] Moreover, the control unit 22 of the terminal 40 includes a secondA communication control unit 221A and a secondB communication control unit 221B as a specific configuration of the second communication control unit 221. The secondA communication control unit 221A controls the secondA communication unit 23A. The secondB communication control unit 221B controls the secondB communication unit 23B. In the following description, the secondA communication control unit 221A and the secondB communication control unit 221B are collectively referred to as the second communication control unit 221.

[0056] The second communication control unit 221 controls the second communication unit 23. Specifically, the second communication control unit 221 communicates with the information notification device 30 via the second communication unit 23. For example, the second communication control unit 221 transmits a notification start instruction to the information notification device 30. In addition, the second communication control unit 221 transmits a notification end instruction to the information notification device 30.

[0057] Next, the operation of the information notification device 30 of the information notification system 2 according to the second embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 6 is a flowchart showing an example of the operation of the information notification device 30 of the information notification system 2 according to the second embodiment.

[0058] 6, first, the first communication control unit 121 of the information notification device 30 receives a notification start instruction from the user's terminal 20 via the first communication unit 13 (step S201). Next, the first communication control unit 121 identifies one or more communicable exterior cameras 50 within a predetermined range (step S202). Next, the first communication control unit 121 acquires camera metadata of each exterior camera 50 (step S203).

[0059] Next, the image acquisition unit 122 acquires images captured by one or more vehicle-mounted cameras 11 mounted on the user vehicle, and also acquires images captured by each of the exterior cameras 50 from each of the exterior cameras 50 (step S204).

[0060] Next, the calculation unit 124 calculates the positional relationship between each of the on-board cameras 11 and each of the outside cameras 50 based on the images captured by each of the on-board cameras 11 and each of the outside cameras 50 (step S205).

[0061] Next, the notification unit 125 creates a surrounding environment image of the user's vehicle based on the images captured by each in-vehicle camera 11 and each exterior camera 50 (step S206). For example, the notification unit 125 creates a surrounding environment image as shown in FIG. 5. An icon Y1 indicating the user's vehicle is placed in the surrounding environment image. The icon Y1 indicating the user's vehicle is a rectangular frame surrounding the user's vehicle. Next, the notification unit 125 transmits a notification including the surrounding environment image to the terminal 20 (step S207). Then, when the display control unit 222 of the terminal 20 receives the notification including the surrounding environment image, it displays the surrounding environment image on the display unit 24.

[0062] Next, the detection unit 123 detects the user from either the image captured by each in-vehicle camera 11 or the image captured by each outside vehicle camera 50, using user recognition information registered in advance in the storage unit 14 (step S208).

[0063] Next, if a user is detected from either the image captured by each in-vehicle camera 11 or the image captured by each exterior camera 50 (YES in step S208), the process proceeds to step S209. On the other hand, if the user is not detected by the detection unit 123 (NO in step S208), the calculation unit 124 proceeds to step S212.

[0064] Next, in the process of step S209, the calculation unit 124 calculates the positional relationship between the user's vehicle and the user and the user's traveling direction based on the image captured by the in-vehicle camera 11 or the image captured by the exterior camera 50 in which the user is detected.

[0065] Next, the notification unit 125 creates a surrounding environment image reflecting the calculated information, that is, the positional relationship between the user vehicle and the user and the user's traveling direction (step S210). For example, in the surrounding environment image shown in FIG. 5, an icon Y2 representing the user is arranged. In the surrounding environment image, the positional relationship between the user vehicle and the user is represented by a dotted circle indicating the distance from the user vehicle and the arrangement positions of the icon Y1 representing the user vehicle and the icon Y2 representing the user. In the surrounding environment image, the protruding direction of the icon Y2 representing the user represents the user's traveling direction. Next, the notification unit 125 notifies the terminal 20 of an instruction to display a notification by the surrounding environment image (step S211). Then, when the display control unit 222 of the terminal 20 receives the instruction to display a notification by the surrounding environment image, the display control unit 222 displays the surrounding environment image on the display unit 24.

[0066] Next, the first communication control unit 121 determines whether or not a notification end instruction has been received from the terminal 20 via the first communication unit 13 (step S212). If it is determined that a notification end instruction has been received (YES in step S212), the process ends. On the other hand, if it is determined that a notification end instruction has not been received (NO in step S212), the process returns to step S206. In the process of step S206 after the process of step S212, the calculation unit 124 updates the surrounding environment image based on the newly acquired images captured by each in-vehicle camera 11 and each outside vehicle camera 50.

[0067] The information notification device 30 of the information notification system 2 according to the second embodiment can detect the user in a wider area by further using images captured by one or more exterior cameras 50. Therefore, the information notification device 30 can calculate the positional relationship between the user and the user's vehicle in a wider area and provide the calculated information to the user. Furthermore, even if the image captured by the in-vehicle camera 11 cannot detect the user due to an obstacle such as another vehicle, the information notification device 30 can detect the user by using the image captured by the exterior camera 50 with a different shooting position or angle of view. Therefore, the information notification device 30 can calculate the positional relationship between the user and the user's vehicle with higher accuracy and provide the calculated information to the user.

[0068] (Third embodiment) Next, the configuration of an information notification system 3 (not shown) according to the third embodiment will be described. Compared to the configuration of the information notification system 2 according to the second embodiment shown in Fig. 4, the information notification system 3 has an information notification device 60 (not shown) instead of the information notification device 30. The information notification device 60 has a similar configuration to the information notification device 30, but further has the following functions. The following description will focus on the different functions between the two devices, and will omit a description of the similar functions.

[0069] The detection unit 123 of the information notification device 60 detects other vehicles that are different from the moving user vehicle from either the images captured by one or more in-vehicle cameras 11 or the images captured by one or more exterior cameras 50. Here, the detection unit 123 determines whether the other vehicle is moving based on whether the speed of the other vehicle is equal to or higher than a threshold. The speed of the other vehicle can be calculated from the moving distance of the object (i.e., the other vehicle) in the time direction.

[0070] When a moving vehicle is detected, the calculation unit 124 calculates the positional relationship between the moving vehicle and the user based on the images captured by each in-vehicle camera 11 and the images captured by each outside vehicle camera 50. Specifically, as described above, the calculation unit 124 calculates the positional relationship between each in-vehicle camera 11, each outside vehicle camera 50, and the user. Then, when a moving vehicle is detected from the image captured by the in-vehicle camera 11, the calculation unit 124 calculates the positional relationship between the in-vehicle camera 11 and the moving vehicle. The calculation unit 124 calculates the positional relationship between the moving vehicle and the user based on the calculated information and the positional relationship between the in-vehicle camera 11 in which the moving vehicle is detected and the user. Also, when a moving vehicle is detected from the image captured by the outside vehicle camera 50, the calculation unit 124 calculates the positional relationship between the outside vehicle camera 50 in which the moving vehicle is detected and the moving vehicle. The calculation unit 124 calculates the positional relationship between the other moving vehicle and the user based on the calculated information and the positional relationship between the user and the outside camera 50 that detected the other moving vehicle.

[0071] The notification unit 125 transmits a notification including the positional relationship between the other moving vehicle and the user to the user's terminal 20. For example, the notification unit 125 creates a surrounding environment image that reflects the positional relationship between the other moving vehicle and the user. FIG. 7 is a diagram showing an example of a surrounding environment image created by the information notification device 30 of the information notification system 3 according to the third embodiment. As shown in FIG. 7, in addition to an icon Y1 indicating the user's vehicle and an icon Y2 indicating the user, an icon Y3 indicating the other moving vehicle is arranged in the surrounding environment image. In the surrounding environment image, the icon Y3 indicating the other moving vehicle is, for example, an arrow and is arranged near the image of the other moving vehicle. The notification unit 125 transmits a display instruction for a notification by the surrounding environment image to the terminal 20 via the first communication unit 13.

[0072] Furthermore, the notification unit 125 determines whether or not the other vehicle in motion may pose a danger to the user based on the positional relationship between the other vehicle in motion and the user. For example, when the distance between the other vehicle in motion and the user is equal to or less than a predetermined threshold, the notification unit 125 determines that the other vehicle may pose a danger to the user. Furthermore, the notification unit 125 can determine whether or not the other vehicle in motion may pose a danger to the user based on whether or not the user is in the traveling direction of the other vehicle in motion. Specifically, the notification unit 125 identifies the traveling direction of the other vehicle from the orientation of the other vehicle in motion or the traveling direction of the other vehicle in the time direction, and detects whether or not the user is in an area surrounded by two straight lines parallel to the traveling direction of the other vehicle. The width between the two straight lines is, for example, the same as the vehicle body width. Here, the notification unit 125 may determine whether the other moving vehicle may pose a danger to the user by using a combination of the determination of whether the distance between the other moving vehicle and the user is equal to or less than a predetermined threshold value and the determination of whether the user is in the traveling direction of the other moving vehicle, or by using only one of them. When using both, the notification unit 125 may determine that the other moving vehicle may pose a danger to the user when the distance between the other moving vehicle and the user is equal to or less than a predetermined distance threshold value and the user is in the traveling direction of the other moving vehicle. Furthermore, the notification unit 125 may determine that the other moving vehicle may pose a danger to the user when any one of the above two conditions is satisfied. Then, when it is determined that the other moving vehicle may pose a danger to the user, the notification unit 125 transmits a vehicle approach danger notification to the terminal 20, indicating that the vehicle is approaching and the user is in danger. For example, the notification unit 125 creates a surrounding environment image reflecting a warning to the user. As shown in FIG. 7, an icon Y4 indicating a moving vehicle that may pose a danger to the user is arranged in the surrounding environment image. In the surrounding environment image, the icon Y4 indicating a moving vehicle that may pose a danger to the user is a warning mark and is arranged superimposed on the image of the moving vehicle. In addition, in the surrounding environment image, icons Y5 indicating a warning to the user, which are warning marks, are arranged in the four corners. In addition, in the surrounding environment image, the background color is changed to, for example, red. Then, the notification unit 125 transmits a display instruction for a notification by the surrounding environment image to the terminal 20 via the first communication unit 13. Note that the notification unit 125 may transmit a sound output instruction for warning to the terminal 20. In addition, the notification unit 125 may transmit a vibration output instruction for warning to the terminal 20. In this embodiment, the function of the information notification system 3 when another moving vehicle is in front of the user has been described using Fig. 7, but the same applies to the case when another moving vehicle is behind the user. In either case, the information notification system 3 can reduce the risk of collision with another vehicle by notifying the user of the other moving vehicle.

[0073] Next, the operation of the information notification device 60 of the information notification system 3 according to the third embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 8 is a flowchart showing an example of the operation of the information notification device 60 of the information notification system 3 according to the third embodiment.

[0074] As shown in FIG. 8, the information notification device 60 executes processes of steps S301 to S306, which will be described later, between the processes of steps S211 and S212 described above.

[0075] After the process of step S211, the detection unit 123 of the information notification device 60 detects other moving vehicles from either the images captured by each in-vehicle camera 11 or the images captured by each exterior camera 50 (step S301).

[0076] Next, the calculation unit 124 calculates the positional relationship between the other vehicle and the user from the images captured by each of the in-vehicle cameras 11 and the images captured by each of the exterior cameras 50 (step S302).

[0077] Next, the notification unit 125 creates a surrounding environment image that reflects the calculated information, that is, the positional relationship between the other vehicle and the user (step S303). For example, as shown in Fig. 7, in addition to an icon Y1 indicating the user vehicle and an icon Y2 indicating the user, an icon Y3 indicating the other vehicle that is moving is arranged in the surrounding environment image. In the surrounding environment image, the icon Y3 indicating the other vehicle that is moving is, for example, an arrow and is arranged near the image of the other vehicle that is moving.

[0078] Next, the notification unit 125 transmits a notification including the surrounding environment image to the terminal 20 (step S304). When the display control unit 222 of the terminal 20 receives the notification including the surrounding environment image, the display unit 24 displays the surrounding environment image.

[0079] Next, the notification unit 125 determines whether or not the other moving vehicle may pose a danger to the user based on the positional relationship between the other moving vehicle and the user (step S305). For example, when the distance between the other moving vehicle and the user is equal to or less than a predetermined threshold, the notification unit 125 determines that the other vehicle may pose a danger to the user.

[0080] When it is determined that the other moving vehicle may pose a danger to the user (YES in step S305), the notification unit 125 transmits a vehicle approach danger notification to the terminal 20, indicating that the user is in danger due to the approach of a vehicle (step S306). For example, the notification unit 125 creates a surrounding environment image reflecting a warning to the user. As shown in FIG. 7, an icon Y4 indicating a moving vehicle that may pose a danger to the user is arranged in the surrounding environment image. In the surrounding environment image, the icon Y4 indicating a moving vehicle that may pose a danger to the user is a warning mark and is arranged superimposed on the image of the other moving vehicle. In addition, in the surrounding environment image, icons Y5 indicating a warning to the user, which are warning marks, are arranged in the four corners. In addition, in the surrounding environment image, the background color is changed to, for example, red. The notification unit 125 may transmit a sound output instruction for warning to the terminal 20. In addition, the notification unit 125 may transmit a vibration output instruction for warning to the terminal 20. Then, after the process of step S306, the process proceeds to step S211. On the other hand, if it is determined that the other vehicle does not pose a danger to the user (NO in step S305), the process proceeds to step S212.

[0081] Therefore, the information notification device 60 of the information notification system 3 according to the third embodiment can visually and audibly notify the user that another moving vehicle is approaching and posing a danger. Therefore, the information notification device 30 can reduce the risk of, for example, collision with another vehicle when the user searches for the user's vehicle.

[0082] (Fourth embodiment) Next, the configuration of an information notification system 4 (not shown) according to a fourth embodiment will be described. Compared to the configuration of the information notification system 1 according to the first embodiment shown in FIG. 1, the information notification system 4 has an information notification device 70 (not shown) instead of the information notification device 10. The information notification device 70 has a similar configuration to the information notification device 10, but further has the following functions. The following description will focus on the different functions between the two devices, and will omit a description of the similar functions.

[0083] The detection unit 123 of the information notification device 70 detects other vehicles, which are vehicles different from the moving user vehicle, from images captured by one or more vehicle-mounted cameras 11.

[0084] When a moving vehicle is detected, the calculation unit 124 calculates the positional relationship between the moving vehicle and the user based on the images captured by each vehicle-mounted camera 11. Specifically, as described above, the calculation unit 124 calculates the positional relationship between each vehicle-mounted camera 11 and the user. Then, when a moving vehicle is detected from the images captured by the vehicle-mounted camera 11, the calculation unit 124 calculates the positional relationship between the vehicle-mounted camera 11 and the moving vehicle. The calculation unit 124 calculates the positional relationship between the moving vehicle and the user based on the calculated information and the positional relationship between the vehicle-mounted camera 11 in which the moving vehicle was detected and the user.

[0085] The notification unit 125 transmits a notification including the positional relationship between the other moving vehicle and the user to the user's terminal 20. For example, the notification unit 125 creates a graphic that reflects the positional relationship between the other moving vehicle and the user. As shown in FIG. 2, an icon X1 indicating the user's vehicle and an icon X2 indicating the user are arranged in the graphic. In this embodiment, the graphic is further arranged with an icon indicating the other moving vehicle. Then, the notification unit 125 transmits an instruction to display the graphic notification to the terminal 20 via the first communication unit 13.

[0086] Furthermore, the notification unit 125 determines whether or not the other moving vehicle may pose a danger to the user based on the positional relationship between the other moving vehicle and the user. Then, when it is determined that the other moving vehicle may pose a danger to the user, the notification unit 125 transmits to the terminal 20 a vehicle approaching danger notification indicating that the vehicle is approaching and the user is in danger. Specifically, the notification unit 125 creates a graphic reflecting a warning to the user. For example, in the graphic, an icon indicating a warning to the user is superimposed on an icon indicating the other moving vehicle. The notification unit 125 transmits a display instruction for the notification by the graphic to the terminal 20 via the first communication unit 13. Furthermore, the notification unit 125 may transmit to the terminal 20 an instruction to output a sound for warning. Furthermore, the notification unit 125 may transmit to the terminal 20 an instruction to output a vibration for warning.

[0087] Fig. 9 is a flowchart showing an example of the operation of the information notification device 70 of the information notification system 4 according to the fourth embodiment. As shown in Fig. 9, the information notification device 70 of the information notification system 4 according to the fourth embodiment executes the processes of steps S401 to S406 described below between the processes of steps S106 and S107 described above.

[0088] After the process of step S106, the detection unit 123 of the information notification device 70 detects other moving vehicles from images captured by one or more vehicle-mounted cameras 11 (step S401).

[0089] Next, the calculation unit 124 calculates the positional relationship between the other moving vehicles and the user from the images captured by each in-vehicle camera 11 (step S402). Next, the notification unit 125 creates a graphic reflecting the calculated information, that is, the positional relationship between the other moving vehicles and the user (step S403).

[0090] Next, the notification unit 125 transmits an instruction to display a notification using graphics to the terminal 20 (step S404). When the display control unit 222 of the terminal 20 receives the instruction to display a notification using graphics, the display unit 24 displays the graphics.

[0091] Next, the notification unit 125 determines whether or not there is a possibility that the other moving vehicle may pose a danger to the user, based on the positional relationship between the other moving vehicle and the user (step S405). If it is determined that the other moving vehicle may pose a danger to the user (YES in step S405), the notification unit 125 transmits a vehicle approach danger notification to the terminal 20 (step S406). Then, the process proceeds to step S107. On the other hand, if it is determined that the other moving vehicle does not pose a danger to the user (NO in step S405), the process proceeds to step S107.

[0092] Therefore, the information notification device 70 of the information notification system 4 according to the fourth embodiment can also visually and audibly notify the user that another moving vehicle is approaching and posing a danger. Therefore, the information notification device 70 can reduce the risk of, for example, collision with another vehicle when the user is searching for the user's vehicle.

[0093] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention. For example, the control unit 12 of the information notification device 10 of the information notification system 1 according to the first embodiment may include a data registration unit (not shown). The data registration unit receives a registration instruction including user recognition data from the terminal 20, and registers the user recognition data in the storage unit 14 in accordance with the registration instruction. The data registration unit may also receive a deletion instruction including user recognition data from the terminal 20, and delete the user recognition data from the storage unit 14 in accordance with the deletion instruction. In addition, the data registration unit may learn user recognition information using deep learning or the like from images of the user captured on the day by a camera such as the in-vehicle camera 11, and register the learned user recognition information in the memory unit 14.

[0094] Further, each component of the information notification device 10 of the information notification system 1 according to the first embodiment is not limited to a single device, and may be realized by multiple devices. For example, the control unit 12, the first communication unit 13, and the storage unit 14 of the information notification device 10 may be configured as a separate device outside the user's vehicle, and the separate device may communicate with the vehicle-mounted camera 11 having a communication function. Regarding the information notification device 30 according to the second embodiment, the information notification device 60 according to the third embodiment, and the information notification device 70 according to the fourth embodiment, each component is not limited to a single device, and may be realized by multiple devices. Furthermore, in the information notification system 1 according to the first embodiment, the information notification device 10 creates a graphic, for example, as shown in FIG. 2, to be displayed on the terminal device 20, but the location where the graphic is created is not limited to the information notification device 10. For example, the terminal device 20 may create the graphic and display it. Alternatively, an external device such as an external server may create the graphic and the terminal device 20 may display it. Similarly, in the information notification system 2 according to the second embodiment, the location where the graphic, for example, as shown in FIG. 5, to be displayed on the terminal device 40 is created is not limited to the information notification device 30. The same applies to the information notification system 3 according to the third embodiment and the information notification system 4 according to the fourth embodiment.

[0095] Each configuration in the above-described embodiments may be configured by hardware or software, or both, and may be configured by one piece of hardware or software, or may be configured by multiple pieces of hardware or software. Each device and each function (processing) may be realized by a computer having a processor such as a CPU (Central Processing Unit) and a memory serving as a storage device. For example, a program for performing the method in the embodiment (arc bead inspection method) may be stored in the memory, and each function may be realized by executing the program stored in the memory with a processor.

[0096] These programs include instructions (or software codes) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The programs may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The programs may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, a transitory computer-readable medium or a communication medium includes electrical, optical, acoustic, or other forms of propagated signals. [Explanation of symbols]

[0097] 1 Device 1, 2, 3, 4 Information Notification System 10, 30, 60, 70 Information notification device 11. In-vehicle camera 12 Control section 13 First Communications Department 13A 1A Communications Department 13B 1B Communication Department 14 Storage section 20, 40 terminals 21 Operation input section 22 Control section 23 2nd Communications Department 23A 2ndA Communication Department 23B 2B Communication Department 24 Display section 25 Audio output section 26 Vibration section 30 Information notification device 50 Exterior Camera 111 Front Camera 112 Rear Camera 113 Left side camera 114 Right side camera 121 First communication control unit (communication control unit) 121A 1A communication control unit 121B 1st B Communications Control Unit 122 Image acquisition unit 123 Detection unit 124 Calculation section 125 Notification Department 221 Second communication control unit 221A 2ndA communication control section 221B 2nd B Communications Control Unit 222 Display control unit 223 Operation control section 224 Audio output control unit 225 Vibration control unit

Claims

1. A communication control unit that communicates with a terminal used by a user; an image acquisition unit that acquires, when a notification start instruction is received from the terminal, images captured by one or more vehicle-mounted cameras mounted on the user vehicle of the user, the images capturing the surrounding environment of the user vehicle from the one or more vehicle-mounted cameras; a detection unit that detects the user from any of the images captured by the one or more vehicle-mounted cameras; a calculation unit that calculates a positional relationship between the user vehicle and the user based on the images captured by the one or more vehicle-mounted cameras when the user is detected from any of the images captured by the one or more vehicle-mounted cameras; a notification unit that notifies the terminal of information including a positional relationship between the user vehicle and the user. Information notification device.

2. The communication control unit Communicating with one or more external cameras that are different from the on-board camera and are outside the user vehicle and capable of communicating with the user vehicle; the image acquisition unit acquires images captured by the one or more vehicle-mounted cameras from the one or more vehicle-mounted cameras, and acquires images captured by the one or more exterior cameras capturing an environment surrounding the one or more exterior cameras from the one or more exterior cameras; The detection unit detects the user from any one of the images captured by the one or more in-vehicle cameras and the images captured by the one or more exterior cameras; The calculation unit is When the user is detected from either the image captured by the one or more on-board cameras or the image captured by the one or more exterior cameras, a positional relationship between the user's vehicle and the user is calculated based on the image captured by the one or more on-board cameras and the image captured by the one or more exterior cameras. The information notification device according to claim 1 .

3. The detection unit detects another vehicle that is a vehicle different from the user vehicle that is moving, the calculation unit calculates a positional relationship between the other moving vehicle and the user when the other moving vehicle is detected; The notification unit notifies the terminal of information including a positional relationship between the other moving vehicle and the user.

3. The information notification device according to claim 1 or 2.

4. The detection unit detects the other moving vehicle whose distance to the user is equal to or less than a threshold or whose user is present in a traveling direction of the other moving vehicle. The information notification device according to claim 3.

5. The computer Communicating with a terminal used by a user, When a notification start instruction is received from the terminal, images captured by one or more vehicle-mounted cameras mounted on the user vehicle of the user are acquired, the images capturing the surrounding environment of the user vehicle being captured by the one or more vehicle-mounted cameras; Detecting the user from any of the images captured by the one or more vehicle-mounted cameras; When the user is detected from any of the images captured by the one or more vehicle-mounted cameras, a positional relationship between the user vehicle and the user is calculated based on the images captured by the one or more vehicle-mounted cameras; Notifying the terminal of information including a positional relationship between the user vehicle and the user. Information notification method.

Images